News

Incus 6.3 has been released

12th of July 2024

Introduction

The Incus team is pleased to announce the release of Incus 6.3!

image|690x486

The highlight for this release is the initial support for running OCI application containers.
This allows the use of common Docker/OCI images directly through Incus, with those containers living alongside our usual system containers and virtual machines!

As usual, you can try it for yourself online: https://linuxcontainers.org/incus/try-it/
NOTE: The online demo environment can't access the Docker Hub.

Enjoy!

New features

Initial support for OCI application containers

Incus is now capable of accessing application container registries such as the Docker Hub, retrieve images, convert (flatten) them for use by Incus and then create a working containers from them.

This is still very early in our OCI container support and there will likely be quite a few gaps that will need to be filled in based on user feedback, but for many simple cases where people are currently running both Docker and Incus on the same system or where they've been using Docker inside of an Incus container just to run a single piece of software, Incus should now be able to handle that directly.

All of the Incus container configuration options, whether resource limits, system call interception, ... all apply to those containers too. They're also all run in the same safe container environment as our system containers.

stgraber@dakara:~$ incus remote add docker https://docker.io --protocol=oci
stgraber@dakara:~$ incus launch docker:mysql mysql \
> -c environment.MYSQL_DATABASE=wordpress \
> -c environment.MYSQL_USER=wordpress \
> -c environment.MYSQL_PASSWORD=wordpress \
> -c environment.MYSQL_RANDOM_ROOT_PASSWORD=1
Launching mysql

stgraber@dakara:~$ incus list mysql
+-------+---------+----------------------+------------------------------------------+-----------------+-----------+
| NAME  |  STATE  |         IPV4         |                   IPV6                   |      TYPE       | SNAPSHOTS |
+-------+---------+----------------------+------------------------------------------+-----------------+-----------+
| mysql | RUNNING | 172.17.250.26 (eth0) | 2602:fc62:c:250:216:3eff:fefa:468 (eth0) | CONTAINER (APP) | 0         |
+-------+---------+----------------------+------------------------------------------+-----------------+-----------+

stgraber@dakara:~$ incus launch docker:wordpress wordpress \
> -c environment.WORDPRESS_DB_HOST=172.17.250.26 \
> -c environment.WORDPRESS_DB_USER=wordpress \
> -c environment.WORDPRESS_DB_PASSWORD=wordpress \
> -c environment.WORDPRESS_DB_NAME=wordpress
Launching wordpress

stgraber@dakara:~$ incus list wordpress
+-----------+---------+-----------------------+-------------------------------------------+-----------------+-----------+
|   NAME    |  STATE  |         IPV4          |                   IPV6                    |      TYPE       | SNAPSHOTS |
+-----------+---------+-----------------------+-------------------------------------------+-----------------+-----------+
| wordpress | RUNNING | 172.17.250.119 (eth0) | 2602:fc62:c:250:216:3eff:fe61:c1fc (eth0) | CONTAINER (APP) | 0         |
+-----------+---------+-----------------------+-------------------------------------------+-----------------+-----------+
stgraber@dakara:~$

Baseline CPU definition within clusters

One big limitation of Incus' live migration logic so far has been that it expected all servers within a cluster to run identical CPUs. Should the CPU differ between two systems, the live migration would fail or cause crashes later on.

That's because Incus would always expose all the CPU flags from the machine it runs on.
This is good to get the maximum amount of performance on a standalone system, but in a heterogeneous cluster, this doesn't quite work.

With this release, Incus will now automatically compute the set of common CPU flags across all servers for a given CPU architecture and use that as the CPU definition for any instance running with live-migration enabled (migration.stateful=true).

Filesystem support for io.bus and io.cache

The io.bus and io.cache options have been around for VM disks for a little while now.
With io.bus offering the option of virtio-scsi, virtio-blk or nvme and io.cache allowing for none, writeback or unsafe caching.

Those config keys are now also supported when passing in filesystems rather than disks.
Their values in such cases are a bit different with io.bus being one of auto (default), 9p or virtiofs and io.cache supporting none (default), metadata or unsafe.

This effectively allows controlling exactly how a filesystem is exposed to the VM and then tweaking caching behavior when using virtiofs.

Improvements to incus top

Incus 6.2 introduced the new incus top command.
With this release, we're making it more useful by having it work against remote servers, properly support clustered environments and also handling projects.

+---------+---------------+-------------+-----------+-----------+
| PROJECT | INSTANCE NAME | CPU TIME(S) |  MEMORY   |   DISK    |
+---------+---------------+-------------+-----------+-----------+
| default | incus-ui      | 63.40       | 12.76MiB  | 1.54GiB   |
+---------+---------------+-------------+-----------+-----------+
| default | kernel-test   | 1865037.10  | 578.01MiB | 32.84GiB  |
+---------+---------------+-------------+-----------+-----------+
| default | speedtest     | 84.10       | 23.14MiB  | 400.12MiB |
+---------+---------------+-------------+-----------+-----------+
| default | win11         | 1865.11     | 15.51GiB  |           |
+---------+---------------+-------------+-----------+-----------+
| demo    | mysql         | 6.77        | 464.20MiB | 276.62MiB |
+---------+---------------+-------------+-----------+-----------+
| demo    | wordpress     | 1.81        | 53.66MiB  | 386.62MiB |
+---------+---------------+-------------+-----------+-----------+
| vpn     | vpn-dev       | 102.97      | 36.83MiB  | 412.00MiB |
+---------+---------------+-------------+-----------+-----------+
| vpn     | vpn-lab       | 57.29       | 27.03MiB  | 347.75MiB |
+---------+---------------+-------------+-----------+-----------+
Press 'd' + ENTER to change delay
Press 's' + ENTER to change sorting method
Press CTRL-C to exit

Delay: 10s
Sorting Method: Alphabetical

CPU flags in server resources

The resources API which is used to expose a lot of details about the machine's hardware configuration has now been updated to expose the CPU flags.

This was required to implement the baseline CPU feature mentioned previously.
The new data can be found in the API directly and is provided for each CPU core.

stgraber@dakara:~$ incus query /1.0/resources | jq .cpu.sockets[0].cores[0].flags -c
["fpu","vme","de","pse","tsc","msr","pae","mce","cx8","apic","sep","mtrr","pge","mca","cmov","pat","pse36","clflush","mmx","fxsr","sse","sse2","ht","syscall","nx","mmxext","fxsr_opt","pdpe1gb","rdtscp","lm","constant_tsc","rep_good","nopl","xtopology","nonstop_tsc","cpuid","extd_apicid","aperfmperf","rapl","pni","pclmulqdq","monitor","ssse3","fma","cx16","sse4_1","sse4_2","x2apic","movbe","popcnt","aes","xsave","avx","f16c","rdrand","lahf_lm","cmp_legacy","svm","extapic","cr8_legacy","abm","sse4a","misalignsse","3dnowprefetch","osvw","ibs","skinit","wdt","tce","topoext","perfctr_core","perfctr_nb","bpext","perfctr_llc","mwaitx","cpb","cat_l3","cdp_l3","hw_pstate","ssbd","mba","ibrs","ibpb","stibp","vmmcall","fsgsbase","bmi1","avx2","smep","bmi2","erms","invpcid","cqm","rdt_a","rdseed","adx","smap","clflushopt","clwb","sha_ni","xsaveopt","xsavec","xgetbv1","xsaves","cqm_llc","cqm_occup_llc","cqm_mbm_total","cqm_mbm_local","clzero","irperf","xsaveerptr","rdpru","wbnoinvd","cppc","arat","npt","lbrv","svm_lock","nrip_save","tsc_scale","vmcb_clean","flushbyasid","decodeassists","pausefilter","pfthreshold","avic","v_vmsave_vmload","vgif","v_spec_ctrl","umip","pku","ospke","vaes","vpclmulqdq","rdpid","overflow_recov","succor","smca","fsrm","debug_swap"]

Unified image support in incus-simplestreams

The incus-simplestreams tool which is used to manage a static web server hosting Incus images using the simplestreams index format has now been updated to support not just split images but also unified images.

Incus images can either be made of two files, one containing the metadata files and one containing the rootfs or root disk, or a single tarball which contains both the metadata and then the rootfs or root disk as a directory/file inside of that single tarball.

To add a unified image to the server, simply call incus-simplestreams add with a single file rather than the usual two.

Completion of libovsdb transition

For the past 4-5 releases, we've been slowly migrating more and more logic from direct calls to the ovs-vsctl, ovn-nbctl and ovn-sbctl command line tools to instead using a native OVSDB client.

This work is now complete and Incus no longer requires any of the OVS/OVN tools be present on the system to interact with OVN.

The new logic keeps a persistent connection to the relevant databases, significantly reducing the time and CPU overhead needed to interact with OVN. This persistent connection will also allow receiving and reacting to events directly from OVN, something which wasn't possible with the previous approach.

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • incus/project: Fix bad --show-access output
  • cmd/incus-user: Avoid double user-user- in network description
  • Translated using Weblate (German)
  • Translated using Weblate (Japanese)
  • incus/admin_sql: Fix description
  • incus/storage_bucket: Fix string quoting
  • incus/profile: Fix examples
  • incus/project: Fix examples
  • incus/snapshot: Improve restore example
  • incus/storage_bucket: Fix typoes in examples
  • incus/storage_bucket: Fix export example
  • incus/exec: Add some examples
  • i18n: Update translation templates
  • incus-user: Don't needlessly update the default profile
  • incus/top: Support remote servers
  • incus/top: Properly handle projects
  • incus/top: Handle clusters
  • incusd/instance/qemu: Avoid endianness issues with vsockIDInt
  • internal/linux: Define some IOCTLs
  • incusd/instance/qemu: Don't use hardcoded ioctl
  • incusd/storage/btrfs: Don't use hardcoded ioctl
  • incusd/devices: Simplify ioctl logic
  • shared/cliconfig: Remove old migration logic
  • shared/cliconfig: Generalize logic
  • incusd/seccomp: Fix sysinfo logic on 32bit platforms
  • shared/cliconfig: Always fill in the protocol
  • incus: Generalize image server logic
  • incus/console: Re-shuffle logic a bit
  • incus: Handle stopped containers in --console
  • incus/console: Don't export an internal function
  • doc: update documentation for forming cluster with existing server
  • github: Cleanup workflow file
  • github: Build go tip
  • github: Change Go releases in tests
  • test/lint/golangci: Properly pull the parent ref
  • cmd/incusd: Fix typo in forknet
  • api: resources_cpu_flags
  • shared/api: Add Flags to ResourceCPUCore
  • doc/rest-api: Refresh swagger YAML
  • incusd/resources: Add CPU Flags to ResourceCPUCore
  • incusd/network/ovn: Port CreateLogicalRouterRoute to libovsdb
  • incusd/network/ovn: Port DeleteLogicalRouterRoute to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Port DeleteLogicalRouterPort to libovsdb
  • incusd/network/ovn: Remove LogicalRouterPortDeleteIPv6Advertisements
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Port DeleteLogicalSwitch to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Remove logicalSwitchFindAssociatedPortGroups
  • doc/instances_console: Tweak wording on SPICE clients
  • incusd/network/ovn: Special handling for Load Balancer table
  • incusd/network/ovn: Align functions context handling
  • incusd/network/ovn: Port DeleteLogicalSwitchDHCPOption to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchPortLocation to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchPortUUID to libovsdb
  • incusd/network/ovn: Port GetLogicalRouterPortHardwareAddress to libovsdb
  • incusd/network/ovn: Add GetLogicalRouter
  • incusd/network/ovn: Port DeleteLoadBalancer to libovsdb
  • incusd/network/acl: Update for OVN function changes
  • incusd/network: Update for OVN function changes
  • incusd/network: Simplify OVN network deletion logic
  • incus/network_load_balancer: Fix example
  • i18n: Update translation templates
  • incusd/network/ovn: Port UpdateLogicalSwitchIPAllocation to libovsdb
  • incusd/network/ovn: Port UpdateLogicalSwitchDHCPv4Revervations to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchDHCPv4Revervations to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchDHCPOptions to libovsdb
  • incusd/network/ovn: Port UpdateLogicalSwitchDHCPv4Options to libovsdb
  • incusd/network/ovn: Port UpdateLogicalSwitchDHCPv6Options to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/networks: Properly finalize OVN networks
  • incusd/networks: Properly record description
  • incusd/response: Add Code function
  • incusd/operations: Implement Code function
  • incusd: Implement Code function
  • incus-agent: Implement Code function
  • client: Fix OIDC re-authentication on POST
  • client: Fix OIDC re-authentication on websocket
  • incus/network: Add missing stdin handling
  • i18n: Update translation templates
  • lxd-to-incus: Handle volume config keys
  • incusd/project: Don't fail creation on authorizer
  • doc/instance_units: Clarify usage
  • incusd/network/ovn: Port logicalSwitchPortACLRules to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchPorts to libovsdb
  • incusd/network/ovn: Port UpdateLogicalSwitchPortOptions to libovsdb
  • incusd/network/ovn: Port CreatePortGroup to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/device/nic: Update for OVN function changes
  • incusd/network/acl: Update for OVN function changes
  • incusd/network/ovn: Port GetPortGroupsByProject to libovsdb
  • incusd/network/ovn: Port CreateAddressSet to libovsdb
  • incusd/network/ovn: Port UpdateAddressSetAdd to libovsdb
  • incusd/network/ovn: Port UpdateAddressSetRemove to libovsdb
  • incusd/network/ovn: Port DeleteAddressSet to libovsdb
  • incusd/network/acl: Update for OVN function changes
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Port UpdateLogicalSwitchPortLinkRouter to libovsdb
  • incusd/network/ovn: Port UpdateLogicalSwitchPortLinkProviderNetwork to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchIPs to libovsdb
  • incusd/network/ovn: Port GetLogicalSwitchPortDNS to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Port UpdateLogicalSwitchPortDNS to libovsdb
  • incusd/network/ovn: Port UpdatePortGroupMembers to libovsdb
  • incusd/network/ovn: Port UpdateLogicalRouterPolicy to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Port CreateLoadBalancer to libovsdb
  • incusd/network/ovn: Port GetLogicalRouterRoutes to libovsdb
  • incusd/network/ovn: Port DeleteLogicalRouterPeering to libovsdb
  • incusd/network: Update for OVN function changes
  • incusd/apparmor: Update for current QEMU
  • incusd/apparmor: Allow /dev/shm in forkproxy
  • incusd/network/ovn: Port CreateLogicalRouterPeering to libovsdb
  • incusd/network: Update for OVN function changes
  • Translated using Weblate (Chinese (Simplified))
  • incusd/network/ovn: Port logicalSwitchPortDeleteDNSOperations to libovsdb
  • incusd/network/ovn: Port DeleteLogicalSwitchPortDNS to libovsdb
  • incusd/network/ovn: Port logicalSwitchPortDeleteOperations to libovsdb
  • incusd/network/ovn: Port CleanupLogicalSwitchPort to libovsdb
  • incusd/network/ovn: Port aclRuleDeleteOperations to libovsdb
  • incusd/network/ovn: Port aclRuleAddOperations to libovsdb
  • incusd/network/ovn: Port ClearPortGroupPortACLRules to libovsdb
  • incusd/network/ovn: Port UpdatePortGroupPortACLRules to libovsdb
  • incusd/network/ovn: Port UpdateLogicalSwitchACLRules to libovsdb
  • incusd/network/ovn: Port UpdatePortGroupACLRules to libovsdb
  • incusd/network/acl: Update for OVN function changes
  • incusd/network: Update for OVN function changes
  • incusd/network/ovn: Remove nbctl
  • api: disk_io_bus_cache_filesystem
  • incusd/device/disk: Extend io.bus option
  • incusd/device/disk: Extend io.cache option
  • incusd/device/disk: Add support for io.cache on virtiofs
  • incusd/device/disk: Add support for io.bus on filesystems
  • incusd/instance/driver_qemu: Handle 9p being disabled
  • doc: Update configs
  • doc/installing: Update Debian/Ubuntu build instructions
  • doc/installing: Mention installing Go from upstream
  • incusd/instance/edk2: Add new package to track EDK2 firmwares
  • incusd/instance/qemu: Update to the new edk2 package
  • incusd/apparmor: Update to the new edk2 package
  • doc: Cleanup OVMF/EDK2 handling to cover aarch64
  • doc/installing: Use Incus 6.0.0 as example
  • incusd/instance/qemu: Fix handling of virtiofs-only disks
  • incus/storage_volume: Tweak help messages
  • i18n: Update translation templates
  • incus/storage_volume: Fix lint
  • doc/installing: Mention incus-tools package
  • incus-simplestreams: Add support for unified images
  • incus-simplestreams: Tweak help message
  • incus-simplestreams: Refactor unified logic
  • gomod: Update dependencies
  • incusd/apparmor: Allow devpts mounts
  • incusd: Improve profile rename errors
  • incusd/sys: Add cluster resources cache path
  • incusd/daemon: Locally cache other server resources
  • incusd/instance/drivers/qmp: Add QueryCPUModel
  • incusd/instance/qemu: Use cluster CPU flags for migration.stateful
  • incus-user: Use shorter interrface name for long UIDs
  • incusd/device/network: Fix Tap interface MTU when in OVN
  • incusd/isntance: Don't expose all internal flags in INFO message
  • incusd/instance/lxc: Allow calling Update from a Create operation
  • cmd/incusd: Add forknet dhcp
  • shared/subprocess: Allow building on Windows
  • api: instance_oci
  • client: Add basic OCI registry client
  • incus: Add OCI remote support
  • shared/cliconfig: Add OCI remote support
  • incusd: Add OCI registry support
  • incusd/instance/lxc: Basic OCI support
  • internal/instance: Add volatile.container.oci
  • incusd/instance/lxc: Add volatile.container.oci
  • incus: Add support for volatile.container.oci
  • incusd/instance: Handle OCI config on create from image
  • tests: Add basic OCI test
  • gomod: Update dependencies
  • doc: Update configs
  • doc: Add OCI to wordlist
  • i18n: Update translation templates
  • shared/subprocess: Fix gofmt
  • incusd/storage/lvmcluster: Don't allow buckets
  • incusd/storage/lvmcluster: Don't exclusively lock ISO volumes
  • incusd/device/disk: Allow attaching the same ISO to multiple instances
  • incusd/device/disk: Allow live-migration with agent/cloud-init disks
  • incusd/instance/qemu: Fix live-migration with agent/cloud-init disks
  • incusd/device/disk: Don't crash on uninitialized pool
  • incusd/storage/lvmcluster: Always use shared access
  • incusd/instance/lxc: Don't report filesystem metrics when no per-instance value
  • incus/top: Set interval to 10s (minimum server-side is 8)
  • incus/top: Hide zero values
  • incusd/device/disk: Mark virtual disks as always migratable
  • tests: Update metrics test for recent change

Documentation

The Incus documentation can be found at:
https://linuxcontainers.org/incus/docs/main/

Packages

There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Installing the Incus server on Linux

Incus is available for most common Linux distributions. You'll find detailed installation instructions in our documentation.

https://linuxcontainers.org/incus/docs/main/installing/

Homebrew package for the Incus client

The client tool is available through HomeBrew for both Linux and MacOS.

https://formulae.brew.sh/formula/incus

Chocolatey package for the Incus client

The client tool is available through Chocolatey for Windows users.

https://community.chocolatey.org/packages/incus/6.3.0

Winget package for the Incus client

The client tool is also available through Winget for Windows users.

https://winstall.app/apps/LinuxContainers.Incus

Support

Monthly feature releases are only supported up until the next release comes out. Users needing a longer support length and less frequent changes should consider using Incus 6.0 LTS instead.

Community support is provided at: https://discuss.linuxcontainers.org
Commercial support is available through: https://zabbly.com/incus
Bugs can be reported at: https://github.com/lxc/incus/issues

Incus 6.0.1 LTS has been released

28th of June 2024

Introduction

The Incus team is pleased to announce the release of Incus 6.0.1!

This is the first bugfix release for Incus 6.0 which is supported until June 2029.

Changes

As usual this bugfix releases focus on stability and hardening.

Minor improvements have also been backported, specifically anything which does not require data migration, database changes or cause any unexpected change to user facing behavior.

The number of such improvements will decrease over time within the LTS branch.

Some of the highlights for this release are:

  • Extended source syntax for ZFS pools (allows mirror & raidz1/raidz2)
  • Cross-project listing on all objects (instances, profiles, images, storage volumes/buckets, networks, ...)
  • Additional functions exposed to instance placement scriptlet
  • All create sub-commands in the CLI now accept YAML input
  • All list sub-commands in the CLI now accept customizable columns
  • The migration.stateful config key was expanded to containers too
  • Stateless network ACLs are now supported on OVN
  • New timestamp exposed for instance uptime
  • New incus top command (uses existing metric API)
  • System load information in incus info --resources
  • PCI devices information in incus info --resources
  • Ability to query who has access to a given project or instance
  • Forceful deletion of projects
  • Improved alias handling in incus-simplestreams

The full list of commits is available below:

Detailed changelog
  • doc/support: Update for LTS
  • incusd/network: Remove bridge.driver=native requirement for extended external_interfaces syntax
  • doc/network/bridge: Update extended external_interfaces documentation
  • incusd/storage/drivers/zfs: Simplify dataset receive and fix progress handling
  • gomod: Update dependencies
  • mini-oidc: Merge into main gomod
  • gomod: Update dependencies
  • go.mod: Bump package major version
  • global: Update Go package to v6
  • test: Update godeps.list
  • README: Update for godoc URLs
  • doc/rest-api: Refresh swagger YAML
  • cmd/incus: Fix import ordering
  • incusd: Remove unneeded import renames
  • incusd/instance: Fix duplicate import
  • doc/projects: Tweak examples
  • shared/api: Remove container resources (deprecated)
  • doc/rest-api: Refresh swagger YAML
  • incus/create: Remove dead code
  • i18n: Update translation templates
  • incusd/daemon: Remove old migration logic
  • incusd: Stop mentioning containers in resources
  • doc/rest-api: Use instances API in example
  • incusd/db/cluster: containers URLs aren't valid in Incus
  • incusd/instances: Don't start instances when evacuated
  • doc/installing: Sort source-build distro instructions
  • doc/installing: Add OpenSUSE source instructions
  • doc/installing: Add Alpine instructions
  • incus/aliases: fix completion regression
  • incus/snapshot: Fix deletion of snapshots containing colons
  • incusd/instance/drivers: Have SR-IOV get stable MACs
  • incusd/device/nic_sriov: Use stable MAC
  • incus/profile: Add support for creating from YAML
  • i18n: Update translation templates
  • incusd/instance/lxc: Only apply soft cgroup limits on cgroup1
  • incus/admin: Don't hide the sql command
  • grafana: Refresh dashboard
  • doc/metrics: Mention Loki in Grafana setup
  • grafana: Better filter Loki events by project
  • incusd/loki: Use hostname as default instance property on standalone systems
  • incusd/loki: Re-order config fields
  • incusd/loki: Allow overriding the location field
  • incusd/loki: Set location field to local hostname on standalone systems
  • incus/projct: Add support for creating project from yaml
  • i18n: Update translation templates
  • incusd/network/ovs: Port GetOVNEncapIP to libovsdb
  • incusd/network/ovs: Add some comments to GetOVNBridgeMappings
  • incusd/network/ovs: Port AddOVNBridgeMapping to libovsdb
  • incusd/network/ovs: Port RemoveOVNBridgeMapping to libovsdb
  • incusd/network/ovs: Port GetHardwareOffload to libovsdb
  • incusd/network/ovs: Port GetBridgePorts to libovsdb
  • incusd/network/ovs: Port UpdateBridgePortVLANs to libovsdb
  • incusd/network/ovs: Port AssociateInterfaceOVNSwitchPort to libovsdb
  • incusd/network/ovs: Switch Installed to checking for unix socket
  • incusd/network: Update for OVS function changes
  • Add missing SecureBoot firmware names
  • incus/snapshot: Add support for creating project from yaml
  • i18n: Update translation templates
  • api: network_zones_all_projects
  • shared/api: Add Project field to NetworkZone
  • incusd/network_zones: Add support for all-projects
  • doc/rest-api: Refresh swagger YAML
  • client: Add GetNetworkZonesAllProjects
  • incus/network_zone: Add --all-projects flag to list
  • i18n: Update translation templates
  • tests: Add all-projects network zone test
  • incusd/network/ovn: Port LogicalRouterAdd to libovsdb
  • incusd/network/ovn: Remove LogicalRouterAdd
  • incusd/network: Replace LogicalRouterAdd usage with CreateLogicalRouter
  • incusd/network/ovn: gofmt
  • incusd/cluster: Disable networks during evacuation
  • incusd/cgroup: Set hugepages reserved limits
  • incusd/storage/drivers: Introduce SparseFileWrapper
  • incusd/storage/drivers/vfs: Use SparseFileWrapper on backup import
  • incusd/storage/drivers/vfs: Use SparseFileWrapper on volume migration
  • incus/storage: Support creating storage pool from yaml
  • i18n: Update translation templates
  • incus/info: Show CPU architecture as separate line
  • i18n: Update translation templates
  • incus/cluster_group: Add yaml support for cluster group create
  • i18n: Update translation templates
  • gitignore: Add JetBrains
  • api: storage_zfs_vdev
  • doc: Update ZFS support for multiple block devices and vdev types in doc/reference/storage_zfs.md
  • incusd/storage/zfs: Add support for vdev type and multiple block devices
  • incus/info: Sorting network interfaces
  • incus/network_acl: Add cmd.Example for network acl create
  • i18n: Update translation templates
  • incus/network_forward: Add yaml example for create
  • i18n: Update translation templates
  • incus/config_template: Add file support for create
  • i18n: Update translation templates
  • incus/network_integrations: Add yaml support for create
  • i18n: Update translation templates
  • incusd/storage/s3: Use 'mc' client
  • incusd/storage: Switch to use minio's 'mc' client
  • gomod: Update dependencies
  • github: Download MinIO client
  • doc: Avoid MyST 3.0.0
  • incus/doc/installing.md: Add Docker information
  • doc: Add Podman to wordlist
  • incus/cluster: Add columns to list
  • i18n: Update translation templates
  • incus/project: Add customazible columns to list
  • i18n: Update translation templates
  • api: container_migration_stateful
  • internal/instance: Add migration.stateful to containers
  • incusd/instance/lxc: Add checks for migration.stateful
  • doc: Update configs
  • incus/network_load_balancer: add Example to create
  • i18n: Update translation templates
  • incus/network_zone: Add example for create command
  • i18n: Update translation templates
  • doc: Fix bad sphinx requirements
  • incusd/instances/qemu: Tweak secureboot firmware list
  • cmd/incus/admin_cluster: Add libexec path for incusd
  • incus/storage: Show usage when no driver passed
  • incusd/storage/drivers/dir: Tweak path validation
  • incusd/backup: Show profile list on lookup error
  • incusd/apparmor/lxc: Allow access to binfmt_misc
  • incusd/apparmor/lxc: Refresh generated rules
  • incusd/storage: Handle instance volume size on import
  • incus/profile: Add customizable columns to list
  • i18n: Update translation templates
  • incus/project: Fix help message for list
  • i18n: Update translation templates
  • api: profiles_all_projects
  • shared/api: Add Project field to Profile
  • client: Add GetProfilesAllProjects
  • incus/profile: Add all-projects to list
  • incusd/db/cluster: Add Project field to Profile
  • incusd/profile: Add all-projects support
  • doc/rest-api: Refresh swagger YAML
  • i18n: Update translation templates
  • incus/storage_volume: Clarify volume errors
  • incusd/apparmor/lxc: Fix access to kernel/security/apparmor
  • api: instances_scriptlet_get_instances
  • doc/instances/scriptlet: Add get_instances
  • incusd/scriptlet: Add get_instances
  • api: instances_scriptlet_get_cluster_members
  • doc/instances/scriptlet: Add get_cluster_members
  • incusd/scriptlet: Add get_cluster_members
  • api: Add network_acl_stateless
  • doc/network_acl: Add allow-stateless action
  • incusd/network/acl: Add allow-stateless action
  • incusd/network/ovn/nb: Port DeleteLogicalRouter to libovsdb
  • incusd/network/ovn/nb: Port CreateLogicalRouterSNAT to libovsdb
  • incusd/network: Update for OVS function changes
  • incusd/network/acl: Properly run instance counting outside of ACL loop
  • incusd/network/ovn: Wait up to 1s for dynamic IPs
  • incusd/network/ovn/nb: Port DeleteLogicalRouterNAT to libovsdb
  • incusd/network: Update for OVS function changes
  • shared/archive: Fix typo
  • incusd/cluster: Re-organize joining logic
  • incusd/cluster: Ignore OVN networks during joining
  • shared/archive: Properly anchor exclude rules
  • incusd/project: Rework low-level permission checks
  • incus/storage_bucket: Add example for storage bucket create
  • i18n: Update translation templates
  • incus/network_peer: Add example for create command
  • i18n: Update trasnlation templates
  • api: instance_state_started_at
  • shared/api: Add StartedAt to InstanceState
  • doc/rest-api: Refresh swagger YAML
  • incusd/instance: Add StartedAt to InstanceState
  • incus/info: Add Started field
  • incus/list: Add started at column
  • i18n: Update translation templates
  • Makefile: Cleanup gomod update
  • gomod: Update dependencies
  • tests/mini-oidc: Bump to go-jose/v4
  • client/connection: Add support for the socket existing in /run/incus
  • incusd/instance/lxc: Add gendoc comments for image restrictions
  • incusd/instance/qemu: Add gendoc comments for image restrictions
  • doc: Update configs
  • doc/image-handling: Use gendoc data
  • incus/storage_bucket: Add yaml support for key create
  • i18n: Update translation templates
  • incusd/instance/qemu: Fix StartedAt handling
  • incus/storage: Customizable columns in list
  • i18n: Update translation templates
  • incusd/network/ovn: Port LogicalRouterSNATAdd and LogicalRouterDNATSNATAdd to libovsdb
  • incusd/network: Update for OVN function changes
  • api: instances_scriptlet_get_project
  • doc/instances/scriptlet: Add get_project
  • incusd/scriptlet: Add get_project
  • api: networks_all_projects
  • shared/api: Add Project field to Network
  • client: Add GetNetworksAllProjects
  • incus/network: Add all-projects
  • incusd/db: Add GetNetworksAllProjects
  • incusd/networks: Add all-projects
  • internal/instance: Add gendoc for limits.kernel
  • doc: Update configs
  • doc: Use gendoc for limits.kernel
  • api: network_acls_all_projects
  • shared/api: Add Project field to NetworkACL
  • client: Add GetNetworkACLsAllProjects
  • incus/network/acl: Add all-projects
  • incusd/db: Add GetNetworkACLsAllProjects
  • incusd/network/acl: Set Project field
  • incusd/network_acls: Add all-projects
  • api: storage_buckets_all_projects
  • shared/api: Add Project field to StorageBucket
  • client: Add GetStoragePoolBucketsAllProjects
  • incus/storage/bucket: Add all-projects
  • incusd/db/storage_buckets: Fill Project field
  • incusd/storage_buckets: Add all-projects
  • i18n: Update translation templates
  • incusd/networks: Fix import shadowing
  • doc/rest-api: Refresh swagger YAML
  • client: Align GetProfilesAllProjects with GetProfiles
  • client: Align GetNetworkZonesAllProjects with GetNetworkZones
  • client: Standardize the GetNetworkAllocation functions
  • incus/network_allocations: Update for client changes
  • incusd/device/usb: Add gendoc for the USB device
  • doc: Update configs
  • doc: Use gendoc for USB devices
  • api: resources_load
  • shared/api: Add Load to resources API
  • doc/rest-api: Refresh swagger YAML
  • incusd/resources: Add load information
  • incus/info: Add load information
  • i18n: Update translation templates
  • incusd/device/unix: Add gendoc comments
  • doc: Update configs
  • doc/devices_unix_block.md: Use gendoc data
  • doc/devices_unix_char.md: Use gendoc data
  • doc/devices_unix_hotplug.md: Use gendoc data
  • incus/top: Add new command
  • i18n: Update translation templates
  • incusd/network/zone: add gendoc for network zone
  • doc: Update configs
  • doc: Use gen doc for network zones
  • incusd/device/unix: Run gofmt
  • incus/info: Add PCI devices to --resources
  • i18n: Update translation templates
  • incusd/device/disk: Add gendoc comments
  • doc: Update configs
  • doc/devices/disk: Use gendoc
  • incus/network: Customizable columns in list
  • i18n: Update translation templates
  • incusd/network_zones: Fix listing of zones within a project
  • incusd/instance/lxc: Fix handling of migration.stateful
  • gomod: Update dependencies
  • incusd/network/ovs: Fix bad VLANMode value
  • fix: close resources
  • incusd/instance: Fix building on 32bit architectures
  • incus/network_zone: Add example for network zone record create
  • i18n: Update translation template
  • incus/storage_volume: Add yaml support for create
  • i18n: Update translation templates
  • cmd/incus/info: Fix runtime error when chassis, motherboard and firwmare information is not available
  • Translated using Weblate (German)
  • incusd/instance/qemu: Allow setting protection.delete when running
  • doc/api-extension: Fix typo
  • shared/api: Introduce Access structs
  • api: instance_access
  • incusd/auth: Introduce GetInstanceAccess
  • incusd/instance: Add access endpoint
  • api: project_access
  • incusd/auth: Introduce GetProjectAccess
  • incusd/project: Add access endpoint
  • doc/rest-api: Refresh swagger YAML
  • client: Add GetInstanceAccess
  • client: Add GetProjectAccess
  • incus/info: Fix description of --show-log
  • incus/info: Add --show-access
  • incus/project: Add --show-access to info
  • i18n: Update translation templates
  • incusd/auth/fga: Rename manager by admin in model
  • incusd/auth/fga: Rework permission model
  • incusd/auth/fga: Rebuild model
  • tests: Fix for permission changes
  • incusd/instance/agent-loader: Support installing to /etc
  • incusd/apparmor/lxc: Fix rule syntax
  • incus-simplestreams add: added flags: --no-default-alias, --alias. #875
  • incus/storage_volume/snapshot: Support YAML for creation
  • i18n: Update translation templates
  • shared/idmap: Make get_userns_fd configure the userns
  • incus-migrate: Handle valid CA certificates
  • incusd/instances_post: Fix migrating into remote cluster
  • incusd/apparmor: Detect nosymfollow support
  • incusd: Set SELinux label on socket
  • incus/network: Align attach-profile with attach
  • create_detached_idmapped_mount: avoid double close
  • incusd/instance/qemu: Extend missing QEMU error
  • doc/installing: Mention extra packages for VMs
  • incusd/storage/btrfs: Fix btrfs argument order
  • incusd/seccomp/sysinfo: Handle 32bit on 64bit
  • api: projects_force_delete
  • incusd/api_project: Add force delete endpoint
  • doc/rest-api: Refresh swagger YAML
  • client: Introduce DeleteProjectForce
  • cmd/incus/project: Add --force to delete
  • i18n: Update translation templates
  • incusd/project: Simplify projectIsEmpty
  • incusd/db: Introduce GetNetworkZoneURIs
  • incusd/db: Introduce GetStorageBucketURIs
  • incusd/api_project: Fix UsedBy
  • incusd/api_project: Add force deletion logic
  • incus/completion: Reduce API calls
  • incus/publish: Complete snapshot names
  • incus/completion: Fix import shadowing
  • Translated using Weblate (French)
  • Makefile: Pin go-acme/lego for Go 1.21
  • Update dependencies
  • cmd/incus/console: Cleanup --show-log
  • incusd/instance_console: Remove old check
  • incusd/instance_console: Handle missing log file
  • incusd/instance_console: Don't fail on empty logs
  • incusd/instance_console: Cleanup error message
  • i18n: Update translation templates
  • incusd/device/sriov: Line up code with comment
  • incus/project: Fix bad --show-access output
  • cmd/incus-user: Avoid double user-user- in network description
  • Translated using Weblate (German)
  • Translated using Weblate (Japanese)
  • incus/admin_sql: Fix description
  • incus/storage_bucket: Fix string quoting
  • incus/profile: Fix examples
  • incus/project: Fix examples
  • incus/snapshot: Improve restore example
  • incus/storage_bucket: Fix typoes in examples
  • incus/storage_bucket: Fix export example
  • incus/exec: Add some examples
  • i18n: Update translation templates
  • incus-user: Don't needlessly update the default profile
  • incus/top: Support remote servers
  • incus/top: Properly handle projects
  • incus/top: Handle clusters
  • incusd/instance/qemu: Avoid endianness issues with vsockIDInt
  • internal/linux: Define some IOCTLs
  • incusd/instance/qemu: Don't use hardcoded ioctl
  • incusd/storage/btrfs: Don't use hardcoded ioctl
  • incusd/devices: Simplify ioctl logic
  • shared/cliconfig: Remove old migration logic
  • shared/cliconfig: Generalize logic
  • incusd/seccomp: Fix sysinfo logic on 32bit platforms
  • shared/cliconfig: Always fill in the protocol
  • incus: Generalize image server logic
  • incus/console: Re-shuffle logic a bit
  • incus: Handle stopped containers in --console
  • incus/console: Don't export an internal function
  • doc: update documentation for forming cluster with existing server
  • github: Cleanup workflow file
  • github: Build go tip
  • github: Change Go releases in tests
  • test/lint/golangci: Properly pull the parent ref
  • cmd/incusd: Fix typo in forknet
  • api: resources_cpu_flags
  • shared/api: Add Flags to ResourceCPUCore
  • doc/rest-api: Refresh swagger YAML
  • incusd/resources: Add CPU Flags to ResourceCPUCore
  • doc/instances_console: Tweak wording on SPICE clients
  • incus/network_load_balancer: Fix example
  • i18n: Update translation templates
  • incusd/networks: Properly finalize OVN networks
  • incusd/networks: Properly record description
  • incusd/response: Add Code function
  • incusd/operations: Implement Code function
  • incusd: Implement Code function
  • incus-agent: Implement Code function
  • client: Fix OIDC re-authentication on POST
  • client: Fix OIDC re-authentication on websocket
  • incus/network: Add missing stdin handling
  • i18n: Update translation templates
  • lxd-to-incus: Handle volume config keys
  • incusd/project: Don't fail creation on authorizer
  • doc/instance_units: Clarify usage
  • incusd/apparmor: Update for current QEMU
  • incusd/apparmor: Allow /dev/shm in forkproxy
  • Translated using Weblate (Chinese (Simplified))
  • doc/installing: Update Debian/Ubuntu build instructions
  • doc/installing: Mention installing Go from upstream
  • doc/installing: Use Incus 6.0.0 as example

Support and upgrade

The Incus 6.0 branch is supported until June 2029. It's always strongly recommended to keep up and run the latest LTS bugfix release.

Downloads

Incus 6.2 has been released

31st of May 2024

Introduction

The Incus team is pleased to announce the release of Incus 6.2!

This release contains the second wave of changes contributed by students of the University of Texas at Austin and a few other features and improvements.

image|690x454

As usual, you can try it for yourself online: https://linuxcontainers.org/incus/try-it/

Enjoy!

New features

New incus top command

A new incus top command was added. This builds on top of Incus' built-in OpenMetrics endpoint and allows for a refreshing view of the instance list, including CPU, memory and disk usage.

+---------------+-------------+-----------+-----------+
| INSTANCE NAME | CPU TIME(S) |  MEMORY   |   DISK    |
+---------------+-------------+-----------+-----------+
| foo           | 6.73        | 12.44MiB  | 341.88MiB |
+---------------+-------------+-----------+-----------+
| speedtest     | 32.79       | 23.84MiB  | 373.50MiB |
+---------------+-------------+-----------+-----------+
| v1            | 67130.91    | 254.54MiB | 1.25GiB   |
+---------------+-------------+-----------+-----------+
Press 'd' + ENTER to change delay
Press 's' + ENTER to change sorting method
Press CTRL-C to exit

Delay: 5s
Sorting Method: Alphabetical

This work was contributed by University of Texas at Austin students.

System load information in resources API

A new section was added to the resources API to expose server load information (1min, 5min, 10min) as well as total process count.

This is particularly useful for placement and auto-balancing logic as it allows for getting a good glimpse at how busy the various servers are solely from the Incus API.

stgraber@castiana:~$ incus info --resources
System:
  UUID: 05006c9c-7863-ee11-9e1b-224425600022
  Vendor: Framework
  Product: Laptop 13 (AMD Ryzen 7040Series)
  Family: Laptop
  Version: A5
  SKU: FRANDGCP05
  Serial: FRANDGCPA5340500AZ
  Type: physical
  Chassis:
      Vendor: Framework
      Type: Notebook
      Version: A5
      Serial: FRANDGCPA5340500AZ
  Motherboard:
      Vendor: Framework
      Product: FRANMDCP05
      Serial: FRANMDCPA534040120
      Version: A5
  Firmware:
      Vendor: INSYDE Corp.
      Version: 03.05
      Date: 03/29/2024

Load:
  Processes: 519
  Average: 0.80 0.77 0.71

[snip...]

This work was contributed by University of Texas at Austin students.

Ability to query access information for instances and projects

Two new APIs were added to allow querying the access list of a project or even a specific instance.

This integrates with our OpenFGA support and provided a sufficiently recent version of OpenFGA, will show you exactly who can access an instance and what role they have.

stgraber@castiana:~$ incus info --show-access foo
- identifier: stgraber@stgraber.org
  role: admin
  provider: openfga

stgraber@castiana:~$ incus project info --show-access default
- identifier: stgraber@stgraber.org
  role: admin
  provider: openfga

This work was contributed by University of Texas at Austin students.

Forceful deletion of projects

When dealing with a lot of busy projects, deleting them can become rather frustrating due to having to track down and delete everything they contain in the right order.

To address that, we now have incus project delete --force which will instruct Incus itself to delete everything in the correct order before deleting the project itself.

This is obviously an extremely dangerous thing to do. The command line tool will always ask for confirmation that you indeed want this project fully gone.

stgraber@castiana:~$ incus project delete demo
Error: Only empty projects can be removed.

stgraber@castiana:~$ incus project delete demo --force
Remove demo and everything it contains (instances, images, volumes, networks, ...) (yes/no): yes
Project demo deleted

New get_project scriptlet function

For those using our scriplet instance placement feature (instances.placement.scriptlet), a new function has now been added, get_project.

This allows retrieving all the details (api.Project) for a specific project and is particularly useful if you want project restrictions or limits to impact the placement decision.

Documentation: https://linuxcontainers.org/incus/docs/main/explanation/clustering/#instance-placement-scriptlet
This work was contributed by University of Texas at Austin students.

Querying objects across projects

Incus has long supported listing all instances regardless of projects.
Then recently this was extended to also cover storage volumes, images, profiles, network zones and operations.

With Incus 6.2, all remaining object collections now support this, adding:
- Storage buckets
- Networks
- Network ACLs

The CLI was updated to match, so all list commands interacting with objects that can be project-specific now also support --all-projects.

This work was contributed by University of Texas at Austin students.

PCI devices in incus info --resources

All PCI devices are now included in the incus info --resources output.
In the past, only those devices that were included in the GPU or disk sections were readily available.

This work was contributed by University of Texas at Austin students.

Improved alias handling in incus-simplestreams

The initial incus-simplestreams implementation would automatically generate our standard looking alias, basically DISTRIBUTION/RELEASE/VARIANT but that's not suitable for all environments and so you now have two new arguments to incus-simplestreams add:

  • --no-default-alias to disable the above alias
  • --alias to define a custom alias (can be passed multiple times)

Feeding YAML to create commands in the incus CLI

This work was started with Incus 6.1 and is now complete.

All create commands as well as incus init and incus launch now support reading an initial configuration as YAML from stdin.

This enables much easier scripting of complex deployments.

Customizable column lists in the CLI

Another piece of work which started with Incus 6.1 and is now complete.

All CLI commands that have a list function now support the --column/-c flag.

This work was contributed by University of Texas at Austin students.

More automatically generated documentation

Not something that should be generally noticeable to most users, but we've been slowly moving our documentation to be generated directly from comments in our code, limiting the risk of it getting outdated or out of sync.

With Incus 6.2, the following are now generated in that way:

  • Network zones
  • Image restrictions
  • Kernel limits
  • Devices
    • disk
    • unix-block
    • unix-char
    • unix-hotplug
    • usb

This work was contributed by University of Texas at Austin students.

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • incusd/instance/lxc: Add gendoc comments for image restrictions
  • incusd/instance/qemu: Add gendoc comments for image restrictions
  • doc: Update configs
  • doc/image-handling: Use gendoc data
  • incus/storage_bucket: Add yaml support for key create
  • i18n: Update translation templates
  • incusd/instance/qemu: Fix StartedAt handling
  • incus/storage: Customizable columns in list
  • i18n: Update translation templates
  • incusd/network/ovn: Port LogicalRouterSNATAdd and LogicalRouterDNATSNATAdd to libovsdb
  • incusd/network: Update for OVN function changes
  • api: instances_scriptlet_get_project
  • doc/instances/scriptlet: Add get_project
  • incusd/scriptlet: Add get_project
  • api: networks_all_projects
  • shared/api: Add Project field to Network
  • client: Add GetNetworksAllProjects
  • incus/network: Add all-projects
  • incusd/db: Add GetNetworksAllProjects
  • incusd/networks: Add all-projects
  • internal/instance: Add gendoc for limits.kernel
  • doc: Update configs
  • doc: Use gendoc for limits.kernel
  • api: network_acls_all_projects
  • shared/api: Add Project field to NetworkACL
  • client: Add GetNetworkACLsAllProjects
  • incus/network/acl: Add all-projects
  • incusd/db: Add GetNetworkACLsAllProjects
  • incusd/network/acl: Set Project field
  • incusd/network_acls: Add all-projects
  • api: storage_buckets_all_projects
  • shared/api: Add Project field to StorageBucket
  • client: Add GetStoragePoolBucketsAllProjects
  • incus/storage/bucket: Add all-projects
  • incusd/db/storage_buckets: Fill Project field
  • incusd/storage_buckets: Add all-projects
  • i18n: Update translation templates
  • incusd/networks: Fix import shadowing
  • doc/rest-api: Refresh swagger YAML
  • client: Align GetProfilesAllProjects with GetProfiles
  • client: Align GetNetworkZonesAllProjects with GetNetworkZones
  • client: Standardize the GetNetworkAllocation functions
  • incus/network_allocations: Update for client changes
  • incusd/device/usb: Add gendoc for the USB device
  • doc: Update configs
  • doc: Use gendoc for USB devices
  • api: resources_load
  • shared/api: Add Load to resources API
  • doc/rest-api: Refresh swagger YAML
  • incusd/resources: Add load information
  • incus/info: Add load information
  • i18n: Update translation templates
  • incusd/device/unix: Add gendoc comments
  • doc: Update configs
  • doc/devices_unix_block.md: Use gendoc data
  • doc/devices_unix_char.md: Use gendoc data
  • doc/devices_unix_hotplug.md: Use gendoc data
  • incus/top: Add new command
  • i18n: Update translation templates
  • incusd/network/zone: add gendoc for network zone
  • doc: Update configs
  • doc: Use gen doc for network zones
  • incusd/device/unix: Run gofmt
  • incus/info: Add PCI devices to --resources
  • i18n: Update translation templates
  • incusd/device/disk: Add gendoc comments
  • doc: Update configs
  • doc/devices/disk: Use gendoc
  • incus/network: Customizable columns in list
  • i18n: Update translation templates
  • incusd/network_zones: Fix listing of zones within a project
  • incusd/instance/lxc: Fix handling of migration.stateful
  • gomod: Update dependencies
  • incusd/network/ovs: Fix bad VLANMode value
  • fix: close resources
  • incusd/instance: Fix building on 32bit architectures
  • incus/network_zone: Add example for network zone record create
  • i18n: Update translation template
  • incus/storage_volume: Add yaml support for create
  • i18n: Update translation templates
  • cmd/incus/info: Fix runtime error when chassis, motherboard and firwmare information is not available
  • Translated using Weblate (German)
  • incusd/instance/qemu: Allow setting protection.delete when running
  • doc/api-extension: Fix typo
  • shared/api: Introduce Access structs
  • api: instance_access
  • incusd/auth: Introduce GetInstanceAccess
  • incusd/instance: Add access endpoint
  • api: project_access
  • incusd/auth: Introduce GetProjectAccess
  • incusd/project: Add access endpoint
  • doc/rest-api: Refresh swagger YAML
  • client: Add GetInstanceAccess
  • client: Add GetProjectAccess
  • incus/info: Fix description of --show-log
  • incus/info: Add --show-access
  • incus/project: Add --show-access to info
  • i18n: Update translation templates
  • incusd/auth/fga: Rename manager by admin in model
  • incusd/auth/fga: Rework permission model
  • incusd/auth/fga: Rebuild model
  • tests: Fix for permission changes
  • incusd/instance/agent-loader: Support installing to /etc
  • incusd/apparmor/lxc: Fix rule syntax
  • incus-simplestreams add: added flags: --no-default-alias, --alias. #875
  • incus/storage_volume/snapshot: Support YAML for creation
  • i18n: Update translation templates
  • shared/idmap: Make get_userns_fd configure the userns
  • incus-migrate: Handle valid CA certificates
  • incusd/instances_post: Fix migrating into remote cluster
  • incusd/apparmor: Detect nosymfollow support
  • incusd: Set SELinux label on socket
  • incus/network: Align attach-profile with attach
  • create_detached_idmapped_mount: avoid double close
  • incusd/instance/qemu: Extend missing QEMU error
  • doc/installing: Mention extra packages for VMs
  • incusd/storage/btrfs: Fix btrfs argument order
  • incusd/seccomp/sysinfo: Handle 32bit on 64bit
  • api: projects_force_delete
  • incusd/api_project: Add force delete endpoint
  • doc/rest-api: Refresh swagger YAML
  • client: Introduce DeleteProjectForce
  • cmd/incus/project: Add --force to delete
  • i18n: Update translation templates
  • incusd/project: Simplify projectIsEmpty
  • incusd/db: Introduce GetNetworkZoneURIs
  • incusd/db: Introduce GetStorageBucketURIs
  • incusd/api_project: Fix UsedBy
  • incusd/api_project: Add force deletion logic
  • incus/completion: Reduce API calls
  • incus/publish: Complete snapshot names
  • incus/completion: Fix import shadowing
  • Translated using Weblate (French)
  • Makefile: Pin go-acme/lego for Go 1.21
  • Update dependencies
  • cmd/incus/console: Cleanup --show-log
  • incusd/instance_console: Remove old check
  • incusd/instance_console: Handle missing log file
  • incusd/instance_console: Don't fail on empty logs
  • incusd/instance_console: Cleanup error message
  • i18n: Update translation templates
  • incusd/device/sriov: Line up code with comment

Documentation

The Incus documentation can be found at:
https://linuxcontainers.org/incus/docs/main/

Packages

There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Installing the Incus server on Linux

Incus is available for most common Linux distributions. You'll find detailed installation instructions in our documentation.

https://linuxcontainers.org/incus/docs/main/installing/

Homebrew package for the Incus client

The client tool is available through HomeBrew for both Linux and MacOS.

https://formulae.brew.sh/formula/incus

Chocolatey package for the Incus client

The client tool is available through Chocolatey for Windows users.

https://community.chocolatey.org/packages/incus/6.2.0

Winget package for the Incus client

The client tool is also available through Winget for Windows users.

https://winstall.app/apps/LinuxContainers.Incus

Support

Monthly feature releases are only supported up until the next release comes out. Users needing a longer support length and less frequent changes should consider using Incus 6.0 LTS instead.

Community support is provided at: https://discuss.linuxcontainers.org
Commercial support is available through: https://zabbly.com/incus
Bugs can be reported at: https://github.com/lxc/incus/issues

Incus 6.1 has been released

7th of May 2024

Introduction

The Incus team is pleased to announce the release of Incus 6.1!

This is our first feature release following Incus 6.0 LTS.
As a reminder, feature releases are only supported until the next one comes out, usually on a monthly cadence. Critical production environments should stay on the LTS release instead.

In this release, we have a lot of small quality of life improvements throughout. A lot of those being first contributions from students of the University of Texas at Austin. Expect a lot more of those in Incus 6.2!

image|690x459

As usual, you can try it for yourself online: https://linuxcontainers.org/incus/try-it/

Enjoy!

New features

Creation of complex ZFS pools

The source key when creating storage pools using our zfs driver has now been extended to allow the creation of more complex vdevs including striping, mirroring, raidz1 and raidz2.

Example syntax:

  • /dev/sda,/dev/sdb (striping, RAID0)
  • mirror=/dev/sda,/dev/sdb (mirroring, RAID1)
  • raidz1=/dev/sda,/dev/sdb,/dev/sdc,/dev/sdd,/dev/sde (raidz1, RAID5)
  • raidz2=/dev/sda,/dev/sdb,/dev/sdc,/dev/sdd,/dev/sde (raidz2, RAID6)

This combined with the data from incus info --resources will now make it possible to deploy complex storage pools all through the API.

Listing of profiles across projects

As part of an effort to add cross-project querying of all API objects, it is now possible to list profiles across all projects.

At the API level, this is support for all-projects=true on the /1.0/profiles API endpoint, at the CLI level this looks like:

stgraber@dakara:~$ incus profile list --all-projects
+-----------------+---------+---------------------------------------+---------+
|     PROJECT     |  NAME   |              DESCRIPTION              | USED BY |
+-----------------+---------+---------------------------------------+---------+
| default         | default | Default Incus profile                 | 10      |
+-----------------+---------+---------------------------------------+---------+
| demo            | default | Default Incus profile                 | 12      |
+-----------------+---------+---------------------------------------+---------+
| lab-cgroup      | default | Default Incus profile                 | 2       |
+-----------------+---------+---------------------------------------+---------+
| lab-lvm-cluster | default | Default Incus profile                 | 3       |
+-----------------+---------+---------------------------------------+---------+
| lab-ovn-ic      | default | Default Incus profile                 | 10      |
+-----------------+---------+---------------------------------------+---------+
| vpn             | default | Default Incus profile for project vpn | 2       |
+-----------------+---------+---------------------------------------+---------+

This feature was contributed by University of Texas at Austin students.

Listing of network zones across projects

As part of an effort to add cross-project querying of all API objects, it is now possible to list network zones across all projects.

At the API level, this is support for all-projects=true on the /1.0/network-zones API endpoint, at the CLI level this looks like:

stgraber@dakara:~$ incus network zone list --all-projects
+---------+--------------------------+-------------+---------+
| PROJECT |           NAME           | DESCRIPTION | USED BY |
+---------+--------------------------+-------------+---------+
| default | default.demo.example.net |             | 0       |
+---------+--------------------------+-------------+---------+
| foo     | foo.demo.example.net     |             | 0       |
+---------+--------------------------+-------------+---------+

This feature was contributed by University of Texas at Austin students.

Additional functions made available to the instance placement scriptlet

Incus supports customizing instance placement through the use of a python-like script called a scriptlet. When used, the scriptlet is exposed some information about the instance, potential targets and the reason for the request.

On top of those arguments, a number of functions are also exposed to those scriptlets.
That includes the ability to log information, the function to actually make the final placement decision and the ability to fetch some basic load information about the candidate servers.

Now this is being extended through two additional functions:

  • get_instances(location, project) => []api.Instance
  • get_cluster_members(group) => []api.ClusterMember

Those are all optional arguments, so they can be used to list all instances or all cluster members as well, allowing a lot of flexibility in placement scripts.

Documentation: https://linuxcontainers.org/incus/docs/main/explanation/clustering/#instance-placement-scriptlet
This feature was contributed by University of Texas at Austin students.

Feeding YAML to create commands in the incus CLI

A number of Incus commands already support reading a YAML file through their standard input as part of a create command, but this isn't very consistent nor well documented, we're now in the process of making things consistent and this release now has support for reading a YAML object definition in the following commands:

  • incus create & incus launch
  • incus cluster group create
  • incus network acl create
  • incus network forward create
  • incus network integration create
  • incus network load-balance create
  • incus network peer create
  • incus network zone create
  • incus profile create
  • incus project create
  • incus snapshot create
  • incus storage create
  • incus storage bucket create

For all of those, YAML data similar to what's showed in the matching show command can be fed through stdin at creation time to configure the object as part of its creation.

Customizable columns in the incus CLI

Something else we're slowly making consistent in the CLI is the ability to choose what columns to display in our list commands.

This has also been expanded with the following commands now supporting it:

  • incus cluster list
  • incus config trust list
  • incus image list
  • incus list
  • incus profile list
  • incus project list
  • incus storage volume list
  • incus storage volume snapshot list
  • incus warning list

This feature was contributed by University of Texas at Austin students.

migration.stateful configuration key for containers

The migration.stateful configuration key has been expanded to also apply to containers now.

It is now required to have it set to true to access any feature requiring the recording and restoration of process state in containers (CRIU), which basically means stateful stop, stateful snapshots and live migration.

This change is unlikely to affect many users as CRIU's ability to live-migrate or perform stateful dumps of Incus containers is extremely limited and so generally seen as not functional.
The change does have the benefit of providing clearer errors to users who accidentally request an action which would make use of CRIU.

This feature was contributed by University of Texas at Austin students.

Stateless ACLs on OVN

A new allow-stateless action has now been added to Incus' network ACL rules.

As the name implies, this leads to the creation of a stateless rule inside of OVN.
This is great for situations where stateful rules may come with a heavy cost and where a matching stateless rule is possible (e.g. DNS interactions).

This feature was contributed by University of Texas at Austin students.

Instance uptime (startup time) tracking

A new StartedAt field has been added to the instance state data.
This exposes the timestamp at which the instance was started and is also available in incus info and incus list.

stgraber@dakara:~$ incus info speedtest | grep Started
Started: 2024/04/29 11:03 EDT

stgraber@dakara:~$ incus list -cnstU
+-------------+---------+-----------------+----------------------+
|    NAME     |  STATE  |      TYPE       |      STARTED AT      |
+-------------+---------+-----------------+----------------------+
| centos3     | STOPPED | CONTAINER       |                      |
+-------------+---------+-----------------+----------------------+
| centos4     | STOPPED | CONTAINER       |                      |
+-------------+---------+-----------------+----------------------+
| fga         | STOPPED | VIRTUAL-MACHINE |                      |
+-------------+---------+-----------------+----------------------+
| incus-ui    | RUNNING | CONTAINER       | 2024/05/07 16:54 EDT |
+-------------+---------+-----------------+----------------------+
| kernel-test | RUNNING | VIRTUAL-MACHINE | 2024/05/07 15:43 EDT |
+-------------+---------+-----------------+----------------------+
| keybase     | STOPPED | CONTAINER       |                      |
+-------------+---------+-----------------+----------------------+
| ovn-test    | RUNNING | VIRTUAL-MACHINE | 2024/05/07 15:43 EDT |
+-------------+---------+-----------------+----------------------+
| speedtest   | RUNNING | CONTAINER       | 2024/04/29 11:03 EDT |
+-------------+---------+-----------------+----------------------+
| void        | STOPPED | VIRTUAL-MACHINE |                      |
+-------------+---------+-----------------+----------------------+
| win11       | STOPPED | VIRTUAL-MACHINE |                      |
+-------------+---------+-----------------+----------------------+

This feature was contributed by University of Texas at Austin students.

Improvement to network handling during evacuation

When performing a cluster evacuation, all the networks will now be shut down at the end of the evacuation and only started back as part of the restoration action.

This is particularly useful in OVN environments as it ensures that an evacuated Incus server doesn't act as a virtual router for any of the defined networks, making system shutdown/reboot less likely to cause network glitches.

This feature was contributed by University of Texas at Austin students.

Complete changelog

Here is a complete list of all changes in this release:

Full commit list
  • doc/support: Update for LTS
  • incusd/network: Remove bridge.driver=native requirement for extended external_interfaces syntax
  • doc/network/bridge: Update extended external_interfaces documentation
  • incusd/storage/drivers/zfs: Simplify dataset receive and fix progress handling
  • gomod: Update dependencies
  • mini-oidc: Merge into main gomod
  • gomod: Update dependencies
  • go.mod: Bump package major version
  • global: Update Go package to v6
  • test: Update godeps.list
  • README: Update for godoc URLs
  • doc/rest-api: Refresh swagger YAML
  • cmd/incus: Fix import ordering
  • incusd: Remove unneeded import renames
  • incusd/instance: Fix duplicate import
  • doc/projects: Tweak examples
  • shared/api: Remove container resources (deprecated)
  • doc/rest-api: Refresh swagger YAML
  • incus/create: Remove dead code
  • i18n: Update translation templates
  • incusd/daemon: Remove old migration logic
  • incusd: Stop mentioning containers in resources
  • doc/rest-api: Use instances API in example
  • incusd/db/cluster: containers URLs aren't valid in Incus
  • incusd/instances: Don't start instances when evacuated
  • doc/installing: Sort source-build distro instructions
  • doc/installing: Add OpenSUSE source instructions
  • doc/installing: Add Alpine instructions
  • incus/aliases: fix completion regression
  • incus/snapshot: Fix deletion of snapshots containing colons
  • incusd/instance/drivers: Have SR-IOV get stable MACs
  • incusd/device/nic_sriov: Use stable MAC
  • incus/profile: Add support for creating from YAML
  • i18n: Update translation templates
  • incusd/instance/lxc: Only apply soft cgroup limits on cgroup1
  • incus/admin: Don't hide the sql command
  • grafana: Refresh dashboard
  • doc/metrics: Mention Loki in Grafana setup
  • grafana: Better filter Loki events by project
  • incusd/loki: Use hostname as default instance property on standalone systems
  • incusd/loki: Re-order config fields
  • incusd/loki: Allow overriding the location field
  • incusd/loki: Set location field to local hostname on standalone systems
  • incus/projct: Add support for creating project from yaml
  • i18n: Update translation templates
  • incusd/network/ovs: Port GetOVNEncapIP to libovsdb
  • incusd/network/ovs: Add some comments to GetOVNBridgeMappings
  • incusd/network/ovs: Port AddOVNBridgeMapping to libovsdb
  • incusd/network/ovs: Port RemoveOVNBridgeMapping to libovsdb
  • incusd/network/ovs: Port GetHardwareOffload to libovsdb
  • incusd/network/ovs: Port GetBridgePorts to libovsdb
  • incusd/network/ovs: Port UpdateBridgePortVLANs to libovsdb
  • incusd/network/ovs: Port AssociateInterfaceOVNSwitchPort to libovsdb
  • incusd/network/ovs: Switch Installed to checking for unix socket
  • incusd/network: Update for OVS function changes
  • Add missing SecureBoot firmware names
  • incus/snapshot: Add support for creating project from yaml
  • i18n: Update translation templates
  • api: network_zones_all_projects
  • shared/api: Add Project field to NetworkZone
  • incusd/network_zones: Add support for all-projects
  • doc/rest-api: Refresh swagger YAML
  • client: Add GetNetworkZonesAllProjects
  • incus/network_zone: Add --all-projects flag to list
  • i18n: Update translation templates
  • tests: Add all-projects network zone test
  • incusd/network/ovn: Port LogicalRouterAdd to libovsdb
  • incusd/network/ovn: Remove LogicalRouterAdd
  • incusd/network: Replace LogicalRouterAdd usage with CreateLogicalRouter
  • incusd/network/ovn: gofmt
  • incusd/cluster: Disable networks during evacuation
  • incusd/cgroup: Set hugepages reserved limits
  • incusd/storage/drivers: Introduce SparseFileWrapper
  • incusd/storage/drivers/vfs: Use SparseFileWrapper on backup import
  • incusd/storage/drivers/vfs: Use SparseFileWrapper on volume migration
  • incus/storage: Support creating storage pool from yaml
  • i18n: Update translation templates
  • incus/info: Show CPU architecture as separate line
  • i18n: Update translation templates
  • incus/cluster_group: Add yaml support for cluster group create
  • i18n: Update translation templates
  • gitignore: Add JetBrains
  • api: storage_zfs_vdev
  • doc: Update ZFS support for multiple block devices and vdev types in doc/reference/storage_zfs.md
  • incusd/storage/zfs: Add support for vdev type and multiple block devices
  • incus/info: Sorting network interfaces
  • incus/network_acl: Add cmd.Example for network acl create
  • i18n: Update translation templates
  • incus/network_forward: Add yaml example for create
  • i18n: Update translation templates
  • incus/config_template: Add file support for create
  • i18n: Update translation templates
  • incus/network_integrations: Add yaml support for create
  • i18n: Update translation templates
  • incusd/storage/s3: Use 'mc' client
  • incusd/storage: Switch to use minio's 'mc' client
  • gomod: Update dependencies
  • github: Download MinIO client
  • doc: Avoid MyST 3.0.0
  • incus/doc/installing.md: Add Docker information
  • doc: Add Podman to wordlist
  • incus/cluster: Add columns to list
  • i18n: Update translation templates
  • incus/project: Add customazible columns to list
  • i18n: Update translation templates
  • api: container_migration_stateful
  • internal/instance: Add migration.stateful to containers
  • incusd/instance/lxc: Add checks for migration.stateful
  • doc: Update configs
  • incus/network_load_balancer: add Example to create
  • i18n: Update translation templates
  • incus/network_zone: Add example for create command
  • i18n: Update translation templates
  • doc: Fix bad sphinx requirements
  • incusd/instances/qemu: Tweak secureboot firmware list
  • cmd/incus/admin_cluster: Add libexec path for incusd
  • incus/storage: Show usage when no driver passed
  • incusd/storage/drivers/dir: Tweak path validation
  • incusd/backup: Show profile list on lookup error
  • incusd/apparmor/lxc: Allow access to binfmt_misc
  • incusd/apparmor/lxc: Refresh generated rules
  • incusd/storage: Handle instance volume size on import
  • incus/profile: Add customizable columns to list
  • i18n: Update translation templates
  • incus/project: Fix help message for list
  • i18n: Update translation templates
  • api: profiles_all_projects
  • shared/api: Add Project field to Profile
  • client: Add GetProfilesAllProjects
  • incus/profile: Add all-projects to list
  • incusd/db/cluster: Add Project field to Profile
  • incusd/profile: Add all-projects support
  • doc/rest-api: Refresh swagger YAML
  • i18n: Update translation templates
  • incus/storage_volume: Clarify volume errors
  • incusd/apparmor/lxc: Fix access to kernel/security/apparmor
  • api: instances_scriptlet_get_instances
  • doc/instances/scriptlet: Add get_instances
  • incusd/scriptlet: Add get_instances
  • api: instances_scriptlet_get_cluster_members
  • doc/instances/scriptlet: Add get_cluster_members
  • incusd/scriptlet: Add get_cluster_members
  • api: Add network_acl_stateless
  • doc/network_acl: Add allow-stateless action
  • incusd/network/acl: Add allow-stateless action
  • incusd/network/ovn/nb: Port DeleteLogicalRouter to libovsdb
  • incusd/network/ovn/nb: Port CreateLogicalRouterSNAT to libovsdb
  • incusd/network: Update for OVS function changes
  • incusd/network/acl: Properly run instance counting outside of ACL loop
  • incusd/network/ovn: Wait up to 1s for dynamic IPs
  • incusd/network/ovn/nb: Port DeleteLogicalRouterNAT to libovsdb
  • incusd/network: Update for OVS function changes
  • shared/archive: Fix typo
  • incusd/cluster: Re-organize joining logic
  • incusd/cluster: Ignore OVN networks during joining
  • shared/archive: Properly anchor exclude rules
  • incusd/project: Rework low-level permission checks
  • incus/storage_bucket: Add example for storage bucket create
  • i18n: Update translation templates
  • incus/network_peer: Add example for create command
  • i18n: Update trasnlation templates
  • api: instance_state_started_at
  • shared/api: Add StartedAt to InstanceState
  • doc/rest-api: Refresh swagger YAML
  • incusd/instance: Add StartedAt to InstanceState
  • incus/info: Add Started field
  • incus/list: Add started at column
  • i18n: Update translation templates
  • Makefile: Cleanup gomod update
  • gomod: Update dependencies
  • tests/mini-oidc: Bump to go-jose/v4
  • client/connection: Add support for the socket existing in /run/incus

Documentation

The Incus documentation can be found at:
https://linuxcontainers.org/incus/docs/main/

Packages

There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Installing the Incus server on Linux

Incus is available for most common Linux distributions. You'll find detailed installation instructions in our documentation.

https://linuxcontainers.org/incus/docs/main/installing/

Homebrew package for the Incus client

The client tool is available through HomeBrew for both Linux and MacOS.

https://formulae.brew.sh/formula/incus

Chocolatey package for the Incus client

The client tool is available through Chocolatey for Windows users.

https://community.chocolatey.org/packages/incus/6.1.0

Winget package for the Incus client

The client tool is also available through Winget for Windows users.

https://winstall.app/apps/LinuxContainers.Incus

Support

At this early stage, each Incus release will only be supported up until the next release comes out. This will change in a few months as we are planning an LTS release to coincide with the LTS releases of LXC and LXCFS.

Community support is provided at: https://discuss.linuxcontainers.org
Commercial support is available through: https://zabbly.com/incus
Bugs can be reported at: https://github.com/lxc/incus/issues

Incus 6.0 LTS has been released

4th of April 2024

Introduction

It's with great pride and pleasure that the Incus team is announcing the release of Incus 6.0 LTS!

Incus is a modern system container and virtual machine manager developed and maintained by the same team that first created LXD. It's released under the Apache 2.0 license and is run as a community led Open Source project as part of the Linux Containers organization.

Incus provides a cloud-like environment, creating instances from premade images and offers a wide variety of features, including the ability to seamlessly cluster up to 50 servers together.

It supports multiple different local or remote storage options, traditional or fully distributed networking and offers most common cloud features, including a full REST API and integrations with common tooling like Ansible, Terraform/OpenTofu and more!

image|690x489

This is a major milestone for Incus as it marks our first release with extended support, suitable for use in production environments where monthly feature releases aren't suitable.

It joins LXC 6.0 LTS and LXCFS 6.0 LTS in wrapping up this round of LTS releases.

Just like its sister projects, Incus 6.0 LTS will be supported until June 2029.
The first 2 years will feature bug and security fixes as well as minor usability improvements, delivered through occasional point releases (6.0.x). After that initial two years, Incus 6.0 LTS will move to security only maintenance for the remaining of its 5 years of support.

As usual, you can try it for yourself online: https://linuxcontainers.org/incus/try-it/

Enjoy!

PS: Incus was made possible thanks to the work of over 70 individual contributors!

Changes since Incus 0.7

Swap limits for containers

The existing limits.memory.swap configuration key for containers has been extended to also allow for byte amounts.

This now makes its behavior be as follows:

  • limits.memory.swap=true => Container memory may be swapped (default)
  • limits.memory.swap=false => Container shouldn't get swapped (minimal swappiness)
  • limits.memory.swap=256MiB => Container can use up to 256MiB of swap space (in addition to its memory limit set through limits.memory)

Example (cgroup2 system):

stgraber@dakara:~$ incus launch images:debian/12 d12 -c limits.memory=1GiB
Launching d12
stgraber@dakara:~$ incus exec d12 bash
root@d12:~# free -m
               total        used        free      shared  buff/cache   available           
Mem:            1024          21         983           0          19        1002
Swap:              0           0           0
root@d12:~#
exit
stgraber@dakara:~$ incus config set d12 limits.memory.swap=128MiB
stgraber@dakara:~$ incus exec d12 bash
root@d12:~# free -m      
               total        used        free      shared  buff/cache   available
Mem:            1024          21         983           0          19        1002
Swap:            128           0         128
root@d12:~#
exit

New shell completion mechanism

With this release, we complete the migration away from a hand-maintained bash completion script and over to generate completion scripts directly in our command line tool.

Completion profiles are now available for:

  • bash
  • fish
  • powershell
  • zsh

The profile can be retrieved by calling incus completion <shell> (e.g. incus completion bash) though this will generally be done by packagers as part of the Incus package build process.

Creation of external bridge interfaces

The managed network bridge configuration syntax for external interfaces, bridge.external_interfaces has now been extended to allow for the creation and attachment of VLAN interfaces.

stgraber@dakara:~$ incus network set incusbr0 bridge.external_interfaces=vlan60/enp35s0/60
stgraber@dakara:~$ ip link show dev vlan60
269: vlan60@enp35s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master incusbr0 state LOWERLAYERDOWN mode DEFAULT group default qlen 1000
    link/ether 00:23:a4:01:01:6f brd ff:ff:ff:ff:ff:ff
stgraber@dakara:~$ incus network unset incusbr0 bridge.external_interfaces
stgraber@dakara:~$ ip link show dev vlan60
Device "vlan60" does not exist.
stgraber@dakara:~$

Live-migration of VMs with attached disks (from remote storage)

As an extension to our ever improving VM live-migration support, virtual-machines with additional disks attached to them which come from a "remote" storage pool (ceph or lvmcluster) will now be live-migratable alongside the virtual machine.

No user action is required for this to happen, you'll simply now notice that virtual machines that previously would have refused to live-migrate through either a manual incus move --target or a cluster evacuation will now happily live-migrate to another server.

System information in incus info --resources

A new System section is now visible in incus info --resources

stgraber@dakara:~$ incus info --resources
System:
  UUID: 88eecd60-34fc-9f97-48f5-fc34979f48f6
  Vendor: ASUS
  Product: System Product Name
  Family: To be filled by O.E.M.
  Version: System Version
  SKU: SKU
  Serial: System Serial Number
  Type: physical
  Chassis:
      Vendor: Default string
      Type: Desktop
      Version: Default string
      Serial: Default string
  Motherboard:
      Vendor: ASUSTeK COMPUTER INC.
      Product: ProArt B550-CREATOR
      Serial: 210382121300122
      Version: Rev X.0x
  Firmware:
      Vendor: American Megatrends Inc.
      Version: 2803
      Date: 04/28/2022

 [snip...]

Having access to this information is particularly useful in clustered environments where incus info --resources can be used with the --target argument to query specific servers, check that all firmwares are up to date and check what machines one is dealing with.

This feature was contributed by University of Texas at Austin students.

USB devices in incus info --resources

A new USB devices section is now visible in incus info --resources

stgraber@dakara:~$ incus info --resources
[snip...]

USB devices:
  Device 0:
    Vendor: Intel Corp.
    Vendor ID: 8087
    Product: AX200 Bluetooth
    Product ID: 0029
    Bus Address: 1
    Device Address: 6
  Device 1:
    Vendor: Corsair
    Vendor ID: 1b1c
    Product: H150iRGBPROXT
    Product ID: 0c22
    Bus Address: 1
    Device Address: 5
  Device 2:
    Vendor: ASUSTek Computer, Inc.
    Vendor ID: 0b05
    Product: AURA LED Controller
    Product ID: 19af
    Bus Address: 1
    Device Address: 2
  Device 3:
    Vendor: Realtek Semiconductor Corp.
    Vendor ID: 0bda
    Product: TX42C500
    Product ID: 4933
    Bus Address: 5
    Device Address: 2
  Device 4:
    Vendor: Blue Microphones
    Vendor ID: b58e
    Product: Yeti Stereo Microphone
    Product ID: 9e84
    Bus Address: 5
    Device Address: 15
  Device 5:
    Vendor: Yubico.com
    Vendor ID: 1050
    Product: YubiKey FIDO+CCID
    Product ID: 0406
    Bus Address: 5
    Device Address: 29
  Device 6:
    Vendor: Logitech, Inc.
    Vendor ID: 046d
    Product: HD Pro Webcam C920
    Product ID: 082d
    Bus Address: 5
    Device Address: 17
  Device 7:
    Vendor: Powerware Corp.
    Vendor ID: 0592
    Product: Powerware UPS
    Product ID: 0002
    Bus Address: 7
    Device Address: 2

That information comes in very handy when adding a USB device to a container or virtual machine.

This feature was contributed by University of Texas at Austin students.

Changes since LXD 5.0 LTS

For those coming from the LXD 5.0 LTS release, here is a concise list of what to expect as far as features having been removed and what has been added both in subsequent LXD feature releases and then through Incus.

Feature removal

A number of features that were Ubuntu or Canonical specific were removed as part of the creation of the Incus project. A number of legacy APIs have also been removed at the same time.
You'll find the full list in the Incus 0.1 announcement.

Highlights:

  • shiftfs has been removed in favor of VFS idmap shifting
  • Canonical Candid authentication has been removed in favor of OpenID Connect
  • Canonical RBAC authorization has been removed in favor of OpenFGA
  • Canonical MAAS network integration has been removed (under/unused feature)
  • Ubuntu Fan networking has been removed in favor of OVN
  • core.trust_password has been removed in favor of trust tokens for security reasons

Feature additions

Here are a few highlights from the many new features introduced within the 2 years since the release of LXD 5.0 LTS.

Complete changelog

Here is a complete list of all changes since Incus 0.7:

Full commit list
  • Translated using Weblate (Japanese)
  • Translated using Weblate (Japanese)
  • incus/image: Fix column handling with --all-projects
  • Replace util.ValueInSlice with slices.Contains
  • shared/util: Delete ValueInSlice function
  • incus/image: Fix column handling with --all-projects
  • incusd/instance/qemu: Relocate image requirement checks
  • doc/images: Add requirements.cdrom_agent
  • incusd/instance/qemu: Add support for requirements.cdrom_agent
  • incusd/device/disk: Fix incorrect block volume usage
  • Translated using Weblate (Japanese)
  • incusd/network/ovn: Use ParseIPToNet instead of manual IPToNet and net.ParseIP
  • incusd/network/ovn: Use listenAddressNet in family check
  • incusd/instance/drivers: Disable architecture check on incus cp with snapshots
  • Translated using Weblate (French)
  • incusd/network/bridge: Set local address on all VXLAN tunnels
  • incus/instance/qemu: Fix RecordOutput
  • incus: add completions for instance actions and snapshots
  • incus: add completions for profiles
  • incusd/network/ovn: Introduce get helper
  • incusd/network/ovn: Add some missing indices
  • incusd/network/ovn: Use get helper
  • incusd/network/ovn: Fix LogicalSwitchPortIPs logic
  • incusd/network/bridge: Fix gofmt
  • incusd/network/ovn: Fix gofmt
  • cmd/incus: Use proper timestamp check
  • cmd/incus: Use consistent date format and timezone
  • client: Rename network_peer for consistency
  • cmd/incusd: Rename network_peer to network_peers
  • shared/api: Rename network_allocation for consistency
  • incusd/db: Fix comment typoes
  • incusd/db/generate: Fix bad camel case handling
  • incusd/db/network_peers: Fix duplicate type definitions
  • incusd/auth: Drop Permission type
  • incusd/auth: Add boilerplate doc strings
  • incusd/images: Properly handle null creation and expiry dates
  • incus: add completions for remotes
  • incus: add completions for projects
  • incusd/images: Fix reporting of images in multiple projects
  • github: Add static build of lxd-to-incus
  • lxd-to-incus: Add support for Alpine service name
  • lxd-to-incus: Re-organize target list
  • lxd-to-incus: Add support for APK
  • Makefile: Add OVN IC to update-ovsdb
  • incusd/network: Update OVS/OVN schemas
  • incusd/network/ovn: Add IC clients
  • incusd/network/ovn: Add GetName to NB client
  • incusd/network/ovn: Add GetGateways to ICSB
  • incusd/network/ovn: Introduce new errors
  • incusd/network/ovn: Add CreateTransitSwitch and DeleteTransitSwitch to ICNB
  • incusd/device/gpu_sriov: Add locking
  • incusd/device/gpu_sriov: Re-locate vfio-pci loading
  • incusd/device/gpu_sriov: Rework VF allocation logic
  • incus/remote: Add a generate-certificate sub-command
  • i18n: Update translation templates
  • incusd/drivers/qmp: Add SetBlockThrottle
  • incusd/device/disk/config: Add DiskLimits
  • incusd/device/disk: Re-shuffle limit parsing
  • incusd/device/disk: Add disk limits on VMs
  • incusd/device/disk: Support live limits update for VMs
  • incusd/instance/qemu: Support disk I/O limits
  • incus/remote: Add missing docstrings
  • incusd/certificates: Improve token handling when clustered
  • cmd/incusd/api_1.0: Update context
  • cmd/incusd/api_cluster: Update context
  • cmd/incusd/api_internal: Update context
  • cmd/incusd/daemon: Update context
  • cmd/incusd/api_project: Update context
  • cmd/incusd/certificates: Update context
  • cmd/incusd/images: Update context
  • cmd/incusd/instance: Update context
  • cmd/incusd/network: Update context
  • cmd/incusd/operations: Update context
  • cmd/incusd/profiles: Update context
  • cmd/incusd/storage: Update context
  • cmd/incusd/warnings: Update context
  • incusd/devices: Skip isolated threads from NUMA CPUs
  • incusd/devices: Restrict CPU threads by NUMA node
  • incusd/instance/qemu: Add support for limits.cpu.nodes
  • incusd/device/gpu: Add support for limits.cpu.nodes for VF selection
  • incusd: Fix import shadowing
  • incusd/images: Fix potential race condition
  • incusd/instance/qemu: Add support for NUMA node restrictions for memory
  • incusd/apparmor/qemu: Silence apparmor failures
  • incusd/network/ovs: Introduce new errors
  • incusd/network/ovn/nb: Move SetChassisGroupPriority to new function signature
  • incusd/network/ovn/sb: Move GetLogicalRouterPortActiveChassisHostname to new function signature
  • incusd/network/ovs: Move GetBridge to new function signature
  • incusd/network/ovs: Move CreateBridge to new function signature
  • incusd/network/ovs: Move DeleteBridge to new function signature
  • incusd/network/ovs: Move CreateBridgePort to new function signature
  • incusd/network/ovs: Move GetChassisID to new function signature
  • incusd/network/ovs: Move GetOVNBridgeMappings to new function signature
  • incusd/network: Update for function changes
  • incusd/device/nic: Update for function changes
  • incusd: Update for function changes
  • doc: Fix bad snapshot syntax
  • Translated using Weblate (French)
  • doc: Fix token creation procedure
  • incusd/network/ovn/nb: Add GetLogicalSwitch
  • incusd/network/ovn/nb: Replace ChassisGroupChassisDelete with SetChassisGroupPriority
  • incusd/network/ovn/nb: Port CreateLogicalRouterPort to OVSDB
  • incusd/network/ovn/nb: Replace LogicalRouterPortLinkChassisGroup with CreateLogicalRouterPort
  • incusd/network/ovn/nb: Port CreateChassisGroup to OVSDB
  • incusd/network/ovn/nb: Port CreateLogicalSwitch to OVSDB
  • incusd/network/ovn: Update for function changes
  • incusd/network/ovn: Remove state references
  • incusd/state: Add OVNNB and OVNSB handles
  • incusd: Update to use state for OVN
  • incusd/device: Make init function return error
  • incusd/device: Add OVN check on nicOVN
  • client: Still return response on RawQuery error
  • incus/query: Respect --raw for errors
  • incusd/network/acl: Add OVN check
  • incusd/network: Make init function return error
  • incusd/network: Add OVN check on ovn driver
  • incusd/api: Re-order config checks
  • incusd: Add OVN loader
  • Translated using Weblate (French)
  • incusd/network/ovn/nb: Port CreateLogicalSwitchPort to OVSDB
  • incusd/network/ovn/nb: Port DeleteLogicalSwitchPort to OVSDB
  • incusd/network/ovn/nb: Port DeleteLogicalRouterPort to OVSDB
  • incusd/network/ovn: Update for function changes
  • incusd/network/ovs: Port GetOVNSouthboundDBRemoteAddress to OVSDB
  • incusd/network/ovs: Port DeleteBridgePort to OVSDB
  • incusd/network/ovs: Port GetInterfaceAssociatedOVNSwitchPort to OVSDB
  • incusd/network/ovs: Align GetChassisID with other functions
  • incusd: Update for OVS function changes
  • incusd/network/ovn/icsb: Fix bad DB schema
  • incusd/network/ovn/nb: Introduce GetLogicalRouterPort
  • incusd/network/ovn/nb: Extend OVNSwitchPortOpts to handle router ports
  • incusd/network/ovn/nb: Change type of RouterPort field to OVNRouterPort
  • incusd/network/ovn/nb: Port DeleteChassisGroup to OVSDB
  • incusd/network/ovn/icnb: Update DeleteTransitSwitch to handle missing switches
  • incusd/network/ovn: Update for function changes
  • Translated using Weblate (French)
  • incus/completion: do not add a space after remote names completion
  • incusd/device/disk: Disable virtiofsd caching
  • incus-agent: Cleanup mount logic
  • Translated using Weblate (French)
  • incus: expose parseVolume to entire package
  • incus: add completions for storage pools and volumes
  • incusd/device/gpu_sriov: Fix default handling
  • doc/packaging: Add mention of documentation
  • incusd/auth: Fix --all-projects for restricted users
  • doc: Add third party tools page
  • gomod: Update dependencies
  • incusd/auth/tls: Prevent project modifications
  • doc: Update wordlist
  • internal/usbid: allow path override of usb.ids path
  • incus/completion: fix image names completion
  • doc/environment: document INCUS_USBIDS_PATH
  • incusd/instance/qemu/agent: Check for semanage
  • incusd/project: Fix config name in ImageProjectFromRecord
  • incus/restart: Fix long description
  • i18n: Update translations
  • lxd-to-incus: Handle common existing bridges
  • shared/simplestreams: Remove defaultOS
  • shared/simplestreams: Add NewLocalClient
  • incus-simplestreams: Introduce new command
  • incus-simplestreams: Simplify delete logic
  • doc: Re-organize image server doc
  • doc: Add section for incus-simplestreams
  • incusd/seccomp: Add support for pidfd threads
  • incus: add completions for clusters
  • incus: add completions for cluster groups
  • incus: add completions for cluster roles
  • incus: add completions for config devices
  • incus: add completions for config templates
  • update translations
  • doc: Update references to mage docs
  • doc/backup: Remove bad reference
  • incus: add completions for network acls
  • shared/api: Add new structs to support configuration metadata
  • client: Add GetMetadataConfiguration
  • incusd: Rename documentation.go -> metadata.go
  • doc/rest-api: Refresh swagger YAML
  • shared/api/metadata: Add GetKeys to simplify usage
  • incusd: Add support for JWT authentication
  • gomod: Update dependencies
  • tests: Add tls2jwt tool
  • tests: Add JWT authentication test
  • api: auth_tls_jwt
  • doc/authentication: Add section on JWT
  • doc/instances: Remove size.state requirement for live migration
  • incusd/instance/qemu: Allow live migration without size.state
  • shared/idmap: Support uid/gid in subuid/subgid
  • shared/cliconfig: Copy clientcerts on remote copy
  • shared/cliconfig: Add HasRemoteClientCertificate
  • shared/cliconfig: Support per-remote client certificates
  • doc: Add clientcerts
  • incusd/cluster/config: Add oidc.claim
  • incusd/auth/oidc: Add support for using a specific claim as username
  • incusd: Pass OIDC claim to verifier
  • api: oidc_claim
  • doc: Update configs
  • doc/howto/instances: Mention extra resources in ISO guidea
  • doc/installing: Add Debian backport
  • doc: Add backported to dictionary
  • lxd-to-incus: Add support for LXD 5.21
  • shared/cliconfig: Ensure client certificate key is 0600
  • api: device_usb_serial
  • doc: Add busnum, devnum and serial to USB devices
  • shared/api: Add Serial to ResourcesUSBDevice
  • incusd/resources: Add USB Serial
  • incusd/devices/usb: Add serial, busnum and devnum options
  • doc/rest-api: Refresh swagger YAML
  • incusd/instance/qemu: Fix handling of > 64 limits.cpu
  • incusd/device/gpu_sriov: Implement NUMA fallback
  • incus: add completions for network forwards
  • incus: add completions for network load balancers
  • shared/validate: Remove stringInSlice
  • shared/validate: Add And and Or functions
  • shared/util: Move ParseUint32Range
  • incusd/project: Update for ParseUint32Range
  • doc/instance_options: Remove mention of limits.cpu.nodes from container-only section
  • incusd/devices: Better handle bad config
  • api: numa_cpu_balanced
  • internal/instance: Add support for balanced NUMA nodes
  • doc: Update configs
  • incusd/instance/common: Add NUMA balancing
  • incusd/instance/lxc: Add support for balanced NUMA allocation
  • incusd/instance/qemu: Add support for balanced NUMA allocation
  • incusd/devices: Add support for balanced NUMA allocation
  • incusd/device/gpu_sriov: Simplify NUMA logic
  • doc/cloud-init: Don't mention non-existing remotes
  • doc/howto/images_remote: Fix wording around image servers
  • doc/benchmark: Fix install command
  • incusd/instance/common: Fix CanMigrate mutating devices
  • incusd/instance/qemu: Reduce agent queries
  • incusd/metrics: Don't filter out all server metrics
  • incusd/auth/tls: Include project restrictions for metrics certificates
  • incusd/auth/tls: Return project-aware checker for metrics
  • incusd/metrics: Use project-specific checker if no global access
  • internal/server/instance/lxd: add support for image.requirments.nesting
  • api: add image_restriction_nesting
  • doc/images: introduce requirements.nesting
  • Show the count values in snapshot count mismatch error
  • incus/admin/init: Use btrfs subvol in --auto
  • incus-migrate: Clarify that disk image files must be raw
  • incusd/network/ovn/icnb: Fix comment
  • incusd/project: Re-format the comments
  • incusd/project: Fix bad default value
  • doc: Update configs
  • incus/migrate: Add CSM support
  • incusd/storage/backend: Better handle name conflicts
  • incus-migrate: Support using the local server
  • api: network_integrations
  • shared/api: Add type and target_integration fields to NetworkPeersPost
  • incusd/db/cluster: Add networks_integrations
  • incusd/db/cluster: Re-generate schema
  • incusd/db/cluster: Add generated DB code for network integrations
  • incusd/db: Update network peer DB query functions
  • client: Add check for network_integrations in CreateNetworkPeer
  • incus/network/peer: Add support for network peer types
  • shared/api: Add network integrations
  • client: Add network integration functions
  • incus/network: Introduce support for integrations
  • incusd/auth: Add network integration functions
  • shared/api: Add lifecycle events for network integrations
  • incusd/lifecycle: Add network integration events
  • incusd: Add network integration API
  • incusd/db: Add GetNetworkPeersURLByIntegration
  • incusd/network_integration: Add UsedBy field
  • incusd/network_integrations: Add validator
  • incusd/network/ovn: Add support for peering with OVN IC
  • incusd/project: Add restricted.networks.integrations
  • incusd/project: Add NetworkIntegrationAllowed
  • incusd/network/integrations: Respect project restrictions
  • incusd/network/ovn: Add support for integration restrictions
  • incusd/auth/openfga: Update the model
  • incusd/auth/openfga: Update the generated model
  • incusd/auth/openfga: Handle model updates
  • incusd: Remove openfga.store.model_id
  • incusd/db/cluster: Remove openfga.store.model_id
  • doc/ovn_peers: Add remote peering
  • doc: Add documentation for network integrations
  • doc/rest-api: Refresh swagger YAML
  • i18n: Update translation templates
  • doc: Update configs
  • gomod: Update dependencies

Documentation

The Incus documentation can be found at:
https://linuxcontainers.org/incus/docs/main/

Installation

There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Linux packages

Incus is available for most common Linux distributions. You'll find detailed installation instructions in our documentation.

https://linuxcontainers.org/incus/docs/main/installing/

Homebrew package for the Incus client

The client tool is available through HomeBrew for both Linux and MacOS.

https://formulae.brew.sh/formula/incus

Chocolatey package for the Incus client

The client tool is available through Chocolatey for Windows users.

https://community.chocolatey.org/packages/incus/6.0.0

Winget package for the Incus client

The client tool is also available through Winget for Windows users.

https://winstall.app/apps/LinuxContainers.Incus

Migrating from LXD

A lxd-to-incus migration tool allows for in-place migration from LXD to Incus.
It's been tested with LXD versions as low as 4.0 LTS and as high as the latest LXD 5.21 bugfix release.

It allows for a very quick migration from LXD over to Incus, automatically checking for potential conflicts ahead of time.

asciicast

More details can be found here: https://linuxcontainers.org/incus/docs/main/howto/server_migrate_lxd/

Support

Incus 6.0 LTS will be supported for a total of 5 years (until June 2029).

During the first 2 years, new point releases will be issued including a mix of bug and security fixes as well as some minor usabiltiy improvements. After that initial 2 years (after Incus 7.0 LTS is released), Incus 6.0 LTS will transition to security fixes only for the remaining 3 years.

This matches what we've been doing for our other projects (LXC and LXCFS) over the past 10 years.

Community support is provided at: https://discuss.linuxcontainers.org
Commercial support is available through: https://zabbly.com/incus
Bugs can be reported at: https://github.com/lxc/incus/issues

Older news