Back to the news overview

Incus 0.5 has been released¶

26.01.2024

Introduction¶

The Incus team is pleased to announce the release of Incus 0.5!

This is our first release of 2024 and it's quite a busy one! It's also the first release to feature no change coming from LXD following their decision to re-license to AGPLv3.

This release comes with a number of welcome improvements to the Incus CLI, a number of new virtual machine features, more options to handle cluster evacuations and host shutdown and some other smaller features and improvements!

On top of that, we've got quite a lot of bugfixes as well as a number of database improvements which should yield noticeable performance improvements especially in clusters.

As usual, you can try it for yourself online: https://linuxcontainers.org/incus/try-it/

Enjoy!

Highlights¶

Ansible, Terraform/OpenTofu and Packer¶

Over the past few months, Incus support has grown quite a bit in common tools!

• A connection plugin for Incus can now be found in Ansible.
• We have an official provider for Terraform and OpenTofu
• There is an available Packer plugin for Incus

Linux distribution packages¶

Since the last release of Incus, additional packages are now available for:

• Arch Linux
• Debian (testing/unstable)
• Ubuntu (noble)
• Void Linux

You'll find all instructions in our installation guide.

Translations¶

We've spent a bit of time cleaning up translations and setting up Weblate for Incus.
It's now easier than ever to log into Weblate and translate the Incus CLI into your language.
All changes are automatically submitted for inclusion through Github.

Upgrade notes¶

subuid/subgid entries¶

A longstanding bug in the idmap parser was causing everything but the first large entry for the root user to be discarded when parsing /etc/subuid and /etc/subgid.

This was then causing issues for the few users that have a legitimate reason to split their uid/gid allocation in half, mostly those using remote authentication on the host system.

This bug has been resolved, but this has a side effect of making Incus containers to fail to start on systems with an invalid subuid/subgid configuration.

If you notice that your containers won't start anymore, go look at /etc/subuid and /etc/subgid and make sure that there is one large entry for the root user, it must be at least 65536 uid/gid large. More importantly, make sure that there is no conflict/overlap in allocations given to the root user.

In most cases, the easiest is to remove all the root entries from those two files and replace them with a single very large entry:

root:1000000:1000000000

New features¶

New incus file create command¶

A new incus file create command was added which provides an easy way to create empty files, symlinks and directories without having to transfer an existing local directory tree.

stgraber@dakara:~$ incus file create demo/root/file
stgraber@dakara:~$ incus file create --type=symlink demo/root/symlink /etc/hosts
stgraber@dakara:~$ incus file create --type=directory demo/root/dir
stgraber@dakara:~$ incus exec demo -- ls -lh /root
total 2.5K
drwxr-xr-x 2 root root  2 Jan 26 03:38 dir
-rw-r--r-- 1 root root  0 Jan 26 03:37 file
lrwxrwxrwx 1 root root 10 Jan 26 03:38 symlink -> /etc/hosts

New incus snapshot show command¶

A new incus snapshot show command makes it easy to look at the configuration data that's included as part of an Incus instance snapshot.

As a reminder, Incus snapshots don't only contain the filesystem state, but also include all the instance configuration (config keys, devices, ...) at the time of the snapshot.

stgraber@dakara:~$ incus snapshot create demo s1
stgraber@dakara:~$ incus snapshot list demo
+------+----------------------+----------------------+----------+
| NAME |       TAKEN AT       |      EXPIRES AT      | STATEFUL |
+------+----------------------+----------------------+----------+
| s1   | 2024/01/25 22:39 EST | 0000/12/31 19:03 LMT | NO       |
+------+----------------------+----------------------+----------+
stgraber@dakara:~$ incus snapshot show demo s1
expires_at: 0001-01-01T00:00:00Z
architecture: x86_64
config:
  image.architecture: amd64
  image.description: Ubuntu jammy amd64 (20240125_07:42)
  image.os: Ubuntu
  image.release: jammy
  image.serial: "20240125_07:42"
  image.type: squashfs
  image.variant: default
  volatile.base_image: f9e9abeb4fc8691edf48078616a1aae628c6d5938b715e361c6b47cda0474679
  volatile.cloud-init.instance-id: f724feba-245a-424b-bc51-43167258dc2a
  volatile.eth0.host_name: vethecbb346e
  volatile.eth0.hwaddr: 00:16:3e:06:67:f0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 8b8a1c75-977b-4197-9ad7-507b899432e9
  volatile.uuid.generation: 8b8a1c75-977b-4197-9ad7-507b899432e9
created_at: 2024-01-26T03:39:09.583020489Z
devices: {}
ephemeral: false
expanded_config:
  image.architecture: amd64
  image.description: Ubuntu jammy amd64 (20240125_07:42)
  image.os: Ubuntu
  image.release: jammy
  image.serial: "20240125_07:42"
  image.type: squashfs
  image.variant: default
  volatile.base_image: f9e9abeb4fc8691edf48078616a1aae628c6d5938b715e361c6b47cda0474679
  volatile.cloud-init.instance-id: f724feba-245a-424b-bc51-43167258dc2a
  volatile.eth0.host_name: vethecbb346e
  volatile.eth0.hwaddr: 00:16:3e:06:67:f0
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 8b8a1c75-977b-4197-9ad7-507b899432e9
  volatile.uuid.generation: 8b8a1c75-977b-4197-9ad7-507b899432e9
expanded_devices:
  eth0:
    name: eth0
    network: incusbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
last_used_at: 0001-01-01T00:00:00Z
name: s1
profiles:
- default
stateful: false
size: 53760

More shell completion options¶

We're slowly transitioning from a single hand-written bash completion script for the incus command line tool, to instead using a much more dynamic way of handling shell completion.

Initial shell completion profiles can be retrieved with:

• incus completion bash
• incus completion fish
• incus completion powershell
• incus completion zsh

Bash users are probably still better off using the hand-written completion script at this point, but we're hopeful that the new dynamically generated completion profiles will take over in the next release or two.

Support for multiple VM agent binaries¶

It's now possible for Incus to provide multiple agent binaries to its virtual machines.

This is useful in two scenarios:

• Handling multiple operating systems
• Handling multiple architectures

At this stage, the focus is on multiple architectures. With this new ability, you can now have 32bit virtual machines running on your system and have them fetch a 32bit of the agent binary.

stgraber@castiana:~$ incus exec debian32 bash
root@debian32:~# uname -a
Linux debian32 6.1.0-17-686-pae #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) i686 GNU/Linux
root@debian32:~# 
exit
stgraber@castiana:~$ incus exec debian32 bash
root@debian32:~# uname -m
i686
root@debian32:~# mount -t 9p agent /mnt
root@debian32:~# ls -lh /mnt
total 34M
-rwxr-xr-x 1 root root 17M Jan 24 10:10 incus-agent.linux.i686
-rwxr-xr-x 1 root root 18M Jan 24 10:10 incus-agent.linux.x86_64

Support for virtio-blk as a disk io.bus¶

After adding NVME support in Incus 0.2, we're now expanding that mechanism to also offering virtio-blk as a disk I/O bus in our virtual machines.

To use it, set the io.bus property on the disk device to be virtio-blk.

stgraber@dakara:~$ incus launch images:debian/12 demo --vm
Launching demo
stgraber@dakara:~$ incus storage volume create default demo size=5GiB --type=block
Storage volume demo created
stgraber@dakara:~$ incus config device add demo extra disk pool=default source=demo io.bus=virtio-blk
Device extra added to demo
stgraber@dakara:~$ incus exec demo bash
root@demo:~# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
sda      8:0    0   10G  0 disk 
├─sda1   8:1    0  100M  0 part /boot/efi
└─sda2   8:2    0  3.9G  0 part /
vda    253:0    0    5G  0 disk

Support for USB network device pass-through in VMs¶

When using nictype=physical for a virtual machine with the parent network device being connected over the USB bus, Incus will now detect the situation and internally convert this into a USB device pass-through to the virtual machine.

stgraber@castiana:~$ incus launch images:debian/12 demo --vm
Launching demo
stgraber@castiana:~$ incus config device add demo eth1 nic nictype=physical parent=enx207bd2a0f9eb
Device eth1 added to demo
stgraber@castiana:~$ incus exec demo bash
root@demo:~# apt install usbutils
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libusb-1.0-0
The following NEW packages will be installed:
  libusb-1.0-0 usbutils
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 142 kB of archives.
After this operation, 492 kB of additional disk space will be used.
Do you want to continue? [Y/n]·
Get:1 http://deb.debian.org/debian bookworm/main amd64 libusb-1.0-0 amd64 2:1.0.26-1 [62.6 kB]
Get:2 http://deb.debian.org/debian bookworm/main amd64 usbutils amd64 1:014-1 [79.7 kB]
Fetched 142 kB in 1s (124 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libusb-1.0-0:amd64.
(Reading database ... 20425 files and directories currently installed.)
Preparing to unpack .../libusb-1.0-0_2%3a1.0.26-1_amd64.deb ...
Unpacking libusb-1.0-0:amd64 (2:1.0.26-1) ...
Selecting previously unselected package usbutils.
Preparing to unpack .../usbutils_1%3a014-1_amd64.deb ...
Unpacking usbutils (1:014-1) ...
Setting up libusb-1.0-0:amd64 (2:1.0.26-1) ...
Setting up usbutils (1:014-1) ...
Processing triggers for libc-bin (2.36-9+deb12u3) ...
root@demo:~# lsusb -tv
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 5000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
    |__ Port 4: Dev 2, If 0, Class=Communications, Driver=cdc_ncm, 5000M
        ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet
    |__ Port 4: Dev 2, If 1, Class=CDC Data, Driver=cdc_ncm, 5000M
        ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
root@demo:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:16:3e:e7:f7:2d brd ff:ff:ff:ff:ff:ff
3: enx207bd2a0f9eb: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 20:7b:d2:a0:f9:eb brd ff:ff:ff:ff:ff:ff

New cluster evacuation options (force-stop and stateful-stop)¶

A couple of new cluster evacuation options have been added.

Those can be selected on a per-instance basis through the cluster.evacuate instance configuration key.

force-stop causes the instance to be immediately stopped without giving it a chance at a clean shutdown. This only really makes sense in cases where the instance is effectively stateless as it won't have any chance to flush any ongoing state to disk.

stateful-stop causes the instance's state to be written to disk before stopping the instance. On restore, the instance state is restored too, causing the instance to just continue where it left off.
This option is currently primarily targeted at virtual machines as stateful stop for containers is quite difficult to achieve.

Ability to configure the host instance shutdown action¶

A new instance configuration key, boot.host_shutdown_action, has been introduced which supports:

• stop (normal shutdown behavior)
• force-stop (see above)
• stateful-stop (see above)

This makes it particularly easy to have a number of virtual machines going through stateful stop on host shutdown and then being restored on boot.

Ability to start instances as part of creation¶

A small API optimization was made which now allows for instances to be started as part of the creation request, saving an API call and making it easier for those scripting the Incus API.

incus launch now makes use of this too.

Configurable Loki instance name¶

When sending events to Loki, Incus provides a set of default labels.

Those include both an instance and a location label .Worth noting that here instance refers to the Loki event source instance, not an Incus instance.

So far, those would only differ in the somewhat unlikely event that a server would be forwarding an event originating from another server in a cluster.

Instead, in clustered environments, it makes a lot more sense to have a way to provide a cluster name of some kind, so that if multiple clusters use the same Loki instance, they can easily be filtered.

To that effect, we've introduced a new loki.instance server configuration key which, when set, will override the instance label.

The default Grafana dashboard has also been updated to filter the Loki events under the assumption that the Loki instance label will match the Prometheus job name.

Extended HEAD support on files¶

The HEAD method on the Incus instance file API now returns the file size through the Content-Length header.

The primary use for this is for those building some kind of file manager on top of the Incus instance file API as it now allows for not just showing the name and file type but also the size of any regular files.

Use of /run/incus for runtime data¶

Up until now, Incus has stored some amount of runtime data in the instance log directories under /var/log/incus/. Other than it obviously not being the correct location for it, it was also causing some issues with systems that aggressively rotate and expire log files.

To solve this, Incus 0.5 will now place runtime data in /run/incus, keeping /var/log/incus only for actual log files.

In the future, more data will likely be relocated from /var/lib/incus to /run/incus as well.

Complete changelog¶

Here is a complete list of all changes in this release:

Documentation¶

The Incus documentation can be found at:
https://linuxcontainers.org/incus/docs/main/

Packages¶

There are no official Incus packages as Incus upstream only releases regular release tarballs. Below are some available options to get Incus up and running.

Installing the Incus server on Linux¶

Incus is available for most common Linux distributions. You'll find detailed installation instructions in our documentation.

https://linuxcontainers.org/incus/docs/main/installing/

Homebrew package for the Incus client¶

The client tool is available through HomeBrew for both Linux and MacOS.

https://formulae.brew.sh/formula/incus

Chocolatey package for the Incus client¶

The client tool is available through Chocolatey for Windows users.

https://community.chocolatey.org/packages/incus/0.5

Winget package for the Incus client¶

The client tool is also available through Winget for Windows users.

https://winstall.app/apps/LinuxContainers.Incus

Support¶

At this early stage, each Incus release will only be supported up until the next release comes out. This will change in a few months as we are planning an LTS release to coincide with the LTS releases of LXC and LXCFS.

Community support is provided at: https://discuss.linuxcontainers.org
Commercial support is available through: https://zabbly.com/incus
Bugs can be reported at: https://github.com/lxc/incus/issues