Back to the news overview

LXD 3.12 has been released

5th of April 2019

Introduction

The LXD team is very excited to announce the release of LXD 3.12!

This is one of the more feature packed releases and if you are a cluster user, there should be a lot to be happy about!

We have taken a look through all LXD commands and how they work against clusters, improved our APIs where they were lacking and tweaked the commands to give cluster operators a better experience.

But cluster improvements are far from the only thing improved with this LXD releases.

We've also finally got shiftfs support! This feature we've been planning for well over a year is finally there when combined with a suitable kernel. With this, LXD containers don't need any slow shifting on initial startup, reducing the filesystem delta and making container creation so much faster!

Lastly, resource reporting was significantly improved, both in the API and the CLI. We now have more details about the CPU topology, especially NUMA for multi-socket systems and are also now exposing GPU configuration.

Enjoy!

New features

Cluster: Aggregated DHCP leases

LXD managed networks that span multiple cluster members now show a unified view of their DHCP leases, showing hostname, MAC, address and the cluster member's name for each lease.

root@edfu:~# lxc network list-leases lxdfan0
+----------+-------------------+--------------+---------+----------+
| HOSTNAME |    MAC ADDRESS    |  IP ADDRESS  |  TYPE   | LOCATION |
+----------+-------------------+--------------+---------+----------+
| a1       | 00:16:3e:2b:de:8c | 240.31.0.206 | DYNAMIC | edfu     |
+----------+-------------------+--------------+---------+----------+
| a2       | 00:16:3e:01:99:58 | 240.34.0.124 | DYNAMIC | djanet   |
+----------+-------------------+--------------+---------+----------+
| a3       | 00:16:3e:b4:8b:94 | 240.36.0.96  | DYNAMIC | nuturo   |
+----------+-------------------+--------------+---------+----------+
| a4       | 00:16:3e:52:13:2b | 240.31.0.212 | DYNAMIC | edfu     |
+----------+-------------------+--------------+---------+----------+
| a5       | 00:16:3e:45:54:80 | 240.34.0.68  | DYNAMIC | djanet   |
+----------+-------------------+--------------+---------+----------+
| a6       | 00:16:3e:d1:81:e3 | 240.36.0.90  | DYNAMIC | nuturo   |
+----------+-------------------+--------------+---------+----------+

Cluster: Events now show location

Event messages are now all marked with the name of the originating cluster member as their location.

location: edfu
metadata:
  class: task
  created_at: "2019-04-05T04:13:21.212580932Z"
  description: Creating container
  err: ""
  id: 0c8e4a7d-ef7b-41a0-b949-7030f9aa6827
  location: edfu
  may_cancel: false
  metadata: null
  resources:
    containers:
    - /1.0/containers/a10
  status: Running
  status_code: 103
  updated_at: "2019-04-05T04:13:21.212580932Z"
timestamp: "2019-04-05T04:13:21.223834434Z"
type: operation

Additionally LXD will now only forward log messages of importance WARN or higher to other members, keeping the INFO and DBUG messages local to reduce network chatter. This behavior can be changed by starting the LXD daemon in debug mode, at which point all log levels will be broadcasted again.

Cluster: Operations now show location

Another area that now benefits from clear tracking of cluster members is operations, as can be seen in lxc operation list:

root@edfu:~# lxc operation list
+--------------------------------------+-----------+-------------------+---------+------------+----------------------+----------+
|                  ID                  |   TYPE    |    DESCRIPTION    | STATUS  | CANCELABLE |       CREATED        | LOCATION |
+--------------------------------------+-----------+-------------------+---------+------------+----------------------+----------+
| 36c11142-52d8-4c1e-a342-63657096cdec | WEBSOCKET | Executing command | RUNNING | NO         | 2019/04/05 04:19 UTC | edfu     |
+--------------------------------------+-----------+-------------------+---------+------------+----------------------+----------+
| 701175cf-df82-4ef5-8078-a25d83b770b3 | WEBSOCKET | Executing command | RUNNING | NO         | 2019/04/05 04:19 UTC | djanet   |
+--------------------------------------+-----------+-------------------+---------+------------+----------------------+----------+

This now makes it clear what cluster member is busy doing what and should simplify making sure that a system isn't actively used before performing maintenance on it.

Cluster: Support for --target in more commands

The following commands have now grown support for --target:

  • lxc config edit/get/show/set/unset
  • lxc info [--resources]
  • lxc network info
  • lxc storage info

This makes it possible to configure some member-specific daemon configuration options, query cluster member runtime information and system resources, get detailed network statistics and storage usage.

Shiftfs support

This is a feature we've been looking forward to for years and that we are really excited to finally see coming to completion. shiftfs allows for an unprivileged container experience that doesn't need any shifting of the filesystem, instead having the kernel do it on the fly.

This requires kernel support through the shiftfs filesystem which is currently a custom patchset that will be present in the Ubuntu 19.04 kernel.

LXD automatically detects support for this and will transparently start using it whenever possible.

Kernel features now exported over API

For some time now, LXD has been detecting a number of optional kernel features on startup and would print an overview then. That same information is now exposed over the API and visible in lxc info.

  kernel_features:
    netnsid_getifaddrs: "true"
    shiftfs: "true"
    uevent_injection: "true"
    unpriv_fscaps: "true"

Improved CPU reporting

The server resources API now exposes CPU sockets and NUMA node information, making it easier to do CPU pinning for containers.

root@djanet:~# lxc info --resources --target edfu
CPUs:
  Socket 0:
    Vendor: GenuineIntel
    Name: Intel(R) Xeon(R) CPU           E5430  @ 2.66GHz
    Cores: 4
    Threads: 4
    Frequency: 1999Mhz (max: 2336Mhz)
    NUMA node: 0
  Socket 1:
    Vendor: GenuineIntel
    Name: Intel(R) Xeon(R) CPU           E5430  @ 2.66GHz
    Cores: 4
    Threads: 4
    Frequency: 1999Mhz (max: 2336Mhz)
    NUMA node: 1

Memory:
  Free: 18.37GB
  Used: 557.76MB
  Total: 18.93GB

GPU:
  Vendor: ASPEED Technology, Inc. (1a03)
  Product: ASPEED Graphics Family (2000)
  PCI address: 0000:06:03.0
  Driver: ast (4.15.0-47-generic)
  NUMA node: 0

The output of lxc info --resources has also been tweaked to adapt to the hardware present on the system.

GPU reporting

As you may have noticed in the previous listing, GPUs are now present in the system resources output. Additional information can also be seen for NVIDIA cards:

root@vm10:~# lxc info --resources
CPU:
  Vendor: GenuineIntel
  Name: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz
  Cores: 2
  Threads: 4
  Frequency: 2400Mhz
  NUMA node: 0

Memory:
  Free: 8.14GB
  Used: 225.81MB
  Total: 8.36GB

GPUs:
  Card 0:
    Vendor: NVIDIA Corporation (10de)
    Product: GK208B [GeForce GT 730] (1287)
    PCI address: 0000:00:07.0
    Driver: nvidia (418.56)
    NUMA node: 0
    NVIDIA information:
      Architecture: 3.5
      Brand: GeForce
      Model: GeForce GT 730
      CUDA Version: 10.1
      NVRM Version: 418.56
      UUID: GPU-6ddadebd-dafe-2db9-f10f-125719770fd3
  Card 1:
    Vendor: NVIDIA Corporation (10de)
    Product: GK208B [GeForce GT 730] (1287)
    PCI address: 0000:00:09.0
    Driver: nvidia (418.56)
    NUMA node: 0
    NVIDIA information:
      Architecture: 3.5
      Brand: GeForce
      Model: GeForce GT 730
      CUDA Version: 10.1
      NVRM Version: 418.56
      UUID: GPU-253db1df-f725-a174-99d4-a8933288c39e

Snapshot expiry now visible in lxc info

On top of showing when a snapshot was taken, snapshots that have an expiry will now show their expiry in the listing too.

root@djanet:~# lxc info a1
Name: a1
Location: edfu
Remote: unix://
Architecture: x86_64
Created: 2019/04/05 04:07 UTC
Status: Stopped
Type: persistent
Profiles: default
Snapshots:
  snap0 (taken at 2019/04/05 04:20 UTC) (expires at 2019/04/05 05:20 UTC) (stateless)
  snap1 (taken at 2019/04/05 04:50 UTC) (expires at 2019/04/05 05:50 UTC) (stateless)
  snap2 (taken at 2019/04/05 04:55 UTC) (expires at 2019/04/05 05:55 UTC) (stateless)
  snap3 (taken at 2019/04/05 04:52 UTC) (stateless)
  snap4 (taken at 2019/04/05 05:00 UTC) (expires at 2019/04/05 06:00 UTC) (stateless)

Bugs fixed

  • client: Optimize copies on same nodes
  • client: Properly generate events URL
  • doc: Fix typo in api-extensions.md
  • doc: Inform about ZFS pool default compression
  • doc: Introduce volatile.idmap.current
  • doc: Fix typo in faq.md
  • doc: Tweak markdown format in storage.md
  • doc: Update documentation for snapshots.pattern
  • i18n: Update translations from weblate
  • i18n: Update translation templates
  • lxc: Use shared.IsSnapshot
  • lxc/action: skip containers with intended state
  • lxc/config: Use shared.IsSnapshot
  • lxc/launch: Show start progress
  • lxd: Don't leak netlink fds
  • lxd: Drop initialShiftRootfs and always shift on start
  • lxd/backups: Attempt to delete storage on failure
  • lxd/backups: Cleanup on failure
  • lxd/backups: Re-order checks for backup.yaml
  • lxd/cluster: Export Snapshot function
  • lxd/cluster: Initialize candid on join
  • lxd/cluster: Limit log message forwarding
  • lxd/containers: Cleanup shifting
  • lxd/containers: Cleanup template application
  • lxd/containers: Export container location
  • lxd/containers: Fix crash on refresh of non-existing
  • lxd/containers: Fix owner/mode of container path
  • lxd/containers: Handle mid-remap containers
  • lxd/containers: Properly handle tar shifting
  • lxd/containers: Stop proxy before storage
  • lxd/containers: Use LXC hook version 1
  • lxd/devices: Cleanup GPU structs
  • lxd/devices: Track vendor/product names and driver
  • lxd/images: Don't keep an in-memory simplestreams cache
  • lxd/internal: Expose raft-snapshot
  • lxd/internal: Have GC endpoint release memory
  • lxd/main_forkproxy: Fix epoll
  • lxd/migration: Shift CRIU files to current map
  • lxd/migration: Fix handling of missing profiles
  • lxd/networks: Bring mtu device up
  • lxd/patches: Fix names of pool volume LVs
  • lxd/resources: Fix bad CPU reporting
  • lxd/response: Simplify SmartError
  • lxd/storage: Make use of shared.IsSnapshot
  • lxd/storage: Remove setUnprivUserACL
  • lxd/storage: Rename ShiftIfNecessary to resetContainerDiskIdmap
  • lxd/storage: Rename shiftRootfs to initialShiftRootfs
  • lxd/storage: Add helper function to get volume snapshots
  • lxd/storage: Fix copying and moving volume snapshots
  • lxd/storage/btrfs: Fix volume copy with snapshots
  • lxd/storage/ceph: Always unmap after use
  • lxd/storage/ceph: Fix copying existing volume snap
  • lxd/storage/ceph: Fix volume copy with snapshots
  • lxd/storage/ceph: Only freeze if needed
  • lxd/storage/dir: Fix volume copy with snapshots
  • lxd/storage/lvm: Fix LV naming
  • lxd/storage/lvm: Fix volume copy with snapshots
  • lxd/storage/lvm: Pass nouuid for xfs backups
  • lxd/storage/zfs: Fix volume copy with snapshots
  • lxd/storage/zfs: Run rename in clean mntns
  • lxd/tasks: Avoid races on startup
  • lxd-p2c: Workaround for broken /proc/self/exe
  • shared: Switch ParseNumberFromFile to simple read
  • shared/api: Drop StoragePool from Resources struct
  • shared/api: Sort ServerEnvironment struct
  • shared/idmap: Use separate uid and gid entries
  • shared/osarch: Add Plamo x86 arch
  • shared/simplestreams: Align JSON struct for images.json
  • shared/simplestreams: Align JSON struct for index.json
  • shared/utils: Do not chown terminal master fd
  • tests: Add volume copy tests
  • tests: Allow up to 15s for container reboot
  • tests: Fix race condition in proxy test
  • tests: Make proxy tests work with shiftfs
  • tests: Make security tests work with shiftfs
  • tests: Remove dead code
  • tests: Update resources test

Try it for yourself

This new LXD release is already available for you to try on our demo service.

Downloads

The release tarballs can be found on our download page.