4 min read lxd, canonical, containers

Shipping Selective state recursion optimization to LXD

A feature I worked on for LXD that lets clients ask the server for only the instance-state fields they need. Shipped end-to-end across the LXD server, pylxd, and lxd-ui.

LXD 6.7 ships today, and one of the changes in the release notes is a new instances_state_selective_recursion API extension I worked on. In plain terms: when lxc list (or any other client) asks for instance state, it can now tell the server which fields it actually wants, and the server skips the work for the rest. The biggest gain is on lxc list itself — listing instances no longer pays for the state.disk and state.network walks if nothing on the screen renders them.

The feature touches four components (three repos): the LXD server, the lxc CLI inside it, the pylxd Python SDK, and the lxd-ui React dashboard. This post is the technical write-up.

Why is recursion expensive ?

When you ask LXD GET /1.0/instances?recursion=2, the server walks every instance and gathers its full state: disk usage, per-NIC counters, processes, the whole thing. On a host with a few hundred containers the disk and network paths dominate that walk. Most callers don’t need them — lxc list only renders a couple of columns by default, dashboards mostly want names and status, automation usually just wants IPs — but the work happens anyway, and the result gets serialised, sent over the wire, and discarded.

The feature adds a contract that lets the caller say what it needs and the server compute only that.

The wire format

Rather than invent a new query parameter, the extension piggybacks on the existing recursion= parameter using a semicolon-separated form:

recursion=2                              # default — all fields, as before
recursion=2;fields=state.network         # only network info
recursion=2;fields=state.disk,state.network
recursion=2;fields=                      # skip all expensive fields (fastest)

That’s the whole server contract. Semicolon, equals, and comma all URL-encode in the usual way. It’s documented in the release notes and the How-to manage instances page.

The PRs

Server: canonical/lxd#17378. Adds the API extension, parses the fields= list out of the recursion parameter, and threads it through the instance-state path. lxc list looks at the columns the user asked for and emits only the matching fields= set, so the CLI now optimises automatically: lxc list -c ns doesn’t trigger the disk walk, lxc list -c nsd does. This was my first contribution against canonical/lxd. Closes #16698.

Server cleanup: canonical/lxd#17679. Follow-up that lands in LXD 6.8. Review on #17378 pointed out two things that were going to bite the next person who extended recursion: the four GetInstancesFull overloads (default / with-project / with-filter / with-project-and-filter) were already painful, and adding a fields parameter to each of them was the wrong direction; and IsRecursionRequest returned a bool when it really wanted to return both the parsed integer level and the parsed fields slice. #17679 collapses the four overloads into a single GetInstancesFull(args GetInstancesFullArgs) and changes IsRecursionRequest to return (int, []string). Same callers, fewer functions, less to maintain.

Python SDK: canonical/pylxd#707. Adds an optional fields= argument to Instance.all() so existing automation can opt in with one line:

client.instances.all(fields=["state.network"])

If the server doesn’t advertise instances_state_selective_recursion in its API extensions list, the client drops the fields= part and sends plain recursion=2, so the new SDK keeps working against older servers. Shipped in pylxd 2.3.9 (12 March 2026). Closes pylxd#706.

React UI: canonical/lxd-ui#1861. The dashboard’s instance list refreshes on a timer, and before this change every refresh asked for the full state. The PR makes the query inspect which columns are visible and ask the server for only the matching state subset, plus a separate access_entitlements call so fine-grained permissions still resolve. Same fallback: when the server doesn’t advertise the extension, the request degrades to plain recursion=2, so it still works against the 5.21 LTS. Shipped in lxd-ui 0.21 (22 April 2026). Closes lxd-ui#1857.

How it shipped over time

Same feature, four releases over three months:

Layer PR Release Date
LXD server #17378 LXD 6.7 26 Feb 2026
LXD server #17679 (cleanup) LXD 6.8 23 Apr 2026
pylxd #707 pylxd 2.3.9 12 Mar 2026
lxd-ui #1861 lxd-ui 0.21 22 Apr 2026

The server PR was the load-bearing one. Until it was on a stable LXD release, neither pylxd nor lxd-ui had anything to opt into. The fallback to plain recursion=2 is what made it safe to merge the SDK and UI sides at their own cadence — they pick up the optimisation against new servers and keep working against old ones.