Repository navigation
feat(compose): install app images into dockerd's containerd - #147
Merged
Merged
Conversation
mike-sul
force-pushed
the
feat/containerd-install
branch
from
October 21, 2026 14:47
d658e21 to
954e58a
Compare
mike-sul
force-pushed
the
feat/containerd-install
branch
from
October 22, 2026 10:06
954e58a to
7dbb13b
Compare
On a dockerd with the containerd image store, docker load keeps every app image compressed twice, in composectl's store and in containerd's content store; the install into the containerd store keeps only the store copy. The note says when the route is used, the install steps, why a crash cannot leave a broken recorded image, and the limits of the route. Assisted-by: Claude Code:claude-fable-5-1 Signed-off-by: Mike Sul <mike.sul@foundries.io>
The install into dockerd's containerd needs the containerd socket, the namespace and the snapshotter that dockerd uses. The Docker v25 client bundled with composectl has no Containerd field in its Info type, and bumping the Docker module is out of scope, so on a local socket `/info` is read with a raw GET through the client's own transport and decoded into the few fields composectl needs. On a tcp host that raw request would not reach the daemon (the client's address and TLS live in its request path), so the typed Info is used there; it has no Containerd field, and such a host is never an install target anyway. The snapshotter now comes from the `driver-type` entry of DriverStatus instead of the `overlayfs` hardcode and `DriverStatus[0][1]`, so a daemon on another snapshotter (erofs, for example) is detected too. The server version is compared numerically with a leading v stripped: the Arduino build of Docker reports `v29.5.2`. Assisted-by: Claude Code:claude-fable-5-1 Signed-off-by: Mike Sul <mike.sul@foundries.io>
On a dockerd with the containerd image store, docker load keeps a second compressed copy of every app image next to the one in composectl's store. Writing the images from the store into the containerd store under a short lease, unpacking them and deleting the lease lets containerd's GC drop the compressed layers, so the store copy is the only persistent one, and layers whose snapshot already exists are not written at all. The manifest carries no layer labels, so only the lease holds the layers. The snapshotter is synced before the image records are created, so a power cut cannot leave an image that looks installed but cannot run, and the next install deletes the leases an interrupted one left. The route is taken only on a local containerd-store daemon of Docker 29.5.2 or newer, the oldest version that lists images whose layers are not in containerd; other daemons keep docker load. The containerd v1.7.15 client is imported; it adds indirect modules only. Assisted-by: Claude Code:claude-fable-5-1 Signed-off-by: Mike Sul <mike.sul@foundries.io>
Every operation that writes to or deletes from dockerd's containerd has to be serialized with an install, otherwise an uninstall or a prune can delete what a concurrent install is counting on. rm, uninstall and prune now take the same flock as install, and so do `update cancel` around its image and blob removal and `update complete` around its final store prune (its uninstall and remove calls lock on their own). prune also deletes the leases that an interrupted install left behind, which hold compressed layers and snapshots that nothing else references. Assisted-by: Claude Code:claude-fable-5-1 Signed-off-by: Mike Sul <mike.sul@foundries.io>
The containerd install route only runs against the dockerd with the containerd image store, so the existing tests do not reach it. The new tests skip on a graphdriver dockerd and cover: an install from the store of an image pinned by a manifest and one pinned by an index, status, run, check --install and uninstall afterwards; an update that shares base layers with the installed version; a reinstall after `/images/prune` removed the images; an install killed after its lease and first snapshot exist, then recovered by the next install; a second composectl waiting for the install lock; and `update cancel` after the install. TestCtrdAktualizrLiteFlow runs aktualizr-lite's command lines in its order and checks that prune removes the leases of an interrupted install. The retry after a base snapshot disappears between the scan and the unpack is tested in pkg/compose through the synchronous progress callback, so that the removal happens at a known point with no data race. That test shares the daemons and the store with the integration tests, so the e2e target runs the packages one at a time. Assisted-by: Claude Code:claude-fable-5-1 Signed-off-by: Mike Sul <mike.sul@foundries.io>
mike-sul
force-pushed
the
feat/containerd-install
branch
from
October 22, 2026 10:33
7dbb13b to
2cc3bb7
Compare
mike-sul
marked this pull request as ready for review
October 22, 2026 10:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On a dockerd with the containerd image store,
composectl installgoes throughdocker load, which keeps a second compressed copy of every app image in containerd's content store next to the one in composectl's store. This PR streams the images from the store straight into dockerd's containerd under an install lease, unpacks them and deletes the lease, so containerd's GC drops the compressed layers and the store copy stays the only persistent one. Layers whose snapshot already exists are skipped, which makes an update that shares base layers cheaper thandocker load.The route is taken only on a local containerd-store dockerd of Docker 29.5.2 or newer; other daemons keep
docker load. Install, uninstall, rm and prune are serialized by a flock under the store. The snapshotter is synced before the image records are written, so a power cut cannot leave an image that looks installed but cannot run, and the next install deletes the leases an interrupted one left behind.Test: the e2e suite with
CONTAINERD_SNAPSHOTTER=trueruns the newTestCtrdInstall*integration tests on dockerd 29. The design note with the rig checks and the power-cut trials on the board isdocs/containerd-install-plan-2026-10-08.md. First of a stack of five PRs.