Repository navigation
vm disk attach: attach a snapshot's COW copy to a running VM (--from-snapshot / --from-dir) #284
Description
Need
A running VM wants to read a hibernated or archived VM's root filesystem without waking it. The sandbox read-only reader (cocoonstack/sandbox#365) does this today by reaching into cocoon internals: it exports the whole snapshot, keeps cow.raw and cocoon.json by name, parses the sidecar for the layer blob paths, hardlinks the blobs out of the cocoon root so vm disk attach accepts them, and reassembles the overlay itself. Any change to the snapshot layout, the sidecar, the blob store or the guest disk layout breaks it silently. On a filesystem without reflink the export also copies and fsyncs the memory image, sized like guest memory, only to delete it.
Running VMs are not in scope: their reads go to the VM itself (confirmed with the sandbox side).
Design
The reader VM is booted from the same image as the sleeping VM, so it already has the image's read-only disks: the erofs layers for OCI, the qcow2 base for cloudimg. The only thing cocoon has to hand over is a private writable copy of the sleeping VM's COW disk.
cocoon vm disk attach VM --from-snapshot SNAPSHOT # snapshot in the local DB
cocoon vm disk attach VM --from-dir DIR # an export dir, possibly from another node
cocoon vm disk detach VM --name NAME # existing; also removes cocoon's copy
Identical for OCI and cloudimg:
- Image identity: the snapshot's layers, in order, and its
image_blob_idsmust equal the target VM's, read from the record reloaded under the same ops lock as the copy and the attach. Digests only, never paths, so another node's export works when the image is present locally (cocoon.jsonis read without the local-root path validation; only basenames are used). A mismatch is refused with both sides in the error. - The
role=cowentry ofcocoon.jsonnames the file:cow.raw(OCI) oroverlay.qcow2(cloudimg). It is reflinked into the target VM's run dir with its extension kept, so the existing attach path picks raw vs qcow2. The copy is not fsynced; it is throwaway. The memory image is never read.--from-snapshotholds the snapshot's read lease for the copy. - A qcow2 copy always gets its backing path rewritten to the local base blob (
qemu-img rebase -u, header only): an export from another node, or a snapshot imported from one, carries that node's path. - The copy is hot-attached read-write through the existing disk attach path. Pre-check, copy and attach run under the VM's ops lock, so a retry after a lost result finds the disk attached and returns it instead of truncating the file under a live device. Firecracker is not supported: its drives carry no serial, and the guest locates disks by serial.
- The copy is removed by detach (after the eject), by
vm rmwith the run dir, and swept on stop. The target VM's own record already pins the image blobs, so nothing new is pinned and GC is unchanged. snapshot.json(the export envelope) gains an additive field listing what--from-dirneeds, so an archive consumer fetches only those and never names cocoon's files itself. The cow file name follows the snapshot:cow.rawfor OCI,overlay.qcow2for cloudimg.
"disk_attach_files": ["cocoon.json", "cow.raw"]Output is the existing attach result plus the copy's format:
{"vm": "...", "name": "...", "id": "cocoon-disk-...", "format": "raw"}format is raw for OCI (an ext4 filesystem whose /upper is the overlay upper) or qcow2 for cloudimg (a whole GPT disk; mount its root partition by device path, not by UUID or label, which the reader's own root shares). The disk is attached read-write so the guest's journal replay lands on the copy; the guest mounts read-only. Partition layout and overlay assembly stay guest-side knowledge. Dirty page cache at pause time is not on disk; a guest sync before the hibernate stays the caller's job.
Cost: on a reflink filesystem (XFS, btrfs) the copy is a metadata operation, milliseconds regardless of COW size. Without reflink (ext4) it is a sparse copy proportional to the COW's allocated bytes, held under the VM's ops lock, the same degradation vm clone and hibernate already have there. No in-place attach is planned for ext4.
Acceptance
--from-snapshoton a hibernated OCI VM attaches one raw disk; the guest mounts it and overlays/upperover its own layers; the files match the sleeping VM. Same for a cloudimg VM with the qcow2 overlay and its root partition.- The memory image is not read: on a non-reflink filesystem the time scales with the COW's allocated size only. Timings on XFS and ext4 are recorded.
- After the guest's read-only mount (journal replay included) the snapshot's files are byte-identical to before.
- A target VM of a different image, or of the same layers in a different order, is refused with an error naming both sides.
--from-diron an export dir holding onlysnapshot.json,cocoon.jsonand the cow file, produced on a node with a different--root, attaches when the image is local; a qcow2 copy attaches after the backing rewrite.- Repeating the attach returns the attached disk and leaves its file untouched.
- Detach,
vm rmand stop remove the copy.snapshot rmof the source after attach does not affect the reader. - Existing
vm disk attach --pathand the export tar are unchanged.
Out of scope
- Running VMs, and any disk-only capture primitive for them.
- Attaching the snapshot's layers or data disks.
- Restoring memory.
- Changing the
ResolveExternalVolumeguard for operator volumes.
Activity
[-]snapshot export: export only the named files (--files)[/-][+]vm disk attach --from-snapshot: give a running VM a snapshot's disks without exposing the snapshot layout[/+][-]vm disk attach --from-snapshot: give a running VM a snapshot's disks without exposing the snapshot layout[/-][+]vm disk attach: attach a snapshot's COW copy to a running VM (--from-snapshot / --from-dir)[/+]