Skip to content

vm disk attach: attach a snapshot's COW copy to a running VM (--from-snapshot / --from-dir) #284

Closed

Description

@CMGS

Need

A running VM wants to read a hibernated or archived VM's root filesystem without waking it. The sandbox read-only reader (cocoonstack/sandbox#365) does this today by reaching into cocoon internals: it exports the whole snapshot, keeps cow.raw and cocoon.json by name, parses the sidecar for the layer blob paths, hardlinks the blobs out of the cocoon root so vm disk attach accepts them, and reassembles the overlay itself. Any change to the snapshot layout, the sidecar, the blob store or the guest disk layout breaks it silently. On a filesystem without reflink the export also copies and fsyncs the memory image, sized like guest memory, only to delete it.

Running VMs are not in scope: their reads go to the VM itself (confirmed with the sandbox side).

Design

The reader VM is booted from the same image as the sleeping VM, so it already has the image's read-only disks: the erofs layers for OCI, the qcow2 base for cloudimg. The only thing cocoon has to hand over is a private writable copy of the sleeping VM's COW disk.

cocoon vm disk attach VM --from-snapshot SNAPSHOT   # snapshot in the local DB
cocoon vm disk attach VM --from-dir DIR             # an export dir, possibly from another node
cocoon vm disk detach VM --name NAME                # existing; also removes cocoon's copy

Identical for OCI and cloudimg:

  1. Image identity: the snapshot's layers, in order, and its image_blob_ids must equal the target VM's, read from the record reloaded under the same ops lock as the copy and the attach. Digests only, never paths, so another node's export works when the image is present locally (cocoon.json is read without the local-root path validation; only basenames are used). A mismatch is refused with both sides in the error.
  2. The role=cow entry of cocoon.json names the file: cow.raw (OCI) or overlay.qcow2 (cloudimg). It is reflinked into the target VM's run dir with its extension kept, so the existing attach path picks raw vs qcow2. The copy is not fsynced; it is throwaway. The memory image is never read. --from-snapshot holds the snapshot's read lease for the copy.
  3. A qcow2 copy always gets its backing path rewritten to the local base blob (qemu-img rebase -u, header only): an export from another node, or a snapshot imported from one, carries that node's path.
  4. The copy is hot-attached read-write through the existing disk attach path. Pre-check, copy and attach run under the VM's ops lock, so a retry after a lost result finds the disk attached and returns it instead of truncating the file under a live device. Firecracker is not supported: its drives carry no serial, and the guest locates disks by serial.
  5. The copy is removed by detach (after the eject), by vm rm with the run dir, and swept on stop. The target VM's own record already pins the image blobs, so nothing new is pinned and GC is unchanged.
  6. snapshot.json (the export envelope) gains an additive field listing what --from-dir needs, so an archive consumer fetches only those and never names cocoon's files itself. The cow file name follows the snapshot: cow.raw for OCI, overlay.qcow2 for cloudimg.
"disk_attach_files": ["cocoon.json", "cow.raw"]

Output is the existing attach result plus the copy's format:

{"vm": "...", "name": "...", "id": "cocoon-disk-...", "format": "raw"}

format is raw for OCI (an ext4 filesystem whose /upper is the overlay upper) or qcow2 for cloudimg (a whole GPT disk; mount its root partition by device path, not by UUID or label, which the reader's own root shares). The disk is attached read-write so the guest's journal replay lands on the copy; the guest mounts read-only. Partition layout and overlay assembly stay guest-side knowledge. Dirty page cache at pause time is not on disk; a guest sync before the hibernate stays the caller's job.

Cost: on a reflink filesystem (XFS, btrfs) the copy is a metadata operation, milliseconds regardless of COW size. Without reflink (ext4) it is a sparse copy proportional to the COW's allocated bytes, held under the VM's ops lock, the same degradation vm clone and hibernate already have there. No in-place attach is planned for ext4.

Acceptance

  • --from-snapshot on a hibernated OCI VM attaches one raw disk; the guest mounts it and overlays /upper over its own layers; the files match the sleeping VM. Same for a cloudimg VM with the qcow2 overlay and its root partition.
  • The memory image is not read: on a non-reflink filesystem the time scales with the COW's allocated size only. Timings on XFS and ext4 are recorded.
  • After the guest's read-only mount (journal replay included) the snapshot's files are byte-identical to before.
  • A target VM of a different image, or of the same layers in a different order, is refused with an error naming both sides.
  • --from-dir on an export dir holding only snapshot.json, cocoon.json and the cow file, produced on a node with a different --root, attaches when the image is local; a qcow2 copy attaches after the backing rewrite.
  • Repeating the attach returns the attached disk and leaves its file untouched.
  • Detach, vm rm and stop remove the copy. snapshot rm of the source after attach does not affect the reader.
  • Existing vm disk attach --path and the export tar are unchanged.

Out of scope

  • Running VMs, and any disk-only capture primitive for them.
  • Attaching the snapshot's layers or data disks.
  • Restoring memory.
  • Changing the ResolveExternalVolume guard for operator volumes.

Activity

  1. changed the title [-]snapshot export: export only the named files (--files)[/-] [+]vm disk attach --from-snapshot: give a running VM a snapshot's disks without exposing the snapshot layout[/+] on Oct 18, 2026
  2. changed the title [-]vm disk attach --from-snapshot: give a running VM a snapshot's disks without exposing the snapshot layout[/-] [+]vm disk attach: attach a snapshot's COW copy to a running VM (--from-snapshot / --from-dir)[/+] on Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions