Repository navigation
feat(gateway): Every grant worker confines itself before it reads any input - #26
Merged
Merged
Conversation
ADR 0001 defines the equivalent OS sandbox the per-grant isolation contract allows on hosts that cannot create namespaces, such as ECS Fargate. GLOSSARY.md starts with the four terms it uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#22) Every grant worker, the startup probe's included, now applies the base layers of ADR 0001 to itself before it reads its control socket: non-dumpable memory, Landlock with no rules granted, and a seccomp allow-list whose default kills the process. Startup logs the worker sandbox and each layer, and a grant worker that dies of SIGSYS is logged as killed by the filter and its grant fenced. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…proval (#22) The memory-read checks now opt out of Yama's descendant-only rule, so the worker sandbox is what refuses them, and an unconfined reader checks non-dumpable memory on its own. The Gateway's own grant worker is shown confined before anything is sent to it. After a filter kill, a fresh approval restores service. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…on needs (#22) Review round 1. Docker's default profile answers clone3 with ENOSYS, so the allow-list now lets clone start a thread and nothing else. fcntl is down to the three commands a grant worker uses, which keeps F_SETOWN from having the kernel signal another process. rt_sigreturn is allowed, so a crash a handler sees first is not logged as a filter kill. The startup probe reports from a thread with an async runtime, the worker-end log lines name the grant, and a failed no_new_privs is reported under the allow-list, the layer it belongs to. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The seccomp allow-list is written per architecture and CI ran only x86-64. A new job runs the Gateway crate's tests, the containment test among them, and the runtime test on ubuntu-24.04-arm. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The strict handbook build failed on docs/adr/0001, a page outside the navigation. ADRs are maintainer records, read in the repository. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Round 3. It installs cryptography alone, builds the Gateway explicitly, and its comment says where x86-64 runs each test. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A container profile older than Landlock answers its version query with EPERM. ADR 0001 makes Landlock a backstop the Gateway starts without, so that host is now reported as Landlock unavailable, as one with no Landlock is. A failure after the kernel has named an ABI is still an error. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #22
Summary
Every grant worker now locks itself down before it reads anything: its memory cannot be read by another process, Landlock gives it no file or network access where the kernel has Landlock, and the kernel kills it on any system call a grant worker does not need. The Gateway logs these layers at startup, and when the kernel kills a grant worker this way the log says so and that grant needs fresh approval.
Choices the ticket left open:
confine()takes no worker sandbox argument. The base layers are the same under both sandboxes, so the argument would do nothing yet. Operators can choose the filter sandbox #23 Operators can choose the filter sandbox adds the choice.no_new_privsis set before Landlock, not after it. The ticket lists it with the allow-list, third. An unprivilegedlandlock_restrict_selffails withEPERMwithout it (landlock_restrict_self(2)), so it has to come second. A failure to set it is reported under the allow-list, the layer it belongs to.killandtgkillonly for the grant worker's own process ID.sendtoonly with a null destination, since Rust writes a stream with it.fcntlonly for the three commands a grant worker uses, which keepsF_SETOWNfrom having the kernel signal another process.ioctlonly forFIONBIO,socketpaironly forAF_UNIX.cloneonly withCLONE_THREAD, for hosts whose profile answersclone3withENOSYS, as Docker's default does.tests/runtime.pyis extended, not rewritten: it now asserts the startup log line and adds a third pass that deliversSIGSYSto a grant worker. Its existing assertions are as they were.gateway-arm64job runs the Gateway crate's tests and the runtime test onubuntu-24.04-arm. The owner added this criterion to Grant workers confine themselves under the namespace sandbox #22 after the session's retro.docs/adr/is left out of the handbook build, likedocs/specs/anddocs/agents/. The strict build failed on ADR 0001, a page outside the navigation.Left for #23 Operators can choose the filter sandbox: the startup error still reads
Gateway worker sandbox failed to start (...). When the allow-list kills the startup probe, that message carries the signal and no layer name. #23 owns the error that names the sandbox and the layer.The untracked folder
crates/countersign-gateway/prototype-isolation-probe/(a strayCargo.lock) was in the worktree before this work and is not part of it.From the session's retro, as the owner answered:
clone3fallback; Operators can choose the filter sandbox #23 Operators can choose the filter sandbox already plans that job.rustfmtandclippybinaries; reinstalled, a fault of that machine alone.Extras for the owner
clone3.clone3takes its flags in a structure seccomp cannot read, so a compromised grant worker can fork whereclone3exists. The child inherits every layer and can signal only the original worker; without bubblewrap it would outlive the supervisor's kill. Answeringclone3withENOSYSforces the readableclone, which is already thread-only here, but ADR 0001 decision 5 says a call outside the allow-list kills, so this changes the ADR.EPERMfails startup. OnlyENOSYSandEOPNOTSUPPfrom Landlock's version query count as "unavailable". A container profile older than the Landlock system calls answersEPERM. ADR 0001 decision 1: "It is a backstop: the Gateway logs the ABI it got and still starts without Landlock."assert site.cutoff.wait(1)after Bob's retirement fails about two runs in three here, onmain's code too. It passes in CI. No source settles it.Evidence
cargo test --workspace --lockedcargo test --workspace --lockedin CI on x86-64SIGSYSis logged as killed by the filter, and its grant is fenced and needs fresh approvalpython crates/countersign-gateway/tests/runtime.py target/debug/countersign-gatewayin CIGLOSSARY.mdare merged with this changegit diff --stat origin/main...HEADgateway-arm64job, run 44662603958EPERMis reported as Landlock unavailable, and the grant worker is still confinedtests/worker_sandbox.rs: stream and threads, file, supervisor's environment, TCP, UDP, running a program, signalling another process, signalling itself,SIGSYS)allow_list()not called:udp 54904: expected a refusal, got GotThrough;signal 0 9: expected a refusal, got Killed(9);trace-me: expected death by SIGSYS, got GotThroughPR_SET_DUMPABLEnot set:left: GotThrough, right: Errno(1)confine():the grant worker waited for input unconfinedENOSYSno longer mapped to unavailable:the stand-in never served its streamleft: "seccomp allow-list: Operation not permitted (os error 1)"EPERMcounted as unavailable:Landlock: Operation not permitted (os error 1),the stand-in never served its streamclone3clonerule:the probed grant worker ended with signal: 31 (SIGSYS)clonerule, on the host withoutclone3:the stand-in never served its streamfcntlunrestricted:signal-on-io 723768: expected a refusal, got GotThroughrt_sigreturn:left: Killed(31), right: GotThrougheprintln!removed:AssertionError: []grant worker ended (exit status: 159), thenAssertionErrorat the wait for the filter-kill lineNot shown red on its own: the x32 check (an allow-list kills an x32 number with or without the explicit range check), and the two memory-read checks with only the allow-list removed (non-dumpable memory still refuses them; the unconfined controls show the reads get through with no worker sandbox).
At commit 9fb3b25, the head, which adds the Landlock
EPERMwidening:cargo test --locked -p countersign-gatewaylocally on aarch64 givesworker_sandbox: 22 passed, 0 failed, and the same binary gives 22 of 22 indocker run ubuntu:24.04under Docker's default profile. A fresh criteria judge read the widening's diff and called the criterion met, with no path that treats a Landlock failure after the version query as unavailable. No review round read this commit: three had run.At commit 4f3ab9f: the
gateway-arm64job of run 44662603958 (ubuntu-24.04-arm) passedcargo test --locked -p countersign-gatewayand the runtime test, and its x86-64 job passed too. At 69e7cff the same job printedworker_sandbox: 21 passed, 0 failed, thensidecar Gateway mTLS passed,shared Gateway mTLS passed,filter-killed grant worker fenced, so the runtime test does not flake on CI's arm64 as it does on the local machine. The Documentation workflow (mkdocs build --strict) failed at 69e7cff on ADR 0001 outside the navigation and passes at 4f3ab9f (run 44662603962). The new job was not shown red: a CI job has nothing to break but its own steps.At commit e685181, whose Rust and Python files the head leaves unchanged:
cargo test --workspace --lockedin CI (run 44655780164, ubuntu-latest, x86-64): pass.worker_sandbox: 23 passed, 0 failed, witha_system_call_in_the_x32_range_kills_itanda_system_call_from_a_foreign_architecture_kills_itboth run, not skipped. Windows job: pass.python crates/countersign-gateway/tests/runtime.py target/debug/countersign-gatewayin CI (run 44655780170, ubuntu-24.04, x86-64, the real Gateway under bubblewrap):sidecar Gateway mTLS passed,shared Gateway mTLS passed,filter-killed grant worker fenced. Log:Gateway worker sandbox "namespaces": namespaces, non-dumpable, Landlock ABI v7, seccomp allow-listandGateway grant alice: grant worker killed by the worker sandbox's filter (SIGSYS); grant fenced.cargo test --locked -p countersign-gateway, local, aarch64, kernel 6.17: unit 3 and 4,sidecar5,upstream1,worker_sandbox21 passed, 0 failed (the two x86-64 checks are compiled out). No skips.worker_sandboxbinary indocker run ubuntu:24.04under Docker 29.2.1's default seccomp profile, aarch64: 21 passed, 0 failed. The first try there, on the build without theclonerule, was killed atclone(auditsyscall=220), which is what round 1's finding 1 predicted.assert site.cutoff.wait(1)after Bob's retirement, which also fails 2 of 3 runs onmain's code here (third extra above).strace -D -fover the whole runtime test, local: exactly onekilled by SIGSYS, the deliberate one.bwrap ... sh -c 'kill -SYS $$'exits 159. Recorded besidekilled_by_filter.cargo +stable fmt --all -- --checkandcargo +stable clippy --workspace --all-targets --locked -- -D warnings: clean.Not run: Fargate (kernel 6.1, Landlock ABI v2), which is #24 Qualify the filter sandbox on Fargate with a maintained harness; the foreign-architecture check on arm64, where a 64-bit process cannot make a 32-bit call.
Merge Danger
Door: two-way. A revert removes the base layers; nothing stored changes.
Blast Radius: every grant worker, on every deployment.
The merge changes the Gateway executable and so its image; no configuration or infrastructure changes, and there is no flag. From this merge a system call a grant worker newly needs, from a dependency or C library update, kills grant workers until it is added to the allow-list. The startup probe, the runtime test and the containment test are what catch that first.
Reviews
Correctness, Standards, Spec and Security, three rounds (pinned c83a7e3, e685181, then 8b57275): 31 findings, each with its outcome
Correctness, round 1
clone3is allowed but notclone, so on a host that answersclone3withENOSYSthe first thread kills the grant worker. Fixed:cloneis allowed withCLONE_THREAD.rt_sigreturnis missing, so returning from a signal handler is a filter kill. Fixed.ENOSYSandEOPNOTSUPPcount as Landlock unavailable;EPERMfrom a container profile fails startup. Fixed:EPERMon the version query counts as unavailable./etc/hostnameare each handled.recvhangs the test if the grant worker dies before reporting. Fixed: a read timeout.launchkeeps anexitedflag beside two channels for one fact. Fixed: the flag is gone.reportandreported, side by side."namespaces". Pushed back: the worker sandbox choice and its type arrive with Operators can choose the filter sandbox #23 Operators can choose the filter sandbox.Standards, round 1
child, a word the glossary avoids. Fixed:process, in the lines this change adds.Layer::NoNewPrivsis not one of the ADR's base layers. Fixed: removed; the failure is reported under the allow-list.startreturns a bare tuple. Pushed back: a private function with two callers; the lifecycle state it sat beside is reduced by 8.host,done,answer). Fixed:setup,checked,fail_with.Reaped.Spec, round 1
SIGSYS. Pushed back: the watcher has to exist to seeSIGSYS, a worker that ended was already fenced at its next connection, and the second line is what tells a crash from a filter kill.Security, round 1
No finding at reportable confidence. Below it, and fixed here:
fcntlwas unrestricted, soF_SETOWNcould have the kernel signal another process. Fixed: three commands only.Correctness, round 2
No findings.
Standards, round 2
Spec, round 2
exitedguard inattachwas removed. Pushed back: it answers finding 8; a send to a grant worker that ended fails and fences on the spot, and the watcher fences when it ends.clone3stays unconditional, so a new process is refused only throughclone. Carried over to The allow-list tells a thread from a new process under clone3 #28 The allow-list tells a thread from a new process under clone3Security, round 2
No finding at reportable confidence. Below it: the
clone3point of finding 26, and that theclonerule would passCLONE_THREADwith a namespace flag, which needs a capability no grant worker has.Correctness, round 3 (the widening; Standards and Spec found nothing, and Spec judged the added criterion met)
cryptographyalone.cargo testhaving built the Gateway. Fixed: an explicit build step.The three round-3 fixes change the workflow only, in commit 4f3ab9f. No review round read them; the job's pass at that commit is their check.
Tracking
Epic: E1 Run grant workers in a filter sandbox on hosts without namespaces
Spec: #21 Run grant workers in a filter sandbox on hosts without namespaces
Ticket: #22 Grant workers confine themselves under the namespace sandbox
Flag: none
🤖 Generated with Claude Code