Repository navigation
feat(gateway): An operator can run the Gateway under the filter sandbox where namespaces are refused - #34
Merged
Conversation
The Gateway configuration takes an optional `worker_sandbox`: "namespaces" (the default) or "filter". Under "filter" the supervisor starts each grant worker itself, with no bubblewrap: a cleared environment, no descriptor beyond the control socket and the standard streams, and a parent-death signal in place of bubblewrap's --die-with-parent. The grant worker then confines itself with the base layers, as under the namespace sandbox. The Gateway never falls back from one sandbox to the other. When the chosen sandbox cannot start, startup exits with `worker sandbox "<sandbox>" failed: <layer>: <cause>` and a pointer to the operator guide's new section on choosing a worker sandbox. A new job, gateway-filter-sandbox, runs the containment test and the runtime test under "filter" in a container under Docker's default security profile, on x86-64 and arm64, and proves that "namespaces" fails at startup there with the named error. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ap at hand (#23) A container whose profile refuses one call of the Landlock layer: the Gateway under "filter" exits with the named error and starts no grant worker another way. The developer guide's line on the script moves to a paragraph of its own. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…nels launch (#23) - A grant worker that cannot apply a base layer exits with a status that names the layer, so the startup probe no longer reads it out of the standard error text, where a warning ahead of it hid it. - The filter sandbox's launch no longer needs Linux 5.11: where the kernel or the container profile refuses close_range, the descriptors listed before the fork are marked close-on-exec one by one. The docs drop the kernel floor. - The container script takes the executables' paths from cargo, mounts cargo's target directory, and gains two steps: the filter sandbox fails by name where the host refuses the allow-list, and passes on no descriptor where close_range is refused. - The runtime test's --startup-fails takes the layer it expects, and looks for the other sandbox's quoted value, not the bare word. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…23) Where close_range is refused, the grant worker's launch now lists the open descriptors itself, between fork and exec, by system calls alone, and fails when it cannot list or mark one. The listing no longer happens in the supervisor ahead of the fork, where a descriptor opened in between would have been missed. The container script removes its temporary profile, and it and the developer guide say what its two last steps do. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
4 tasks
sodre
marked this pull request as ready for review
November 20, 2026 05:36
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #23
Summary
An operator on a host that refuses namespaces, such as ECS Fargate, can set
worker_sandboxto"filter"in the Gateway configuration, and the Gateway starts and serves grants there without bubblewrap. With the field left out nothing changes: the namespace sandbox stays the default, the Gateway never switches from one sandbox to the other, and a sandbox that cannot start stops the Gateway withworker sandbox "<sandbox>" failed: <layer>: <cause>.<layer>isnon-dumpable,Landlockorseccomp allow-listwhen a base layer failed,namespaceswhen bubblewrap or its namespaces did, andlaunchwhen the filter sandbox could not start the grant worker at all.Choices the ticket left open:
"filter". bubblewrap's--die-with-parentgave this under the namespace sandbox. Started directly, a grant worker would wait forever on its control socket after the supervisor was killed, holding a grant's traffic keys. The launch sets a parent-death signal, and the runtime test now asserts it under both sandboxes./proc/self/fditself between fork and exec and marks each one, and fails when it cannot. Spec story 9 asks that older kernels are not locked out.launchas a layer name. ADR 0001's layers cannot name a failure to start the process, and the error's form needs one. The operator guide defines it where the operator reads it.scripts/filter-sandbox-container.sh, so a maintainer can run the same thing locally. It builds on the host and runs inubuntu:24.04as an unprivileged user. Two images: one without bubblewrap (the filter sandbox does not need it), one with (the namespace sandbox fails there because the host refuses namespaces, not because a program is missing). Two last steps leave Docker's default profile for one quirk each: a host that refuses the allow-list, and a kernel withoutclose_range.Extras for the owner
maintoo.assert site.cutoff.wait(1)in the shared-mode retirement part ofcrates/countersign-gateway/tests/runtime.pyfailed 2 of 6 runs ofmain's test againstmain's binary on this arm64 host, and at a similar rate on this branch under both sandboxes. It predates this ticket and nothing here touches it. No source settles it.Evidence
worker_sandbox,"namespaces"(default) and"filter"; any other value fails loading"filter", grant workers start without bubblewrap and without/usr/bin/bwrap, in sidecar and shared modescripts/filter-sandbox-container.shstep 3, in an image with no bubblewrap"filter"(sidecar, shared, concurrent users, cutoff, revocation)gateway-filter-sandboxon36d2eda, x86-64 and arm64 runners (https://github.com/zeroae/countersign/actions/runs/44679962288):sidecar Gateway mTLS passed,shared Gateway mTLS passed,filter-killed grant worker fencedunderGateway worker sandbox "filter"25 passed, 0 failedon x86-64,23 passed, 0 failedon arm64"namespaces"fails at startup there with the named errorworker sandbox "namespaces" failed: namespaces: bwrap: No permissions to create new namespace, ..., sidecar and sharedworker_sandboxand both valuesmkdocs build --strict, which resolves the row's link to the new sectionmkdocs build --strict; the startup error's link ends in the section's anchor,operate/#choose-a-worker-sandbox, which the startup tests assertmkdocs build --strict, which resolves the row's linkgit diff origin/main...HEAD -- docs/specs/2026-11-17-countersign-design.md: one line, naming the namespace sandbox and the filter sandboxruntime.py,configuration_loading)AssertionError: namespacesruntime.py,launched_directly)"filter"routed through bubblewrap:AssertionError: Name: bwrap--worker-sandbox namespaces --startup-fails namespaces, script step 4)try "filter"added to the message:AssertionError; with"namespaces"routed to the direct launch, a fallback: the Gateway started,TimeoutExpired--worker-sandbox filter --startup-fails 'seccomp allow-list', script step 5, image with bubblewrap)TimeoutExpired, exit 1. With the exit-status mapping shifted by one:worker sandbox "filter" failed: Landlock: ...,AssertionErrorruntime.py, after the filter-kill scenario)AssertionError: A grant worker outlived its supervisorfilter_sandbox_launch_passes_on_no_descriptor_and_no_environment)close_rangestarting at 1,000,000: FAILED; withenv_clearremoved: FAILEDclose_rangeis refused (script step 6)What else ran, on an arm64 host (Linux 6.17, Docker 29.2.1, bubblewrap 0.9.0), at
36d2eda:scripts/filter-sandbox-container.sh: exit 0, six steps. The container refuses namespaces (bwrap: No permissions to create new namespace). Containment test: 23 passed, 0 failed. Runtime test under"filter":sidecar Gateway mTLS passed,shared Gateway mTLS passed,filter-killed grant worker fenced, with the logGateway worker sandbox "filter": non-dumpable, Landlock ABI v7, seccomp allow-list. Step 4 printedworker sandbox "namespaces" failed: namespaces: bwrap: No permissions to create new namespace, ...; step 5 printedworker sandbox "filter" failed: seccomp allow-list: Invalid argument (os error 22); each followed bySee "Choose a worker sandbox" in the operator guide: https://zeroae.github.io/countersign/operate/#choose-a-worker-sandbox.runtime.py target/debug/countersign-gateway, with no option and with--worker-sandbox filter, on the host: all three lines each. Each needed retries for the flake under Extras.cargo test --locked -p countersign-gateway: 14 unit and integration tests passed and the containment test's 23, 0 ignored, 0 failed.cargo fmt --all -- --check,cargo clippy --locked -p countersign-gateway --all-targets -- -D warnings,mkdocs build --strict: clean.Probe (2026-11-19): on
main,runtime.pywithmain's binary, six runs: four passed, two failed atassert site.cutoff.wait(1).GitHub, on
36d2eda: all 28 checks passed. The jobgateway-filter-sandboxran all six steps of the script on x86-64 and arm64 runners, and the workspace jobs rancargo test --workspace,pnpm lintandpnpm test.Not run: nothing
Merge Danger
Door: two-way. A revert removes the field; a configuration that sets
worker_sandboxthen fails to load as an unknown field, so an operator who set it removes it first. No release carries the field yet.Blast Radius: Gateway startup and grant worker launch.
The merge ships no image and no infrastructure. It adds one CI job to the end-to-end workflow. For a Gateway that leaves
worker_sandboxout, two things change at the next release: a failed startup probe printsworker sandbox "namespaces" failed: namespaces: <cause>in place ofGateway worker sandbox failed to start (...), and a grant worker that cannot apply a base layer exits with 64, 65 or 66 in place of 1. No flag.Reviews
Correctness, Standards, Spec and Security, three rounds: 24 findings, each with its outcome. Round 3 found nothing.
Pinned commits: round 1
7b85f451de2439a1ddf69de642118de9c93b9e80, round 211bed34dbeba058e81e5e6fef99615da8a59af6a, round 336d2edaa9166242fa5142f244f0b08b93b3f6a33. Security found nothing in any round.Correctness, round 1
--cap-drop ALL,--new-sessionand--chdir /. Pushed back: the allow-list leaves a confined grant worker no system call that a capability, a session or a working directory changes (signals only to itself, no file or socket call,ioctlonly forFIONBIO), it confines itself before reading any input, and the operator guide keeps the supervisor unprivileged../target. Fixed: it takes each executable's path and the target directory from cargo, and mounts the directory.HOMEbeing set and standard input being open. Fixed: it duplicates a descriptor it opened, and reads the launched environment withenv.WorkerSandbox::outer_layerserves the startup log and the error.Standards, round 1
launchis a layer that neither ADR 0001 nor the glossary defines. Pushed back: the error's form needs a layer for a grant worker that could not be started at all, the ticket itself says the supervisor "launches each grant worker directly", and the operator guide defines the name beside the others.sandbox(sandbox, ...)anddirect. Fixed:launch_command,namespace_sandbox,filter_sandbox.docker runrepeatedinside. Fixed:insidetakes the profile.configuration_loading. Fixed.do_testtakes a third flag. Pushed back: the function held several scenarios behind flags before this ticket, and splitting the runtime test is not this ticket's work.Spec, round 1
--die-with-parentgave under the namespace sandbox; without it a grant worker holding traffic keys outlives a killed supervisor. Named in the Summary as a choice the ticket left open.Correctness, round 2
Standards, round 2
Spec, round 2
fcntl's result, and said nothing. Fixed: the listing runs between fork and exec by system calls alone, and the launch fails when it cannot list or mark a descriptor.Tracking
Epic: E1 Run grant workers in a filter sandbox on hosts without namespaces
Spec: #21 Run grant workers in a filter sandbox on hosts without namespaces
Ticket: #23 Operators can choose the filter sandbox
Flag: none
🤖 Generated with Claude Code