Skip to content

fix(gateway): A confined grant worker cannot start a process by clone3 - #30

Merged
sodre merged 2 commits into
mainfrom
feat/28-clone3-enosys
Nov 20, 2026
Merged

sodre merged 2 commits into
mainfrom
feat/28-clone3-enosys

Conversation

@sodre

@sodre sodre commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Closes #28

Summary

A compromised grant worker can no longer start a second process with clone3. Operators change nothing: grant workers serve connections on threads as before.

 seccomp allow-list (every grant worker, both worker sandboxes)
   foreign architecture, x32     -> kill
+  clone3                        -> fails with ENOSYS; the C library retries with clone
-  clone3                        -> allowed (a thread, or a new process)
   clone with CLONE_THREAD       -> allowed
   anything else off the list    -> kill

Choices the ticket left open:

  • ENOSYS, not "keep clone3". The ticket asked for the decision. Under the filter sandbox a second process would outlive the supervisor's kill of the grant worker and keep the grant's streams, so allowing it has no good reason to record. ADR 0002 holds the decision and supersedes ADR 0001's decision 5 for this one call. It is written as accepted; merging this is the acceptance.
  • The no-clone3 test setup stays. The allow-list now gives the answer that setup stood in for, so its three tests overlap the confined ones. It is kept because it runs the whole containment test as on a host without clone3 (Docker's default profile, where Operators can choose the filter sandbox #23 will run it).

Extras for the owner

  • The runtime test fails on main on an arm64 workstation. runtime.py:331 (assert site.cutoff.wait(1), the shared-mode cutoff on a retirement notice) failed in 7 of 8 runs of origin/main's own build, and in 5 of 6 on this branch, on an aarch64 host (kernel 6.17, load average about 5). No source settles it.

Evidence

Criterion Ticked Shown by
An ADR records the decision for clone3 yes docs/adr/0002-…, read by the test "Starting a process by clone3 fails as unsupported", which asserts the ENOSYS the ADR decides
If ENOSYS: a confined grant worker cannot start a new process by clone3, shown by the containment test with a raw clone3 call without CLONE_THREAD, and its threads still work yes Starting a process by clone3 fails as unsupported
The containment test and the runtime test pass on x86-64 and arm64 yes This pull request's CI at ea413ce: workspace (ubuntu-latest) (containment test, x86-64), curl (runtime test, x86-64), gateway-arm64 (both, arm64)
Test Red Green
Starting a process by clone3 fails as unsupported (starting_another_process_by_clone3_fails_as_unsupported) Against the allow-list before the change: left: GotThrough, right: Errno(38), 22 passed, 1 failed ok
The same test on a host without clone3 The test as first committed, run whole under the no-clone3 setup (<test> stand-in no-clone3 <test> run): stand-in 0, clone3: expected success, got Errno(38), 22 passed, 1 failed ok: this host has no clone3: the call fails unconfined too, 23 passed, 0 failed

All on arm64, glibc 2.39, kernel 6.17, at ea413ce:

  • cargo test --locked -p countersign-gateway: containment test 23 passed, 0 failed (including its_stream_arrives_and_its_threads_work, whose thread now starts by clone); the other suites 13 passed, 0 failed, 0 ignored.
  • cargo clippy -p countersign-gateway --all-targets --locked -- -D warnings and cargo fmt --all -- --check: clean.
  • python3 crates/countersign-gateway/tests/runtime.py target/debug/countersign-gateway, the real Gateway under the namespace sandbox: passed once at 936db20 (sidecar Gateway mTLS passed, shared Gateway mTLS passed, filter-killed grant worker fenced) and once in six runs at ea413ce. The other five failed at runtime.py:331, after sidecar Gateway mTLS passed. origin/main built apart fails the same line in 7 of 8 runs (see the extra).

This pull request's CI at ea413ce, runs 44666958013 (CI) and 44666957972 (Sidecar mTLS): workspace (ubuntu-latest) ran cargo test --workspace --locked on x86-64, curl ran runtime.py on x86-64, and gateway-arm64 ran cargo test -p countersign-gateway and runtime.py on arm64. All passed, as did every other job.

Not run: nothing. The runtime test's failures on the arm64 workstation are carried over to #32.

Merge Danger

Door: two-way. A revert allows clone3 again; nothing stored changes.

Blast Radius: every grant worker's thread start.

The merge changes the Gateway binary and so its image at the next release. No configuration, flag or infrastructure changes. A C library that does not fall back from clone3 to clone on ENOSYS could not start threads in a grant worker; glibc and musl both do.

Reviews

Correctness, Standards, Spec and Security, two rounds: 6 findings, all fixed

Round 1 pinned 936db20; round 2, the fix round, pinned ea413ce and found nothing under any heading. Security was read in the session against the diff, without the bundled review's sub-tasks, and found nothing in either round.

Correctness, round 1

  1. The new test's unconfined baseline needs a raw clone3 to succeed, so it fails on a host that refuses clone3 itself. Fixed: the test notes such a host and skips the baseline, shown red and green under the no-clone3 setup.
  2. The no-clone3 setup and its three tests now overlap the confined ones, and its doc called it a Docker-only quirk. Fixed: the doc says what the setup still covers; the tests stay.
  3. allow_list()'s doc comment still said every call outside it kills. Fixed: it names the clone3 exception and ADR 0002.
  4. The confined half of the new test copied each_of_two by hand and lost its messages. Fixed: it calls each_of_two.

Standards, round 1

  1. Possible Duplicated Code: the hand-copied loop of finding 4. Fixed.
  2. Possible Duplicated Code: the no-clone3 setup of finding 2. Fixed: the doc says what the setup still covers.

Tracking

Epic: E1 Run grant workers in a filter sandbox on hosts without namespaces
Spec: #21 Run grant workers in a filter sandbox on hosts without namespaces
Ticket: #28 The allow-list tells a thread from a new process under clone3
Flag: none

🤖 Generated with Claude Code

sodre and others added 2 commits November 19, 2026 22:10
The allow-list answers clone3 with ENOSYS, so the C library starts threads
with clone, whose flags the filter checks for CLONE_THREAD. ADR 0002 records
it as the one call outside the allow-list that does not kill.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ames its exception (#28)

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@sodre
sodre marked this pull request as ready for review November 20, 2026 03:50
@sodre
sodre merged commit 05d4738 into main Nov 20, 2026
34 checks passed
@sodre sodre mentioned this pull request Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The allow-list tells a thread from a new process under clone3

1 participant