Skip to content

fix(dynamic-test): the recorded build-egress ruling — the falsifier's second disjunct, the accepted state pinned so it cannot drift (#710) - #875

Open
gadievron wants to merge 1 commit into
masterfrom
fix/710-build-egress-allowlisting
Open

gadievron wants to merge 1 commit into
masterfrom
fix/710-build-egress-allowlisting

Conversation

@gadievron

Copy link
Copy Markdown
Collaborator

What

Closes #710 — #247 closed runtime egress for the dynamic tester and named the build half as an accepted, deferred residual ("Build-time RUN egress stays OPEN… A build-egress allowlisting proxy is deferred"), tracked nowhere but a code comment.

The falsifier (authored at filing): "a dynamic-test build whose RUN steps resolve only allowlisted hosts (or a recorded ruling that build egress is accepted by design)."

The decision this issue asked for — and how it was made

One implementation of the first disjunct (a loopback-bound host-side proxy routed via the build's proxy env) was built and then killed by executed evidence (the adversarial review round, with Docker available, running the production argv shape — the probe records live in the T1 review chain, runs/710/_t1_r1_review.txt):

  1. A host-side proxy is unreachable from the RUN steps — Docker's predefined proxy build-args do reach the RUN env (verified), but http://127.0.0.1:<port> there is the build container's own loopback, not the host's: every dependency build fails with ConnectionRefused — the generator's own templates (pip/npm/go) all break. The routing that would reach a host-side proxy (--network host on the build) puts the RUN steps in the host's netns — the build-time posture this module exists to prevent — and binding the proxy on a non-loopback interface to make it bridge-reachable exposes it to containers on the host's bridge networks (a bind/topology exposure — not firewall-measured here).
  2. The proxy-env control is advisory, not structural — the Dockerfile is LLM-authored from untrusted findings: a single ENV http_proxy= line (measured) or a raw socket (measured) egresses freely. A cooperating-client control cannot close a channel the content author controls.
  3. (design judgment, honestly labelled — not executed) The real structural control — a build network-namespace with a proxy sidecar, or --network none with vendored deps — either breaks every dependency build or is a disproportionate re-architecture for an opt-in, defense-in-depth surface.

Structural proxy architectures (a build netns with a proxy sidecar, or --network none with vendored deps) were declined by design judgment — honestly labelled as such in the record — not disproved by measurement. So this lands the second disjunct — the recorded ruling, with the evidence named where the residual comment lived, plus drift pins holding the enumerated protections below.

The recorded ruling

docker_executor.py (the single-build site) now carries the decision, its evidence honestly labelled (two executed findings + one design judgment), and the accepted posture: the RUNTIME sandbox is the hard boundary (--network none), the env allowlist limits the inherited environment variables available to build-time interpolation (a name-agnostic scrub — the pin holds the provider-secret-dropped / docker-essentials-kept contract), and the build context is minimal.

The drift pins (tests/test_issue710_build_egress_ruling.py, 7 tests)

The pins hold the enumerated protections — the argv exact-equality, the _safe_build drop, the flag presence, and the --tmpfs count — not a full drift-proof — the enumerated protections only (the resource-cap values are held by the #732 suite; the env-allowlist membership by the #247 hardening suite):

  • The single-container build argv pinned exactly (["docker","build","-t",tag,"."]) — any added flag (e.g. --network host) is a deliberate diff the pin names.
  • The compose build argv pinned exactly.
  • The runtime argv closed (--network none + every cap, --tmpfs counted ×2).
  • The env allowlist in force (a provider secret dropped, docker essentials kept).
  • The compose FILE-side route dropped (_safe_build drops build.network/extra_hosts/entitlements — the review round's catch: no pin held this drop; a one-line drift would silently reopen the --network host posture. Verified load-bearing: the pin fires on a simulated passthrough drift).
  • The ruling's presence (reads the executor source: the ruling text, the honest labelling, the runtime-boundary clause — the one assertion-red witness at base).
  • No proxy-routing residue (no --build-arg/HTTP_PROXY in the executor; the proxy module is gone).

The executable AST is unchanged from base (checked with docstrings stripped — a normalization-identity, not a byte comparison) — the deliverable is the record + the pins, never a behavior change to the #247 machinery.

Evidence (runs/710)

Parity

The legs' own terminal lines, pasted verbatim (the full logs in the bundle — base-parity-full.log, head-parity-settled.log [the WARM run], head-parity-settled-cold.log, head-parity-go.log, collect-diff.txt):

base (at 9143061f, the full core suite):

FAILED tests/patch/test_run_cve_batch.py::test_interrupt_then_resume - Assert...
FAILED tests/test_llm_sdk_contract_floor.py::test_installed_sdks_match_the_requirements_pins
FAILED tests/test_llm_sdk_contract_floor.py::test_anthropic_pin_tied_to_requirements
3 failed, 13280 passed, 38 skipped, 2 xfailed, 1 warning in 1677.49s (0:27:57)

head, warm (at a053fa18, the same suite): the standing trio identity-matched; the +8 passing over base = the 7-test ruling suite + the 1 parametrized no-hardcoded-paths guard item; the skips identical (38 = 38):

FAILED tests/patch/test_run_cve_batch.py::test_interrupt_then_resume - Assert...
FAILED tests/test_llm_sdk_contract_floor.py::test_installed_sdks_match_the_requirements_pins
FAILED tests/test_llm_sdk_contract_floor.py::test_anthropic_pin_tied_to_requirements
3 failed, 13288 passed, 38 skipped, 2 xfailed, 1 warning in 1592.12s (0:26:32)

the go module (at the head): head-parity-go.log ends EXIT:0 — all packages ok.

collect-diff: base 13,324 collected → head 13,332 — +8 added / 0 removed, exactly the ruling suite's 7 tests + the parametrized guard item (collect-diff.txt).

the cold-run disclosure: the head worktree's FIRST full run showed 3 failed, 13146 passed, 180 skipped — the +142 skips over base are the first-run provisioning class, fully decoded by the warm run above (38 = 38); the cold log retained as head-parity-settled-cold.log.

… second disjunct, the accepted state pinned so it cannot drift (#710)

#247 closed RUNTIME egress (--network none / internal compose nets) and
named the build half as deferred: 'Build-time RUN egress stays OPEN... A
build-egress allowlisting proxy is deferred.' The falsifier (authored at
filing): 'a dynamic-test build whose RUN steps resolve only allowlisted
hosts (or a recorded ruling that build egress is accepted by design).'

The host-proxy alternative (the first disjunct) was IMPLEMENTED and then
KILLED BY EXECUTED EVIDENCE (the adversarial review round's probes):
(1) a host-side proxy pointed at via Docker's predefined proxy build-args
is UNREACHABLE from the RUN steps — 127.0.0.1 inside a build container is
the CONTAINER's own loopback, and the only routing that reaches the host
(--network host) reopens the runtime posture this module exists to
enforce — measured: every dependency build fails with ConnectionRefused;
(2) even with routing, the proxy-env control is ADVISORY, not structural —
the Dockerfile is LLM-authored from untrusted findings, and a single
'ENV http_proxy=' line or a raw socket egresses freely (both executed);
(3) the real structural control (a build network-namespace with a proxy
sidecar, or --network none with vendored deps) either breaks every
dependency build or is a disproportionate re-architecture for an opt-in,
defense-in-depth surface.

This lands the SECOND disjunct — the recorded ruling, with the executed
evidence named where the residual comment lived, plus the DRIFT PINS: the
accepted state (the build invocations run with default networking,
exactly as #247 left them; the runtime argv closed; the env allowlist in
force) is pinned test-side so it cannot drift silently — any flag added
to a build, any runtime cap removed, any allowlist entry loosened, is a
deliberate diff that fires the pins.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant