[MaxGhenis]

WIP. Do not merge before 2026-10-29 12:00Z (UK hold on shared compile modules), and land it before NZ's G8-r2c, which depends on it.

What this does

Adds an optional precalibration_stages transport resource: the rules bindings whose declared outputs may feed calibration. Both countries' income producers enter calibration through it (the joint d1952 ruling).

• Belgium: rule-computed professional income and tax on tax units, which are unit-entity semantics the ancestry guard refuses today.
• NZ (r2c): person-entity producers. Today's guard already admits them; the list declares them and restricts what flows from them.

Spec: S5-SPEC revision 2, reviewed by the NZ hub. The repo doc is docs/transport-precalibration-stages.md.

• Absent or "stages": []: the existing guard runs unchanged, and outcomes and messages match a frozen copy of it.
• A listed unit-entity stage admits:
  • its own nodes, engine reference and outputs;
  • group-encoder outputs that only listed unit-stage nodes read.
• A listed person stage admits nothing new.
• Every listed stage passes on only its declared outputs. A node outside the stage that reads a column whose values the stage's own nodes wrote upstream of calibration, other than a declared output, refuses. So does one that reads any artifact a stage node produces: outputs are columns, so artifacts are never declared.
  • The rule follows each column read to every writer whose values it receives, as the executor's writer chain does (executor._writers_of). That is the owner in the reader's population version, then, while that owner keeps earlier values, the writers below it. A rewrite, a masked owner (an output with a rows mask) and an EXPAND claim keep them. The version's EXPAND counts as a writer of its cells.
  • A structural node reads its base. A rewrite reads its incumbent, and a claim (materialized_expand_outputs) reads its cell, both without a Slice.
  • Another binding with the same variable name, or another node owning a column of the same name in another version, is unaffected.
  • The composed pass resolves exactly, and is checked against the executor's own static writer lists (executor._input_writers): on every property graph, the writer and EXPAND examples, and the composed NZ and entitlement graphs.
  • Where a declaration leaves something unknown, the declaration pass falls back:
    • an unknown version (no population, an undeclared version, a cycle of bases) counts every declared writer;
    • an owner without a population version counts in every version;
    • an EXPAND whose expand_cells is not a literal list writes every column read through its version;
    • a non-literal materialized_expand_outputs claims every non-rewrite output.
• A stage changes columns, not the population (since commit 14): a node of a listed stage upstream of calibration, calibration nodes included, may not be structural (CREATE, FILTER, EXPAND or REWEIGHT). Both passes refuse it first, naming the node, its kind and its stage binding. A structural node also writes what every later kernel of its version receives without a Slice (ids, memberships, weights, strata, the frame metadata and the mass log), and the executor lets no other node write those. A structural node outside every stage is checked like any other node.
• Derived outputs (optional derived_outputs): columns a stage node computes that are not binding variables, such as NZ's w2b simulate.solve_zero@1 gross-up output. Each must be owned by a stage node upstream of calibration (a stage claim that owns an EXPAND cell counts). A stage may list "outputs": [] and only derived outputs, which is NZ's Schedule 1 shape: the tax stays inside the solve. This is option (b) of the NZ hub's ask.
• Hold-out isolation: a stage's outputs, derived outputs and module must not be hold-out measures. The hold-out, selection and digest checks apply to every upstream node. In the composed pass they read each node's declared selections: its transport_graph row or the entitlement row the extension instantiated it from (entitlement._rows), whose params are resolved before the graph exists. A code extension's node has no declaration, so it gets the engine-reference and inventory checks only.
• The prepared engine_refs inventory refuses upstream of calibration, on stage nodes too: a mapping with an engine_refs key, at any depth, or a string holding the quoted key "engine_refs" and a colon, as written or after decoding JSON escapes layer by layer. That covers spaced or nested JSON, braces or the key written as escapes (\u007b, \u0065ngine_refs), JSON embedded in a string, and the data a resource selection names. Plain text naming engine_refs is not the map.
  • Each layer is one linear pass. Decoding stops after 32 layers, and a string still escaped then counts as carrying the map, so the scan costs at most 33 passes.
  • An encoder that escapes a backslash as \\ doubles them per layer, but one that writes \u005c grows linearly: 33 layers of the quoted key take 344 characters. So the bound is reachable, and harmless text that deep refuses, as does prose that quotes the key and a colon. None of the 1.65 million strings in the 179 build JSON files carries the map, and none holds an escape after parsing.
• Scope: only nodes in the ancestry of a calibration node (every calibrate.* node and the calibration endpoint) are checked. A stage binding's nodes outside the ancestry of every calibration, such as extension checkpoints, are unaffected. A node after the central fit that a variant calibration reads is checked.
• Malformed declarations refuse: with stages listed, every field the checks read is validated first, on every declared node and rules binding: node id, kernel, links, structural, params selections, inputs, outputs (with any rows mask), artifact_inputs and sources; binding id, entity, rulespec_path and variables; and the calibration endpoint (text when present; an explicit null refuses). In the composed pass, entitlement rows need distinct text ids, none a skeleton id, and well-formed params. A malformed one refuses as "node 'x' has malformed " instead of being read character by character. The guard without stages is unchanged.
• No hashes change: no node parameter or hash moves, so the NZ, BE, UK, US and toy outputs are unchanged.

Invariants (tested)

• I1, conservativeness:
  • With no stages, the result is exactly today's guard.
  • With only person stages listed, an accepted graph is also accepted by today's guard.
  • Unit stages admit only their own nodes, refs, declared outputs and encoder outputs that only they read.
• I2, monotonicity (corrected from S5-SPEC revision 2): dropping a person stage never refuses what listing it admitted.
  • While another stage stays listed, dropping a unit stage never admits what listing it refused. The exception is when the refusal was that stage's own declaration error: its schema, a hold-out measure, or a derived output no upstream stage node produces.
  • Dropping the last stage gives exactly today's guard (I1), which can admit more. Its declaration pass doesn't follow column reads, and a resource selection leaves no marker in compiled params. The spec's original I2 ("removing a stage never admits") is false for person stages, since a listed person stage adds restrictions.
• I3, hold-out isolation: with stages listed, no hold-out source and no declared hold-out selection (in a transport_graph or entitlement row) is upstream of calibration, and no stage output or module carries a hold-out measure.
  • Without stages the guard is unchanged. It still misses a holdout_references selection that calibration reaches only through column reads, and any hold-out selection in an entitlement row.
  • That gap predates this PR and is shown on its no-stage path, which equals main. It is filed as separate tasks, so I1 identity holds here.
• I4, determinism: with stages listed, shape checks run first over every declared node and binding, so a malformed declaration refuses naming its first malformed node. Then, in each pass, the first structural stage node upstream of calibration refuses, then the first derived output no upstream stage node produces, and otherwise the first offending node in declaration order. The composed pass runs only after the declaration pass admits: it checks the entitlement rows' shapes and reads the compiled EXPAND cells and claims, refuses the first node upstream of calibration with a parameter still JSON-escaped after 32 layers, then applies the same order to the graph's nodes. One message is not fixed: with several links to undeclared nodes, which one a refusal names follows set iteration, as without stages. Without stages, the existing guard's iteration is unchanged.
• I5, population changes: with stages listed, no node of a listed stage upstream of calibration is structural, in either pass. So the ids, memberships, weights, strata and frame metadata that kernels receive without a Slice come, upstream of calibration, only from nodes outside every stage. The executor's writer lists cover column reads only, which is why this needs its own rule.
• I6, the frame mass log: a transport run refuses any node whose admitted mass log differs from the log it received. That covers a stage's node or not, calibration included, whether it ran or was restored from a store a plain run wrote. Any node's receipt can append to the log, and later structural nodes receive it without a Slice. The guard can't see that, so the runner enforces it. No transport kernel appends.
• I7, stage membership: the composed pass's layered engine-reference scan finds every reference that decoding JSON from each quote or brace (_embeds_engine_ref) finds, tested over fixed and random references. A reference's JSON-escaped text counts wherever it appears. The scan is linear in the parameter's length. A parameter still escaped after 32 layers refuses upstream of calibration only, where bindings are read.

The Hypothesis property (2,000 examples in each of four domains) compares the guard with three things in both passes, and checks the composed pass's writers against the executor's on every graph:

• an independent reference predicate;
• the frozen guard without stages, for I1;
• the same model with one random stage dropped, for I2.

The mixed domain draws encoders, unit and person rules, selectors and hold-out nodes, with hold-out sources on stage nodes, variable names shared between bindings, and derived outputs. The rules_only domain drops hold-out data, so most graphs pass the other checks and producer attribution is exercised in every run. Since 20dfb3e the domains also draw population versions: filter nodes, a version closed once a filter branches from it, and shared_<entity> columns owned in different versions, with each read naming the producer it resolves to. Since 3cb8172 they also draw plain non-stage owners of a shared column, person rewrites in a filter version (read of the incumbent) and filter inputs from the base. A third domain, versions, makes those likely, so version attribution decides most outcomes. Since ebe6f3d a fourth, writers, draws masked owners and EXPANDs whose cell the next node claims, with reads naming every writer. Its focused mode keeps refusals to about one cause. Since commit 14, EXPANDs are mostly outside every stage, and any structural node is a person binding's node one time in eight, so listed structural stage nodes refuse in the reference too. A claim with a binding owns a new cell as that binding's derived output. Otherwise the cell may overlay a person column the EXPAND sees, so the claim receives every writer below it.

Commits

1. 293bec1: the build, from the spec on a GPT-6.1 Sol build lane. It adds 19 test functions (56 cases), the frozen-guard differentials and the property.
2. 7b933e9: review fix. The build had treated every node of a listed stage as a root, including nodes after calibration. That would refuse a sound graph wherever a listed binding also runs in an extension checkpoint beside hold-out data or unit rules.
   • Two new example tests and the property fail against the old roots (checked).
   • The commit also tightens the property with the I1 and I2 checks and adds the doc.
3. 062d89c: fixes the four findings of the GPT-6.1 Sol review of 7b933e9 (REQUEST_CHANGES), and adds derived_outputs.
   • Major: spaced JSON let the inventory through. It is now recognised by decoded structure.
   • Major: listing an unused binding that shares a variable name restricted another stage's internal reads, which also broke I2. Rule 3 is now producer-attributed.
   • Minor: P1 missed hold-out sources on stage nodes. The generator now draws them, and the rules_only domain was added.
   • Minor: malformed benefit-unit entities raised TypeError with no stages. Entities are now compared, never hashed.
4. ead5c9c: lets a stage declare only derived outputs ("outputs": []), since otherwise NZ's Schedule 1 stage would have to list the tax variable. A stage with neither still refuses.
5. 16c35df: the NZ hub's item 4. A regression checks that a node after the central fit that feeds a variant calibration is checked in both passes, and the doc now says "outside the ancestry of every calibration node".
6. 3d8e9f9: closes an artifact channel. A node naming a listed unit stage could pass an artifact straight to calibration: today's guard refuses that graph, and the stage path admitted it, against I1.
   • Rule 3 now covers artifact edges in both passes, and the composed rows carry them.
   • The property draws artifact edges apart from column reads. That exposed the no-stage gap above, so P1 compares the no-stage case with the frozen guard only.
7. c4e23bd: refuses malformed declarations on the stage path. The NZ hub's review lane asked what a malformed artifact_inputs entry does; at 3d8e9f9 it raised KeyError or TypeError, as today's guard and _node still do. Probing it found three spellings the stage path accepted where their list form refuses, because a string was read character by character:
   • a string columns hid a read of an undeclared stage output;
   • a string selection path hid selected data embedding an engine reference;
   • a string sources hid a hold-out source.
     A repeated node id hid the first node's reads, and a string binding variables matched stage outputs by substring.
   • The fix validates every field the checks read, before either pass. Resource selections follow activation's rules (text resource, list path, known encoding), and binding selections (binding with field) name both as text. Engine-reference documents and object-valued prepared data are untouched.
   • A new example test isolates the composed pass's artifact edges, which only the property test covered before.
8. 20dfb3e: fixes the five findings of the GPT-6.1 Sol review of c4e23bd (REQUEST_CHANGES: three majors, a minor and a nit).
   • Major: braces written as \u007b escapes hid the inventory, and selected resource data was never checked for it. Every string is now decoded, and the selection loop refuses selected data carrying the inventory, in both passes.
   • Major: the composed pass recovered selections only from transport_graph, so a variant target compiler in the real entitlement fixture could select holdout_references and reach calibration through artifact edges. The composed pass now reads the entitlement rows' declared selections; that graph fails compose_transport_graph, and the unmodified fixture still composes (92 nodes).
   • Major: rule 3 conflated population versions. A stage's own person.shared in version v2 refused as another stage's private column of the same name in create. Rule 3 is now attributed through version lineage, as above.
   • Minor: an explicit null calibration endpoint was ignored. It now refuses.
   • Nit: the I4 doc now says shape checks come first.
9. c05685c: the NZ hub's review of c4e23bd (SIGN_OFF_WITH_ASKS) found the same three holes as round 2 (its M1, M2 and m1, all fixed in 20dfb3e) and listed doc sentences the code contradicted (m2). This commit closes the m2 items still open:
   • A non-text or repeated node id now refuses naming it ("node 'x' is declared twice", "node id None is not text"), in the declaration and in the entitlement rows, so "a malformed field refuses, naming the node" holds. Rules binding ids refuse the same way.
   • The doc maps each node list to its own fields, and says the guard without stages checks selections only in the rows its declaration pass reaches through population, base and artifact links.
   • The guard's docstring says which view the derived-output check uses.
10. 3cb8172 (tests only): a property-only mutant pass at 20dfb3e killed 9 of 32 and missed the version-blind rule 3. The generator drew a graph that tells it from the reference about once in 2,000 examples, and never drew a rewrite or a structural node's inputs. The generator now draws them (above), and the property alone kills 14 of 32. In ten property-only runs it killed the version-blind rule 3, a compiled rewrite resolved at its own version and compiled structural reads at their own version every time, and rewrites ignored in either pass nine times.
11. bfd5e85 (doc only): the NZ hub's sign-off noted that "every field the checks read is validated first" overstated the code. Three values get no shape check, because the checks read them exactly as composition does: an output's rewrite flag (by truthiness, as the compiler reads it; Owned doesn't type-check it), a selection path (through mappings only; composition refuses anything else), and a prepared value (only "engine_refs" counts, the one prepared parameter composition admits). The doc now says so. A bool check on rewrite would refuse a shape composition accepts.
12. ebe6f3d: fixes the five findings of the round-3 Sol review of 3cb8172 (REQUEST_CHANGES).

• Major: a masked owner of a stage's private column hid the values it leaves outside its mask. The reviewer executed this: calibration received the stage's private value. Reads now follow every writer, as above.
• Major: a materialized EXPAND claim forwarded an EXPAND's private cell with no Slice, again executed. Claims are now reads, and EXPANDs are writers of their cells.
• Minor: the inventory scan was quadratic on harmless strings with many quotes (32 KB took 24 s). It is now layered decoding, at most 33 linear passes.
• Minor: the non-text extension-row id refusal had no test; it has one now.
• Nit: I4 now says which message is not fixed.
• Also: structural and output rows shape checks, the executor differential, and the writers property domain.

13. 339445f (tests only): the round-4 mutant sweep at ebe6f3d killed 53 of 54. The survivor, I5, put the quadratic suffix scan back into the inventory helper. At the linear test's 32 KB it took 0.47 s on an idle host, under the test's 1 s bound; its cost quadruples per doubling (7.5 s at 128 KB).

• The test now runs at 128 KB in two cases: the helper alone, and the declaration pass on a person-only stage spec. Both take 0.01 s at the head and about 5 s with I5, which both cases now kill.
• The old test also timed main's engine-reference scan (see "Known limitation" below), because its spec listed a benefit-unit stage. The guard case lists only a person stage, so that scan has no reference to look for: its four calls return at once and decode nothing.

14. d42fbcb: answers the round-4 Sol review of ebe6f3d. That run stopped on a provider filter after about an hour; its draft (REQUEST_CHANGES: one major, two minors, nits) is in jobs/s5-review-4/partial-review.txt of the Belgium hub.

• Major: a stage could change the population. Kernels receive ids, memberships, weights, strata, the frame metadata and the mass log without a Slice, and the executor's writer lists hold no read for them. The reviewer executed it: a stage EXPAND overlays person_household_id, calibration declares no Slice and receives [10, 20], and both passes accepted. Only structural nodes write those values: the executor refuses a NONE node that owns an id or membership column or declares a weight transition, and a kernel's frame, keep, expand, weights and strata results are for CREATE, FILTER, EXPAND and REWEIGHT nodes. So a listed stage's node upstream of calibration may not be structural (above). NZ checked that none of its stage nodes is structural (517 guard calls over 13 probes).
• A stage EXPAND now always refuses, so EXPAND cells no longer count as a stage's production. An EXPAND still counts as a writer of its cells in read resolution.
• Minor (the guard is quadratic through _embeds_engine_ref): unchanged here; see "Known limitation". The test the reviewer saw fail was fixed in 339445f.
• Minor (the 32-layer rationale): corrected in the doc and the code comment (above).
• Nits: binding refusals name the binding, post-calibration nodes still get shape checks, and surrogate pairs decode as two code points; the doc now says so.
• Tests: each structural kind refuses as a stage node, declared and compiled-only; unbound structural nodes and bound ones that are no ancestor of calibration pass; the reviewer's three-node graph runs through run_graph (calibration receives the stage's membership, the writer lists hold no read of it, and both passes refuse). _expand_case moved the EXPAND outside the stage, overlaying A's cell, so the claim chain is still tested, with a read-first shape for claims that keep the values below them. P1 is reworked as above.

15. 9959390 (doc and test only): NZ noted that the calibration ancestry includes its roots, so a calibration node that carries a stage binding is checked as the stage's node (a REWEIGHT calibration refuses as structural), while the doc said only "upstream of calibration". The doc now says each calibration node is in its own ancestry. A new test pins it: a REWEIGHT calibration carrying A's engine reference refuses naming cal in both passes (the composed pass alone when only cal is declared), and without it passes.

16. 1d1801e: fixes the round-5 Sol review of 9959390 (REQUEST_CHANGES: one major, two minors, a nit).

• Major: the frame mass log. A non-structural stage node can pass an undeclared value to calibration through the frame mass log. Its receipt's frame_mass_log_append records reach every later structural node of its lineage without a Slice, because the executor gives a structural node its base version's cumulative log. The reviewer executed it: stage A's node appends a conserving record whose reason is its private value, calibration sets its weight from it, and both passes accepted.
  • The UK runtime appends from non-structural nodes (UKStageKernel), so the graph package can't refuse appends in general.
  • So the transport runner (cli._run) passes run_graph a population observer. It refuses any node whose admitted mass log differs from the log it received, cold or restored, before the node is persisted.
  • No kernel outside uk_runtime appends (git grep frame_mass_log_append), so no transport run changes.
  • The doc's premise that only structural nodes write the mass log is corrected.
• Minor: the scan was quadratic on S5's new path. The composed pass found a node's stage bindings with one _embeds_engine_ref call per binding, which decodes JSON from every quote. A 16 KB harmless parameter took 1.8 s.
  • It now uses _embedded_engine_refs: escapes are decoded layer by layer, as in the inventory scan, in one linear pass per layer for all bindings, up to 32 layers.
  • A parameter still escaped after 32 layers refuses.
  • The shared decoder now decodes a surrogate pair to one character, as json.loads does.
  • The inherited unit-binding calls are unchanged, so the guard without stages still equals main.
• Minor: multi-cell claims. The two_claims shape claims a public cell first and a private one second. It is in the outcome test and the executor differential.
• Nit: the changelog says post-calibration nodes still face the shape checks.
• New tests:
  • the runner refuses each appender (stage node, non-stage node, calibration), cold and restored, while both guard passes accept and a plain run delivers the value;
  • the composed person-stage path makes zero raw_decode calls at 64,000 escaped quotes;
  • the 32-layer refusal;
  • a Hypothesis property that the layered scan finds whatever _embeds_engine_ref finds (nested json.dumps, ASCII escaping with surrogate pairs, objects, lists, text embedding).

17. 973c674: fixes the round-6 Sol review of 1d1801e (APPROVE, with one minor and a nit).

• Nit: the escape bound refused after calibration. The composed pass refused a node with a parameter still escaped after 32 layers before it knew the ancestry. So a node after calibration, which the doc says faces only the shape checks, refused on a harmless 162-byte note.
  • The ancestry is now computed first, from predecessors and roots only, and the bound refuses only upstream of calibration, the calibration nodes included.
  • No check uses a non-ancestor's bindings. The structural rule's loop computes them for every node, but tests them only for ancestors.
  • The one refusal-order change: in the composed pass, an undeclared calibration endpoint now refuses before the bound on a graph with both faults.
• Minor: no test needed the escaped-text match. The reviewer found a reference the match alone finds: a newline, a backslash and n, written as its escaped text after a stray backslash. Decoding pairs that backslash with the text's first escape, so no layer holds the reference. Without the match, the node left its stage and its derived output refused as unproduced. My own 40,000-case fuzz had not found it. Tests only:
  • that case through the guard;
  • a property that a reference's escaped text counts in any surrounding text, with the reviewer's case and a non-ASCII variant as explicit examples, so the kill doesn't depend on random draws;
  • the _embeds_engine_ref differential now draws random references too, with quoteless and stray-backslash spellings. It passed 20,000 examples at 1d1801e.
• The 32-layer test covers a stage node and the calibration node (both refuse) and a post-calibration node (accepted).

18. 3b97bea (comments and docs only): words the round-7 review's two points exactly. The code comment says the checks use only an ancestor's bindings. The docs bullet and the property's docstring say the escaped-text match holds within the 32-layer bound: a parameter with any string still escaped after it gives no membership, and upstream of calibration it refuses. compose.py and cli.py parse to the same AST as at 973c674, and the test file differs only in that docstring.
19. 91d46e1 (docs only): the round-8 review found the same "read" for "used" slip in two older sentences, the docs' membership paragraph and the bound test's docstring. A post-calibration node's bindings are computed but not used. Both now say so. The test file differs only in that docstring.
20. 6197d74 (format only): formats compose.py with the ruff that uv.lock pins (0.15.16). That version wraps the multi-line lambda in _is_rows in parentheses. The module's AST is identical before and after. An independent review of Belgium's OK for microcosm#1397 found the drift. CI runs only ruff check, which passed at 91d46e1.
21. 42bb291 (merge of main at c0513db): brings in main's 444 commits since dce4aa9. S5's own edits survive unchanged except one line: main already supplies the CompiledGraph import, so compose.py's added import becomes context. This PR differs from main on exactly six paths.

Tests run

• test_transport_composed_graph.py, single file, -n 0, with the normal conftest: 210 passed at 293bec1, and the 58 pre-calibration tests pass at 7b933e9.

• At 062d89c: test_transport_composed_graph.py plus test_transport_graph.py, -n 4, 259 passed.

• At ead5c9c: test_transport_composed_graph.py, -n 4, 245 passed. At 16c35df: 249 passed.

• At 3d8e9f9: test_transport_composed_graph.py plus test_transport_graph.py, -n 4, 268 passed. The property passes on six fixed Hypothesis seeds.

• At c4e23bd, single process (-p no:xdist) because of the host reservation: every pre-calibration and ancestry test in test_transport_composed_graph.py, the property included, 149 passed. The full required files are queued behind the reservation and the fleet disk hold.

• Mutation check, now sixteen mutants. At c4e23bd every one is killed by the example tests alone, with the property excluded. They are:

  • inventory detection off;
  • the embedded-JSON scan off;
  • rule 3 by binding variable name;
  • skipping the hold-out source check on stage nodes;
  • hashing unit entities;
  • the derived presence check off;
  • derived outputs not exempt from rule 3;
  • requiring non-empty outputs again;
  • roots without the calibrate.* nodes in the declaration pass (killed by the variant-calibration regression);
  • the artifact rule off;
  • the composed rows without artifact edges (killed by the new composed-only artifact test; at 3d8e9f9 only the property caught it);
  • the node-row checks off (15 tests);
  • repeated node ids allowed;
  • the binding checks off;
  • the selection checks off;
  • the calibration endpoint check off.

  At 3d8e9f9 the property test, run by itself, also killed six of the first eleven: rule 3 by name, the hold-out source check, two of the derived-output mutants, and both artifact mutants.

• At 20dfb3e, single process (-p no:xdist) because of the host reservation: the 173 pre-calibration and ancestry tests in test_transport_composed_graph.py pass, the property at 2,000 examples per domain included.

• Mutation check at 20dfb3e, 32 mutants, every one killed by the example tests alone (property excluded). Results are in jobs/s5-build/mutants-r3-examples/ and mutants-r3-n8/ of the Belgium hub. Beyond the sixteen above (one now split into the version-blind rule 3 of c4e23bd), they are:

  • the escaped-inventory scan skipped;
  • the selected-inventory check off;
  • extension rows off, their params unchecked, their ids unchecked;
  • rewrites ignored, in the declaration and in the composed pass;
  • compiled structural nodes reading their own version;
  • floating producers (no population) ignored;
  • the unknown-version and base-cycle fallbacks counting nothing;
  • no base walk, in the declaration and in the composed pass;
  • a compiled rewrite resolved at its own version;
  • a null endpoint ignored.

  A property-only pass over the 32 mutants is running.

• At c05685c, single process: 177 pre-calibration and ancestry tests pass. The NZ hub's probe scripts (q1_nzfit, q1_real_compose, q3_branches, q2_probes, q2e_nz), run unmodified, give the same outputs as at 20dfb3e. Against the hub's c4e23bd outputs:

  • the V1 targets holdout_references case on the real entitlement fixture now refuses naming nz.v.V1.targets (no stages: still accepts);
  • all four rewrite-only reads of an undeclared stage column now refuse, in both passes;
  • the cross-branch b.cal cases now accept.

  The 32-mutant example sweep at c05685c kills all 32 (jobs/s5-build/mutants-c056-examples/).

• At 3cb8172, single process: 178 pre-calibration and ancestry tests pass (the property now has three domains). Property-only mutant results: jobs/s5-build/mutants-c056-property-v2/ (all 32) and -v4/ (ten repeats of the version mutants).

• At ebe6f3d, single process: all 363 tests in test_transport_composed_graph.py pass (111 s), 225 of them pre-calibration and ancestry tests, the property at 2,000 examples in each of four domains included.

• At ebe6f3d, the S2-D item 6 required files, one file at a time with -n 4 (jobs/s5-build/required-ebe6f3d1a/). All pass, and the worktree was clean throughout:

  • test_country_spec 136;
  • test_nz_spec_package 1,192;
  • test_spec_engine_country_bundles 21;
  • test_spec_only_country_packages 8;
  • test_transport_population_kernels 30;
  • test_transport_dependent_children 24;
  • test_transport_graph 15;
  • test_transport_composed_graph 363;
  • test_transport_target_kernels 15.

• Mutation sweep at ebe6f3d, 54 mutants (jobs/s5-mutants-r4/summary.txt, all pre-calibration and ancestry tests): 53 killed. Beyond the 32 of 20dfb3e, the new ones are:

  • each writer kind ignored, in the declaration and in the composed pass (masks, rewrites keeping the incumbent, claims read, claims keeping, EXPAND as a writer, cells produced);
  • unknown cells writing nothing, and selected claims claiming nothing;
  • EXPAND parameter errors swallowed;
  • the structural and rows shape checks off;
  • the non-text extension id unchecked;
  • one decoding layer only, the key unquoted, no colon, no \u escapes, deep strings passing, and no layer bound.

  The property's writers domain kills W1–W12. The executor differential kills every compiled writer mutant (W2, W4, W6, W8, W10, the compiled rewrites, no compiled base walk). The survivor, I5, is fixed in 339445f and killed there (jobs/s5-mutants-r4/at-339445fbc/).

• At 339445f: all 364 tests in test_transport_composed_graph.py pass (-n 4). The only change from ebe6f3d is that test.

• At d42fbcb, under Fleet ops' Waiver 3 for DISK HOLD populace-data: registry-driven loader for published populations #4 (one file, then a short mutant sweep): all 382 tests in test_transport_composed_graph.py pass (-n 4, 78 s).

• Mutants of the new rule (jobs/s5-mutants-r5/). The run stopped at the disk floor after two of eight; the rest, and a rerun of the 52 round-4 mutants that still apply, follow the lift. (W11 and W12 mutated the cell production this commit removes.)

  • The rule off in the declaration pass: killed by 10 tests (each kind declared, stage_expand, the executed membership test, P1 in all four domains).
  • The compiled rows without structural: killed by 8 tests (each kind compiled-only, P1 in all four domains).

• The round-5 mutant sweep finished at 9959390, once DISK HOLD populace-us-2024-9f1260b-20260611: eCPS-free build provenance #6 lifted (jobs/s5-mutants-r5/summary.txt). It ran in place with -n 2 while load1 > 60, restoring compose.py byte for byte after each mutant. Of 63 mutants, the 9 structural-rule mutants (S1–S9) and the 54 round-4 mutants that still apply, 61 are killed. W9 and W13 survive as equivalent; the round-5 reviewer confirmed W9.

• At 9959390 (DISK HOLD populace-data: registry-driven loader for published populations #4 still in force, so no pytest): the new test's four cases pass, called in memory. Under a mutant that exempts the calibration nodes from the rule (ancestors - roots), its two bound cases fail.

• CI at 9959390: all 10 jobs pass (run 40181956868). Both engine-free jobs (3.13, 3.14) run test_transport_composed_graph.py, the new test included, with 21,356 passed each.

• At 1d1801e: the new and touched tests pass (30 selected, -n 2). They are the runner, two-claim, linear-scan, 32-layer and property tests, the executor differentials and the EXPAND outcome test. The reviewer's mass-log leak, run through cli._run, refuses cold and restored; a plain run_graph still delivers it.

• At 1d1801e, locally: all 399 tests in test_transport_composed_graph.py pass (-n 2, rc 0). Of the round-6 mutant sweep (jobs/s5-mutants-r6/), the reviewer's O3 and O4 (only the first claimed cell read, compiled and declared) ran before Fleet ops' 22:24Z focus mode. Both are killed, by the two-claim writer differential and the two-claim outcome test. I stopped the sweep between mutants for the focus; its other 16 mutants rerun at 973c674.

• CI at 1d1801e: all 10 jobs pass (run 40230656382). The round-6 reviewer read the engine-free 3.14 log: the full file, no keyword selector, 21,391 passed.

• At 973c674: the new and touched tests pass (24 selected, -n 2): the 32-layer, escaped-text, scan, linear, mass-log and two-claim tests.

• At 973c674, locally: all 403 tests in test_transport_composed_graph.py pass (-n 2, rc 0).

• Mutant sweep at 973c674 (jobs/s5-mutants-r7/summary.txt): in place, -n 2, restoring both files byte for byte after each run. 19 of 20 are killed:

  • The claims (the round-5 reviewer's): O3 and O4, reading only the first claimed cell, are killed by the two-claim writer differential and outcome test.
  • The scan:
    • the old per-binding scan (M1): killed by the linear test, with zero raw_decode calls, and the bound;
    • no escaped-text match (M2): the guard regression and the escaped-text property;
    • no decoding (M3), and two layers without refusing (M3b): the bound and the differential;
    • no surrogate pairs (M4), keys unscanned (M8), lists unscanned (M9): the differential;
    • past the bound returns what was found (M5), and None as empty (M6): the bound, on the stage node and calibration;
    • ASCII escaped text (M7): the property's non-ASCII example.
  • The bound's scope: global again (M10) is killed by the post-calibration case, calibration nodes exempted (M11) by the calibration case.
  • The runner's mass-log check:
    • observer off (C1): 6 of the 8 mass-log cases;
    • structural nodes unchecked (C3), or compared to themselves (C5): the calibration appender, cold and restored;
    • non-structural nodes unchecked (C4), or compared to themselves (C6): the a and b appenders, cold and restored.
  • C2 survives as equivalent: comparing the log's length only. The frame API only appends records, so the length changes whenever the records do. The round-6 reviewer reached the same conclusion independently.

• test_ci_integration_job.py: 9 passed (it reads the agent guide).

• ruff check and ruff format --check with uv.lock's ruff 0.15.16: clean at 6197d74. Earlier rounds used ruff 0.14.10, which doesn't make the lambda wrapping that commit 20 adds.

Known limitation (predates this PR)

The guard's engine-reference scan (_embeds_engine_ref) is on main unchanged: this PR only threads allowed_references through it. Since 1d1801e, S5's own composed membership scan no longer uses it (I7); the unit-binding calls below are main's. Like the old inventory scan, it decodes JSON from every quote of a parameter that is not JSON.

• On a long parameter full of escaped quotes, that is quadratic: the declaration pass took 0.34 s at 32 KB and 5.2 s at 128 KB.
• It applies whenever a benefit-unit engine reference is looked for, with or without stages.

Rewriting it would change the guard without stages, which this PR keeps identical to main (I1), so it is a separate task. That task keeps every outcome identical, checked by a differential against a frozen copy, and lands after this PR.

Plan

• Independent review on a Sol lane, round 1: REQUEST_CHANGES with four findings, fixed in 062d89c.
• Sol review, round 2, at c4e23bd: REQUEST_CHANGES with five findings, fixed in 20dfb3e.
• The NZ hub reviews the guard at c4e23bd: SIGN_OFF_WITH_ASKS; its asks are fixed in 20dfb3e and c05685c, and its probes pass at c05685c.
• Sol review, round 3, at 3cb8172: REQUEST_CHANGES with two majors (masked owners, EXPAND claims), two minors and a nit, fixed in ebe6f3d.
• Sol review, round 4, at ebe6f3d (20261025-032306-s5-review-4). It stopped on a provider filter; its draft found one major, two minors and nits, answered in d42fbcb.
• Sol review, round 5, at 9959390 (covering 339445f, d42fbcb and 9959390): REQUEST_CHANGES with one major (the mass log), two minors and a nit, fixed in 1d1801e. Its evidence is in jobs/s5-review-5/review/ of the Belgium hub.
• Sol review, round 6, at 1d1801e (job 20261025-132808-s5-review-6b): APPROVE. Round 5's major and both minors are closed. Two nonblocking findings, a coverage gap (the escaped-text match) and a doc conflict (the escape bound after calibration), are fixed in 973c674. Its evidence is in jobs/s5-review-6/review/. It executed the composed skeleton (27 nodes), extended (31) and entitlement (92) graphs through execute_transport_graph with the mass-log observer: all pass. It found no other undeclared carrier.
• The full file at 1d1801e, locally and in CI (above).
• Sol review, round 7, at 973c674 (job 20261025-163042-s5-review-7): APPROVE, no findings, and both round-6 findings closed. Its evidence is in jobs/s5-review-7/review/.
  • It probed twelve compiled graphs. A carrier that feeds calibration, a variant calibration included, refuses; one outside every calibration ancestry is accepted.
  • The both-fault graph's order change is as stated.
  • Its 40,000-case scanner fuzz, half with random references (controls, JSON syntax, non-BMP, lone surrogates), found no miss.
  • All 17 sampled mutants are killed: 16 by example tests, S8 by P1. The escaped-text mutants are killed by the explicit examples alone, with no generated draws.
  • Two wording points, neither a defect: this body said no check reads a non-ancestor's bindings, now "uses"; and the escaped-text property holds within the 32-layer bound, since a parameter deeper than that makes the scan refuse. The code comment and doc bullet get the same wording before landing.
• The NZ hub's SIGN_OFF holds at 973c674 and at its docs-only child 3b97bea, with no asks (NZ-VERDICT-973c.md, covering 9959390..973c674).
  • Its 14 probes plus q8–q10 ran at 973c674: all 20 output files are identical to its d42fbcb run (14,862 leaves, 0 differences).
  • q8: 161 guard calls on 18 NZ graphs, with 0 writer mismatches against the executor. q9: no structural stage node. q10: all 11 cases give the same outcome and message.
  • By its read, every node_bindings read in check() is ancestry-guarded, and no NZ node carries a parameter escaped past 32 layers.
  • The frame_mass_log_append writers are only in uk_runtime at main and at NZ's three heads, and no transport module imports them, so no NZ node appends. NZ takes on that its w2b, w3 and r2c nodes never change the mass log.
  • For 3b97bea: compose.py's AST is identical, the test AST is identical with docstrings masked, and the docs change is one bullet.
  • git merge-tree of either head with each of NZ's three heads is clean.
  • The SIGN_OFF extends to 91d46e1: NZ confirmed the change is exactly "read" to "used" twice, the test AST is unchanged with docstrings masked, and the merge-tree with its three heads is clean.
  • It extends to 6197d74 (NZ-VERDICT-973c.md, addendum 15:4xZ). One compose.py hunk (+4/−2) wraps the _is_rows lambda; ast.dump is identical at 91d46e1 and 6197d74. The merge-tree is clean against all six current NZ heads: Allocate Census residuals through shared reference profiles #1375 840d933, Add feed-derived NZ reference ageing factors (G8-r2a) #1382 c4ddcb4, Add exact income-band intervals and boundary diagnostics #1394 fca69c1, Normalize donor money with exact declared factors #1395 db02ae8, Add receipt contract v2 and selection-aware reference preflight #1397 84c2082 and Reconcile regional Census family-type counts under d1932 #1398 9c7c4a2.
• The full file and the 20-mutant sweep at 973c674, locally (above).
• Sol review, round 8 (delta), at 3b97bea (job 20261025-210918-s5-review-8): APPROVE. Its AST comparison found no executable change and one changed docstring. It confirmed each new sentence against the code, including all 1,554 references the property can draw (at most 14 characters). One P3, the read/used wording in two older sentences, is fixed in 91d46e1. Its evidence is in jobs/s5-review-8/review/.
• Sol review, round 9 (delta), at 91d46e1 (job 20261025-214457-s5-review-9): APPROVE, no findings. The delta is exactly the two words, and no executable AST has changed since 973c674. Round 8's P3 is closed. It swept all 342 prose units in the S5 diff (docs, changelog, comments, docstrings, messages) and found no other sentence that says post-calibration bindings are unread or that the bound refuses outside calibration's ancestry. Its evidence is in jobs/s5-review-9/review/.
• CI at 91d46e1: all 10 checks pass, MERGEABLE.
• CI at 6197d74: all 10 checks pass, MERGEABLE.
• CI at 42bb291: all 10 checks pass, MERGEABLE, merge state CLEAN.
• Sol review, round 10, at the final head 42bb291 (job 20261031-201113-s5-review-10): APPROVE, with one nonblocking P3. Its evidence is in jobs/s5-review-10/review/.
  • Merge: it compared S5's ordered edits before and after the merge, on all six paths and in both directions. The CompiledGraph import is the only difference.
  • Semantics: it searched paths, row shapes, kernels and G8's new graphs on the merged head, and found no merge defect.
  • Tests: test_transport_composed_graph.py gave 432 passed and 1 failed on a loaded host (-n 2). The failure is test_precalibration_ancestry_composed_stage_scan_is_linear at assert elapsed < 1, which saw 1.75 s. Its acceptance and decoder-count assertions passed, an isolated rerun passed, and CI passes. test_country_spec.py gave 136 passed and test_spec_only_country_packages.py 28.
  • P3: that wall-clock cutoff, and the same pattern at two other tests, predate this merge. A separate change replaces them with work counts.
• The NZ hub's SIGN_OFF holds at d42fbcb. Its 14 probes rerun with identical outputs (17 files, 14,609 leaves). Its composed-vs-executor._input_writers differential: 161 calls on 18 NZ graphs, 0 mismatches. No NZ stage node is structural. A new probe, q10, makes NZ's rates filter, JS gross-up EXPAND, V1 rates REWEIGHT and entitlement CREATE structural in turn; each refuses with this wording. A filter bound only through an embedded engine reference is refused by the composed pass alone, and post-calibration and unread nodes pass. Its note on the calibration roots is answered in 9959390, and the sign-off extends there: the source is unchanged, and the new test matches its q10 run on nz.calibrate.
• The full file at 9959390, in CI (above).
• The remaining six structural mutants, the calibration-root mutant (S9) and the round-4 mutants: 61 of 63 killed, W9 and W13 equivalent (above).
• The NZ hub's delta check of ebe6f3d: SIGN_OFF holds. All 14 of its probes rerun. Verdicts are unchanged, and the read sets only grew: writers below rewrites. Its own differential of the composed pass against executor._input_writers covers 161 calls on 18 NZ graphs, with 0 mismatches. The old and new inventory detectors give no hits across the S5, Add exact income-band intervals and boundary diagnostics #1394 and Normalize donor money with exact declared factors #1395 NZ JSON.
• The NZ hub extends its SIGN_OFF to 339445f. It confirmed the delta is test-only. Its longest build/nz JSON string is 1,361 characters, with at most 6 quotes, so NZ is far from the engine-reference scan's slow case.
• The NZ hub signs off at 3cb8172, and the sign-off covers bfd5e85 (NZ checked the doc-only commit's three claims against the code): SIGN_OFF. Its GPT-6.1 Sol lane at 20dfb3e found no blocker or major issue. All 13 planned NZ donor rewrites, V3's receipt rewrite and the r2c nodes accept. All 118 V1–V5 pre-calibration rows get exact versions. There are no inventory false positives across the 20 build/nz JSON files, and I1 holds. Its one optional nit is fixed in bfd5e85. A non-stage node that rewrites a stage's private column upstream of calibration now refuses.
• S2-D item 6 required files, single file each, at ebe6f3d: all nine pass (above). test_transport_composed_graph.py passes again at 339445f.
• Re-run against the merge order before landing: the G8 stack landed on main ahead of S5, and round 10 and CI ran on the merge of that main.
• Mark ready after 10/29 12:00Z.

🤖 Generated with Claude Code