Repository navigation
fix(report): the multi-language reachability figures are per-language — each language's filter record carried through the merge, the scan-wide figures summed across languages (#707) - #870
Conversation
… — each language's filter record carried through the merge, the scan-wide figures summed across languages (#707) PR #251's disclosed-but-never-filed residual: on a two-language scan, pipeline_stats.reachable_units was language A's kept count while total_units spanned A+B — the derived reduction percentage described A's slice while labelled scan-wide, and reachability_filter_applied read true from A's record even when B ran unfiltered. The merge spread only the first language's metadata and built per_language without the reachability record. The fix (the issue's own direction): the merge carries each language's reachability_filter record into metadata.per_language[lang] (present- only — an unfiltered language keeps no key, never a fabricated record); the reporter's reader builds the scan-wide aggregate when any record is present — reachable_units and original_units are the sums (an UNFILTERED language contributing its whole unit count to BOTH sides, its units being all reachable, so the numerator's language scope always matches the denominator's), the reduction percentage re-derived scan-wide. The numeric telemetry keys are PER-KEY present-only (a key emitted only when some record carried it — never a fabricated zero, the #602 contract); the advisories are JOINED and ATTRIBUTED per language (never first-wins — dropping language B's advisory re-opened the #323/#520 vector); a MIXED-LEVEL scan claims no single effective_processing_level (a warning instead, the scanner's own refilter-aggregate precedent); a MALFORMED record (no integer reachable_units) is named in a warning and treated as unfiltered (its units both sides, never denominator-only fabricated pruning); a top-level record already carrying its own per_language block (the scanner's newer scan-wide aggregate) outranks the parse-time records. Single-language datasets fall back to the top-level record. The falsifier holds BOTH disjuncts: the scan-wide figures' denominator matches the numerator's language scope, and the artifact now carries the additive reachability_per_language decomposition (each language's own kept count) beside the sums.
…y the numerator
The malformed-record gate admitted a record on ONE count:
if not isinstance(rec.get("reachable_units"), int) or \
isinstance(rec.get("reachable_units"), bool):
while the summing loop below adds each side INDEPENDENTLY — `kept` to
reachable_total, `orig` to original_total, each behind its own isinstance
check. So a record carrying a VALID integer reachable_units beside a
missing or non-integer original_units passed the gate, landed in
`records[lang]`, and then contributed to the NUMERATOR and not to the
DENOMINATOR. The language's own `units` count is never consulted on that
path, because a dict record with a valid kept count is routed to
`records[lang]` and the `units` fallback belongs to the UNFILTERED branch.
Two things could therefore ship that must not: more reachable units than
total units, and a NEGATIVE reduction percentage. That is the exact
invariant #707 exists to establish — the PR's own stated falsifier is "the
scan-wide figures' denominator matches the numerator's language scope" —
failing on a narrower input than the falsifier was written against.
EXECUTED through the real merge and the real _load_reachability_metadata
(fixture: python {units 20, rf 10/20} + go {units 500, kept 50, NO
original_units}):
head ccd9328 : reachable=60 original=20 reduction=-200.0 <-- inverted
this commit : reachable=510 original=520 reduction=1.9 <-- sane
canonical 2-lang fixture: 50.0% on BOTH sides, unchanged
and identically for original_units given as the string "500", as a bool,
as a float, as None and as a list.
THE REPAIR: the gate validates every field the reduction READS. The fields
are named ONCE, in `_REDUCTION_COUNT_FIELDS`, and checked by
`_unusable_reduction_counts` — the defect was that the gate knew one member
of a set the arithmetic reads two members of, so a list the gate iterates
cannot fall out of step the way two hand-written isinstance pairs did. A
rejected record falls to the UNFILTERED branch, which is this PR's own
documented policy for a malformed record ("its units both sides, never
denominator-only fabricated pruning"); the mirrored direction, numerator-
only, fabricates NEGATIVE pruning and is now named as such.
The malformed-record sentence moves into `_malformed_record_warning` so the
three emitting sites cannot drift, and the reason clause names the field
that actually failed. With reason == "no integer reachable_units" it
reproduces the pre-repair text BYTE-FOR-BYTE: the numerator case's wording
is unchanged, and a regression test pins that byte-equality.
WHY NO AUTHORED TEST CAUGHT IT, and the hole this closes: the suite's
record helper is `_rf(kept: int, original: int, ...)` — BOTH counts are
required positional arguments, so no fixture in the file could express the
defective record. 19/19 passed at the head and 19/19 on the control. The
suite gains `_partial_rf(**fields)`, the deliberate counterpart that CAN
omit a count, and the PR's falsifier is now asserted as a predicate
(reachable <= original AND reduction >= 0) over six malformed shapes
rather than stated as a sentence. This is the second consecutive defect in
this series hidden by a helper that cannot express the bad shape; the
inexpressiveness is what the new helper removes.
Reachability, stated against this change rather than for it: every one of
the ELEVEN production record producers writes original_units beside
reachable_units — the canonical reachability_filter_record
(entry_point_detector.py:648), parsers/swift/test_pipeline.py:248 and
core/parser_adapter.py:593,659, with rust/zig/go/c/ruby/php/javascript all
routing through the canonical one. So this gate can only ever reject a
record NO production path emits (a legacy, hand-assembled, third-party or
truncated dataset.json), and the canonical corpus is untouched. The defect
was latent, not an active regression; the gate is correspondingly in the
over-seed direction.
Rule-40 receipt (instrument checked FIRST, hash-bound
eb5c07f4f713f5a9 -> 555733f047c049a7), three dimensions — the #707 suite,
the inversion predicate, the canonical fixture: 6 mutants, 6 KILLED,
0 survivors.
I0 a deliberate no-op comment -> NOT scored a kill (instrument valid)
M1 _REDUCTION_COUNT_FIELDS narrowed to ("reachable_units",) -- THE
DEFECT ITSELF -> KILLED (3 failed,
the predicate speaking the exact divergence: assert 60 <= 20)
M2 the bool tooth removed -> KILLED (2 failed)
M3 narrowed to ("original_units",) -> KILLED (3 failed)
M4 the gate disabled (predicate returns []) -> KILLED (6 failed)
M5 the warning wording drifted -> KILLED (1 failed)
M6 the reason clause hardcoded to the numerator field -> KILLED (1 failed)
Control restored BYTE-EXACT to 555733f047c049a7 after every mutant. M5 and
M6 kill on the suite dimension only and pass the inversion predicate, which
is correct: they touch wording, not arithmetic.
Review-arc commits: this commit (the hunks below had NO independent
reviewer — the reviewer became the author under the ADDENDUM 42 grant).
authored-by: arc(ses_e8704fffdffehwcool1yeMiODl)
Agent: PR-MERGE ses_e8704fffdffehwcool1yeMiODl
99ba15d to
f8b403d
Compare
R3/R4 — PR 870, at head
|
| commit | what |
|---|---|
84636e33 |
the authored #707 work, rebased onto a959dd3b (unchanged content) |
f8b403d9 |
the denominator-gate repair + 4 tests (+166/−15) |
THE FINDING. The malformed-record gate admitted a record on one count (reachable_units), while the summing loop adds each side independently. So a record with a valid integer reachable_units beside a missing or non-integer original_units passed the gate, landed in records[lang], and contributed to the numerator and not the denominator. Two things could ship that must not: more reachable units than total units, and a negative reduction percentage. That is the exact invariant #707 exists to establish — its own stated falsifier is "the scan-wide figures' denominator matches the numerator's language scope".
EXECUTED through the real merge and the real _load_reachability_metadata (fixture python {units 20, rf 10/20} + go {units 500, kept 50, NO original_units}):
| tree | reachable | original | reduction |
|---|---|---|---|
head ccd93281 |
60 | 20 | −200.0 ← inverted |
this head f8b403d9 |
510 | 520 | 1.9 ← sane |
| canonical 2-language fixture | 15 | 30 | 50.0 (unchanged) |
Identical inversion for original_units as the string "500", as a bool, as a float, as None and as a list.
THE REPAIR. The gate validates every field the reduction reads; the fields are named once in _REDUCTION_COUNT_FIELDS and checked by _unusable_reduction_counts, so a list the gate iterates cannot fall out of step with the arithmetic the way two hand-written isinstance pairs did. A rejected record falls to the unfiltered branch — this PR's own documented policy for a malformed record. The malformed sentence moves into _malformed_record_warning so the three emitting sites cannot drift, and with reason no integer reachable_units it reproduces the pre-repair text byte-for-byte (a regression test pins that byte-equality).
WHY NO AUTHORED TEST CAUGHT IT. The suite's helper is _rf(kept: int, original: int) — both counts are required positional arguments, so no fixture could express the defective record. 19/19 passed at the head and 19/19 on the control. The suite gains _partial_rf(**fields), the deliberate counterpart that can omit a count, and the falsifier is now asserted as a predicate over six malformed shapes. Three of the four new tests are RED on ccd93281 (assert 60 <= 20) and GREEN here; the fourth is the wording control that must pass on both sides.
REACHABILITY, stated against the change rather than for it. All eleven production record producers write original_units beside reachable_units — the canonical reachability_filter_record (entry_point_detector.py:648), parsers/swift/test_pipeline.py:248, core/parser_adapter.py:593,659, with rust/zig/go/c/ruby/php/javascript routing through the canonical one. So the gate can only reject a record no production path emits; the canonical corpus is untouched and the defect was latent, not an active regression.
RULE-40 (instrument checked FIRST, hash-bound eb5c07f4f713f5a9 → 555733f047c049a7), three dimensions — the #707 suite, the inversion predicate, the canonical fixture: 6 mutants, 6 KILLED, 0 survivors. I0 a deliberate no-op was not scored a kill. M1 the gate narrowed back to numerator-only — the defect itself — KILLED, the predicate speaking the exact divergence. M2 the bool tooth removed, M3 denominator-only, M4 the gate disabled, M5 wording drift, M6 the reason hardcoded — all KILLED. Control restored byte-exact after each. One mutant (M5, first attempt) produced a file hash equal to the control, i.e. the edit never applied; it was re-run rather than scored as a survivor.
SUITE: 13,259 passed / 41 skipped / 2 xfailed on the repaired tree. The 9 failures (tests/patch/test_run_cve_batch.py ×5, tests/patch/test_run_metadata.py::TestCollectGitInfo ×2, tests/test_llm_sdk_contract_floor.py ×2) reproduce identically on the unrepaired head — the control was run before the attribution, not after — and are not attributable to this change. ruff check . clean. CI 19/19 at this head, run_attempt > 1 census = 0, so no rerun family was invoked and none is claimed.
THE REBASE, DISCLOSED. The repair's first push (99ba15dc) was a fast-forward, but it then read mergeable: dirty: my helper block and 869's landed _finding_cwe block insert at the same anchor, so the two independent additions conflicted. The pre-repair head ccd93281 merged master cleanly — the conflict was mine, not the author's. Resolved by 14b's named rebase onto a959dd3b, keeping both blocks; the author's commit rebased without conflict and its content is unchanged. behind_by is now 0, so the rebase served as F3 and no separate update-branch was issued. Every verification leg was re-run on the rebased bytes.
THE PUSH. [PUSH-GRANT] form=prose scope=870 branch=fix/707-per-language-reachability parent=ccd93281, riding ADDENDUM 42's committed "the 14a(iii) grant made explicit". The author lane was probed immediately before each act and read ACTIVE (mid-turn) — but its live work is OpenAnt-709 (issue #709), it holds no claim on this branch, and its last push here was 9.73 h earlier; the head was unmoved across three endpoints. Lease pinned, readback byte-equal, master never touched. Letter-deviation disclosed: the literal [TAKEOVER-GRANT] PR=… head=… token is still absent — the grant names clause 14a(iii) and declares it explicit, which §1273 admits as "the maintainer's explicit waiver of the literal form", but it is a clause-naming, not the token. Recorded rather than papered over.
What
Closes #707 — PR #251's "Scope honesty" residual, disclosed but never filed: on a two-language scan,
pipeline_stats.reachable_unitswas language A's kept count whiletotal_unitsspanned A+B — the derived reduction percentage described A's slice while labelled scan-wide, andreachability_filter_appliedread true from A's record even when B ran unfiltered. The merge spread only the first language's metadata and builtper_languagewithout the reachability record.The fix
The merge carries each language's
reachability_filterrecord intometadata.per_language[lang](present-only — an unfiltered language keeps no key, never a fabricated record).The reader builds the scan-wide aggregate when any record exists:
reachable_unitsandoriginal_unitsare the sums — an unfiltered language contributes its whole unit count to both sides (its units are all reachable), so the numerator's language scope always matches the denominator's;reachability_per_languagemap — the falsifier's first disjunct;"python: …; javascript: …") — never first-wins, which silently dropped language B's advisory and re-opened the Detector blackout advisory has no deterministic surface: absent from the Go terminal summary and exit envelope, and rendered in SUMMARY_REPORT.md only at the LLM's discretion (PRs #233/#238's deferred follow-up) #323/Keep-all net keyed on ANY real seed — consider keying on STRUCTURAL seeds only (the #506 follow-up; changes filtering for every incidental-only corpus) #520 misdiagnosis vector;effective_processing_level(a warning naming the levels — the scanner's own refilter-aggregate precedent);reachable_units) is named in a warning and the language treated as unfiltered — its units both sides, never denominator-only fabricated pruning;per_language(the scanner's newer post-LLM aggregate) outranks the parse-time records and its record-dict values are projected to kept ints — the artifact's{lang: kept-int}contract holds on both paths.Single-language datasets fall back to the top-level record (the pre-#707 shape, still written) — unchanged.
Disclosed limits
The #602 denominator marker (
reachability_baseline_languages) is not emitted on the parse-time merge path — no producer writes the per-record key (only the scanner's aggregate derives it), so the merge path can emitunits_newly_reachablewithout the marker: absence, not fabrication; the mirror-the-scanner remediation is recorded. The first-wins regression is caught by the reader-level sibling pin (the artifact-level advisory pin's fixture carries B's advisory only — its docstring cites the sibling). Thepruned_by_filecap matches the scanner's top-20.Evidence (runs/707)
6f4b1f60..ccd93281: 19 nodes / 4 witnesses, all assertion-red at base (the reviewer re-ran the suite at base: the witnesses red by assertion, the legacy pins green).--llm-reachabilityruns, the advisory double-append — every finding executed, remediated, and mutation-probed; the terminal round plus the post-audit pin round both TERMINAL DRY.go_parserbuild race (the same first-run environment class as the npm race — both pass on the warmed re-run); the collect-diff: +20 items, 0 removed; the Go module:EXIT 0.96 passed in 0.21s(this suite's 19 + the merge + report suites).