Skip to content

fix(report): the finding display IDs are identity-derived — the #416-deferred half of #314's suggestion 3, now a rename, not a redesign (#706) - #869

Open
gadievron wants to merge 1 commit into
masterfrom
fix/706-identity-derived-display-ids
Open

gadievron wants to merge 1 commit into
masterfrom
fix/706-identity-derived-display-ids

Conversation

@gadievron

Copy link
Copy Markdown
Collaborator

What

Closes #706 — the half of #314's third suggestion that PR #416 deferred as "a schema-design change": with the identity derivation already sitting one line above the positional assignment, the deferred work is a rename, not a redesign.

A positional VULN-NNN is not stable across runs: any change to the finding set renumbers everything after the insertion point, and consumers that carry an ID across runs — a triage spreadsheet, a suppression list, a cross-reference in a disclosure body — silently retarget. #416 closed the mis-join hazard with the identity check; the instability of the user-facing identifier was untouched.

The fix

The display ID is VULN- + the identity key's leading hex, derived two-pass and order-independent: each identity takes the shortest width (6/8/10/12 chars) whose prefix is unique against every other identity in the set — so two findings sharing a 6-hex prefix both extend to 8, the short form names nobody, and no id ever names a different finding (the reviewer caught the first draft's order-dependent ladder silently retargeting through the fix's own mechanism). A genuine duplicate identity falls to a positional #n ladder, deterministic within the run.

Disclosed residual — read this as the cross-run contract it is: an ID may lengthen when a finding whose identity shares its 6-hex prefix joins the set (the short form then names nobody; the survivor extends to 8). Within the run the assignment is order-independent; across runs the stability is conditional on no prefix-colliding identity joining — a visible diff in the ID, never a silent retarget (no ID ever names a different finding; the audit's counterexample is exactly this case, disclosed here rather than claimed away).

The falsifier holds for the issue's own shape: a finding's operator-visible ID is identical across two runs of the same commit with a (non-colliding) finding inserted before it — pinned through the real build_pipeline_output, twice, including the colliding-insertion variant that pins the lengthening.

The identity extraction is one shared helper — the emission loop's own read verbatim (the non-dict-element guard, the full CWE fallback chain) — so the pre-pass and the loop read the same triple. One disclosed input shape remains inconsistent (the CWE-from-results case below): the shared helper is the same code, but the pre-pass's call site does not supply the results-row fallback, so for that shape the identity and the emitted cwe_id disagree — the follow-up is recorded, not claimed away. (The reviewer caught the first pre-pass draft crashing on vulnerabilities=["str"] — an existing regression test now guards it.)

The consumers — form-agnostic by construction, pinned

  • fix(report): the dynamic-test merge verifies finding identity — a positional-ID mis-join is refused and surfaced #416's dynamic-test join reads the id from the pipeline output and verifies the identity_key: the stale-file refusal still refuses through the new form; the agreeing merge still attaches.
  • The patch CLI looks the finding up by its id string (an old-form id does not resolve — ValueError, no silent fallback).
  • The #N linkifier links the report's positional anchors, not the ids.
  • The Go report side never sees the id (the Finding struct has no ID field); the HTML badge stays the positional #N.

Disclosed limits

The HTML report shows only the positional #N badge — the stable id's operator surfaces are pipeline_output.json, the patch artifact filenames, and the dynamic-test markdown. The cwe-from-results shape (a confirmed row whose CWE lives only on the results row) derives its identity from cwe 0 while the emitted cwe_id says otherwise — low production reachability (the verifier writes both rows from the same dict), a disagreeing dynamic-results file is refused by the merge (surfaced, never silent), and the one-line follow-up is recorded. No numeric-suffix parser was found in the searched repository scope at the pushed head (the reviewer's rg sweep over VULN-\d, VULN-\(, suffix slices, and id-keyed sorts across libs/ and apps/, non-test, non-vendor).

Evidence (runs/706)

  • RED gate at dbb79937..66e16800: 10 nodes / 4 witnesses, all assertion-red at base — including the real-loop falsifier (two builds, an insertion before, the survivors identical; the reviewer re-derived its behavioral red with the symbol pin stripped).
  • C4 at the pushed head: 4 behavioral hunks in the producer (the helper, the loop, the seen-set threading, the extraction) + 9 prose claims (the docstring + the docs' examples); the test scope includes the neighboring reporter regression suite.
  • fable adversarial review: 3 rounds — the order-dependent retarget (executed), the drifted pre-pass crash (caught by the reviewer's full-suite run, an existing regression test failing at the intermediate head), the position-leak mutant (killed by the loop-level prefix-collision pin), the three stale old-form spots — terminal round TERMINAL DRY, three disclosed notes in reconciliation.
  • T8 standing-promises table (5 rows incl. the probed MISS — the cwe-from-results shape), C2 anchor queries (consumed==total), A4 census, reconciliation (4 rows: the chain, the notes, the fix-scoped-test lesson).
  • Parity — the legs' own terminal lines (the full logs in the bundle):
    # base (dbb79937)
    3 failed, 13233 passed, 38 skipped, 2 xfailed, 1 warning in 1676.35s
    # head (66e16800)
    3 failed, 13242 passed, 40 skipped, 2 xfailed, 1 warning in 1645.89s
    
    the standing trio (test_interrupt_then_resume + the two llm_sdk_contract_floor pins) identity-matched in both legs; the 2 xfails unchanged (the earlier cycles' strict-xfail pins, now in master); the +9 passing = this suite's 10 tests + the 1 parametrized no-hardcoded item, minus 2 Go-binary tests that flipped pass→skip in the fresh head worktree (the go_parser build provisioning raced the collection — the same first-run environment class as the npm race; both tests pass on re-run in the warmed worktree); the collect-diff: +11 items, 0 removed; the Go module: EXIT 0.
  • The battery — the runs' own terminal lines:
    $ pytest tests/test_issue706_identity_display_ids.py tests/test_reporter_nondict_vuln.py -q
    12 passed in 3.30s
    $ pytest tests/test_issue706_identity_display_ids.py tests/test_reporter_nondict_vuln.py tests/test_issue314_dynamic_merge_provenance.py tests/report/ tests/patch/ -q
    7916 passed, 4 skipped, 1 warning in 151.86s
    

…deferred half of #314's suggestion 3, now a rename, not a redesign (#706)

A positional VULN-NNN is not stable across runs: any change to the
finding set renumbers everything after the insertion point, and
consumers that carry an ID across runs (a triage spreadsheet, a
suppression list, a cross-reference in a disclosure body) silently
retarget. #416 closed the mis-join hazard with the identity check but
deferred the display half as 'a schema-design change' — with the
identity key derived one line above the positional assignment, the
deferred work was a rename.

The display ID is now VULN- + the identity key's leading hex, derived
TWO-PASS and ORDER-INDEPENDENT: each identity takes the shortest width
(6/8/10/12 chars) whose prefix is unique against every OTHER identity
in the set, so two findings sharing a 6-hex prefix BOTH extend to 8 —
the short form names nobody, and no id ever names a different finding
(the fable-r1 F1 catch: a seen-set ladder made a prefix collision
resolve by list order, silently retargeting through the fix's own
mechanism). A genuine duplicate identity falls to a positional #n
ladder, deterministic within the run. DISCLOSED RESIDUAL: an ID may
LENGTHEN when a colliding finding joins the set — a visible diff,
never a silent retarget. The falsifier holds: a finding's
operator-visible ID is identical across two runs of the same commit
with a finding inserted before it.

The consumers are form-agnostic by construction and are pinned as
such: #416's dynamic-test join reads the id from the pipeline output
and verifies the identity_key (the stale-file refusal still refuses
through the new form); the patch CLI looks the finding up by its id
string; the #N linkifier links the report's positional anchors, not
the ids; the Go report side never sees the id (the Finding struct has
no ID field). The migration surface: the producer loop, the comments,
and the docs' examples (PIPELINE_MANUAL, report/README,
dynamic_tester/README, the root README's patch example, the
autopatcher's run_traced usage).
sys.path.insert(0, str(_CORE_ROOT))

from core.reporter import finding_identity_key # noqa: E402
import core.reporter as _prod_reporter # noqa: E402
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Finding display IDs are still positional — the identity key exists beside them, so the schema change #416 deferred is now a rename, not a redesign

1 participant