[mevichitra]

Summary

Adds a long-span ("drift") mode to the finding-identity harness (research/finding_identity/). Consecutive commit pairs turned out to be too easy: in the pilot, 690 of 695 persisting findings had only moved. Real baselines are created once and compared with much later versions. python -m research.finding_identity drift <repo> --anchor-back 200 --spans 1,10,50,100,200 picks an anchor commit c0 and scores every identity scheme between c0 and c_k for each span k. It uses the same outcome taxonomy and metrics as the consecutive mode, reported per span.

Chained ground truth. The harness follows each anchor finding's line through every intermediate first-parent commit using linemap.map_lines on that step's diff, and follows renames per step (git diff -M). It does not use a direct c0→c_k diff. A finding persists when its chain is unbroken and a finding with the same rule sits at the mapped line in c_k. It counts as edited if any step edited the line. Only c0 and the c_k are scanned. Intermediate commits are read only as blobs (via git cat-file --batch) of files holding tracked findings.

Based on claude/nice-allen-ptg63r (#51, not yet merged), so this PR targets that branch.

Type of change

• Bug fix (false positive / false negative correction)
• New rule (attach rule ID from the tracker)
• Engine / runner change
• CLI change
• Docs only
• Tooling / CI (research harness only, nothing under src/)

Checklist

• git commit -s — DCO sign-off on all commits
• pytest passes locally (629 passed), plus python -m pytest research/finding_identity/tests (20 passed)
• ruff check research/ and ruff format --check research/ clean
• MYPYPATH=src mypy --strict research/finding_identity clean
• News fragment: not needed (research code only)

What changed

• drift.py (new): trace_files (file lineage and first change per anchor file), chain_lines (line chains with renames and an edited flag), per-span comparison, and the drift CLI arguments.
• summarize.py: additive. Adds one table per span, scheme × span pivots for false-new and hidden rate, a pooled "ALL" table per span, and a long-format CSV (label, span, scheme, …rates) for plotting. The existing functions are unchanged.
• matching.py: the scoring part of evaluate() is now a separate score() function, so both modes share the outcome taxonomy. evaluate() behaves exactly as before.
• gitrepo.py: adds files(commit) (ls-tree).
• cli.py: adds the drift and drift-summary subcommands. Existing code is not restructured, and schemes.py and scan.py are untouched.
• README.md: documents the mode, the chained ground truth, what gets compared, the cost, and the limits.

Test evidence

$ python -m pytest research/finding_identity/tests -v
test_drift.py::test_chained_ground_truth PASSED
test_drift.py::test_span_1_insertion_only_breaks_line_numbers PASSED
test_drift.py::test_span_2_edit_loses_the_edited_finding PASSED
test_drift.py::test_span_3_file_rename_breaks_every_path_based_key PASSED
test_drift.py::test_span_4_function_rename_breaks_scoped_keys PASSED
test_drift.py::test_span_5_new_finding_hidden_by_text_keys PASSED
test_drift.py::test_summary_metadata PASSED
test_drift.py::test_details_and_drift_summary PASSED
test_drift.py::test_spans_must_fit_the_anchor PASSED
test_drift.py::test_chain_follows_a_gradual_rewrite_a_direct_diff_loses PASSED
test_drift.py::test_chain_breaks_on_delete_and_follows_renames PASSED
test_drift.py::test_trace_files_handles_a_rename_swap PASSED
(+ 8 existing tests) 20 passed

The synthetic repository has five commits after the anchor: lines inserted above both findings, an edit of the flagged eval line, lib.py → pkg/lib.py, a rename of the function around the exec line, and a new function whose line equals the anchor eval line's original text. For each span the tests assert the chained truth (persisting/edited/introduced, renamed files) and each scheme's outcome:

• line loses findings on the insertion.
• Every content scheme loses the edited finding.
• Every scheme loses the renamed file's finding, because keys contain the path, while the truth still follows it.
• Scoped keys lose the finding in the renamed function.
• line_text and cg_no_scope hide the new finding.
• github loses exec once code is appended after it.

The unit tests also show that the chain follows a three-step gradual rewrite that a direct v0→v2 line diff leaves unmapped.

Results: anchor 200 first-parent commits back

Ran with categories security,performance,quality. Results are in the git-ignored research/finding_identity/results/drift/ and are not committed.

repo	ref	anchor c0	c_200	anchor findings in compared files (k=200)	runtime
pallets/flask	main	cd1b68c (2023-04-25)	d086db8 (2026-10-07)	40	8 s
DefectDojo/django-DefectDojo	dev	08db92cd (2026-09-03)	559a677f (2026-10-14)	259	36 s

Pooled (both repositories), by span

false-new %       k=1     k=10     k=50    k=100    k=200
---------------------------------------------------------
line           100.00    86.36    76.92    53.85    61.17
line_text        0.00     0.00     3.42     1.92     1.37
github           0.00     0.00     4.27     4.33     3.78
codeguard        0.00     0.00     3.42     1.92     1.37
cg_no_scope      0.00     0.00     3.42     1.92     5.50
cg_no_mask       0.00     0.00     3.42     1.92     1.37
cg_raw           0.00     0.00     3.42     1.92     1.37

hidden %          k=1     k=10     k=50    k=100    k=200
---------------------------------------------------------
line              n/a      n/a     0.00     0.00     0.00
line_text         n/a      n/a     0.00     6.67     2.04
github            n/a      n/a     0.00     0.00     0.00
codeguard         n/a      n/a     0.00     6.67     2.04
cg_no_scope       n/a      n/a     0.00     6.67    26.53
cg_no_mask        n/a      n/a     0.00     6.67     2.04
cg_raw            n/a      n/a     0.00     6.67     2.04

Per-span tables: pallets/flask

flask: anchor cd1b68cf6ebb, 200 first-parent commits behind HEAD

span 1 (c_1 = 73739a29f4c1): 0 files with findings, 0 anchor findings, 0 persisting (0 moved, 0 edited), 0 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line                 n/a      n/a       n/a       n/a        n/a        n/a
line_text            n/a      n/a       n/a       n/a        n/a        n/a
github               n/a      n/a       n/a       n/a        n/a        n/a
codeguard            n/a      n/a       n/a       n/a        n/a        n/a
cg_no_scope          n/a      n/a       n/a       n/a        n/a        n/a
cg_no_mask           n/a      n/a       n/a       n/a        n/a        n/a
cg_raw               n/a      n/a       n/a       n/a        n/a        n/a

span 10 (c_10 = 37badc3ce8b0): 2 files with findings, 5 anchor findings, 5 persisting (5 moved, 0 edited), 0 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               60.00    60.00       n/a       n/a       0.00       0.00
line_text           0.00     0.00       n/a       n/a       0.00      40.00
github              0.00     0.00       n/a       n/a       0.00       0.00
codeguard           0.00     0.00       n/a       n/a       0.00       0.00
cg_no_scope         0.00     0.00       n/a       n/a       0.00      40.00
cg_no_mask          0.00     0.00       n/a       n/a       0.00       0.00
cg_raw              0.00     0.00       n/a       n/a       0.00       0.00

span 50 (c_50 = faef9a0fcef3): 8 files with findings, 16 anchor findings, 14 persisting (14 moved, 0 edited), 3 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line              100.00   100.00       n/a      0.00       0.00      24.24
line_text           0.00     0.00       n/a      0.00       0.00      36.36
github              7.14     7.14       n/a      0.00       0.00      24.24
codeguard           0.00     0.00       n/a      0.00       0.00      24.24
cg_no_scope         0.00     0.00       n/a      0.00       0.00      48.48
cg_no_mask          0.00     0.00       n/a      0.00       0.00      24.24
cg_raw              0.00     0.00       n/a      0.00       0.00      24.24

span 100 (c_100 = 088b58dbceb0): 10 files with findings, 28 anchor findings, 26 persisting (26 moved, 0 edited), 4 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               61.54    61.54       n/a      0.00       0.00      13.79
line_text           0.00     0.00       n/a     25.00       0.00      48.28
github             15.38    15.38       n/a      0.00       0.00      13.79
codeguard           0.00     0.00       n/a     25.00       0.00      13.79
cg_no_scope         0.00     0.00       n/a     25.00       0.00      62.07
cg_no_mask          0.00     0.00       n/a     25.00       0.00      13.79
cg_raw              0.00     0.00       n/a     25.00       0.00      13.79

span 200 (c_200 = d086db856be1): 12 files with findings, 40 anchor findings, 38 persisting (38 moved, 0 edited), 28 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               86.84    86.84       n/a      0.00       0.00       7.55
line_text           0.00     0.00       n/a      3.57       0.00      43.40
github             13.16    13.16       n/a      0.00       0.00       7.55
codeguard           0.00     0.00       n/a      3.57       0.00      32.08
cg_no_scope        31.58    31.58       n/a     46.43       0.00      73.58
cg_no_mask          0.00     0.00       n/a      3.57       0.00       9.43
cg_raw              0.00     0.00       n/a      3.57       0.00       9.43

false-new %       k=1     k=10     k=50    k=100    k=200
---------------------------------------------------------
line              n/a    60.00   100.00    61.54    86.84
line_text         n/a     0.00     0.00     0.00     0.00
github            n/a     0.00     7.14    15.38    13.16
codeguard         n/a     0.00     0.00     0.00     0.00
cg_no_scope       n/a     0.00     0.00     0.00    31.58
cg_no_mask        n/a     0.00     0.00     0.00     0.00
cg_raw            n/a     0.00     0.00     0.00     0.00

hidden %          k=1     k=10     k=50    k=100    k=200
---------------------------------------------------------
line              n/a      n/a     0.00     0.00     0.00
line_text         n/a      n/a     0.00    25.00     3.57
github            n/a      n/a     0.00     0.00     0.00
codeguard         n/a      n/a     0.00    25.00     3.57
cg_no_scope       n/a      n/a     0.00    25.00    46.43
cg_no_mask        n/a      n/a     0.00    25.00     3.57
cg_raw            n/a      n/a     0.00    25.00     3.57

Per-span tables: DefectDojo/django-DefectDojo (--ref dev)

django-DefectDojo: anchor 08db92cd7230, 200 first-parent commits behind dev

span 1 (c_1 = 10d3ef238948): 1 files with findings, 2 anchor findings, 1 persisting (1 moved, 0 edited), 0 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line              100.00   100.00       n/a       n/a       0.00       0.00
line_text           0.00     0.00       n/a       n/a       0.00       0.00
github              0.00     0.00       n/a       n/a       0.00       0.00
codeguard           0.00     0.00       n/a       n/a       0.00       0.00
cg_no_scope         0.00     0.00       n/a       n/a       0.00       0.00
cg_no_mask          0.00     0.00       n/a       n/a       0.00       0.00
cg_raw              0.00     0.00       n/a       n/a       0.00       0.00

span 10 (c_10 = 74c0b8c13599): 5 files with findings, 18 anchor findings, 17 persisting (17 moved, 0 edited), 0 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               94.12    94.12       n/a       n/a       0.00       0.00
line_text           0.00     0.00       n/a       n/a       0.00      51.43
github              0.00     0.00       n/a       n/a       0.00       0.00
codeguard           0.00     0.00       n/a       n/a       0.00      11.43
cg_no_scope         0.00     0.00       n/a       n/a       0.00      51.43
cg_no_mask          0.00     0.00       n/a       n/a       0.00      11.43
cg_raw              0.00     0.00       n/a       n/a       0.00      11.43

span 50 (c_50 = 2f9ef9532132): 33 files with findings, 105 anchor findings, 103 persisting (99 moved, 4 edited), 7 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               73.79    72.73    100.00      0.00       0.97       0.00
line_text           3.88     0.00    100.00      0.00       0.00      13.95
github              3.88     0.00    100.00      0.00       0.00       0.00
codeguard           3.88     0.00    100.00      0.00       0.00       3.72
cg_no_scope         3.88     0.00    100.00      0.00       0.00      15.81
cg_no_mask          3.88     0.00    100.00      0.00       0.00       3.72
cg_raw              3.88     0.00    100.00      0.00       0.00       3.72

span 100 (c_100 = 67850879713b): 48 files with findings, 185 anchor findings, 182 persisting (178 moved, 4 edited), 11 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               52.75    51.69    100.00      0.00       0.55       0.00
line_text           2.20     0.00    100.00      0.00       0.00      12.17
github              2.75     0.56    100.00      0.00       0.00       0.00
codeguard           2.20     0.00    100.00      0.00       0.00       8.47
cg_no_scope         2.20     0.00    100.00      0.00       0.00      15.87
cg_no_mask          2.20     0.00    100.00      0.00       0.00       6.35
cg_raw              2.20     0.00    100.00      0.00       0.00       6.35

span 200 (c_200 = 559a677f183c): 69 files with findings, 259 anchor findings, 253 persisting (249 moved, 4 edited), 21 introduced
scheme       false-new %  (moved)  (edited)  hidden %  mispair %  collide %
---------------------------------------------------------------------------
line               57.31    56.63    100.00      0.00       3.16       0.00
line_text           1.58     0.00    100.00      0.00       0.00      10.13
github              2.37     0.80    100.00      0.00       0.00       0.00
codeguard           1.58     0.00    100.00      0.00       0.00       9.01
cg_no_scope         1.58     0.00    100.00      0.00       0.00      19.51
cg_no_mask          1.58     0.00    100.00      0.00       0.00       4.50
cg_raw              1.58     0.00    100.00      0.00       0.00       4.50

false-new %       k=1     k=10     k=50    k=100    k=200
---------------------------------------------------------
line           100.00    94.12    73.79    52.75    57.31
line_text        0.00     0.00     3.88     2.20     1.58
github           0.00     0.00     3.88     2.75     2.37
codeguard        0.00     0.00     3.88     2.20     1.58
cg_no_scope      0.00     0.00     3.88     2.20     1.58
cg_no_mask       0.00     0.00     3.88     2.20     1.58
cg_raw           0.00     0.00     3.88     2.20     1.58

hidden %          k=1     k=10     k=50    k=100    k=200
---------------------------------------------------------
line              n/a      n/a     0.00     0.00     0.00
line_text         n/a      n/a     0.00     0.00     0.00
github            n/a      n/a     0.00     0.00     0.00
codeguard         n/a      n/a     0.00     0.00     0.00
cg_no_scope       n/a      n/a     0.00     0.00     0.00
cg_no_mask        n/a      n/a     0.00     0.00     0.00
cg_raw            n/a      n/a     0.00     0.00     0.00

Notes for reviewer

• The numbers are small. Only files changed within the span are compared, so flask has 40 anchor findings at k=200 and DefectDojo has 259. The k=1 spans hold 0 and 2 findings, which is too few to say anything. One anchor per run means one commit pair per span, so variance across anchors is not measured yet.
• Edited findings are rare, even over 200 commits: 4 of 291 persisting findings. Every content scheme loses all of them (false-new 100% within "edited"). On moved-only findings, codeguard, line_text and cg_no_mask/cg_raw lose none. github loses 2.4% at k=200, because its hash reads ~100 characters after the line, so changes below a finding break it. line loses 50–100%.
• cg_no_scope on flask at k=200 (31.6% false-new, 46.4% hidden) comes from key collisions, not from a ground-truth error. Two test files hold 12 CG-PERF-004 findings each whose statements normalize to the same key (73.6% collision rate). The scheme pairs equal keys in line order, so when new copies are added, the truly persisting findings go unmatched and the new ones are matched to old ones.
• The one false_known behind the 25% / 3.6% hidden rate on flask is src/flask/blueprints.py:19 (Blueprint.__init__, CG-QUAL-005). The first-parent step that merged the sansio split (1d8b53f) moved the old class to sansio/blueprints.py and wrote a new subclass at the old path. In one step git sees a modification of blueprints.py, not a rename, so the truth calls the new __init__ introduced, while every key built from the line text (all schemes except line and github) matches it to the old one. Both readings are defensible. This is the documented "code moved to another file" limit of a line-diff ground truth.
• No file renames occurred in the compared files of either run (renamed_files = 0). The rename path is covered by the synthetic test.