[mevichitra]

Summary

Turns the finding-identity harness into the material for a 6-page IEEE paper. It re-runs the 35-repository corpus with every identity scheme, adds the shipped tools' analyzers and SonarQube's branch-analysis behaviour, runs the controlled experiment (RQ2) on the corpus, and generates the paper's table, figure and numbers into research/finding_identity/paper/. Every generated file there reproduces byte for byte from the commands in paper/outline.md.

Type of change

• Bug fix (false positive / false negative correction)
• New rule (attach rule ID from the tracker)
• Engine / runner change
• CLI change
• Docs only
• Tooling / CI (research code under research/, not part of the released package)

Checklist

• git commit -s — DCO sign-off on all commits
• pytest passes locally
• ruff check research/ and ruff format --check research/ clean
• MYPYPATH=src mypy --strict research/finding_identity clean
• News fragment: not needed for research/

What changed

Harness

• batch --analyzer bandit|semgrep replays the corpus with that analyzer's findings. Bandit skips the JavaScript repositories. A result directory counts as done only for the analyzer named in its analyzer.json.
• transform <local repository> --ref <commit> exports that commit's Python files, so RQ2 runs on the same pinned commits as RQ1. Corpus clones have no working tree.
• sonar_branch: SonarQube's tracker as a branch analysis runs it. A renamed file keeps its findings only if SonarQube's file move detection accepts the pair: line counts within (0.84×, 1.18×) and line-hash similarity ≥ 85. This ports FileMoveDetectionStep and SourceSimilarityImpl from 26.10.0.133078. sonar stays the pull-request mode, which takes renames from the SCM. Replay and drift pass renamed down to the matchers.

Paper kit

• paper_kit.py writes Table II (table_rq1.tex), numbers.md, rq1.csv, the pooled RQ2 matrix (rq2.csv) and Fig. 1 (fig_rq2.pdf). It uses cluster-bootstrap intervals over repositories, splits the false-new rate by renamed files, and can leave rules out (Bandit's B101).
• gt_sample.py draws a seeded, stratified sample of 100 ground-truth decisions for a manual check (paper/ground_truth_check/).
• paper/outline.md (page budget, section contents, reproduction commands) and paper/claims.md (every citable number with its interval and source).

Results

CodeGuard findings in 35 repositories (23,237 persisting / 1,974 introduced); Bandit findings in 25 Python repositories. Rates in %, with cluster-bootstrap 95% intervals.

scheme	false new	same path	renamed	hidden	Bandit findings without B101: hidden
line	52.6 (48.5–57.6)	52.0	100	1.2	1.9
github	3.1 (2.0–4.9)	1.9	100	0.8	3.3
codeguard	1.8 (0.7–3.4)	0.6	100	1.9	2.5
sonar	0.4 (0.3–0.6)	0.4	2.8	1.5	4.5
sonar_branch	0.6 (0.3–1.0)	0.4	20.2	1.4	4.5
bandit	4.4 (3.1–6.3)	3.2	100	20.0	29.2 (20.1–40.9)

• Every key scheme includes the path. Renamed files hold 1.2% of persisting findings but 69% of codeguard's false-new reports.
• Bandit's set-membership baseline hides 29% of new Bandit findings without B101, 36% outside test code, and 51% with B101.
• RQ2 (Fig. 1): each single-key scheme has a blind spot. github loses 77.5% on edits below the line; codeguard loses 100% on function renames and extraction; sonar survives most edits through its fallback passes.
• The line, line_text, github and codeguard-family results are identical to the first corpus run (2026-10-14).

Test evidence

$ python -m pytest research/finding_identity/tests -q
129 passed, 1 skipped          # the skip is a live Semgrep test
$ python -m pytest -q
652 passed
$ ruff check research/ && ruff format --check research/
All checks passed!
$ MYPYPATH=src python -m mypy --strict research/finding_identity
Success: no issues found in 38 source files

New tests cover the analyzer resume check and an end-to-end Bandit batch, transform --ref on a bare repository, and SonarQube's similarity score. That last one includes a seeded comparison with the textbook edit distance and the exact line-count bounds (ceil(100 × 1.18) = 119). They also cover rename gating, the rename split, rule exclusion, and RQ2 pooling.

Notes for reviewer

• sonar_branch models SonarQube's decision only for file pairs that git already detects as renames (git diff -M, 50%). A move SonarQube would find but git would not is not modelled. This is documented in the README.
• CodeGuard's SQL-injection and taint rules put source line numbers in their messages ("built on line N"). That breaks message-based identity (Bandit, SonarQube's message passes) whenever lines shift; it shows up in RQ2's insert_top row for bandit.
• Clones and per-commit results stay git-ignored; paper/ holds only aggregates and the 100-finding sample.

---

Generated by Claude Code