Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/evidence/class-miner-phase0-2026-10/00-preregistration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Class miner Phase 0 — pre-registration

Written 2026-10-20, before any record of the corpus was read for this purpose. Only the ledger schema
(`evidence/ledger/SCHEMA.md`) has been looked at.

Authority: owner decision `GO-CLASS-MINER-PHASE-0`, given after the `KILL-DICTIONARY-REF-PILOT` ruling. This is
not a GO on implementing an analyzer.

## Question

Does a fix corpus of real, confirmed defects contain a **narrow, recurring** defect mechanism? To qualify, Own.NET
must miss it today, and it must justify the next investment.

## Inputs (reuse only, no new miner)

- **Corpus 1.** The 58 bug/fix pairs of the Own.NET direction research, a private repo. Its records are in
`evidence/ledger/records.jsonl` (`source_type = github_pr_fix`), and per-pair facts and Own.NET runs are in
`evidence/pairs/`.
- **Existing tools.** `scripts/` of that repo; `triage_pairs.py` and the `gh` query scripts of #412.
- **Deep check.** At most **12 new PRs** outside corpus 1. They are read through the GitHub API (diff, description,
tests). Nothing is built and no package is downloaded.
- **Budget.** 0.5 agent-day.

## Definitions

- **Narrow class.** One mechanism that can be stated as one rule: the same code shape plus the same failure, for
example "a pooled buffer is not returned on the exception path". A C01–C18 category is too broad to be a class.
- **Confirmed defect.** All three must hold:
1. a merged fix;
2. the diff establishes the mechanism (`ground_truth_level` ≥ `patch_supported`, or the same on reading a
deep-check PR);
3. there is evidence of an observable effect: an issue report, a profiler or dump result, a regression test, or a
maintainer statement. `impact_evidence = none` does not count.
- **Product code only.** Tests, samples and benchmarks do not count.
- **Independent.** Different repository owners. Two defects in one repo count as one.

## Excluded classes (already decided; not reopened)

- C01: event subscription retention (R5 killed).
- C18: SemaphoreSlim.
- The D10 design.
- #382 shape A (canonical first-party calls) and #406 shape B (closure capture): NO-GO within budget.
- #400 stale dictionary ref: KILL.
- #402 CTS inversion (blocked by #407, deferred).

The Meziantou exception-safety defect C-1b (a `default` value is inserted before the factory runs and stays when
the factory throws) **is** eligible. It already has one confirmed instance.

## Recorded per candidate class

- the independent repos and the defects counted;
- the observable effect and its evidence;
- whether a regression test exists;
- the current Own.NET result, taken from the ledger's `ownnet_result`/`ownnet_detail` (no new run unless the
ledger says `not_run`);
- the expected implementation cost (S/M/L), with the existing Own.NET capability it would reuse.

## Decision rules

- **GO-CANDIDATE.** At least **2 independent confirmed defects** of one narrow class, and Own.NET does not detect
them: `not_detected_*`, `unsupported`, or `detected_partial` where the warning does not identify the mechanism.
The output is a recommendation for the owner, not an implementation.
- A class that Own.NET already detects on all its instances is not a gap, and is reported as such.
- Two or more qualifying classes are ranked by the number of independent confirmed defects, then by evidence
strength, then by cost. Ranking does not lower the threshold.
- **STOP.** No class reaches 2 within the corpus plus the 12 deep-check PRs. The search is not widened.
- A class is defined from the fix mechanisms before its Own.NET result is looked up. Classes are not redefined
after the count to reach the threshold. Any merge or split of a class is recorded with its reason.

## Known biases

- Corpus 1 was assembled by queries aimed at resource-lifetime categories (design A), so it over-represents
them.
- One researcher classifies. Ledger levels marked `reviewer_verified(llm)` were set by an earlier agent.
137 changes: 137 additions & 0 deletions docs/evidence/class-miner-phase0-2026-10/REPORT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Class miner Phase 0 — report

**Result: `GO-CANDIDATE` for one class, N3.** N3: a request-scoped object escapes into detached work. This is a
recommendation for the owner, not an implementation decision.

The other classes that reached the threshold rank below N3 on count or evidence, or conflict with earlier
decisions (§3).

Rules: [`00-preregistration.md`](00-preregistration.md), written before any record was read.

Budget used:
- the existing corpus (the direction-research ledger) only; no new miner was written;
- 10 of the 12 allowed deep-check PRs;
- well under 0.5 agent-day.

Nothing was built, no package was downloaded, and no production, OwnIR, CI or golden file was touched.

## 1. Inputs

- **Corpus.** `records.jsonl` of the direction research has 220 `github_pr_fix` records. The 58 bug/fix pairs with
base/fix trees and Own.NET runs are among them. The ledger is private research data. Its filtered output is
committed as [`data/eligible.tsv`](data/eligible.tsv), produced by [`scripts/extract.py`](scripts/extract.py).
- **Filter for "confirmed defect".**
- the fix is merged;
- ground truth is at least `patch_supported`, so `candidate`, `rejected` and `unresolved` are out;
- the impact evidence is an issue report, a profiler or dump result, a test, or a maintainer statement;
- the code is product code.

101 records from 81 owners pass. Records in excluded classes (C01, C18) were then set aside.
- **Classes.** Narrow classes were defined from each record's root cause and fix, before its Own.NET result was
looked up.
- **Corpus-record check.** For the top two classes, the diffs of the corpus records were re-read. This matters
because most corpus levels are `reviewer_verified(llm)`, set by an earlier agent.

## 2. Candidate classes

| Class | Mechanism (one rule) | Corpus: independent owners | Deep check | Total | Own.NET today | Competitors | Effect evidence |
|---|---|---:|---|---:|---|---|---|
| **N3** | An object owned by a request/DI scope (`HttpContext`, a scoped service, a scope-provided `IServiceProvider`/`HttpClient`) reaches deliberately detached work — a discarded task `_ = …`, `Task.Run`, `QueueBackgroundWorkItem` — and is used after the scope ends | 3 (mars-vista-api, GanjoorService, ChatCore.NET) | 5 of 6 confirmed | **8** | no rule: 3/3 `not_detected_no_rule` (DI001–005 cover constructor and locator capture only) | no rule in any of the 8 analyzer packages or CodeQL; runtime scope validation does not catch it | production `ObjectDisposedException`s in every instance: 39 events in about 7 months (mars-vista); an import that always failed; staging logs; failure on Azure; a live tenant; silent loss of webhook retries |
| **N1** | A disposable is acquired; a step that can throw runs before ownership is handed over (return, field or dictionary store, registration); there is no dispose on the throw path | 3 (whizbang, rockbot, NexusKit) | 1 of 4 (BareWire) | **4** | missed 3/3. The locals are returned, stored or factory-acquired, and Own.NET tracks only non-escaping `new` locals. The opt-in `--body-throw-edges` tier would not reach them (by reading the model; not run) | CA2000 needs `dispose_analysis_kind=AllPaths` and is off by default; no other rule | one resource leaked per failed attempt (test, issue #1187, maintainer statements). No crash |
| N6 | A long-lived keyed collection (singleton, static or plugin lifetime) is only ever added to | 4 (honua, chaosflix, AI-Cockpit, BB84) | not checked | 4 | no rule | no rule | memory growth: 1 profiler, 2 tests, 1 maintainer statement |
| N7 | A `CancellationTokenSource` is disposed while concurrent code still reads its token | 3 (headless-framework, frame-flow, LiteDB) | not checked | 3 | no rule | no rule | tests |
| N1b | A release placed after a call that can throw in the same teardown sequence; the throw skips the release | 1 (PattN) | daqifi-core#504 (found while checking N1) | 2 | PattN missed (factory acquisition); daqifi not run | none | daqifi: the serial port stays claimed for the life of the process, 13 tests |
| N2 | A singleton constructor-injects a scoped service | 2 missed (affiant, SIBO) + 1 detected | — | 2 | DI001 exists. The misses are model gaps: framework registrations (`AddIdentityCore`), and a `ServiceDescriptor` with a runtime lifetime | none | both fail at startup under Development scope validation, so the framework already catches them |
| N4 | A non-atomic disposed flag under concurrent `Dispose` | 2 (couchbase, AgentInfer) | — | 2 | tool failure / no rule | none | maintainer statements; AgentInfer "not reproduced" |

Deep check, PR by PR, selected by title before reading
([`data/deepcheck-selection.tsv`](data/deepcheck-selection.tsv)). Class membership of every instance, corpus and
deep check, is in [`data/classes.tsv`](data/classes.tsv).

| PR | Class | Verdict | Why |
|---|---|---|---|
| LLabmik/DotNetCloud#43 | N3 | confirmed | scoped `_seriesService` (DbContext) used in a fire-and-forget `Task.Run`; `ObjectDisposedException` |
| DoForYou-ZA/DFY-BE#35 | N3 | confirmed | `_ = _notificationService.NotifyAdminsAsync(…)` on a request-scoped service; `ObjectDisposedException` in staging logs |
| ASADRAJA/Vendorea.PartnerConnect#49 | N3 | confirmed | request-scoped `IServiceProvider` field used inside `Task.Run`; failed reliably on Azure |
| escalated-dev/escalated-dotnet#162 | N3 | confirmed | `_ = RetryLaterAsync(…)` on a scoped dispatcher; the retry hits a disposed DbContext and is lost silently; regression test |
| microting/eform-angular-timeplanning-plugin#1669 | N3 | confirmed | fire-and-forget sends capture a request-scoped service; live `ObjectDisposedException`; tests |
| klitztuch/LanBoard#47 | N3 | **not this class** | event-triggered loads during Blazor prerender while the scope is disposing; no detached work |
| Wizard-Software/BareWire#73 | N1 | confirmed | channel created, `basic.qos` throws before registration, channel never disposed; one leak per failure |
| reqnroll/Reqnroll.IdeSupport#1119 | N1 | **not this class** | a cached proxy field nulled without dispose on failure paths (C02 shape) |
| daqifi/daqifi-core#504 | N1 | **not this class** (→ N1b) | a throwing event handler skips the release in a teardown sequence |
| modelingevolution/micro-plumberd#20 | N1 | **not counted** | shape matches (start fails after the container was created), but it is a test-support library and the resource is external (a Docker container) |

## 3. Ranking and recommendation

Ranking follows the pre-registered order: independent count, then evidence strength, then cost.

**1. N3 — `GO-CANDIDATE`.** Strongest on both count and evidence:
- 8 independent owners;
- each defect was observed failing in production or staging;
- no static tool covers it, and runtime scope validation cannot.

It is close to what Own.NET already models:
- the DI registration graph (DI001–005) supplies scoped-ness;
- the extractor already records closure captures;
- the detached-work entry points are a small, known set of APIs.

Forms by expected cost:
- **Intra-method, M.** A scoped field or `HttpContext` passed to a discarded call, or captured in a lambda handed to
`Task.Run` or a background queue. This covers DFY-BE, Vendorea, DotNetCloud, Ganjoor, microting and mars-vista.
- **One level into a same-class method, M.** escalated (`_ = RetryLaterAsync`).
- **Cross-class, L.** ChatCore needs interprocedural flow, which is close to the #382 territory left at NO-GO.

Caveats, all unresolved:
- **Prevalence is not measured.** The deep check used keyword searches aimed at the class, so it shows recurrence
across owners, not a rate.
- **Mature-code yield is unknown.** 7 of the 8 repos are young or small (0–4 stars). The direction research (D11, private) recorded the same pattern
for C10, whose mature-code yield was 0.
- **The DI graph has a known false-positive problem.** It merges test and product registrations: 7/7 holdout FPs in
D11. The C10 fallback plan (product-only graph, about 1 week) would be a prerequisite.
- **Precision is unknown.** Any design must pre-register a precision gate on an independent sample, as #400 did. The
#400 PoC found its targets and still failed that gate.
- **Some deep-check PR texts are themselves AI-generated** (Vendorea, microting). The effect statements there are the
maintainers' claims and were not reproduced.

**2. N1 — runner-up, not recommended now.**
- 4 independent owners, but the effect is a per-failure leak, with no crash.
- Own.NET would need two things it lacks: tracking of locals that are factory-acquired or later transferred, and
throw edges limited to the window before the transfer.
- The existing throw tier is documented as the "CA2000 firehose". Narrowing it to the transfer window is untested.
- Factory acquisition through third-party APIs needs type-based ownership assumptions.

**3. N6, N7, N1b, N2, N4 — not recommended.**
- **N6.** Only corpus evidence; detection is cheap, but precision risk is high: intentional caches with bounded
keys.
- **N7, N4.** Cross-thread. The direction research's final decision (§16, private) ruled these classes need new architecture and are not worth it now. The count does
not override that decision.
- **N1b.** 2 owners, one of them found incidentally.
- **N2.** The framework already fails these at startup. It remains part of the C10 fallback, not a new class.

The Meziantou C-1b defect (a default inserted before a throwing factory, from #412) found no second instance in this
corpus. It stays a single case.

## 4. Limits

- **The corpus is biased.** It was assembled by queries aimed at resource lifetime (design A), and every ledger
level was set by an earlier agent. Only the N3 and N1 records were re-read.
- **One researcher, no second labeller.**
- **Own.NET results for deep-check PRs are `NOT_RUN`.** Only the corpus records carry ledger-backed Own.NET
results.
- **Silence is not safety.** Classes that did not reach 2 are not shown to be rare; they were not found in this
corpus.

## 5. Files

- [`00-preregistration.md`](00-preregistration.md): the rules, written before any record was read.
- [`scripts/extract.py`](scripts/extract.py) and [`data/eligible.tsv`](data/eligible.tsv): the confirmed-defect
filter over the private ledger and its output (101 records).
- [`data/classes.tsv`](data/classes.tsv): every instance per class, with source and verdict.
- [`data/deepcheck-selection.tsv`](data/deepcheck-selection.tsv): the 10 deep-check PRs, chosen by title before
reading.
- [`scripts/search.sh`](scripts/search.sh), [`scripts/search2.sh`](scripts/search2.sh) and their outputs
[`data/search.out`](data/search.out), [`data/search2.out`](data/search2.out). GitHub search drifts, so a re-run
will not return the same lists.

The texts and diffs of the deep-check PRs are third-party content and are not committed. Each PR is linked by
number and can be read on GitHub.
31 changes: 31 additions & 0 deletions docs/evidence/class-miner-phase0-2026-10/data/classes.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
class source instance owner_counted verdict note
N3 corpus A-james-langridge/mars-vista-api#26 james-langridge confirmed HttpContext passed to a discarded async call; diff re-read
N3 corpus A-ganjoor/GanjoorService#508 ganjoor confirmed injected HttpClient captured by QueueBackgroundWorkItem lambda; diff re-read
N3 corpus A-CalebTek/ChatCore.NET#2 CalebTek confirmed scoped repository used from fire-and-forget Task.Run (cross-class); diff re-read
N3 deep LLabmik/DotNetCloud#43 LLabmik confirmed scoped service in fire-and-forget Task.Run
N3 deep DoForYou-ZA/DFY-BE#35 DoForYou-ZA confirmed _ = scopedService.NotifyAdminsAsync(...)
N3 deep ASADRAJA/Vendorea.PartnerConnect#49 ASADRAJA confirmed request-scoped IServiceProvider field used in Task.Run
N3 deep escalated-dev/escalated-dotnet#162 escalated-dev confirmed _ = RetryLaterAsync(...) on a scoped dispatcher
N3 deep microting/eform-angular-timeplanning-plugin#1669 microting confirmed fire-and-forget sends capture a request-scoped service
N3 deep klitztuch/LanBoard#47 - not_this_class event-triggered loads during prerender scope disposal
N1 corpus A-whizbang-lib/whizbang#1204 whizbang-lib confirmed new client, throwing verification, returned on success; diff re-read
N1 corpus A-MarimerLLC/rockbot#638-2 MarimerLLC confirmed McpClient.CreateAsync, throwing discovery, stored on success; code re-read
N1 corpus A-NexusFFXIV/NexusKit#34-2 NexusFFXIV confirmed BuildServiceProvider, throwing init, returned on success; diff re-read
N1 deep Wizard-Software/BareWire#73 Wizard-Software confirmed channel created, basic.qos throws before registration
N1 deep reqnroll/Reqnroll.IdeSupport#1119 - not_this_class cached proxy field nulled without dispose (C02 shape)
N1 deep daqifi/daqifi-core#504 - not_this_class moved to N1b
N1 deep modelingevolution/micro-plumberd#20 - not_counted test-support library, external resource (Docker container)
N1b corpus A-AmirrezaFarnamTaheri/PattN#10 AmirrezaFarnamTaheri confirmed Dispose after a throwing StopAsync in the same cleanup block
N1b deep daqifi/daqifi-core#504 daqifi confirmed throwing event handler skips the release; found while checking N1
N6 corpus A-honua-io/honua-server#5294 honua-io ledger_only add-only key-membership index
N6 corpus A-praetorianer777/chaosflix-jellyfin#38 praetorianer777 ledger_only cache and per-key lock dictionary never evicted
N6 corpus A-raymondkrahwinkel/AI-Cockpit#583-2 raymondkrahwinkel ledger_only plugin-lifetime dictionary keyed by pane only added to
N6 corpus A-BoBoBaSs84/BB84.IPTV#36-2 BoBoBaSs84 ledger_only unbounded image cache in a long-lived service
N7 corpus A-xshaheen/headless-framework#1206 xshaheen ledger_only CTS disposed while a racing StopAsync uses it
N7 corpus A-charles8051/frame-flow#727-1 charles8051 ledger_only CTS disposed while flush/clock paths read its token
N7 corpus A-litedb-org/LiteDB#2987 litedb-org ledger_only CTS shared via a cache entry disposed by the publisher
N2 corpus A-Sakwala/affiant#20 Sakwala ledger_only missed: ServiceDescriptor runtime lifetime, TryAddSingleton factory, IEnumerable dependency
N2 corpus A-jonasmz/SIBO-AUTH-SSD#9 jonasmz ledger_only missed: framework registration (AddIdentityCore)
N2 corpus A-rrr-pi-rate/StockPriceSheetPrintService#60 rrr-pi-rate detected DI001 full match; not counted as a gap
N4 corpus A-couchbase/couchbase-net-client#193 couchbase ledger_only non-atomic disposed flag (Own.NET tool failure)
N4 corpus A-wileyc427/AgentInfer#8 wileyc427 ledger_only non-atomic disposed flag
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
class pr selected_by_title
N3 LLabmik/DotNetCloud#43 fix: series enrichment ObjectDisposedException - use IServiceScopeFactory for background enrichment
N3 escalated-dev/escalated-dotnet#162 fix(webhooks): retry failed deliveries in a scope of their own
N3 klitztuch/LanBoard#47 Stop live-update DB work from racing request scope disposal
N3 ASADRAJA/Vendorea.PartnerConnect#49 Fix FTP-ingestion disposed-scope crash + price-push UI polish
N3 DoForYou-ZA/DFY-BE#35 fix: prevent disposed DbContext during user registration
N3 microting/eform-angular-timeplanning-plugin#1669 fix: own a DI scope per fire-and-forget handover push send
N1 Wizard-Software/BareWire#73 fix(transport): release RabbitMQ consumer channels on setup failure and idle stop
N1 reqnroll/Reqnroll.IdeSupport#1119 fix(#1034): dispose the brokered service proxy in RunTestOutcomeBridge failure paths
N1 daqifi/daqifi-core#504 fix(transport): a throwing StatusChanged subscriber no longer kills auto-reconnect or leaks the port handle
N1 modelingevolution/micro-plumberd#20 EventStoreServer: a failed start removes its container; labelled owner reaper; exact name lookup
Loading
Loading