WS3: one lockfile model per ecosystem

[Mikola Lysenko (mikolalysenko)]

Implements the landed part of WS3 in docs/design/v5-plan.md: one pure model per lock format under crates/socket-patch-core/src/formats/<fmt>/, plus one formats::registry() table of which files carry wiring. The branch includes release/v5-prerelease (#280, ledger-free hosted) as of b0a6aa8. Per the owner's decision, the remaining WS3 items are follow-ups, listed below.

What lands

• pnpm (formats::pnpm): PnpmLock covers every lock generation (shrinkwrap, 5.x block, 5.4/6.0/9.0 flow, Rush). It backs:

  • entries(), resolves(), wired_refs() and wired_integrity();
  • vendored_in_use(), computed at parse time and CRLF-tolerant;
  • plan_hosted().

  The redirect rewriter, lock inventory, lockfile discovery, repair, both vendored planners and get's pnpm-PnP probe all read through it.

• registry (formats::registry()): one table, which the following now derive from:

  • REDIRECT_CANDIDATE_FILES (pinned by value);
  • repair's WIRING_FILES;
  • discovery's vendored-liveness probe;
  • hosted_memory's root markers and file ecosystems;
  • npm_family::FILES.

• cargo lock (formats::cargo): CargoLock::parse is the one parse. It uses toml_edit's spanned Document and backs:

  • entries(), packages(), dependents(), vendored_in_use(), is_locked() and locked_versions();
  • plan_hosted(), which splices at the recorded spans.

  The separate text grammar is deleted. An oracle test compares the old planner, kept test-only, with the new one. On randomized v1 and v3/v4 locks both produce identical bytes and FileEdits. v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280's upstream Cargo.lock restore now splices at the same spans.

• composer (formats::composer): ComposerLock / ComposerLockPackage. The hosted scanner moves to formats::composer::hosted verbatim.

• gem (formats::gem): GemfileLock plus the hosted lock-source convergence. Fixes the inherited CHECKSUMS writer bug: an uppercase, multi-digest or bare row used to get a second, conflicting row. It is now replaced in place, with round-trip tests for LF and CRLF.

• yarn/bun (formats::yarn, formats::bun):

  • one yarn grammar sniff and one is_berry_lock, which also closes a BOM gap;
  • one BunTextLock prelude;
  • repair's yarn/bun trust anchor now reads the entry models.

• maven/nuget readers moved to formats::maven and formats::nuget. v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280's upstream restore imports them from there.

• The placeholder LockModel::restore_upstream() has been removed; upstream restoration is v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280's.

Validation (b0a6aa8)

• cargo clippy --workspace --all-targets --all-features -- -D warnings: clean.
• cargo test --workspace --all-features --no-fail-fast -j4: 10,218 passed. The 20 failures are all write-failure or chmod tests that cannot fail when run as root in the sandbox. The same set fails on the base branch.
• e2e_redirect_cargo_build now passes (fixed by v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280).
• Base-inherited CI reds:
  • vlt install-proof (vlt_pinned_matrix_agent_get_and_remove), fixed on v5: remove setup (WS7) + patch UI streamlining (WS8) #279;
  • Windows covgap_commands_scan_mod.

Follow-ups (not in this PR)

• cargo manifest half: CargoRegistryPins vs discovery's dependency_entries, and four [package] readers.
• pnpm: the hosted span grammar and the vendored planners' lines:: grammar are still two grammars.
• npm-family: vendored-in-use text probes, and hosted yarn classic/berry chunk splices (move verbatim after adding a berry oracle).
• go: move the shared pure readers into formats::golang.
• maven / nuget: these pairs of duplicate readers give different answers today; each needs an oracle and a decision:
  • NuGet nuget_package_source_keys vs parse_config_source_keys
  • find_maven_dependency_matches vs parse_pom
  • mvn checksum merges vs discovery's reader
  • rewrite_nuget's lock walk vs nuget_lock_entries
• F09 (docs: v5 waste review + repacking-to-depscan design #286 waste review): the Python lock family is missing from the WS3 order.
• F16 (docs: v5 waste review + repacking-to-depscan design #286 waste review): WS1's patch::redirect::upstream/* builds a second per-format grammar outside formats/. Its restore readers should move onto the format models; cargo is done here.

🤖 Generated with Claude Code

https://claude.ai/code/session_018BYsX2VfVNKHvfFAJnFryc

---

Note

Medium Risk
Large refactor of lock parsing, hosted redirects, and install narrowing; behavior is intended to be equivalent but mistakes in resolution or file lists could mis-route patches or skip grants.

Overview
Introduces socket-patch-core/src/formats/ as the single place for lock/wiring knowledge: per-ecosystem models (notably PnpmLock, CargoLock, ComposerLock, extended GemfileLock, plus yarn/bun sniff helpers) and a shared formats::registry() table of which project files are hosted redirect candidates, vendor wiring targets, root markers, and npm flavor probes.

CLI and in-memory hosted stop maintaining parallel file lists and ad-hoc sniffs: REDIRECT_CANDIDATE_FILES and repair’s wiring scan come from registry::HOSTED / VENDORED; pnpm trust-policy helpers and yarn/pnpm grammar detection are re-exported or delegated to the format modules. get’s pnpm-PnP hosted narrowing drops the local pnpm_lock_resolves text probe in favor of PnpmLock::parse + resolves() (tests move to core).

Hosted redirect wires pnpm through formats::pnpm::plan_hosted instead of an in-redirect rewrite_pnpm_lock; cargo/composer/gem hosted logic is relocated or rebuilt on spanned/byte-accurate models (cargo gets a unified CargoLock::plan_hosted; composer hosted scanner lives under formats::composer::hosted). The old npm_family::FILES table in constants is removed in favor of registry role flags, with a pinned-value test guarding the hosted candidate list.

^{Reviewed by Cursor Bugbot for commit b0a6aa8. Configure here.}

[Mikola Lysenko (mikolalysenko)]

CI status on 1c849f3 — one failure was this PR's, now fixed; the rest are inherited from release/v5-prerelease (red on its head 8ae7dc3 too):

• test (macos-latest) compile error — this PR's, fixed in 9c078fa. The clippy cleanup dropped PatchedFixture::{before_hash, after_hash}, but the #[cfg(target_os = "macos")] blob-retention test reads them. The fields are back with allow(dead_code) off macOS.
• test (ubuntu/windows), coverage, test-release: e2e_redirect_cargo_build (cargo_hosted_fresh_checkout_fetch…, cargo_get_uuid_hosted_fresh_checkout_fetch: offline vex exits 0, expected 1). The identical failure is in base run 36352437716, and I reproduced it locally on 8ae7dc3. Windows base also fails covgap_commands_scan_mod. No fix exists yet; this belongs to WS1 (ledger-free hosted, where the offline/no-ledger record path lives), not the lock-model refactor.
• vlt compatibility install-proof (every vlt version × OS): e2e_vlt::vlt_pinned_matrix_agent_get_and_remove (Absent vs Patched). Every install-proof job fails the same way in base run 36352437746. No fix exists yet.

I have no job re-run available beyond the push itself, so 9c078fa is the re-run.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

v5 review at 9c078faa — the shared format registry is useful, but this remains an intermediate consolidation. I did not confirm a new functional regression in the moved code.

Please make “one model / parse once” the implementation boundary, not just the directory layout:

• PnpmLock::vendored_in_use() walks the text again after parsing; Cargo still has separate TOML and hosted text grammars. Inventory, wiring, forward rewrites and upstream restoration should use one parsed representation with byte spans, preserving CRLF and existing format support.
• formats::LockModel currently exposes FORMAT plus a default unsupported restore_upstream(), while #280 adds the actual inverse parsers elsewhere. Wire one real format end to end before keeping a public placeholder abstraction beside a second parser family.
• There is a reproducible inherited Gem checksum bug worth closing here. The reader accepts uppercase digests/additional digest tokens, but the hosted writer at patch/redirect/mod.rs:5063–5070 only matches a lowercase terminal sha256. Its insertion fallback then adds a second conflicting checksum. A probe of the unchanged writer produced one row for lowercase, two for uppercase, and two for sha256 + sha512, with no warnings. This is already present in #277, not introduced by this PR. Use the shared Gem checksum parser in the writer and add those round-trip fixtures.

That last case is the practical payoff of consolidation: a format should not be accepted by discovery but misread by its writer. Keep all supported package-manager versions; remove duplicate grammars rather than compatibility coverage.

Validation: diff/source review and an isolated Rust checksum probe; no full workspace test run.

[Mikola Lysenko (mikolalysenko)]

Thanks for the review. Pushed as of 9301a39:

• Gem checksum bug: fixed (cf050e4). The hosted writer now finds the dep's CHECKSUMS row with formats::gem's split_checksum_entry, scoped to the CHECKSUMS section with exact name (version). It replaces that row in place, keeping its line ending, and records the old row verbatim for revert. gem_checksum_rewrite_replaces_every_spelling_the_reader_accepts covers:

  • lowercase, uppercase, sha256 sha512 and sha256,sha512 entries, and a bare entry;
  • each of those under both LF and CRLF.

  Every case now yields exactly one row, and the shared reader reads it back as the patched pin. A re-run is a no-op. I did not add a tests/fixtures/redirect golden case, because those fixtures are shared with depscan's TS rewriter. Say if you want one added on both sides.

• Placeholder LockModel / restore_upstream(): removed (9301a39). Upstream restoration stays with v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280, which will land it on the format models.

• pnpm vendored_in_use: now answered from the parse (9301a39). PnpmLock::parse computes the vendored-uuid set in one pass over packages/snapshots keys, with \r dropped per line. The v9 backend memoizes that same set, and LockIndex's copy and the LF-only vendored_in_use_lines are deleted.

  This changes behavior for CRLF locks: they used to read as "undeterminable", and now they answer like their LF twin. The unwired-revert guard still refuses while the artifact is consumed, and both CRLF guard tests were updated to assert Some(true).

Not done yet: one span-carrying representation for cargo (and likewise pnpm's hosted span grammar vs. the vendored planners' lines:: grammar). My plan:

1. CargoLock parses once with toml_edit::Document<&str>, keeping each [[package]] table's byte span and its source/checksum value spans, plus v1 [metadata] and [root].
2. plan_cargo_lock locates blocks and twins from those spans instead of the [[package]]\nname = … header search.
3. The randomized cargo_lock_equivalence_tests generator (v1 and v3/v4, CRLF) gets an oracle that keeps today's planner and asserts identical bytes and FileEdits before the old grammar is deleted.

The known behavior change: non-canonical locks (comments or reordered keys inside a block), which the text grammar reports as NotFound today, would start being found. I'll take that as intended unless you object.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Restack request from the v5 coordinator (please act now).

#280 (ledger-free hosted) is now merged into release/v5-prerelease as 686e5fb4. Please merge origin/release/v5-prerelease into this branch, resolve the conflicts, get CI to "base-inherited reds only", and mark the PR ready for review (not draft) when done.

• Resolve toward v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280's model. There is no hosted ledger any more: rollback/remove/vendor restore upstream via patch::redirect::upstream. Don't reintroduce redirect-state.json writes or fragment-replay paths. Keep this PR's own intent on top.
• Landing order: WS5: one VendoredBackend for vendored apply/revert/repair; cut repair's ledger rebuild #283 → WS3: one lockfile model per ecosystem #281 → v5: remove setup (WS7) + patch UI streamlining (WS8) #279 → v5 WS4/WS6: one hosted engine for disk + memory; unified Ledgers view #282, then the design PRs (docs: v5 waste review + repacking-to-depscan design #286 docs, Add the NuGet vendoring design (docs only) #285/Vendored Maven and Gradle: design and prototype #287). After each one lands, the next ones merge the base again. Expect a small second round of conflicts.
• Base-inherited reds that aren't yours to fix: vlt install-proof (vlt_pinned_matrix_agent_get_and_remove; fixed on v5: remove setup (WS7) + patch UI streamlining (WS8) #279) and Windows covgap_commands_scan_mod.
• The repo only allows squash merges. Post a comment with your final head SHA and CI summary when you're ready.

[Mikola Lysenko (mikolalysenko)]

Owner decision (via coordinator): land the families done so far. The cargo span model and the remaining 'parse once' work become follow-up PRs, since the review items are design refinements, not regressions. Priority now: merge the base (#280), resolve conflicts, get green, and mark ready.

[Mikola Lysenko (mikolalysenko)]

Ready for landing. Head: b0a6aa8, which merges release/v5-prerelease at 686e5fb (#280). The PR is marked ready for review.

Restack. The three conflicts were resolved toward #280's model, with no ledger or fragment-replay paths reintroduced:

• the yarn fragment-kind helper is dropped along with the ledger;
• the rollback fixture takes base's shape;
• the scan test imports follow base.

#280's new patch::redirect::upstream/* now reads through formats:::

• its Cargo.lock restore splices at CargoLock's spans instead of the deleted block walker;
• its gem, maven, nuget, composer and pnpm readers are imported from formats::.

CI on b0a6aa8

• Green:

  • test (ubuntu-latest), test (macos-latest), test-release, coverage and clippy;
  • every docker e2e/coverage job;
  • the pnpm, npm, yarn, cargo, go, uv and bun native matrices;
  • the vlt native and e2e_safety_vlt legs.

  e2e_redirect_cargo_build now passes.

• Base-inherited reds only:

  • vlt install-proof (every version and OS): vlt_pinned_matrix_agent_get_and_remove, fixed on v5: remove setup (WS7) + patch UI streamlining (WS8) #279;
  • test (windows-latest): only covgap_commands_scan_mod.

• One re-run is pending: Bun native (macos-latest, 1.4.0). Its first attempt failed every hosted cell on that runner; the same job passed on Ubuntu and Windows for this head, and on WS5: one VendoredBackend for vendored apply/revert/repair; cut repair's ledger rebuild #283 on the same base. One cell logged a DNS error (nodename nor servname provided). I queued a single re-run, which is still waiting for a macOS runner. If it fails again I'll treat it as real.

Locally

• cargo clippy --workspace --all-targets --all-features -- -D warnings: clean.
• The full workspace test run: 10,218 passed. The 20 failures are the root-only write-failure/chmod set, the same as on base.

Follow-ups are listed in the PR body, including F09 and F16 from the #286 review.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#283 landed on release/v5-prerelease as 06437d2; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Merged release/v5-prerelease at 06437d2 (#283). The new head is 66a6b8c, and the PR stays ready.

• The one conflict: WS5: one VendoredBackend for vendored apply/revert/repair; cut repair's ledger rebuild #283 deletes commands/repair_vendor.rs, which this branch had edited. I took the deletion. Its reader edits went with it, since WS5: one VendoredBackend for vendored apply/revert/repair; cut repair's ledger rebuild #283 removed those flavor sniffs.
• Wiring list: the vendored wiring-file list WS5: one VendoredBackend for vendored apply/revert/repair; cut repair's ledger rebuild #283 moved into vendored_backend::repair now comes from formats::registry (VENDORED), as it did before.
• Local checks:
  • clippy with -D warnings is clean;
  • the full workspace test run passed 10,170 tests, with the same 20 root-only write-failure/chmod failures as the previous head (no new failures).
• CI: running on 66a6b8c. I'll report if anything besides the known base-inherited reds (vlt install-proof, Windows covgap_commands_scan_mod) turns up.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Bun native (macos-latest, …) red: runner DNS, not this PR. One macOS Bun job per run fails, on a different Bun version each time:

Commit	Failing job
b0a6aa8 (this PR)	macOS 1.4.0
66a6b8c (this PR)	macOS 1.2.0
#283's head (now on base as 06437d2)	macOS 1.0.0

Every failing run looks the same:

• every hosted cell fails, with urlopen error [Errno 8] nodename nor servname provided; the harness cannot resolve its registry host on that runner;
• every vendored cell on that runner passes;
• the same Bun versions pass on Ubuntu, on Windows and on the other macOS runners.

It is an intermittent macOS runner/harness problem in scripts/backtest-bun.py's hosted leg and already reproduces on the base. This PR does not touch that code. I'll re-run the failed job once when the workflow finishes; I can't re-run it while the rest of the workflow is still going.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

CI finished on 66a6b8c, with base 06437d2. The only reds are the ones inherited from the base, so this PR is ready to land.

• Green:
  • test (ubuntu/macos), test-release, coverage, clippy;
  • every docker e2e and coverage job, plus hosted-e2e;
  • the pnpm, npm, yarn, cargo, go, uv and deno matrices;
  • Bun on every OS and version. The one macOS Bun job that hit the runner DNS flake passed when re-run.
• Inherited from the base:
  • test (windows-latest): the only failing target is covgap_commands_scan_mod.
  • vlt install-proof: vlt_pinned_matrix_agent_get_and_remove fails. It is fixed on v5: remove setup (WS7) + patch UI streamlining (WS8) #279.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

I merged release/v5-prerelease at 0f2de18 (#285, #288, #289, #290); the new head is 2606b70. The merge was clean. In crates/ it changes only the two test files #288 fixed, and none of this PR's code.

The vlt install-proof red is still inherited from the base, but it has changed. The vlt_pinned_matrix_agent_get_and_remove failure (the one #279 fixed) is gone. #288 now lets the e2e tier run, and it shows a new failure on old vlt releases (0.0.0-*, 1.0.0-rc.1…rc.8):

• Five hosted legs fail: vlt_pinned_matrix_hosted_{crlf_lock,idempotence,resave_crlf_rollback,resave_install_rollback,rollback_byte_exact}.
• The error is patched_ref_unattributable, from vex/discover/vlt.rs: those old releases write a lock with no lockfileVersion, which vlt ≥ rc.15 would re-resolve.

This isn't caused by this PR: #279's run 36452380239 on the same base fails the same five tests with the same error on the same versions, and #279 doesn't touch lock models. Neither vex/discover/vlt.rs nor e2e_redirect_vlt_build.rs is changed by this PR. No fix exists yet. The fix is either in how those legs are gated by version, or in the vlt discovery rule. I'll re-run the failed jobs once the workflow finishes. Local clippy and the full test gate are still running on 2606b70, and I'll post the full CI summary when CI settles.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

v5 coordinator: #281 is next to land, but these block it.

1. The branch is missing the current base. release/v5-prerelease is now at 28cebf7c ("Attribute vlt pins and uv overrides in ledger-free hosted rollback", pushed after v5 design: staged patch rollout (socket.yml + scan limit) #290), and 2606b70 doesn't contain it. Please merge origin/release/v5-prerelease again. That commit likely addresses the vlt patched_ref_unattributable failures and the e2e_redirect_uv_build failures now showing on this head.
2. CI on 2606b70 hasn't finished (run 36452391521 is still in progress). Once CI on the new head settles, post a summary showing every remaining red is also red on the base.

When both are done this PR lands on the next coordinator run.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Merged release/v5-prerelease at 28cebf7 ("Attribute vlt pins and uv overrides in ledger-free hosted rollback"). The new head is 8b10d2d; the merge was clean and touches only upstream/mod.rs, hosted_inventory.rs and CLI_CONTRACT.md.

28cebf7 is the base's fix for both reds on 2606b70:

• vlt install-proof: the hosted and migration legs failed on unscoped rollback with patched_ref_unattributable / hosted_wiring_contested. Correction to my earlier comment: this was not only old vlt releases. rc.30 and rc.32 failed too, and v5: remove setup (WS7) + patch UI streamlining (WS8) #279 fails the same way on the same base.
• e2e (ubuntu-latest, e2e_redirect_uv_build, 0.9.30): hosted_uv_transitive_override_manifestless_vex failed with hosted_wiring_contested on pyproject.toml during revert.

Checks on 8b10d2d:

• clippy (-D warnings): clean.
• hosted_inventory: 8/8 pass.
• The core redirect lib tests pass, except for the known root-only vlt_heal unremovable-lock test.
• The real-uv test hosted_uv_transitive_override_manifestless_vex now passes locally (uv 0.8.17).
• On 2606b70, the full workspace run passed 10,170 tests. It failed the same root-only write-failure/chmod set as before, plus stage_local_artifact_caps_oversized_artifact_before_buffering. That test is an RSS-measuring test in vendor/registry_fetch.rs, which this PR doesn't touch; it passes when run alone, so the failure came from -j4 memory load.

I'll post a CI summary once 8b10d2d settles.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

The yarn-classic 1.0.2 / 1.6.0 / 1.7.0 / 1.9.4 reds are inherited from the base and have no fix yet.

The failing suite is mode_migration_npm, in classic_hosted_then_vendored_takeover_round_trips_to_registry and classic_vendored_then_hosted_takeover_leaves_pure_hosted. The test's mount_registry_from_classic_lock panics with no \integrity`, because yarn older than 1.10 writes no integrityline intoyarn.lock`. I reproduced it locally with yarn 1.9.4; 1.22.22 passes.

• v5: remove setup (WS7) + patch UI streamlining (WS8) #279's run 36452380314 on the same base fails the same four versions and passes 1.10.1 and 1.22.22.
• This PR doesn't touch mode_migration_npm.rs; the test came with v5 WS1+WS2: ledger-free hosted mode, upstream-restore rollback, vendor ejects hosted projects #280.

Why I'm not proposing a patch yet: I tried making integrity optional in the mock-registry helper locally. The two tests then get further and fail later (mode_migration_npm.rs:827 and :1076). So pre-1.10 yarn needs a real decision in #280's migration path, either supporting it or gating those tests by version. That goes beyond this PR, and I reverted the local change.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

CI settled on 8b10d2d (base 28cebf7, the current release/v5-prerelease head).

The only reds are ones the base also fails. Each is listed below with the base run that shows the same failure.

Green

• CI (run 36457510256): 242 of 246 jobs pass. That includes test on ubuntu, macOS and Windows, clippy, coverage, test-release and every docker job. The whole e2e tier is green, including every e2e_redirect_uv_build, e2e_redirect_vlt_build, mode_migration_vlt and e2e_vlt leg that was red on 2606b70.
• Compatibility workflows: pnpm, npm, Bun, Go, Poetry, Pipenv and PDM all pass.

Red, also red on the base

1. CI yarn-classic 1.0.2 / 1.6.0 / 1.7.0 / 1.9.4. The failing suite is mode_migration_npm, and the cause is yarn older than 1.10 writing no integrity line. I explained it in the comment above. v5: remove setup (WS7) + patch UI streamlining (WS8) #279's run 36452380314 fails the same four versions, and 1.10.1 and 1.22.22 pass on both. No fix exists yet.
2. vlt install-proof on 0.0.0-1, 0.0.0-11 (ubuntu + windows) and 0.0.0-1 on Node 22.0.0.
   • Three hosted legs fail: hosted_crlf_lock, hosted_idempotence and hosted_rollback_byte_exact. After rollback, vlt-lock.json keeps the mock registry's tarball URL instead of the original registry.npmjs.org one.
   • The base's own dispatch run on 28cebf7 (36454568301) fails exactly these four jobs with the same three assertions.
   • Every other vlt version is green, including rc.30 and rc.32, which 28cebf7 fixed. No fix exists yet.

Locally on this head

• clippy (-D warnings) is clean.
• Full tests on 2606b70 passed 10,170, with only the known root-only failures (details in my earlier comment).

---

Generated by Claude Code