v5: cap new patches per scan, most critical first

[Mikola Lysenko (mikolalysenko)]

Work item B of the staged-rollout design (docs/design/staged-rollout.md, §5, §7.2 B rows, §9.2). scan can now add a limited number of new patches per run, most critical first, and report the rest as deferred. Repeated scans roll a repo forward until everything is patched.

What changes

• scan --max-new-patches <N|none> (env SOCKET_MAX_NEW_PATCHES, socket.yml patches.maxNewPatches). A run adds at most N patches to packages that had none in the project. 0 means upgrades only; none means no cap. Precedence is flag > env > socket.yml > unlimited.

• Classification (§5.1). Every selected row is compared with the merged recorded view: agent manifest, then hosted lockfile pins, then vendor ledger.

  • ALREADY: the writer gets the recorded uuid, so a re-scan never swaps an applied patch for an equal sibling.
  • UPGRADE: the selection supersedes the recorded patch, judged by the new ranking::search_result_supersedes on by-package records. Never capped.
  • NEW: nothing is recorded. Capped.

• Eligibility before budget (§5.2). A NEW row only takes a slot if it can land. Each mode uses its own write-free checks:

  • hosted: grants and purl/url, vlt preflight, takeover refusals, wheel metadata, and the rewrite's confirmation probe
  • vendored: the Bun/vlt preflight
  • agent: the vendored / not-installed partition

  A patch that can't land never blocks the queue. If a lookup fails under a cap, no NEW patch is admitted that run (rollout_incomplete_lookup).

• Order. Eligible NEW packages are ranked in-flight first, then by severity, then advisory count (descending), then ecosystem, base purl and uuid. The order is total and uses no dates. The budget unit is the base purl, so qualifier twins and the same package in several roots cost one slot.

• Budget scope. On disk, PATH directories share one budget, visited in sorted order. In memory, one budget covers every root.

• Output.

  • Every successful scan --json result gets a top-level rollout block: maxNewPatches, counts, and a ranked deferred[].
  • Hosted mode mirrors deferred rows into redirect.skipped[] as rollout_deferred.
  • Human output gets a Rollout: line and next steps (Next up: …).
  • Exit codes don't change.

• updates[] now comes from the by-package records, so it lists exactly the UPGRADE rows the run acts on. It falls back to the batch data only where no by-package offer exists.

• In-memory engine (napi, hosted-bundle).

  • New options: maxNewPatches (a number or "none"), maxNewPatchesCap (a server ceiling that only tightens) and inFlightPatches.
  • The engine classifies each root against its committed .socket/manifest.json, its vendor ledger, and the hosted pins its own lockfiles name. It plans once across all roots and rewrites a root again without its deferred rows.
  • The result gains a session-level rollout block and ProjectResult.deferred[]. index.d.ts is updated.

• Core: socket_patch_core::rollout holds plan_rollout, rollout_cmp, resolve_max_new and canonical_base_purl, with the §9.0 types.

Staged-rollout walkthrough

A repo with 9 patchable packages and --max-new-patches 3. This is the scenario tests/scan_rollout_e2e.rs runs in hosted, agent and vendored mode:

Run	Adds (most severe first)	rollout.counts	Deferred (ranked)
--dry-run	nothing written; same decisions as run 1	{new: 3, deferred: 6, upgrade: 0, already: 0}	c, d, h, a, i, f
1	e (critical ×2), b (critical), g (high ×3)	{new: 3, deferred: 6, upgrade: 0, already: 0}	c, d, h, a, i, f
2	c (high), d (medium), h (medium)	{new: 3, deferred: 3, upgrade: 0, already: 3}	a, i, f
3	a (low), i (low), f (unknown)	{new: 3, deferred: 0, upgrade: 0, already: 6}	—
4	nothing; lockfile byte-identical	{new: 0, deferred: 0, upgrade: 0, already: 9}	—

Human output for run 1 (stdout; warnings go to stderr):

Redirected 3 packages; rewrote 2 files.
Rollout: 3 of 9 new patches applied (maxNewPatches=3 from --max-new-patches); 0 upgrades, 0 already applied.
Commit .npmrc and package-lock.json to keep the redirect.
Reinstall from the updated lockfile (e.g. `npm ci`) so the installed packages pick up the patched artifacts, then run `socket-patch vex` to verify them.
6 new patches deferred; commit these changes and run scan again to apply the next 3.
Next up: roll-c@1.0.0 (high), roll-d@1.0.0 (medium), roll-h@1.0.0 (medium), …

After that, a newer superseding patch for an applied package still lands with --max-new-patches 0 (hosted and agent), and it shows up in updates[].

With socket.yml instead of the flag:

version: 2
patches:
  minSeverity: high
  maxNewPatches: 2   # a PR bot keeps the same 2 until merged, then proposes the next 2

Decisions where the plan left a gap

These are also recorded in docs/design/staged-rollout.md §11:

• A JSON report-only run fetches no by-package data. Its updates[] stays on the batch rungs, and its rollout block has zero counts.
• Discovery only recognizes pins on patch.socket.dev or a --patch-server-url origin. A hosted pin on any other server would read as NEW on every run and stall the rollout. To prevent that, a NEW row counts as ALREADY when a lockfile names its selected uuid. This uses one linear uuid scan per file.
• The in-memory engine has no disk discovery, so it reads pins as the offered uuids each root's own files mention. A pin to a patch the API no longer offers counts as NEW once.
• The in-memory maxNewPatches option reports source: "flag".
• A capped reference-lookup failure that only affects NEW rows defers them with rollout_reference_failed instead of failing the run.
• The Rollout: line only prints when a cap is set. A wet hosted run that can only defer takes no apply lock and writes nothing.

Self-review

Five parallel reviewers (correctness, determinism/churn, UX/output, test quality, perf) went over the diff. Fixed from their findings:

• the recorded view now folds case like discovery and keeps every hosted pin, so a NuGet pin or two pinned pypi twins no longer reads as NEW or as a phantom upgrade
• O(1) lookups replace per-row scans
• the lock is taken for rows an earlier directory admitted
• error envelopes drop rollout
• the in-memory re-rewrite keeps its first-pass skips, and a fully failed root freezes new admissions
• dry-run, incomplete-lookup, zero-cap and shared-budget wording is fixed
• the flag no longer reads the env value
• more tests

Left as documented limits (§11):

• memory refuses a root whose symlinked file only a deferred row touches
• a single-pass optimistic rewrite (perf) is not done yet
• no vlt-withheld e2e case

Integration with work item A

• Work item A landed as v5: socket.yml patch rollout config and filtering #293, which loads the socket.yml policy in scan and the in-memory engine. This branch is merged up to it.
• The file layer is live. SelectionPolicy::max_new_patches() is the socket.yml layer of the cap on disk (both run_scan and the PATH-list budget), and in memory it is resolved after the session's socket.yml loads. --no-socket-yml drops it, and a flag or env cap still wins.
• Pipeline order (§9.0). A's ScanPolicy::select (root, package and severity filters, retained set) runs inside discover_selected. The rollout classifies what it keeps: a retained package is neither NEW nor UPGRADE, and a severity floor removes a NEW row before it takes a slot. The memory engine's select_with_policy now returns Offers and feeds the same classification.
• A memory session that fails on its socket.yml (policyError) has no rollout block (index.d.ts: rollout?).
• Combined e2e (§9.3) in tests/hosted_memory_rollout.rs, on disk and in memory:
  • fixture: includePaths: ["/apps/"], minSeverity: high and maxNewPatches: 2 over apps/one, apps/two and legacy
  • both engines converge in two runs and never touch legacy/
  • then --no-socket-yml --max-new-patches 1: disk spends the slot in directory order, and memory spends it run-wide on legacy's critical patch
• A has landed (v5: socket.yml patch rollout config and filtering #293), so this PR's diff against release/v5-prerelease is work item B only.

Tests

• Core unit tests: rollout_cmp sorts every permutation the same way; plan_rollout with caps 0 / 1 / more than available, ties across ecosystems, carry across directories, ineligible rows, incomplete lookups, in-flight rows and twins; the resolve_max_new precedence table (including a cap on none); search_result_supersedes rungs.
• CLI unit tests: classification rows, the recorded-view matching rules, the rendered block and lines, the uuid scanner.
• E2E (tests/scan_rollout_e2e.rs):
  • a failed reference lookup defers new rows under a cap and still fails uncapped or with applied rows
  • a failed lookup for an already-applied package does not freeze new patches
  • three runs converge in hosted, agent and vendored mode, and a fourth changes nothing
  • dry run equals wet run
  • an upgrade lands with a cap of 0
  • withdrawn and bad_purl top-ranked patches hold no slot
  • a failed detail lookup admits nothing new
  • PATH directories share one budget
  • the env/flag precedence and a malformed env value (exit 2)
  • convergence without --patch-server-url
• Parity and policy (tests/hosted_memory_rollout.rs):
  • disk and memory produce identical rollout and redirect blocks and identical bytes run after run until converged
  • two roots: memory's run-wide ranks vs disk's per-directory budget
  • memory counts a committed manifest, vendor ledger or pin as recorded
  • in-flight patches go first; the server cap tightens the limit
  • socket.yml policy plus cap converge on disk and in memory (§9.3)
• Parser contract rows in cli_parse_scan.rs, and a napi smoke test for maxNewPatches.
• cargo clippy --workspace --all-features --all-targets -- -D warnings is clean.

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes when and which lockfiles/manifests scan mutates across hosted, agent, and vendored modes; misconfigured socket.yml or rollout caps could defer critical patches until later runs, though policy only narrows and invalid config fails closed.

Overview
scan gains staged rollout and repo-root socket.yml patch policy, so teams can narrow what gets patched and drip new fixes in by severity instead of landing everything in one run.

Gradual rollout (--max-new-patches, env, patches.maxNewPatches). After policy-aware selection, each package is ALREADY, UPGRADE, or NEW against the merged recorded state (manifest → hosted pins → vendor ledger). Only NEW rows consume a shared per-invocation budget, ranked by severity and advisory count; upgrades and re-confirms are uncapped. Hosted mode defers after eligibility checks (including a confirmation rewrite probe), mirrors deferrals in redirect.skipped[] and JSON rollout; agent/vendored and the in-memory engine (hosted-bundle, napi) follow the same planner. Failed lookups under a cap defer all new patches (rollout_incomplete_lookup). get ignores the cap and warns when policy would have filtered packages.

socket.yml policy (integrated with rollout). Repo-root patches / projectIgnorePaths narrow paths, ecosystems, packages, severity floor, enabled, and maxNewPatches; invalid or ambiguous files fail scan before I/O (socket_yml_invalid / socket_yml_ambiguous, --no-socket-yml bypass). Discovered projects skip default test/fixture trees; explicit PATHs and negation patterns can re-include. Successful scan --json adds policy and rollout blocks; human output adds policy and rollout lines.

Patch retention / updates[]. Re-scans keep the recorded patch unless the offer meaningfully supersedes it (ranking::search_result_supersedes on by-package records); updates[] tracks the upgrades the run actually applies.

Other: ignore + serde-saphyr for YAML/glob matching; CI test-release timeout 40m → 50m; docs (README, CLI_CONTRACT, CHANGELOG).

^{Reviewed by Cursor Bugbot for commit 03aed58. Configure here.}

[Mikola Lysenko (mikolalysenko)]

[agent] CI status (updated for 1613348): complete and green except the 8 jobs that also fail on the base branch. See the latest [agent] ready to land comment for the per-check base-run links.

• Checks: 415 passed, 6 skipped, 8 failed. coverage and Pipenv compatibility passed.
• The 8 failures:
  • the 4 vlt install-proof jobs, also red in base run 36454568301 at 28cebf7
  • yarn-classic 1.0.2, 1.6.0, 1.7.0 and 1.9.4, also red in base run 36554947505 at b9e106d
• Head: 1613348 merges b9e106d (v5: socket.yml patch rollout config and filtering #293). Its tree is identical to 23c89e8, and the PR's diff is work item B only.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#281 landed on release/v5-prerelease as 73c0c4f; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#282 landed on release/v5-prerelease as 14a9cb0; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] Merging #282 (14a9cb0) is in progress. I'm doing it after A (#293) merges it, not before.

• v5 WS4/WS6: one hosted engine for disk + memory; unified Ledgers view #282 moves the in-memory engine from the CLI into socket_patch_core::hosted::memory and deletes hosted_memory/redirect.rs. A test merge here conflicts in 9 files.
• Most of those conflicts are A's in-memory socket.yml code, not this PR's. That code imports CLI-only items (crate::commands::scan::policy::{policy_block, FilteredEntry}) that core can't depend on. v5: socket.yml patch rollout config and filtering #293 has to port the same code for the same request.
• If I ported A's code here, the two ports would differ and conflict again once v5: socket.yml patch rollout config and filtering #293 lands. So when v5: socket.yml patch rollout config and filtering #293 pushes its v5 WS4/WS6: one hosted engine for disk + memory; unified Ledgers view #282 merge, I'll merge v5/socket-yml-patch-config, which brings in the base too. Then I'll port only this PR's rollout parts onto the unified engine: per-root classification, the run-wide plan and the re-rewrite without deferred rows.
• v5: socket.yml patch rollout config and filtering #293 lands first, so this doesn't delay the landing order. I'm checking v5: socket.yml patch rollout config and filtering #293 every 30–45 minutes.
• CI on the current head c59c8f4 has only the 8 failures the base branch has too (4 vlt install-proof, 4 yarn-classic). test (windows-latest) passed there, so the update_notifier_e2e failure on 0da9e4b didn't repeat.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#291 landed on release/v5-prerelease as f9cb7e1; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] coverage on a6f5008 hit its 35-minute timeout. The cause is a test flake in code this PR doesn't touch, so I've re-run the failed jobs once (run 36497894523).

What hung: vendor::maven_repo::tests::fifo_project_pom_fails_fast_in_revert (log: "has been running for over 60 seconds", then cancelled at 35 minutes).

Why its guard hung too:

• expect_fast in crates/socket-patch-core/src/vendor/maven_repo.rs gives the future 5 seconds.
• When time runs out, it opens the FIFO for writing so a stuck reader can finish. That open doesn't pass O_NONBLOCK.
• revert_maven reads the pom with read_regular_to_string, a guarded non-blocking read, so no reader ever waits on the FIFO. The watchdog's write-open then blocks forever instead of failing the test.
• Result: a revert that takes a little over 5 seconds on a loaded coverage runner turns into a hang that uses up the whole job.

Not from this PR: coverage passed on 2270cb3 in 13 minutes and on the base branch at f9cb7e1 in 11 minutes, and this PR doesn't change vendor/.

Proposed fix (not pushed here, since it's outside this PR's scope). It mirrors release_fifo_reader in cargo_config.rs:

             Err(_) => {
-                let _ = std::fs::OpenOptions::new().write(true).open(fifo);
+                use std::os::unix::fs::OpenOptionsExt;
+                let _ = std::fs::OpenOptions::new()
+                    .write(true)
+                    .custom_flags(libc::O_NONBLOCK)
+                    .open(fifo);
                 panic!("{what}");

With it, a slow revert fails the test after 5 seconds instead of hanging the job. The same re-run also repeats the 8 jobs that also fail on the base branch (4 vlt install-proof, 4 yarn-classic). All other checks on a6f5008 passed, including install-proof (ubuntu-latest, 1.0.0-rc.22).

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#292 landed on release/v5-prerelease as a7b0d00; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#296 landed on release/v5-prerelease as 1e3ace6; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#297 landed on release/v5-prerelease as b97a1c2; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

#293 landed on release/v5-prerelease as b9e106d; please merge origin/release/v5-prerelease again, resolve conflicts, get green, and keep it ready.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Coordinator: #294 is next in the landing order but is blocked. Its only [agent] ready to land comment (2026-09-28 18:24Z) is older than the last push (1613348, which merges b9e106d / #293, at 2026-09-29 10:22Z). CI on 1613348 was also still finishing when I checked. Once CI completes, please post a fresh [agent] ready to land for 1613348 that lists any red checks and the base-branch run where each one also fails.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] ready to land

• Head: 1613348, which merges release/v5-prerelease at b9e106d (v5: socket.yml patch rollout config and filtering #293, work item A). Against the base, the diff is work item B only (48 files).
• What B adds: scan --max-new-patches <N|none> (env SOCKET_MAX_NEW_PATCHES, socket.yml patches.maxNewPatches). Each run adds new patches most severe first, never caps upgrades, and converges over repeated runs. Results are reported in the rollout block, on disk and in the in-memory engine.
• CI on 1613348 is complete: 415 passed, 6 skipped, 8 failed. coverage and Pipenv compatibility passed. Each of the 8 failures also fails on the base branch:

Red check on 1613348	Same check failing on release/v5-prerelease
install-proof (ubuntu-latest, 0.0.0-1)	run 36454568301 @ 28cebf7
install-proof (ubuntu-latest, 0.0.0-1, 22.0.0)	run 36454568301 @ 28cebf7
install-proof (ubuntu-latest, 0.0.0-11)	run 36454568301 @ 28cebf7
install-proof (windows-latest, 0.0.0-11)	run 36454568301 @ 28cebf7
yarn-classic 1.0.2	run 36554947505 @ b9e106d
yarn-classic 1.6.0	run 36554947505 @ b9e106d
yarn-classic 1.7.0	run 36554947505 @ b9e106d
yarn-classic 1.9.4	run 36554947505 @ b9e106d

• Base evidence for the vlt jobs: the vlt compatibility workflow didn't run for b9e106d. Its latest run on release/v5-prerelease is 36454568301 (a workflow_dispatch at 28cebf7), and it has the same four install-proof jobs red. This PR doesn't touch vlt rollback or yarn-classic.
• Reviews: there are no review threads or reviews on the PR.

---

Generated by Claude Code