diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index d9895a2c..21ca5f78 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -120,7 +120,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifact preflight**: before any takeover or write (dry runs included), each granted artifact with a default-registry instance is fetched once as vlt fetches it and must verify, else the dep is withheld (`redirect_vlt_artifact_unverifiable`, see the tag table). **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the npm registry, following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`. -**Takeover reconciliation (every hosted ecosystem, v5.0)**: vendoring over a hosted pin (`vendor`, `scan --mode vendored`, `get --mode vendored`) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed `, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (exit 1 / `partial_failure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed ` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partial_failure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `redirect.warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. +**Takeover reconciliation (every hosted ecosystem, v5.0)**: vendoring over a hosted pin (`vendor`, `scan --mode vendored`, `get --mode vendored`) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed `, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (exit 1 / `partial_failure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed ` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partial_failure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `redirect.warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. **Gem preflight before the takeover**: `scan` / `get --mode vendored` ask the gem vendored backend's own refusals BEFORE the upstream restore — the manifest gate (`gemfile_not_loaded`: a `gems.rb` twin or a `BUNDLE_GEMFILE`-configured manifest) and the Gemfile declaration gate evaluated on the Gemfile and Gemfile.lock text the restore would leave (`gemfile_declaration_not_editable`: a declaration inside a `group` / `platforms` / conditional block, a parenthesized or duplicate declaration, …) — so a hosted gem vendored mode cannot wire is reported `failed ` with the hosted `Gemfile` / `Gemfile.lock` byte-untouched (exit 1 / `partial_failure`), and their `--dry-run` preview reports that gem as `would_refuse` with the same `errorCode` (exit 0, like the Bun / vlt `would_refuse` rows), never `would_vendor`. Pinned against real Bundler by `tests/e2e_redirect_gem_build.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `redirect.warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. ### Scan modes (v5.0) diff --git a/crates/socket-patch-cli/src/commands/get.rs b/crates/socket-patch-cli/src/commands/get.rs index 7c470820..2f85f12a 100644 --- a/crates/socket-patch-cli/src/commands/get.rs +++ b/crates/socket-patch-cli/src/commands/get.rs @@ -1728,7 +1728,11 @@ type LockRefusals = HashMap; /// refusal and the ledger's idempotency skip, which take precedence in the /// fetch loop. A purl the lockfiles pin hosted is left to the vendor loop: /// its takeover restores the upstream lock entry first, and the restore -/// rewrites the very text the gates read. +/// rewrites the very text the gates read. The one exception is a hosted +/// gem the takeover would refuse ([`gem_takeover_refusals_for`]), which +/// is refused here with the takeover's code instead of after its fetch. +/// +/// [`gem_takeover_refusals_for`]: crate::commands::vendor::gem_takeover_refusals_for /// /// Only a package the vendor loop would hand to its backend is refused /// here (see [`crate::commands::vendor::lock_refusals_reaching_backend`]): @@ -1745,44 +1749,65 @@ async fn lock_text_refusals_for( prior: Option<&crate::ecosystem_dispatch::NpmCrawlSnapshot>, ) -> LockRefusals { let cwd = params.cwd.as_path(); - let claimed: Vec = socket_patch_core::patch::redirect::upstream::HostedPin::all( + let origins: Vec = params + .patch_server_url + .iter() + .filter(|url| !url.trim().is_empty()) + .cloned() + .collect(); + let pins = socket_patch_core::patch::redirect::upstream::HostedPin::all( &socket_patch_core::vex::discover_patched_refs_with( cwd, &socket_patch_core::vex::DiscoverOptions { - patch_server_origins: params - .patch_server_url - .iter() - .filter(|url| !url.trim().is_empty()) - .cloned() - .collect(), + patch_server_origins: origins.clone(), }, ) .await, - ) - .into_iter() - .map(|pin| canonical_purl(&pin.purl)) - .collect(); - let candidates: Vec<(&str, &str)> = selected + ); + let claimed: Vec = pins.iter().map(|pin| canonical_purl(&pin.purl)).collect(); + let fetchable: Vec<&PatchSearchResult> = selected .iter() .filter(|sr| bun_refusal.filter(|r| r.applies_to(&sr.purl)).is_none()) .filter(|sr| { detached_ledger_record(RecordStore::Ledger(&ledger.entries), &sr.purl, &sr.uuid) .is_none() }) + .collect(); + let candidates: Vec<(&str, &str)> = fetchable + .iter() .filter(|sr| !claimed.contains(&canonical_purl(&sr.purl))) .map(|sr| (sr.purl.as_str(), sr.uuid.as_str())) .collect(); let refused = socket_patch_core::vendor::lock_text_refusals(cwd, &candidates).await; let options = params.crawler_options(); - crate::commands::vendor::lock_refusals_reaching_backend( - cwd, - refused, - &ledger.entries, - |purls| async move { - crate::commands::vendor::installed_purls(&options, &purls, prior).await - }, - ) - .await + let mut refusals = + crate::commands::vendor::lock_refusals_reaching_backend( + cwd, + refused, + &ledger.entries, + |purls| async move { + crate::commands::vendor::installed_purls(&options, &purls, prior).await + }, + ) + .await; + // A hosted gem the takeover will refuse (#775) is refused here too, so + // its view is never fetched for a package the run cannot vendor. The + // download phase only runs online (`--offline` refuses `get` and `scan` + // before it), so the dry-run restore may resolve the registry entry. + refusals.extend( + crate::commands::vendor::gem_takeover_refusals_for( + cwd, + fetchable + .iter() + .filter(|sr| claimed.contains(&canonical_purl(&sr.purl))) + .map(|sr| sr.purl.as_str()), + &pins, + false, + origins, + ) + .await, + ); + refusals } /// The record a detached ledger entry already carries for `purl` at @@ -3629,7 +3654,12 @@ async fn run_get_vendored( // Dry run: ledger-classification preview only (scan's posture) — no // download, no vendor step, no writes. if args.common.dry_run { - let preview = super::scan::preview_vendor_json(&args.common.cwd, selected).await; + let takeover = super::vendor::gem_takeover_preview_refusals( + &args.common, + selected.iter().map(|p| p.purl.as_str()), + ) + .await; + let preview = super::scan::preview_vendor_json(&args.common.cwd, selected, &takeover).await; if args.common.json { let mut result = serde_json::json!({ "status": "success", diff --git a/crates/socket-patch-cli/src/commands/scan/mod.rs b/crates/socket-patch-cli/src/commands/scan/mod.rs index 183a88e8..80f1cfe3 100644 --- a/crates/socket-patch-cli/src/commands/scan/mod.rs +++ b/crates/socket-patch-cli/src/commands/scan/mod.rs @@ -2717,12 +2717,13 @@ async fn run_scan( }; // The rollout plan (§5.2): deferred NEW rows leave the selection here. - // Vendored eligibility is the wet run's Bun / vlt preflight; the agent - // partition above already removed what cannot land in place. + // Vendored eligibility is the wet run's Bun / vlt / npm-lock / gem + // takeover preflight; the agent partition above already removed what + // cannot land in place. let selected = if report_only { selected } else if vendor { - let refused = vendor_flow::preflight_refused_purls(&args.common.cwd, &selected).await; + let refused = vendor_flow::preflight_refused_purls(&args.common, &selected).await; stage.plan(&rows, |r| !refused.contains(&r.writer.purl)); let deferred = stage.deferred_keys(); selected @@ -2841,7 +2842,12 @@ async fn run_scan( // rendered as `[would-refuse]` lines so a preview never // advertises vendoring the wet run would refuse. let preview = if vendor { - Some(preview_vendor_json(&args.common.cwd, &selected).await) + let takeover = crate::commands::vendor::gem_takeover_preview_refusals( + &args.common, + selected.iter().map(|p| p.purl.as_str()), + ) + .await; + Some(preview_vendor_json(&args.common.cwd, &selected, &takeover).await) } else { None }; diff --git a/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs b/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs index 2bec575e..3f8dccb5 100644 --- a/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs +++ b/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs @@ -78,9 +78,14 @@ type VendorStepResult = Result<(bool, Envelope), VendorStepError>; /// outside the preflights are not predicted), and `would_refuse` never /// flips the run's status or exit code. The preflights (the only disk /// access besides the ledger) run only when the selection holds an npm purl. +/// `takeover_refusals` adds the hosted→vendored takeover refusals the +/// caller resolved (the gem gates of +/// [`crate::commands::vendor::gem_takeover_preview_refusals`]), keyed by +/// the selected purl, as `would_refuse` rows too. pub(crate) async fn preview_vendor_json( cwd: &Path, selected: &[PatchSearchResult], + takeover_refusals: &HashMap, ) -> serde_json::Value { // The ledger load outcome reaches the preflight AS a result, so an // unreadable ledger previews as `vendor_state_unreadable` rather than @@ -111,6 +116,13 @@ pub(crate) async fn preview_vendor_json( "errorCode": r.code, "error": r.detail, }) } + _ if takeover_refusals.contains_key(&p.purl) => { + let (code, detail) = &takeover_refusals[&p.purl]; + serde_json::json!({ + "purl": p.purl, "uuid": p.uuid, "action": "would_refuse", + "errorCode": code, "error": detail, + }) + } _ if p.purl.starts_with("pkg:npm/") && npm_lock_refusal.is_some() => { let (code, detail) = npm_lock_refusal.as_ref().expect("checked by the guard"); serde_json::json!({ @@ -134,26 +146,33 @@ pub(crate) async fn preview_vendor_json( serde_json::json!({ "dryRun": true, "patches": patches }) } -/// The purls of `selected` the wet run's Bun, vlt or npm package-lock preflight would refuse -/// before any download (the `would_refuse` rows of -/// [`preview_vendor_json`]): the vendored planning pass, so a refused NEW -/// patch holds no rollout slot. +/// The purls of `selected` the wet run's Bun, vlt or npm package-lock +/// preflight, or the gem hosted→vendored takeover gate, would refuse before +/// any download (the `would_refuse` rows of [`preview_vendor_json`]): the +/// vendored planning pass, so a refused NEW patch holds no rollout slot. pub(super) async fn preflight_refused_purls( - cwd: &Path, + common: &GlobalArgs, selected: &[PatchSearchResult], ) -> HashSet { + let cwd = common.cwd.as_path(); let state = load_state(cwd).await; let refusal = bun_vendor_preflight_with_ledger(cwd, selected, state.as_ref().map(|s| &s.entries)).await; let vlt_refusals = vlt_vendor_preflight_selected(cwd, selected, state.as_ref().map(|s| &s.entries)).await; let npm_lock_refusal = npm_lock_refusal(cwd, selected).await; + let gem_refusals = crate::commands::vendor::gem_takeover_preview_refusals( + common, + selected.iter().map(|p| p.purl.as_str()), + ) + .await; selected .iter() .filter(|p| { refusal.as_ref().is_some_and(|r| r.applies_to(&p.purl)) || vlt_refusal_for(&vlt_refusals, &p.purl).is_some() || (p.purl.starts_with("pkg:npm/") && npm_lock_refusal.is_some()) + || gem_refusals.contains_key(&p.purl) }) .map(|p| p.purl.clone()) .collect() @@ -540,7 +559,7 @@ async fn run_vendor_json_path( // The planning pass: a patch the preflight refuses holds no slot (it // still reaches the engine, which reports the refusal). let writers = writers_of(&rows); - let refused = preflight_refused_purls(&args.common.cwd, &writers).await; + let refused = preflight_refused_purls(&args.common, &writers).await; stage.plan(&rows, |r| !refused.contains(&r.writer.purl)); let deferred = stage.deferred_keys(); let selected: Vec = writers @@ -552,7 +571,12 @@ async fn run_vendor_json_path( if args.common.dry_run { // No downloads, no backends: classify against the ledger // and preview the GC, exactly like `--apply`'s dry run. - result["vendor"] = preview_vendor_json(&args.common.cwd, &selected).await; + let takeover = crate::commands::vendor::gem_takeover_preview_refusals( + &args.common, + selected.iter().map(|p| p.purl.as_str()), + ) + .await; + result["vendor"] = preview_vendor_json(&args.common.cwd, &selected, &takeover).await; if prune { result["gc"] = gc_json( &args.common, @@ -1274,7 +1298,7 @@ mod preview_tests { #[tokio::test] async fn preview_without_bun_lock_is_plain_would_vendor() { let tmp = tempfile::tempdir().unwrap(); - let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)]).await; + let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)], &HashMap::new()).await; assert_eq!( preview, serde_json::json!({ @@ -1290,7 +1314,12 @@ mod preview_tests { async fn preview_marks_would_refuse_for_refused_bun_tree() { let tmp = tempfile::tempdir().unwrap(); std::fs::write(tmp.path().join("bun.lock"), V1_WORKSPACE_LOCK).unwrap(); - let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM), sel(UUID, PYPI)]).await; + let preview = preview_vendor_json( + tmp.path(), + &[sel(UUID, NPM), sel(UUID, PYPI)], + &HashMap::new(), + ) + .await; let npm = action_of(&preview, NPM); assert_eq!(npm["action"], "would_refuse", "{preview}"); assert_eq!( @@ -1322,7 +1351,7 @@ mod preview_tests { std::fs::write(tmp.path().join("bun.lock"), V1_WORKSPACE_LOCK).unwrap(); seed_entry(tmp.path(), NPM, UUID); - let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)]).await; + let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)], &HashMap::new()).await; assert_eq!( action_of(&preview, NPM)["action"], "would_refuse", @@ -1330,7 +1359,7 @@ mod preview_tests { ); seed_entry(tmp.path(), NPM, OLD_UUID); - let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)]).await; + let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)], &HashMap::new()).await; let rec = action_of(&preview, NPM); assert_eq!(rec["action"], "would_refuse", "{preview}"); assert!( @@ -1344,7 +1373,7 @@ mod preview_tests { ); std::fs::write(tmp.path().join("bun.lock"), wired).unwrap(); seed_entry(tmp.path(), NPM, UUID); - let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)]).await; + let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)], &HashMap::new()).await; assert_eq!( action_of(&preview, NPM)["action"], "already_vendored", @@ -1353,11 +1382,28 @@ mod preview_tests { } /// Malformed bun.lockb: `would_refuse` with the binary format code. + #[tokio::test] + async fn preview_marks_a_refused_takeover_would_refuse() { + const GEM: &str = "pkg:gem/rails@7.0.0"; + let tmp = tempfile::tempdir().unwrap(); + let refusals = HashMap::from([( + GEM.to_string(), + ("gemfile_declaration_not_editable", "indented".to_string()), + )]); + let preview = + preview_vendor_json(tmp.path(), &[sel(UUID, GEM), sel(UUID, NPM)], &refusals).await; + let gem = action_of(&preview, GEM); + assert_eq!(gem["action"], "would_refuse", "{preview}"); + assert_eq!(gem["errorCode"], "gemfile_declaration_not_editable"); + assert_eq!(gem["error"], "indented"); + assert_eq!(action_of(&preview, NPM)["action"], "would_vendor"); + } + #[tokio::test] async fn preview_marks_malformed_lockb_would_refuse() { let tmp = tempfile::tempdir().unwrap(); std::fs::write(tmp.path().join("bun.lockb"), b"\x00binary").unwrap(); - let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)]).await; + let preview = preview_vendor_json(tmp.path(), &[sel(UUID, NPM)], &HashMap::new()).await; assert_eq!( action_of(&preview, NPM)["errorCode"], "vendor_bun_lockb_invalid", diff --git a/crates/socket-patch-cli/src/commands/vendor.rs b/crates/socket-patch-cli/src/commands/vendor.rs index a30f773c..2d97dc5a 100644 --- a/crates/socket-patch-cli/src/commands/vendor.rs +++ b/crates/socket-patch-cli/src/commands/vendor.rs @@ -1035,6 +1035,86 @@ fn emit_eject_refusal(common: &GlobalArgs, code: &'static str, message: &str) -> 1 } +/// The gem vendored backend's refusals a hosted→vendored takeover raises +/// BEFORE it restores `pin` upstream, so a gem vendored mode cannot wire +/// keeps its hosted wiring instead of ending up unpatched in both modes: +/// the manifest gate (a `gems.rb` twin, `BUNDLE_GEMFILE`) and the Gemfile +/// declaration gate on the restored Gemfile (a declaration inside a +/// `group` block, #775). +async fn gem_takeover_refusal( + cwd: &Path, + candidate: &str, + pin: &HostedPin, + restore_opts: &socket_patch_core::patch::redirect::upstream::RestoreOptions, +) -> Option<(&'static str, String)> { + match socket_patch_core::vendor::gem::gem_manifest_refusal(cwd).await { + Some(refusal) => Some(refusal), + None => { + socket_patch_core::vendor::gem::gem_vendor_target_preflight( + cwd, + candidate, + pin, + restore_opts, + ) + .await + } + } +} + +/// [`gem_takeover_refusal`] for the dry-run preview of `scan` / `get +/// --mode vendored`: each selected gem purl the lockfiles still pin hosted +/// whose takeover the wet run would refuse, keyed by the selected purl. +/// Nothing is written (the restore is resolved as a dry run). +pub(crate) async fn gem_takeover_preview_refusals<'a>( + common: &GlobalArgs, + purls: impl Iterator, +) -> HashMap { + let gems: Vec<&str> = purls.filter(|p| p.starts_with("pkg:gem/")).collect(); + if gems.is_empty() { + return HashMap::new(); + } + let pins = HostedPin::all(&crate::commands::discover_wiring(common, &common.cwd).await); + gem_takeover_refusals_for( + &common.cwd, + gems.into_iter(), + &pins, + common.offline, + crate::commands::rollback::patch_server_origins(common), + ) + .await +} + +/// [`gem_takeover_preview_refusals`] over already-discovered hosted `pins`: +/// the vendored download phase reads the pins itself, so it refuses these +/// gems before fetching their views instead of after. +pub(crate) async fn gem_takeover_refusals_for<'a>( + cwd: &Path, + purls: impl Iterator, + pins: &[HostedPin], + offline: bool, + patch_server_origins: Vec, +) -> HashMap { + let mut refusals = HashMap::new(); + let restore_opts = socket_patch_core::patch::redirect::upstream::RestoreOptions { + dry_run: true, + offline, + patch_server_origins, + bun_lockb: true, + }; + for purl in purls.filter(|p| p.starts_with("pkg:gem/")) { + let Some(pin) = pins + .iter() + .find(|pin| canonical_purl(&pin.purl) == canonical_purl(purl)) + else { + continue; + }; + if let Some(refusal) = gem_takeover_refusal(cwd, purl, pin, &restore_opts).await { + refusals.insert(purl.to_string(), refusal); + } + } + refusals +} + /// The hosted pins whose ecosystem `--ecosystems` selects. fn hosted_pins_in_scope(common: &GlobalArgs, pins: Vec) -> Vec { pins.into_iter() @@ -2501,11 +2581,13 @@ pub(crate) async fn vendor_records_reusing( // code and detail, in the dry run and the wet run alike — // so the hosted wiring stays untouched. // The gem backend's manifest refusal, likewise raised before - // the restore (a hosted `gems.rb` project cannot vendor). + // the restore (a hosted `gems.rb` project cannot vendor), and + // its Gemfile declaration refusal, evaluated on the restored + // Gemfile (a gem declared inside a `group` block stays hosted). if candidate.starts_with("pkg:gem/") { - if let Some((code, detail)) = - socket_patch_core::vendor::gem::gem_manifest_refusal(&common.cwd).await - { + let refusal = + gem_takeover_refusal(&common.cwd, candidate, pin, &restore_opts).await; + if let Some((code, detail)) = refusal { has_errors = true; env.record( PatchEvent::new(PatchAction::Failed, candidate.clone()) diff --git a/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs b/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs index 6d590d8e..342dfc4f 100644 --- a/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs @@ -445,6 +445,12 @@ enum Driver { /// environment, so bundler still loads `Gemfile.next` and the run must /// still redirect and attest nothing. ScanVexDualBootEnvGemfile, + /// [`Driver::ScanVex`] on a Gemfile that declares the gem inside a + /// `group :development do … end` block (#775): hosted mode wraps it in + /// a source block inside the group, but vendored mode cannot edit an + /// indented declaration, so a later takeover must refuse before it + /// un-hosts the gem. + ScanVexGroupBlock, } impl Driver { @@ -465,6 +471,7 @@ impl Driver { Driver::ScanVexDualBootEnvGemfile => { "scan --mode hosted (config Gemfile.next, env BUNDLE_GEMFILE=Gemfile)" } + Driver::ScanVexGroupBlock => "scan --mode hosted (gem in a group block)", } } } @@ -733,6 +740,10 @@ async fn redirect_scanned_project( group :test do\n gem \"{DEP}\"\nend\n", server.uri() ), + Driver::ScanVexGroupBlock => format!( + "source \"{}/upstream\"\n\ngroup :development do\n gem \"{DEP}\"\nend\n", + server.uri() + ), Driver::ScanVexEvalGemfile => { std::fs::write(proj.join("Gemfile.common"), format!("gem \"{DEP}\"\n")).unwrap(); format!( @@ -867,6 +878,7 @@ async fn redirect_scanned_project( | Driver::ScanVexDualBootEnvGemfile | Driver::ScanVexDuplicateDeclaration | Driver::ScanVexEvalGemfile + | Driver::ScanVexGroupBlock | Driver::ScanVexMultiLineDeclaration | Driver::ScanVexConditionalDeclaration | Driver::ScanVexScopedConstantModifier @@ -1013,7 +1025,7 @@ async fn redirect_scanned_project( ); } match driver { - Driver::ScanVex => { + Driver::ScanVex | Driver::ScanVexGroupBlock => { assert_eq!(env["vex"]["statements"], 1, "vex block: {env}"); assert_eq!( env["vex"]["verified"], false, @@ -1666,6 +1678,152 @@ async fn gem_hosted_gems_rb_pin_survives_a_refused_vendored_takeover() { vendor_takeover_keeps_the_hosted_gems_rb_pin(&fx); } +/// #775: hosted mode accepts a gem declared inside a `group … do` block, +/// but vendored mode refuses an indented declaration. A takeover (`scan` +/// or `get ` with `--mode vendored`, dry or wet) must raise that +/// refusal BEFORE it restores the hosted pin, or the gem ends up unpatched +/// in both modes. +#[tokio::test(flavor = "multi_thread")] +#[ignore = "host capstone: shells out to a real ruby/gem/bundler (>= 1.17; CHECKSUMS arm >= 2.6); \ + the unpinned `test` job skips it, an e2e job with a pinned toolchain runs it via --ignored"] +async fn gem_hosted_group_block_pin_survives_a_refused_vendored_takeover() { + let Some(fx) = redirect_scanned_project( + "group-block takeover", + Spelling::Gemfile, + true, + true, + None, + Driver::ScanVexGroupBlock, + ) + .await + else { + return; + }; + // The takeover's restore re-derives a CHECKSUMS pin's upstream sha256 + // only for a rubygems.org remote (the shape of the #775 report). Spell + // the mock upstream as rubygems.org in the pair, and serve that + // registry from the mock (`SOCKET_RUBYGEMS_URL`, below). No bundler + // runs after this point. + let upstream = format!("{}/upstream", fx._server.uri()); + for (file, from, to) in [ + ( + "Gemfile", + format!("\"{upstream}\""), + "\"https://rubygems.org\"", + ), + ( + "Gemfile.lock", + format!("{upstream}/"), + "https://rubygems.org/", + ), + ] { + let path = fx.proj.join(file); + let text = std::fs::read_to_string(&path).unwrap(); + assert!( + text.contains(&from), + "{file} names the mock upstream:\n{text}" + ); + std::fs::write(&path, text.replace(&from, to)).unwrap(); + } + for (command, selector) in [("get", Some(UUID)), ("scan", None)] { + for dry_run in [true, false] { + let views = view_requests(&fx).await; + vendor_takeover_keeps_the_hosted_group_pin(&fx, command, selector, dry_run); + // The refusal is known before the download phase: a wet scan + // never fetches the patch view (nor its files) of a gem it + // cannot vendor. `get ` fetches the view once to resolve + // its identifier, and that is all it fetches. + let fetched = view_requests(&fx).await - views; + let allowed = usize::from(command == "get"); + assert!( + fetched <= allowed, + "{command} --mode vendored (dry_run={dry_run}) fetched the refused gem's \ + view {fetched} time(s)" + ); + } + } +} + +/// One refused takeover of the group-block fixture: the +/// `gemfile_declaration_not_editable` refusal (exit 1 wet; a `would_refuse` +/// preview row dry), no revert (nor a preview of one), and the hosted +/// `Gemfile` / `Gemfile.lock` byte-untouched. +fn vendor_takeover_keeps_the_hosted_group_pin( + fx: &RedirectFixture, + command: &str, + selector: Option<&str>, + dry_run: bool, +) { + let before: Vec> = ["Gemfile", "Gemfile.lock"] + .iter() + .map(|f| std::fs::read(fx.proj.join(f)).unwrap()) + .collect(); + let proj = fx.proj.to_str().expect("utf8 tmp path"); + let api = fx._server.uri(); + let mut argv: Vec<&str> = vec![command]; + argv.extend(selector); + argv.extend([ + "--mode", + "vendored", + "--json", + "--yes", + "--cwd", + proj, + "--api-url", + &api, + "--org", + ORG, + "--api-token", + "fake", + "--patch-server-url", + &api, + ]); + if dry_run { + argv.push("--dry-run"); + } + let label = format!("{command} --mode vendored (dry_run={dry_run})"); + let upstream = format!("{api}/upstream"); + let (code, stdout, stderr) = + run_socket_env(&fx.proj, &argv, &[("SOCKET_RUBYGEMS_URL", &upstream)]); + let env: serde_json::Value = serde_json::from_str(&stdout) + .unwrap_or_else(|e| panic!("{label}: not JSON: {e}\nstdout:\n{stdout}\nstderr:\n{stderr}")); + if dry_run { + // The ledger-classification preview: the refusal is a + // `would_refuse` row (which never flips the exit code), never + // `would_vendor`. + assert_eq!(code, 0, "{label}: {env}"); + let row = &env["vendor"]["patches"][0]; + assert_eq!(row["action"], "would_refuse", "{label}: {env}"); + assert_eq!( + row["errorCode"], "gemfile_declaration_not_editable", + "{label}: {env}" + ); + } else { + assert_eq!(code, 1, "{label} must refuse: {env}\nstderr:\n{stderr}"); + assert!( + stdout.contains("gemfile_declaration_not_editable"), + "{label}: the declaration refusal names its cause: {env}" + ); + } + for code in [ + "vendor_takeover_reverted_redirect", + "vendor_would_revert_redirect", + "would_vendor", + ] { + assert!( + !stdout.contains(code), + "{label}: the hosted pin must not be (previewed as) reverted ({code}):\n{stdout}" + ); + } + for (file, before) in ["Gemfile", "Gemfile.lock"].iter().zip(before) { + assert_eq!( + std::fs::read(fx.proj.join(file)).unwrap(), + before, + "{label}: {file} keeps its hosted wiring" + ); + } +} + /// A hosted→vendored takeover of a `gems.rb` project: vendored mode cannot /// wire `gems.rb`, so `get --mode vendored` must refuse BEFORE it restores the hosted /// pin's upstream entry, or the gem ends up unpatched in both modes. diff --git a/crates/socket-patch-core/src/vendor/gem.rs b/crates/socket-patch-core/src/vendor/gem.rs index fb14ceba..5f9357b4 100644 --- a/crates/socket-patch-core/src/vendor/gem.rs +++ b/crates/socket-patch-core/src/vendor/gem.rs @@ -183,6 +183,63 @@ pub async fn gem_manifest_refusal(project_root: &Path) -> Option<(&'static str, } } +/// The per-gem twin of [`gem_manifest_refusal`] for the hosted→vendored +/// takeover: the backend's Gemfile declaration gate ([`plan_gemfile_edit`] +/// and [`refuse_append_of_direct_dependency`], refused as +/// `gemfile_declaration_not_editable`) for `purl`. Evaluated on the files +/// as the takeover's restore of `pin` will leave them: a dry-run +/// [`restore_upstream`] supplies the restored `Gemfile` / `Gemfile.lock` +/// text, so the hosted `source … do` block socket-patch itself wrote is +/// never mistaken for the user's declaration. A gem declared inside a +/// `group` block is the common case: hosted mode wires it, vendored mode +/// cannot, and without this gate the takeover un-hosts it first (#775). +/// Returns `(code, detail)`, exactly the refusal the backend would raise +/// after the restore; `None` when it would not refuse, when the restore +/// itself would refuse (the takeover reports that), or when the files are +/// unreadable (which the backend reports itself). +/// +/// [`restore_upstream`]: crate::patch::redirect::upstream::restore_upstream +pub async fn gem_vendor_target_preflight( + project_root: &Path, + purl: &str, + pin: &crate::patch::redirect::upstream::HostedPin, + opts: &crate::patch::redirect::upstream::RestoreOptions, +) -> Option<(&'static str, String)> { + use crate::patch::redirect::upstream::{restore_upstream, RestoreOptions}; + let (name, version) = parse_gem_purl(purl)?; + // The copy path only shapes a plan that passes; any refusal is decided + // by the declaration alone, so an unsafe uuid is left to the backend. + let copy_rel = format!( + "{}/{name}-{version}", + vendor_uuid_dir_rel("gem", &pin.uuid)? + ); + let dry = RestoreOptions { + dry_run: true, + ..opts.clone() + }; + let restore = restore_upstream(project_root, std::slice::from_ref(pin), &dry).await; + if restore.refused().next().is_some() { + return None; + } + let restored = |file: &str| restore.staged_text.get(file).cloned(); + let gemfile_text = match restored(GEMFILE) { + Some(text) => text?, + None => read_regular_to_string(&project_root.join(GEMFILE)) + .await + .ok()?, + }; + let lock_text = match restored(GEMFILE_LOCK) { + Some(text) => text?, + None => read_regular_to_string(&project_root.join(GEMFILE_LOCK)) + .await + .ok()?, + }; + plan_gemfile_edit(&gemfile_text, &name, &version, ©_rel) + .and_then(|plan| refuse_append_of_direct_dependency(plan, &lock_text, &name)) + .err() + .map(|detail| ("gemfile_declaration_not_editable", detail)) +} + async fn gem_prelude( purl: &str, installed_path: &Path, @@ -7888,6 +7945,87 @@ mod tests { assert!(!root.join(".socket").exists(), "nothing written"); } + // ── #775: the takeover's declaration preflight ───────────────────── + + const TAKEOVER_IDX: &str = "https://patch.socket.dev/patch-registry/gem/11111111-1111-1111-1111-111111111111/9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f/"; + + /// A converged, CHECKSUMS-less hosted lock (restoring it needs no + /// registry lookup, so the preflight runs offline). + fn hosted_takeover_lock() -> String { + format!( + "GEM\n remote: {TAKEOVER_IDX}\n specs:\n rails (7.0.0)\n\nGEM\n remote: \ + https://rubygems.org/\n specs:\n puma (6.0.0)\n\nPLATFORMS\n ruby\n\n\ + DEPENDENCIES\n puma\n rails (= 7.0.0)!\n\nBUNDLED WITH\n 2.4.0\n" + ) + } + + async fn takeover_preflight(gemfile: &str) -> (Option<(&'static str, String)>, PathBuf) { + let dir = tempfile::tempdir().unwrap().keep(); + std::fs::write(dir.join(GEMFILE), gemfile).unwrap(); + std::fs::write(dir.join(GEMFILE_LOCK), hosted_takeover_lock()).unwrap(); + let pin = crate::patch::redirect::upstream::HostedPin { + purl: "pkg:gem/rails@7.0.0".to_string(), + uuid: UUID.to_string(), + files: vec![GEMFILE.to_string(), GEMFILE_LOCK.to_string()], + }; + let opts = crate::patch::redirect::upstream::RestoreOptions { + dry_run: false, + offline: true, + patch_server_origins: Vec::new(), + bun_lockb: true, + }; + let refusal = gem_vendor_target_preflight(&dir, "pkg:gem/rails@7.0.0", &pin, &opts).await; + (refusal, dir) + } + + #[tokio::test] + async fn takeover_preflight_refuses_a_hosted_gem_inside_a_group_block() { + let gemfile = format!( + "source \"https://rubygems.org\"\n\ngem \"puma\"\n\ngroup :development do\n\ + source \"{TAKEOVER_IDX}\" do\n gem \"rails\", \"7.0.0\"\nend\nend\n" + ); + let (refusal, dir) = takeover_preflight(&gemfile).await; + let (code, detail) = refusal.expect("an indented declaration is not editable"); + assert_eq!(code, "gemfile_declaration_not_editable"); + assert!(detail.contains("indented"), "{detail}"); + // A preflight: the hosted pair is never written, even though the + // caller's options are a wet run. + assert_eq!(std::fs::read_to_string(dir.join(GEMFILE)).unwrap(), gemfile); + assert_eq!( + std::fs::read_to_string(dir.join(GEMFILE_LOCK)).unwrap(), + hosted_takeover_lock() + ); + std::fs::remove_dir_all(dir).ok(); + } + + #[tokio::test] + async fn takeover_preflight_passes_a_top_level_hosted_gem() { + // The hosted source block is socket-patch's own wiring: evaluated on + // the restored Gemfile, the declaration is a plain top-level line. + let gemfile = format!( + "source \"https://rubygems.org\"\n\ngem \"puma\"\n\nsource \"{TAKEOVER_IDX}\" do\n \ + gem \"rails\", \"7.0.0\"\nend\n" + ); + let (refusal, dir) = takeover_preflight(&gemfile).await; + assert_eq!(refusal, None); + assert_eq!(std::fs::read_to_string(dir.join(GEMFILE)).unwrap(), gemfile); + std::fs::remove_dir_all(dir).ok(); + } + + #[tokio::test] + async fn takeover_preflight_leaves_a_refused_restore_to_the_takeover() { + // A hand-edited hosted block the restore cannot unwind: the restore + // refuses, and the takeover reports that itself + // (`redirect_revert_failed`), not this gate. + let gemfile = format!( + "source \"https://rubygems.org\"\n\ngroup :test do\nsource \"{TAKEOVER_IDX}\" do\n \ + gem \"rails\", \"~> 7.0\"\nend\nend\n" + ); + let (refusal, dir) = takeover_preflight(&gemfile).await; + assert_eq!(refusal, None); + std::fs::remove_dir_all(dir).ok(); + } + // ── #779: a gem outside the lock's first GEM section ────────────────── /// Bundler 2.2+ writes one GEM section per rubygems source, sorted by