diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 600d7387f..b1d5eb80b 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -185,7 +185,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **Hosted-state visibility (`redirectState`, additive/MINOR).** Every non-hosted-mode, non-vendored-mode `scan --json` SUCCESS envelope (report-only, `--mode agent`/`--sync`, and the zero-discovery envelope) carries an additive top-level `redirectState` object whenever the project's lockfiles pin ≥ 1 hosted patch: `{ mode, records: [{purl, uuid}], wiringLive: [purl] }`. It is a descriptive STATE block, not a warning. `mode` is the constant `"hosted"`. **v5.0 (MAJOR shape change)**: hosted mode keeps no ledger, so `records` lists the hosted pins lockfile discovery finds (one per `(purl, uuid)`, the same discovery `vex` uses: a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` origin), and the v4 `ledger` and `records[].ledgerKey` keys are gone. Each record's `purl` is CANONICALIZED (qualifiers stripped, percent-decoded — e.g. `pkg:npm/@scope/pkg@1.0.0`, `pkg:gem/nokogiri@1.13.3`) to the same spelling `wiringLive` carries, so the join is a plain string compare. `wiringLive` is the subset of those pins among this run's *counted* purls (post-`--ecosystems`-filter) — computed once per run, the same set that feeds `hosted_wiring_retained`. A record missing from `wiringLive` is still wired; it just was not crawled/queried this run (an `--ecosystems` filter, a zero discovery). The key is omitted when no lockfile pins a hosted patch (and under `--global`), and error envelopes (the `--offline` refusal, all-batches-failed) are deliberately minimal and never carry it. A pre-v5 `.socket/vendor/redirect-state.json` is not read. Hosted-mode runs carry the `redirect` sub-object instead (the run's own result), and vendored-mode runs carry the takeover warnings (their takeovers may restore pins mid-run) — neither duplicates a pre-run snapshot that could go stale. -**Agent-flow run-level warnings (additive).** An agent-mode apply (`--mode agent` / `--sync`, `--json`) may add a top-level `warnings[]` array of `{code, detail}` entries to the scan envelope (absent when none fired; each is also mirrored to stderr unless `--silent`). They surface cross-mode state the apply cannot change — never a status or exit-code change (hosted refusals set the precedent: exit 0 + warning). Codes (stable; new codes are additive/MINOR): `vendored_ownership_retained` — vendor-owned package(s) were skipped before download (the per-patch `skipped`/`vendored` records in `apply.patches[]` are unchanged); the detail names the purls and the migration path (`remove `, or `vendor --revert` which unwinds every vendored package, then re-run). `hosted_wiring_retained` — the lockfiles still pin scanned package(s) to a hosted patch (the agent run does not unwind hosted wiring — as of v5.0 that is `socket-patch rollback`'s job, which restores the upstream registry entries, or `remove ` per package); the detail names the purls and the options (stay `--mode hosted`, migrate via `scan --mode vendored`, or `socket-patch rollback`). The warning keys on the hosted pins lockfile discovery finds at scan time, so a flow that restored the upstream entries retires it. The human path prints the same `hosted_wiring_retained` text to stderr after an apply; the vendored counterpart is already covered by its per-package `[skip] … (vendored …)` lines. `ownership_not_restored` (v5.0; `apply` and `rollback` `warnings[]` alike) — a file WAS patched (or restored) but its ownership could not be put back to the original uid/gid (the mode is still restored last); the detail is `: : patched, but ownership could not be restored to uid N gid M: ` and the human line `Warning: ` (stderr, muted by `--silent`); never a status or exit change. +**Agent-flow run-level warnings (additive).** An agent-mode apply (`--mode agent` / `--sync`, `--json`) may add a top-level `warnings[]` array of `{code, detail}` entries to the scan envelope (absent when none fired; each is also mirrored to stderr unless `--silent`). They surface cross-mode state the apply cannot change — never a status or exit-code change (hosted refusals set the precedent: exit 0 + warning). Codes (stable; new codes are additive/MINOR): `vendored_ownership_retained` — vendor-owned package(s) were skipped before download (the per-patch `skipped`/`vendored` records in `apply.patches[]` are unchanged); the detail names the purls and the migration path (`remove `, or `vendor --revert` which unwinds every vendored package, then re-run). `hosted_wiring_retained` — the lockfiles still pin scanned package(s) to a hosted patch (the agent run does not unwind hosted wiring — as of v5.0 that is `socket-patch rollback`'s job, which restores the upstream registry entries, or `remove ` per package); the detail names the purls and the options (stay `--mode hosted`, migrate via `scan --mode vendored`, or `socket-patch rollback`). The warning keys on the hosted pins lockfile discovery finds at scan time, so a flow that restored the upstream entries retires it. The human path prints the same `hosted_wiring_retained` text to stderr after an apply; the vendored counterpart is already covered by its per-package `[skip] … (vendored …)` lines. `ownership_not_restored` (v5.0; `apply` and `rollback` `warnings[]` alike) — a file WAS patched (or restored) but its ownership could not be put back to the original uid/gid (the mode is still restored last); the detail is `: : patched, but ownership could not be restored to uid N gid M: ` and the human line `Warning: ` (stderr, muted by `--silent`); never a status or exit change. `cargo_build_cache_stale` (v5.0; `apply` and `rollback` `warnings[]` alike) — a cargo crate's bytes changed but its compiled copy in a project build directory could not be invalidated (a fingerprint directory that could not be removed, or a `build.build-dir` with a `{workspace-path-hash}` template); the detail names the crates to `cargo clean -p` before the next build, which may otherwise link the stale code; never a status or exit change. `scan --prune` opts into garbage collection. When set, `scan` removes manifest entries for packages no longer present in the crawl, then deletes orphan blob files, and every obsolete diff and package archive, from `.socket/`. A blob is an orphan when no patch left in the manifest references it: the afterHash and beforeHash blobs of every remaining patch are kept, the same retention policy `repair` uses (the beforeHash blobs are an offline rollback's only restore data, and `repair` downloads afterHash blobs only). Diff and legacy package archives are never kept, even for a remaining patch: nothing reads them any more. Off by default (v3.0) so a temporary uninstall doesn't silently destroy manifest state. Only entries whose ecosystem this run actually crawled are eligible: a `pkg:/` with no crawler in this build (a newer CLI's ecosystem in the committed manifest) is exempt — the crawl never looked for them, so their absence is not evidence of removal (same fail-safe as the `--ecosystems` filter, which narrows the query but never the prune's installed set). The pass also reconciles vendored state (runs FIRST, under ONE apply-lock acquisition shared with the manifest prune — lock contention skips the whole pass without failing the scan; `--lock-timeout` is honored and a lock I/O error is reported rather than swallowed; the existence gate — a manifest file OR a vendor ledger file, both cheap stats; an emptied ledger is deleted on save, so its presence is its content proxy — runs BEFORE the lock, so a bare project never gets a `.socket/`; in the vendored scan arms the pass runs AFTER the vendor step): (a) ledger entries still tracked by a manifest record (manifest-mode entries written by standalone `vendor`) whose patch is gone from the manifest are reverted — `detached` entries (every `scan`/`get --mode vendored` entry, v5.0) have no manifest record to lose and are exempt from this leg; (b) EVERY ledger entry whose dependency is no longer in the lockfile graph is reverted and any manifest entry it still had dropped (v5.0: the check is about the lockfile, not the manifest, so embedded-record entries are no longer exempt; a missing or undeterminable lockfile keeps the entry, fail-safe); and (c) orphan `.socket/vendor//` dirs with no ledger entry are swept. The prune never deletes a zero-patch `.socket/manifest.json` (its `{"patches": {}}` + `setup` block stay). The JSON `gc` sub-object gains `revertedVendoredEntries` + `keptVendoredEntries` + `failedVendoredEntries` + `removedVendorOrphanDirs` (wet) / `revertableVendoredEntries` + `vendorOrphanDirs` (preview), plus two ADDITIVE wet-only keys: `skipped: {code, message}` — present exactly when the pass was skipped at the lock (`lock_held` | `lock_io`; every count is then zero) — and `warnings: [{code, detail}]` — `vendor_state_write_failed` / `manifest_write_failed` (entries were reverted but the ledger or manifest rewrite failed), `cleanup_failed` (an orphan sweep failed mid-way), and the reinstall advisories of the vendored reverts (`vendor_bun_reinstall_required`, `vendor_vlt_reinstall_required`; a revert's other warnings, such as `vendor_lock_entry_removed` or a drift keep's, are not repeated here). Human mode prints `GC: skipped (): .`, one `GC: .` line per warning, and `GC: failed to revert N vendored entries: …` (singular for one) for `failedVendoredEntries`. `keptVendoredEntries` lists drift-kept entries the revert deliberately preserved (`vendor_artifact_kept` — undo the drift and re-run `vendor --revert` to finish); the preview cannot see drift (backends return before the wiring replay on dry runs), so `revertableVendoredEntries` may over-promise what a wet run will actually reclaim. @@ -534,7 +534,11 @@ are patched by `socket-patch apply` like every other ecosystem: `.cargo-checksum.json` sidecar is rewritten so `cargo build` accepts the modified files. Rollback restores the original bytes from the `beforeHash` blobs. *(Note: a non-vendored crate patches the **shared** registry cache, which affects other projects on the machine and is reset by `cargo clean` - / a cache prune. Vendor the dependency for a project-local, committable patch.)* + / a cache prune. Vendor the dependency for a project-local, committable patch. After apply or + rollback changes a crate's bytes, that crate's fingerprints in the project's build directories are + deleted so the next `cargo build` recompiles it instead of reusing the cached rlib — cargo never + re-checks registry or `cargo vendor` sources — see docs/ecosystems.md "Cargo: shared registry + cache"; what cannot be invalidated is a `cargo_build_cache_stale` warning.)* - **golang** — `apply` writes a project-local **patched copy** under `.socket/go-patches/@/` and a `go.mod` `replace` directive pointing at it; `go build` links the copy (the module cache is `go.sum`-verified, so in-place patching can't build). Commit `go.mod` + `.socket/go-patches/` + your @@ -1037,7 +1041,7 @@ v5.0 replaces v4's per-purl reverts and whole-ledger reverse replay (`revert_rem | Key | Shape | Meaning | |---|---|---| | `error` | `{code, message}` | Only on `status: "error"` (v5.0, MAJOR: was a string). Codes: `manifest_not_found`, `manifest_invalid`, `manifest_unreadable`, `patch_not_found`, `path_glob_no_match`, `hosted_wiring_contested`, `vendor_ledger_missing`, `rollback_failed`, `lock_held` / `lock_io`, and `path_glob_invalid` (a usage error, exit 2). Per-result `results[*].error` stays a string. | -| `warnings` | `[{code, detail}]` | Run-level warnings, now populated (previously always empty): `reinstall_required`, `hosted_state_not_preservable`, `out_of_scope_copies_restored`, `vendor_state_unreadable`, `cleanup_failed`, `manifest_write_failed`, `legacy_redirect_ledger_kept`, the upstream-restore advisories (`npm_allow_remote_left`, `pnpm_trust_lockfile_left`, `maven_trusted_checksums_left`, `nuget_default_config_left`, `upstream_uv_override_removed`, `upstream_registry_fallback`, `upstream_pnpm_tarball_setting_guessed`, `upstream_gem_stale_cache`, `yarn_berry_node_gyp_unresolved`, `hosted_resolution_orphaned`), `ownership_not_restored` (a restored file whose ownership could not be put back — see the apply warnings), `rollback_record_superseded` (a manifest record superseded by a live hosted pin, left to the hosted leg — see Manifest cleanup), plus vendored/hosted leg advisories. New codes are additive (MINOR) | +| `warnings` | `[{code, detail}]` | Run-level warnings, now populated (previously always empty): `reinstall_required`, `hosted_state_not_preservable`, `out_of_scope_copies_restored`, `vendor_state_unreadable`, `cleanup_failed`, `manifest_write_failed`, `legacy_redirect_ledger_kept`, the upstream-restore advisories (`npm_allow_remote_left`, `pnpm_trust_lockfile_left`, `maven_trusted_checksums_left`, `nuget_default_config_left`, `upstream_uv_override_removed`, `upstream_registry_fallback`, `upstream_pnpm_tarball_setting_guessed`, `upstream_gem_stale_cache`, `yarn_berry_node_gyp_unresolved`, `hosted_resolution_orphaned`), `ownership_not_restored` (a restored file whose ownership could not be put back — see the apply warnings), `cargo_build_cache_stale` (see the apply warnings), `rollback_record_superseded` (a manifest record superseded by a live hosted pin, left to the hosted leg — see Manifest cleanup), plus vendored/hosted leg advisories. New codes are additive (MINOR) | | `vendored` | `[purl]` | **Meaning narrowed (MAJOR)**: vendor-owned purls the run did NOT act on — today exactly the corrupt-vendor-ledger skip. | | `vendoredReverted` | `[purl]` | Ledger entries cleanly reverted this run (unwired + artifact deleted + entry dropped; previewed on dry-run) | | `vendoredPreserved` | `[purl]` | `--preserve-state`: unwired with artifact + ledger entry kept | diff --git a/crates/socket-patch-cli/src/commands/apply.rs b/crates/socket-patch-cli/src/commands/apply.rs index e1aecdf32..8846acf60 100644 --- a/crates/socket-patch-cli/src/commands/apply.rs +++ b/crates/socket-patch-cli/src/commands/apply.rs @@ -1174,6 +1174,31 @@ pub(crate) async fn run_locked( }) })); + // Cargo keys a registry / `cargo vendor` crate's compiled + // artifacts on its package id, never its source bytes: a + // project built before this apply would keep linking the + // pre-patch rlib (#387). Invalidate this project's compiled + // copies of every crate whose bytes just changed. + // The warnings also ride the report, for a nested caller's + // envelope (`scan` / `get`). + let mut cache_warnings: Vec = Vec::new(); + if !args.common.dry_run { + let rewritten = results.iter().filter(|r| { + r.success + && !r.files_patched.is_empty() + && r.package_path != VENDOR_OWNED_MARKER + }); + cache_warnings.extend( + socket_patch_core::utils::cargo_build_cache::invalidate_project( + &args.common.cwd, + rewritten.map(|r| r.package_key.as_str()), + ) + .into_iter() + .map(|(code, detail)| RunWarning { code, detail }), + ); + run_warnings.extend(cache_warnings.iter().cloned()); + } + // Run-level advisories + best-effort fallback-home skips on the // human path: one gated stderr line each. `--silent` is // errors-only, and under `--json` the envelope copies below are @@ -1405,7 +1430,8 @@ pub(crate) async fn run_locked( // The mismatch overwrites, for a nested caller's envelope (the // JSON events above are the standalone apply's copy). - let warnings = mismatch_overwrite_warnings(&results, args.common.dry_run); + let mut warnings = mismatch_overwrite_warnings(&results, args.common.dry_run); + warnings.extend(cache_warnings); // A requested-but-failed VEX flips an otherwise-successful // apply to a non-zero exit (fail-the-command contract). if success && !vex_failed { diff --git a/crates/socket-patch-cli/src/commands/rollback.rs b/crates/socket-patch-cli/src/commands/rollback.rs index 27dc12e8f..8b673ab3f 100644 --- a/crates/socket-patch-cli/src/commands/rollback.rs +++ b/crates/socket-patch-cli/src/commands/rollback.rs @@ -2661,6 +2661,20 @@ pub(crate) async fn rollback_patches_inner( results.push(result); } + // The restored crates' compiled copies in this project's build cache + // still hold the patched code cargo keyed on the package id (#387). + if !common.dry_run { + warnings.extend( + socket_patch_core::utils::cargo_build_cache::invalidate_project( + &common.cwd, + results + .iter() + .filter(|r| r.success && !r.files_rolled_back.is_empty()) + .map(|r| r.package_key.as_str()), + ), + ); + } + superseded_left.sort(); superseded_left.dedup(); Ok(RollbackOutcome { diff --git a/crates/socket-patch-cli/tests/e2e_safety_cargo_build.rs b/crates/socket-patch-cli/tests/e2e_safety_cargo_build.rs index 1df6e17d6..66259e941 100644 --- a/crates/socket-patch-cli/tests/e2e_safety_cargo_build.rs +++ b/crates/socket-patch-cli/tests/e2e_safety_cargo_build.rs @@ -1069,3 +1069,142 @@ fn apply_normalizes_package_prefix_in_cargo_checksum() { "sidecar record must still report .cargo-checksum.json:rewritten; got {cargo}" ); } + +// ── Warm build cache (#387) ─────────────────────────────────────────── + +/// Where the warm-cache scenario's build artifacts live. +enum TargetDirSpec { + /// cargo's default `/target`. + Default, + /// `CARGO_TARGET_DIR` pointing outside the project, seen by both cargo + /// and socket-patch. + Env, + /// `build.target-dir` in the project's `.cargo/config.toml`. + Config, +} + +/// `cargo run --offline --frozen` WITHOUT wiping the build cache: what a +/// developer checkout or a CI job with a cached `target/` does next. +fn warm_run(consumer: &Path, cargo_home: &Path, target_env: Option<&Path>) -> String { + let mut cmd = cargo_e2e_matrix::cargo_command(consumer, cargo_home); + cmd.env_remove("CARGO_BUILD_TARGET_DIR") + .env_remove("CARGO_BUILD_BUILD_DIR") + .args(["run", "-q", "--offline", "--frozen"]); + if let Some(t) = target_env { + cmd.env("CARGO_TARGET_DIR", t); + } + let out = cmd.output().expect("cargo run"); + assert!( + out.status.success(), + "cargo run failed:\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr), + ); + String::from_utf8_lossy(&out.stdout).trim().to_string() +} + +/// #387: build, apply, build — the second build must link the patched +/// crate, not reuse the pre-apply rlib cargo keyed on the package id; then +/// rollback, build — it must link the original again. +fn warm_cache_round_trip(spec: TargetDirSpec) { + if !cargo_e2e_matrix::cargo_available("e2e_safety_cargo_build") { + return; + } + let root = tempfile::tempdir().unwrap(); + let consumer = stage_consumer(root.path()); + let cargo_home = root.path().join(".cargo-home"); + generate_lockfile(&consumer, &cargo_home); + + let ext_target = root.path().join("ext-target"); + let target_env = match spec { + TargetDirSpec::Env => Some(ext_target.as_path()), + _ => None, + }; + if let TargetDirSpec::Config = spec { + let cfg = consumer.join(".cargo/config.toml"); + let text = std::fs::read_to_string(&cfg).unwrap(); + std::fs::write( + &cfg, + format!("[build]\ntarget-dir = \"cfg-target\"\n\n{text}"), + ) + .unwrap(); + } + // socket-patch sees the same CARGO_TARGET_DIR as cargo (an empty value + // reads as unset — the ambient one must not leak in). + let ext = ext_target.to_string_lossy().to_string(); + let sp_env: Vec<(&str, &str)> = vec![ + ("CARGO_HOME", cargo_home.to_str().unwrap()), + ( + "CARGO_TARGET_DIR", + if target_env.is_some() { + ext.as_str() + } else { + "" + }, + ), + ("CARGO_BUILD_TARGET_DIR", ""), + ("CARGO_BUILD_BUILD_DIR", ""), + ]; + + assert_eq!(warm_run(&consumer, &cargo_home, target_env), "world"); + let built_in = match spec { + TargetDirSpec::Default => consumer.join("target"), + TargetDirSpec::Env => ext_target.clone(), + TargetDirSpec::Config => consumer.join("cfg-target"), + }; + assert!( + built_in.is_dir(), + "the baseline build must populate {built_in:?}" + ); + + let (before, _after) = stage_socket_manifest(&consumer); + write_blob( + &consumer.join(".socket"), + &before, + ORIGINAL_LIB_RS.as_bytes(), + ); + let cwd = consumer.to_str().unwrap(); + let (code, stdout, stderr) = + common::run_with_env(&consumer, &["apply", "--json", "--cwd", cwd], &sp_env); + assert_eq!(code, 0, "apply failed:\n{stdout}\n{stderr}"); + let env = parse_json_envelope(&stdout); + assert_eq!( + env["warnings"].as_array().map(Vec::len).unwrap_or(0), + 0, + "a fully invalidated build cache warns about nothing: {env}" + ); + assert_eq!( + warm_run(&consumer, &cargo_home, target_env), + "PATCHED", + "the next build after apply must compile the patched sources, not reuse the \ + pre-apply rlib" + ); + + let (code, stdout, stderr) = + common::run_with_env(&consumer, &["rollback", "--json", "--cwd", cwd], &sp_env); + assert_eq!(code, 0, "rollback failed:\n{stdout}\n{stderr}"); + assert_eq!( + warm_run(&consumer, &cargo_home, target_env), + "world", + "the next build after rollback must compile the original sources, not reuse the \ + patched rlib" + ); +} + +#[test] +#[ignore] +fn warm_build_cache_relinks_after_apply_and_rollback() { + warm_cache_round_trip(TargetDirSpec::Default); +} + +#[test] +#[ignore] +fn warm_build_cache_in_cargo_target_dir_relinks() { + warm_cache_round_trip(TargetDirSpec::Env); +} + +#[test] +#[ignore] +fn warm_build_cache_in_config_target_dir_relinks() { + warm_cache_round_trip(TargetDirSpec::Config); +} diff --git a/crates/socket-patch-core/src/utils/cargo_build_cache.rs b/crates/socket-patch-core/src/utils/cargo_build_cache.rs new file mode 100644 index 000000000..b12ce8f4e --- /dev/null +++ b/crates/socket-patch-core/src/utils/cargo_build_cache.rs @@ -0,0 +1,486 @@ +//! Invalidate cargo's compiled artifacts for crates whose sources an +//! in-place (agent-mode) apply or rollback just rewrote (#387). +//! +//! Cargo treats registry and directory-source (`cargo vendor`) packages as +//! immutable: their build fingerprint is keyed on the package id, never on +//! the source files' mtimes or contents. A project built before `apply` +//! therefore keeps linking the pre-patch `lib-*.rlib` from `target/` +//! — and after `rollback`, keeps linking the patched one. Deleting the +//! crate's fingerprint directories (`/[/]/ +//! .fingerprint/-<16 hex>/`) makes the unit dirty, so the next build +//! recompiles it from the bytes on disk, and every dependent relinks +//! because the rebuilt output is newer than theirs. +//! +//! Only build directories reachable from the project are known here: the +//! `CARGO_TARGET_DIR` / `CARGO_BUILD_TARGET_DIR` / `CARGO_BUILD_BUILD_DIR` +//! overrides, `build.target-dir` / `build.build-dir` in the `.cargo/config` +//! files cargo reads for the project, and `/target`. Every +//! version of the crate's name is invalidated (the fingerprint directory +//! does not record which version it built): a spurious rebuild is cheap, +//! linking stale code is not. + +use std::path::{Path, PathBuf}; + +/// The build directories of the project at `cwd`, and the config values +/// naming one this module cannot resolve (a `{workspace-path-hash}` +/// template) — the caller tells the user to clean those by hand. +#[derive(Debug, Default, PartialEq, Eq)] +pub struct BuildDirs { + /// Existing directories, deduplicated, in discovery order. + pub dirs: Vec, + /// Configured build directories that could not be resolved. + pub unresolved: Vec, +} + +/// What [`invalidate`] did. +#[derive(Debug, Default)] +pub struct Invalidation { + /// Fingerprint directories removed. + pub removed: Vec, + /// Fingerprint directories that could not be removed, with the error. + pub failed: Vec<(PathBuf, String)>, +} + +/// The project's build directories, reading the process environment. +pub fn build_dirs(cwd: &Path) -> BuildDirs { + build_dirs_with(cwd, &|k| std::env::var(k).ok(), cargo_home().as_deref()) +} + +fn cargo_home() -> Option { + match std::env::var("CARGO_HOME") { + Ok(v) if !v.trim().is_empty() => Some(PathBuf::from(v)), + _ => crate::utils::fs::home_dir().map(|h| h.join(".cargo")), + } +} + +/// [`build_dirs`] with an injected environment and `CARGO_HOME`. +pub fn build_dirs_with( + cwd: &Path, + env: &dyn Fn(&str) -> Option, + cargo_home: Option<&Path>, +) -> BuildDirs { + let env = |k: &str| env(k).filter(|v| !v.trim().is_empty()); + let ws_root = workspace_root(cwd); + let config = ConfigDirs::read(cwd, cargo_home); + + let mut candidates: Vec = Vec::new(); + let mut unresolved = Vec::new(); + + // target-dir: env beats config beats the default. + let target_dir = env("CARGO_TARGET_DIR") + .or_else(|| env("CARGO_BUILD_TARGET_DIR")) + .map(|v| cwd.join(v)) + .or_else(|| config.target_dir.clone()) + .unwrap_or_else(|| ws_root.join("target")); + candidates.push(target_dir.clone()); + + // build-dir (cargo 1.91+): where the fingerprints live; defaults to + // the target dir. + let build_dir = env("CARGO_BUILD_BUILD_DIR") + .map(|v| (v, cwd.to_path_buf())) + .or_else(|| config.build_dir.clone()); + if let Some((raw, base)) = build_dir { + match expand_build_dir(&raw, &ws_root, cargo_home) { + Some(p) => candidates.push(base.join(p)), + None => unresolved.push(raw), + } + } + + // The defaults too, in case a build ran without the override this + // process sees (a shell profile vs. CI env): over-invalidating only + // costs a recompile. + candidates.push(ws_root.join("target")); + candidates.push(cwd.join("target")); + + let mut dirs: Vec = Vec::new(); + for c in candidates { + let c = std::fs::canonicalize(&c).unwrap_or(c); + if c.is_dir() && !dirs.contains(&c) { + dirs.push(c); + } + } + BuildDirs { dirs, unresolved } +} + +/// `{workspace-root}` / `{cargo-cache-home}` substituted; `None` for a +/// template naming anything else (`{workspace-path-hash}`). +fn expand_build_dir(raw: &str, ws_root: &Path, cargo_home: Option<&Path>) -> Option { + let mut s = raw.replace("{workspace-root}", &ws_root.to_string_lossy()); + if s.contains("{cargo-cache-home}") { + s = s.replace("{cargo-cache-home}", &cargo_home?.to_string_lossy()); + } + if s.contains('{') { + return None; + } + Some(PathBuf::from(s)) +} + +/// The nearest ancestor of `cwd` holding a `Cargo.lock` (cargo writes it +/// beside the workspace root manifest), else the nearest one holding a +/// `[workspace]` manifest, else `cwd`. +fn workspace_root(cwd: &Path) -> PathBuf { + for dir in cwd.ancestors() { + if dir.join("Cargo.lock").is_file() { + return dir.to_path_buf(); + } + } + for dir in cwd.ancestors() { + let manifest = dir.join("Cargo.toml"); + if let Ok(text) = crate::utils::fs::read_regular_to_string_sync(&manifest) { + if text + .parse::() + .is_ok_and(|d| d.contains_key("workspace")) + { + return dir.to_path_buf(); + } + } + } + cwd.to_path_buf() +} + +/// `build.target-dir` / `build.build-dir` from the config files cargo +/// reads for `cwd`: `.cargo/config.toml` (or legacy `.cargo/config`) in +/// `cwd` and every ancestor, then `$CARGO_HOME/config.toml`; the deepest +/// file that sets a key wins. A relative value resolves against the +/// directory holding the `.cargo` directory. +#[derive(Default)] +struct ConfigDirs { + target_dir: Option, + /// The raw value and the base its relative form resolves against. + build_dir: Option<(String, PathBuf)>, +} + +impl ConfigDirs { + fn read(cwd: &Path, cargo_home: Option<&Path>) -> Self { + let mut files: Vec<(PathBuf, PathBuf)> = cwd + .ancestors() + .map(|d| (d.join(".cargo"), d.to_path_buf())) + .collect(); + if let Some(home) = cargo_home { + let base = home.parent().map(Path::to_path_buf).unwrap_or_default(); + files.push((home.to_path_buf(), base)); + } + let mut out = ConfigDirs::default(); + for (dot_cargo, base) in files { + let text = ["config.toml", "config"].iter().find_map(|f| { + crate::utils::fs::read_regular_to_string_sync(&dot_cargo.join(f)).ok() + }); + let Some(doc) = text.and_then(|t| t.parse::().ok()) else { + continue; + }; + let Some(build) = doc.get("build") else { + continue; + }; + if out.target_dir.is_none() { + if let Some(v) = build.get("target-dir").and_then(|v| v.as_str()) { + out.target_dir = Some(base.join(v)); + } + } + if out.build_dir.is_none() { + if let Some(v) = build.get("build-dir").and_then(|v| v.as_str()) { + out.build_dir = Some((v.to_string(), base.clone())); + } + } + } + out + } +} + +/// Crate names compare the way crates.io does: ASCII case-insensitive, +/// `-` and `_` equivalent. +fn same_crate(a: &str, b: &str) -> bool { + a.len() == b.len() + && a.bytes().zip(b.bytes()).all(|(x, y)| { + let n = |c: u8| { + if c == b'_' { + b'-' + } else { + c.to_ascii_lowercase() + } + }; + n(x) == n(y) + }) +} + +/// Whether `dir_name` is a fingerprint directory of `crate_name`: +/// `-<16 lowercase hex>`. +fn is_fingerprint_of(dir_name: &str, crate_name: &str) -> bool { + let Some((name, hash)) = dir_name.rsplit_once('-') else { + return false; + }; + hash.len() == 16 + && hash + .bytes() + .all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)) + && same_crate(name, crate_name) +} + +/// Whether `dir_name` is a unit-hash directory of the build-dir layout +/// cargo uses since `-Zbuild-dir-new-layout` (`<16 lowercase hex>`). +fn is_unit_hash(dir_name: &str) -> bool { + dir_name.len() == 16 + && dir_name + .bytes() + .all(|b| b.is_ascii_digit() || (b'a'..=b'f').contains(&b)) +} + +fn subdirs(p: &Path) -> Vec { + std::fs::read_dir(p) + .into_iter() + .flatten() + .flatten() + .filter(|e| e.file_type().is_ok_and(|t| t.is_dir())) + .map(|e| e.path()) + .collect() +} + +/// Every profile directory under `build_dir` (`/` and +/// `//`): one holding a `.fingerprint` directory (the +/// classic layout) or a `build/` directory (the new layout). +fn profile_dirs(build_dir: &Path) -> Vec { + let is_profile = |p: &Path| p.join(".fingerprint").is_dir() || p.join("build").is_dir(); + let mut out = Vec::new(); + for level1 in subdirs(build_dir) { + if is_profile(&level1) { + out.push(level1); + continue; + } + out.extend(subdirs(&level1).into_iter().filter(|p| is_profile(p))); + } + out.sort(); + out +} + +/// The fingerprint directories of `crate_names` in one profile directory, +/// in both layouts cargo writes: +/// * classic: `.fingerprint/-<16 hex>/`; +/// * new (`-Zbuild-dir-new-layout`, the nightly default since 1.100): +/// `build//<16 hex>/fingerprint/`. +fn crate_fingerprints(profile: &Path, crate_names: &[String]) -> Vec { + let named = |p: &Path, pred: &dyn Fn(&str, &str) -> bool| { + p.file_name() + .map(|n| n.to_string_lossy().into_owned()) + .is_some_and(|n| crate_names.iter().any(|c| pred(&n, c))) + }; + let mut hits: Vec = subdirs(&profile.join(".fingerprint")) + .into_iter() + .filter(|p| named(p, &is_fingerprint_of)) + .collect(); + for pkg in subdirs(&profile.join("build")) { + if !named(&pkg, &same_crate) { + continue; + } + for unit in subdirs(&pkg) { + let fp = unit.join("fingerprint"); + if unit + .file_name() + .is_some_and(|n| is_unit_hash(&n.to_string_lossy())) + && fp.is_dir() + { + hits.push(fp); + } + } + } + hits.sort(); + hits +} + +/// Remove every fingerprint directory of `crate_names` under `dirs`. +pub fn invalidate(dirs: &[PathBuf], crate_names: &[String]) -> Invalidation { + let mut out = Invalidation::default(); + if crate_names.is_empty() { + return out; + } + for dir in dirs { + for profile in profile_dirs(dir) { + for hit in crate_fingerprints(&profile, crate_names) { + match std::fs::remove_dir_all(&hit) { + Ok(()) => out.removed.push(hit), + Err(e) => out.failed.push((hit, e.to_string())), + } + } + } + } + out +} + +/// The run-warning code for a crate whose compiled copy could not be +/// invalidated: the next build may link the stale one. +pub const STALE_WARNING: &str = "cargo_build_cache_stale"; + +/// Invalidate the project's compiled copies of the cargo crates in +/// `purls` (non-cargo purls are ignored). Returns the run warnings +/// `(code, detail)` for what could not be invalidated, each naming the +/// `cargo clean -p` remedy. +pub fn invalidate_project<'a>( + cwd: &Path, + purls: impl IntoIterator, +) -> Vec<(String, String)> { + let mut names: Vec = purls + .into_iter() + .filter_map(cargo_purl_name) + .map(str::to_string) + .collect(); + names.sort(); + names.dedup(); + if names.is_empty() { + return Vec::new(); + } + let dirs = build_dirs(cwd); + let inv = invalidate(&dirs.dirs, &names); + let mut warnings = Vec::new(); + for (path, err) in &inv.failed { + warnings.push(( + STALE_WARNING.to_string(), + format!( + "could not invalidate cargo's compiled copy at {} ({err}); the next build may \ + link the stale code. Run `cargo clean -p ` for: {}", + path.display(), + names.join(", ") + ), + )); + } + for raw in &dirs.unresolved { + warnings.push(( + STALE_WARNING.to_string(), + format!( + "cargo build-dir `{raw}` could not be resolved, so its compiled copies were not \ + invalidated; the next build may link the stale code. Run `cargo clean -p \ + ` for: {}", + names.join(", ") + ), + )); + } + warnings +} + +/// The crate name of a `pkg:cargo/@` purl. +pub fn cargo_purl_name(purl: &str) -> Option<&str> { + let rest = purl.strip_prefix("pkg:cargo/")?; + let end = rest.find(['@', '?', '#']).unwrap_or(rest.len()); + let name = &rest[..end]; + (!name.is_empty()).then_some(name) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn mkfp(root: &Path, rel: &str) -> PathBuf { + let p = root.join(rel); + std::fs::create_dir_all(&p).unwrap(); + std::fs::write(p.join("lib-x.json"), "{}").unwrap(); + p + } + + #[test] + fn invalidates_only_the_named_crate_in_every_profile_and_triple() { + let t = tempfile::tempdir().unwrap(); + let target = t.path().join("target"); + let a = mkfp(&target, "debug/.fingerprint/cfg-if-0123456789abcdef"); + let b = mkfp(&target, "release/.fingerprint/cfg-if-fedcba9876543210"); + let c = mkfp( + &target, + "x86_64-unknown-linux-gnu/debug/.fingerprint/cfg_if-00000000000000aa", + ); + let keep1 = mkfp(&target, "debug/.fingerprint/cfg-if-extra-0123456789abcdef"); + let keep2 = mkfp(&target, "debug/.fingerprint/app-0123456789abcdef"); + let keep3 = mkfp(&target, "debug/.fingerprint/cfg-if-notahash"); + let inv = invalidate(std::slice::from_ref(&target), &["cfg-if".to_string()]); + assert!(inv.failed.is_empty()); + assert_eq!(inv.removed.len(), 3, "{:?}", inv.removed); + for p in [a, b, c] { + assert!(!p.exists(), "{p:?} must be removed"); + } + for p in [keep1, keep2, keep3] { + assert!(p.exists(), "{p:?} must be kept"); + } + } + + /// The new build-dir layout (`build///fingerprint/`, the + /// nightly default) is invalidated too; the classic layout's build-script + /// dirs (`build/-/`) and other crates are left alone. + #[test] + fn invalidates_the_new_build_dir_layout() { + let t = tempfile::tempdir().unwrap(); + let target = t.path().join("target"); + let a = mkfp(&target, "debug/build/cfg-if/9a1d16c6d1ca6d4f/fingerprint"); + let b = mkfp( + &target, + "aarch64-apple-darwin/release/build/cfg_if/00000000000000aa/fingerprint", + ); + let out = mkfp(&target, "debug/build/cfg-if/9a1d16c6d1ca6d4f/out"); + let keep1 = mkfp(&target, "debug/build/app/9a1d16c6d1ca6d4f/fingerprint"); + let keep2 = mkfp( + &target, + "debug/build/cfg-if-extra/9a1d16c6d1ca6d4f/fingerprint", + ); + let keep3 = mkfp(&target, "debug/build/cfg-if-0123456789abcdef/out"); + let inv = invalidate(std::slice::from_ref(&target), &["cfg-if".to_string()]); + assert!(inv.failed.is_empty()); + assert_eq!(inv.removed.len(), 2, "{:?}", inv.removed); + for p in [a, b] { + assert!(!p.exists(), "{p:?} must be removed"); + } + for p in [out, keep1, keep2, keep3] { + assert!(p.exists(), "{p:?} must be kept"); + } + } + + #[test] + fn build_dirs_honours_env_config_and_workspace_root() { + let t = tempfile::tempdir().unwrap(); + let root = t.path().join("ws"); + let member = root.join("member"); + std::fs::create_dir_all(member.join("target")).unwrap(); + std::fs::create_dir_all(root.join("target")).unwrap(); + std::fs::write(root.join("Cargo.lock"), "").unwrap(); + let none = |_: &str| None; + let d = build_dirs_with(&member, &none, None); + let canon = |p: PathBuf| std::fs::canonicalize(p).unwrap(); + assert_eq!( + d.dirs, + vec![canon(root.join("target")), canon(member.join("target"))] + ); + + // build.target-dir in an ancestor config, relative to its base. + std::fs::create_dir_all(root.join(".cargo")).unwrap(); + std::fs::create_dir_all(root.join("out")).unwrap(); + std::fs::write( + root.join(".cargo/config.toml"), + "[build]\ntarget-dir = \"out\"\nbuild-dir = \"{workspace-root}/bd\"\n", + ) + .unwrap(); + std::fs::create_dir_all(root.join("bd")).unwrap(); + let d = build_dirs_with(&member, &none, None); + assert_eq!(d.dirs[0], canon(root.join("out"))); + assert_eq!(d.dirs[1], canon(root.join("bd"))); + + // The env beats the config. + let ext = t.path().join("ext"); + std::fs::create_dir_all(&ext).unwrap(); + let ext_s = ext.to_string_lossy().to_string(); + let env = move |k: &str| (k == "CARGO_TARGET_DIR").then(|| ext_s.clone()); + let d = build_dirs_with(&member, &env, None); + assert_eq!(d.dirs[0], canon(ext.clone())); + + // An unresolvable template is reported. + std::fs::write( + root.join(".cargo/config.toml"), + "[build]\nbuild-dir = \"{cargo-cache-home}/b/{workspace-path-hash}\"\n", + ) + .unwrap(); + let d = build_dirs_with(&member, &none, Some(t.path())); + assert_eq!( + d.unresolved, + vec!["{cargo-cache-home}/b/{workspace-path-hash}"] + ); + } + + #[test] + fn purl_name() { + assert_eq!(cargo_purl_name("pkg:cargo/cfg-if@1.0.0"), Some("cfg-if")); + assert_eq!(cargo_purl_name("pkg:cargo/a?x=1"), Some("a")); + assert_eq!(cargo_purl_name("pkg:npm/a@1"), None); + } +} diff --git a/crates/socket-patch-core/src/utils/mod.rs b/crates/socket-patch-core/src/utils/mod.rs index b0c0edc8c..f3d49935f 100644 --- a/crates/socket-patch-core/src/utils/mod.rs +++ b/crates/socket-patch-core/src/utils/mod.rs @@ -1,3 +1,4 @@ +pub mod cargo_build_cache; pub mod cargo_workspace; pub mod composer_version; pub mod concurrent; diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 13b9f4f49..532ff6327 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -1046,6 +1046,20 @@ crate that means the **shared** `$CARGO_HOME/registry` cache: the patch affects project on the machine, and is silently reset by `cargo clean` or a cache prune. Use `--mode vendored` for a project-local, committable patch. +Cargo never re-checks a registry or `cargo vendor` crate's source files: it reuses the +crate's compiled artifacts while the package id is unchanged. So after `apply` or +`rollback` changes a crate's bytes, socket-patch deletes that crate's fingerprints +(`.fingerprint/-/`, or `build///fingerprint/` in the newer +build-dir layout; every version, profile and target triple) in the +project's build directories, and the next `cargo build` recompiles it and relinks its +dependents. Those directories are `CARGO_TARGET_DIR` / `CARGO_BUILD_TARGET_DIR`, +`CARGO_BUILD_BUILD_DIR`, `build.target-dir` / `build.build-dir` from the project's +`.cargo/config.toml` files and `$CARGO_HOME/config.toml`, and `/target`. +A fingerprint that can't be removed, or a `build.build-dir` using +`{workspace-path-hash}`, adds a `cargo_build_cache_stale` warning that names the crates +to `cargo clean -p`. Other projects that share the registry cache keep their own build +caches: run `cargo clean -p ` there too. + Run `cargo fetch` before `apply` on a fresh checkout or CI runner. Cargo unpacks a locked crate into `registry/src` only when it fetches or builds, so on a cold or pruned cache there is nothing to patch, and the next `cargo build` would compile the pristine