diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 6506b62ad..33f13a500 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -765,7 +765,7 @@ directory or tarball, a yarn classic registry or `file:` directory block of the like any other pin (#828, #935, #938, #939). The grant token of an attributed pin's own URL is never contested wiring where the same file also names that pin's patch uuid (a uv pin's paired `pyproject.toml` `[tool.uv.sources]` entry, a vlt pin in a `vlt-lock.json` whose pins -are withheld from the lock basis); any other unattributed uuid still is. A yarn berry root `package.json` `resolutions` selector routing to a Socket-hosted tarball that no live `yarn.lock` entry resolves any more, and that matches no descriptor the lock is keyed by (`yarn remove` or `yarn up` of a hosted-pinned package leaves it behind, since yarn never edits `resolutions`), is not contested either: it is Socket's own leftover pin. `list` lists around it with a `hosted_resolution_orphaned` warning, and `rollback` (unscoped, or scoped to its purl) and `remove ` retire it from `package.json` with the same warning code, leaving `yarn.lock` untouched (#1203). `--vex` works as on the manifest-driven +are withheld from the lock basis); any other unattributed uuid still is. A yarn berry root `package.json` `resolutions` selector routing to a Socket-hosted tarball that no live `yarn.lock` entry resolves any more, and that matches no descriptor the lock is keyed by (`yarn remove` or `yarn up` of a hosted-pinned package leaves it behind, since yarn never edits `resolutions`), is not contested either: it is Socket's own leftover pin. `list` lists around it with a `hosted_resolution_orphaned` warning, and `rollback` (unscoped, or scoped to its purl) and `remove ` retire it from `package.json` with the same warning code, leaving `yarn.lock` untouched (#1203). The uv twin is not contested either: a `[tool.uv] override-dependencies` entry under hosted mode's `# socket-patch hosted` comment plus its single unconditional `[tool.uv.sources]` URL (naming that release's artifact), for a package `uv.lock` (or a script lock) no longer holds — `uv remove ` or `uv lock --upgrade-package ` drops a transitive package hosted mode pinned but never edits either line. `list` lists around it with a `hosted_resolution_orphaned` warning naming the metadata file, and `rollback` / `remove` retire the override, the source and the lock's `[manifest]` record of them with the same warning code, leaving a lock `uv lock --locked` accepts; nothing is fetched. A user's own (unmarked) override, or a package the lock still holds, stays contested (#1445). `--vex` works as on the manifest-driven path. Without hosted pins the no-manifest no-op below is unchanged. **Prebuilt vendor artifacts (`--vendor-source`)**: `service` is the default and `auto` is a compatibility alias. `build` is rejected at argument parsing. There is no local construction or fallback. Package-reference POSTs on the configured API/proxy (`--vendor-url` overrides it) return the download URL and integrity; `--patch-server-url` can override the download origin. The CLI verifies transfer integrity and patched-member afterHashes before writing. Pending builds, missing artifacts, service failures, wrong layouts and integrity mismatches fail closed. Fresh acquisition requires the network; healthy committed artifacts remain reusable offline. @@ -1427,7 +1427,7 @@ command's integration tests check its output. | `manifest_write_failed` | rollback `warnings[]` | rollback (v5.0): the post-rollback manifest update could not be written; no entries were removed (no `removed` event) and the run exits `partialFailure` 1. | | `npm_allow_remote_left` / `pnpm_trust_lockfile_left` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (v5.0): no npm-family lock entry is hosted any more, but the project `.npmrc` keeps a top-level `allow-remote=all` (resp. `pnpm-workspace.yaml` keeps `trustLockfile: true`) in a file that is not exactly what hosted mode creates; the file is left untouched (v5 records no provenance), remove the line if nothing else needs it. A file that is exactly hosted mode's own is deleted silently, and a file that is a keyless document plus exactly the `packages:` scaffold and `trustLockfile: true` lines hosted mode splices into one (#1096) gets those lines removed, restoring it byte for byte. | | `maven_trusted_checksums_left` / `nuget_default_config_left` / `upstream_uv_override_removed` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (v5.0): `.mvn` config keeps the trusted-checksums resolver lines because it holds more than hosted mode writes; `nuget.config` now holds only the nuget.org source (delete it if hosted mode created it); a transitive `override-dependencies` entry hosted mode added to `pyproject.toml` (the one under its `# socket-patch hosted` comment) was removed; a user-authored override is never removed and never reported. | -| `hosted_resolution_orphaned` | rollback/remove `warnings[]`; list `warnings[]` (human: `Warning: …` on stderr) | yarn berry (#1203): the root `package.json` keeps a Socket-hosted `resolutions` selector that no live `yarn.lock` entry resolves and no lock descriptor matches (left behind by `yarn remove` / `yarn up`). `rollback` / `remove` removed it (the lock is untouched; an emptied `resolutions` object is dropped); `list` only reports it and names the scoped `remove ` that retires it alone. Never flips the exit. | +| `hosted_resolution_orphaned` | rollback/remove `warnings[]`; list `warnings[]` (human: `Warning: …` on stderr) | yarn berry (#1203): the root `package.json` keeps a Socket-hosted `resolutions` selector that no live `yarn.lock` entry resolves and no lock descriptor matches (left behind by `yarn remove` / `yarn up`). `rollback` / `remove` removed it (the lock is untouched; an emptied `resolutions` object is dropped). uv (#1445): `pyproject.toml` (or a script's PEP 723 block) keeps a hosted `override-dependencies` pin and its `[tool.uv.sources]` URL for a transitive package the lock no longer holds (left behind by `uv remove ` / `uv lock --upgrade-package `). `rollback` / `remove` removed both lines and the lock's `[manifest]` record of them (emptied tables are dropped). `list` only reports either and names the scoped `remove ` that retires it alone. Never flips the exit. | | `upstream_registry_fallback` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore: a yarn berry, pnpm or vlt entry is restored from the version document of the registry the project resolves it against (yarn berry: `npmScopes` / `YARN_NPM_REGISTRY_SERVER` / `npmRegistryServer` from the `.yarnrc.yml` chain, as above, the pnpm lock's sibling `.npmrc` `registry` / `@scope:registry` or pnpm-workspace.yaml `registry` / `registries`, vlt's node registry); that registry could not be read (e.g. it needs credentials) or, for yarn berry, could not be determined (an rc value referencing an environment variable that is set, or an unset one with no default; a flow-style `npmScopes`), so the default registry's document was used and the restored tarball URL may not be the mirror's. | | `upstream_gem_stale_cache` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#1260): a restored gem's `-.gem` is still in Bundler's cache dir (`cache_path`, default `vendor/cache`, resolved as for the Gem stale-install guard) and its sha256 is not the upstream one (the restored `CHECKSUMS` entry, else the rubygems.org compact index), or it could not be checked (`--offline`, a registry error). Bundler installs from that dir first, so a `bundle cache` taken while the hosted pin was live makes every later install fail on the upstream checksum (exit 37) or, on bundler < 2.6 frozen installs, keep installing the patched bytes. The detail names the file. Remedy: delete it, then run `bundle cache` to cache the upstream gem in its place (or `bundle install` if the project does not commit its cache; with the cache dir committed, a frozen install reads only the cache). Read-only: the restore never deletes it. Not raised for an archive whose sha256 matches upstream. | | `upstream_pnpm_tarball_setting_guessed` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#902): nothing showed which pnpm wrote a hosted `pnpm-lock.yaml` (no unpinned registry entry that shows the setting, no `node_modules/.modules.yaml` install record, no package.json `packageManager` pin; for a Rush lock, no rush.json `pnpmVersion`), so its entries were restored with or without `tarball:` by pnpm 10's reading of `lockfileIncludeTarballUrl` (pnpm-workspace.yaml, else `.npmrc` `lockfile-include-tarball-url`), and pnpm 9 (which reads only `.npmrc`) or pnpm >= 11 (which reads only pnpm-workspace.yaml) would have read it the other way. The detail names the lock, the setting followed, the pnpm that disagrees and the entries. Remedy: pin the pnpm (package.json `packageManager`, or reinstall so the install record names it; rush.json `pnpmVersion` for Rush), or give the two files the same value so every pnpm reads it alike; a rollback or remove can then be redone by restoring the lock from version control and re-running. Not raised when evidence decided, when both files read the same on every pnpm, or when the tarball is one pnpm records regardless. | diff --git a/crates/socket-patch-cli/src/commands/list.rs b/crates/socket-patch-cli/src/commands/list.rs index ed0d42416..c5969455f 100644 --- a/crates/socket-patch-cli/src/commands/list.rs +++ b/crates/socket-patch-cli/src/commands/list.rs @@ -395,18 +395,36 @@ pub async fn run(args: ListArgs) -> i32 { eprintln!("Warning: {}", crate::ui::sentence_case(detail)); } } - // A hosted `resolutions` selector no lock installs any more (#1203) is - // no patch either: say how to retire it. + // A hosted `resolutions` selector (#1203) or uv override pin (#1445) + // no lock installs any more is no patch either: say how to retire it. for pin in &inventory.stale { // Named by file and purl, as the contested refusal names files: a // hosted URL carries its grant token, which output never prints. - let detail = format!( - "{} keeps a hosted `resolutions` entry for {} that no lockfile installs any \ - more; `socket-patch remove {}` removes that entry alone", - pin.files.join(", "), - pin.purl, - pin.purl - ); + // A uv pin lives in the lock's paired metadata (and the lock's + // `[manifest]` record of it). + let uv_metadata: Vec = pin + .files + .iter() + .filter_map(|f| socket_patch_core::utils::python_lock::paired_metadata_path(f)) + .collect(); + let detail = if uv_metadata.is_empty() { + format!( + "{} keeps a hosted `resolutions` entry for {} that no lockfile installs any \ + more; `socket-patch remove {}` removes that entry alone", + pin.files.join(", "), + pin.purl, + pin.purl + ) + } else { + format!( + "{} keeps a hosted `override-dependencies` pin and `[tool.uv.sources]` URL for \ + {} that {} no longer installs; `socket-patch remove {}` removes that pin alone", + uv_metadata.join(", "), + pin.purl, + pin.files.join(", "), + pin.purl + ) + }; if args.common.json { warnings.push(RunWarning { code: "hosted_resolution_orphaned".to_string(), diff --git a/crates/socket-patch-cli/tests/in_process_rollback_hosted.rs b/crates/socket-patch-cli/tests/in_process_rollback_hosted.rs index 6d1ce8c70..a75b47608 100644 --- a/crates/socket-patch-cli/tests/in_process_rollback_hosted.rs +++ b/crates/socket-patch-cli/tests/in_process_rollback_hosted.rs @@ -2034,3 +2034,133 @@ fn remove_retires_a_berry_selector_left_by_yarn_remove() { ); } } + +// ── #1445: a uv override pin `uv remove` / `uv lock -P` left behind ───────── +// Hosted uv wires a transitive package (six, through isodate 0.6.1) with a +// marked `override-dependencies` entry plus a `[tool.uv.sources]` URL. After +// `uv lock --upgrade-package isodate` (0.7.2 has no six) or `uv remove +// isodate`, uv drops six from uv.lock but keeps both lines and the lock's +// `[manifest] overrides` record. That used to read as contested wiring: +// `list`, `rollback` and `remove` failed forever with +// `hosted_wiring_contested`, and the printed re-scan remedy changed nothing. + +const SIX_PURL: &str = "pkg:pypi/six@1.16.0"; +const SIX_UUID: &str = "66666666-6666-4666-8666-666666666666"; +const SIX_HOSTED_URL: &str = "http://patch.test/patch/pypi/six/1.16.0/22222222-2222-4222-8222-222222222222/66666666-6666-4666-8666-666666666666/six-1.16.0-py2.py3-none-any.whl"; + +/// The project's pyproject.toml before the hosted run. +const UV_PYPROJECT_PRISTINE: &str = "[project]\nname = \"demo\"\nversion = \"0.1.0\"\n\ + requires-python = \">=3.9\"\ndependencies = [\"isodate>=0.6\"]\n"; + +/// uv.lock as real uv writes it after the relock, with `manifest` (the +/// leftover `[manifest]` block) or none. +fn uv_lock_after_relock(manifest: Option<&str>) -> String { + let manifest = manifest.map(|m| format!("{m}\n\n")).unwrap_or_default(); + format!( + "version = 1\nrevision = 3\nrequires-python = \">=3.9\"\n\n{manifest}\ + [[package]]\nname = \"demo\"\nversion = \"0.1.0\"\nsource = {{ virtual = \".\" }}\n\ + dependencies = [\n {{ name = \"isodate\" }},\n]\n\n\ + [package.metadata]\nrequires-dist = [{{ name = \"isodate\", specifier = \">=0.6\" }}]\n\n\ + [[package]]\nname = \"isodate\"\nversion = \"0.7.2\"\n\ + source = {{ registry = \"https://pypi.org/simple\" }}\n\ + wheels = [\n {{ url = \"https://files.pythonhosted.org/packages/15/aa/0aca39a37d3c7eb941ba736ede56d689e7be91cab5d9ca846bde3999eba6/isodate-0.7.2-py3-none-any.whl\", hash = \"sha256:28009937d8031054830160fce6d409ed342816b543597cece116d966c6d99e15\", size = 22320, upload-time = \"2024-10-08T23:04:09.501Z\" }},\n]\n" + ) +} + +fn write_uv_override_left_by_relock(root: &Path) { + std::fs::write( + root.join("pyproject.toml"), + format!( + "{UV_PYPROJECT_PRISTINE}\n[tool.uv]\noverride-dependencies = [\n \ + # socket-patch hosted: pins a patched transitive dependency; rollback removes it\n \ + \"six==1.16.0\",\n]\n\n[tool.uv.sources]\nsix = {{ url = \"{SIX_HOSTED_URL}\" }}\n" + ), + ) + .unwrap(); + std::fs::write( + root.join("uv.lock"), + uv_lock_after_relock(Some(&format!( + "[manifest]\noverrides = [{{ name = \"six\", url = \"{SIX_HOSTED_URL}\" }}]" + ))), + ) + .unwrap(); +} + +/// Both files are back to what `uv lock` writes without the leftover. +fn assert_uv_override_retired(root: &Path, context: &str) { + assert_eq!( + std::fs::read_to_string(root.join("pyproject.toml")).unwrap(), + UV_PYPROJECT_PRISTINE, + "{context}" + ); + assert_eq!( + std::fs::read_to_string(root.join("uv.lock")).unwrap(), + uv_lock_after_relock(None), + "{context}" + ); +} + +#[test] +#[serial] +fn uv_override_left_by_relock_is_listed_around() { + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path()); + let pyproject = std::fs::read_to_string(tmp.path().join("pyproject.toml")).unwrap(); + let (code, envelope) = run_cli_json(tmp.path(), &["list"]); + assert_eq!(code, 0, "list must succeed: {envelope}"); + let warnings = envelope["warnings"].as_array().cloned().unwrap_or_default(); + let orphaned: Vec<&Value> = warnings + .iter() + .filter(|w| w["code"] == "hosted_resolution_orphaned") + .collect(); + assert_eq!(orphaned.len(), 1, "{envelope}"); + let detail = orphaned[0]["detail"].as_str().unwrap(); + assert!( + detail.contains("pyproject.toml") + && detail.contains("override-dependencies") + && detail.contains(&format!("socket-patch remove {SIX_PURL}")), + "{detail}" + ); + assert!( + !warning_codes(&envelope).contains(&"hosted_wiring_contested".to_string()), + "{envelope}" + ); + assert_eq!( + std::fs::read_to_string(tmp.path().join("pyproject.toml")).unwrap(), + pyproject, + "list writes nothing" + ); +} + +#[test] +#[serial] +fn rollback_retires_a_uv_override_left_by_relock() { + for scope in [&[][..], &[SIX_PURL][..]] { + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path()); + let (code, envelope) = run_rollback_subprocess(tmp.path(), scope); + assert_eq!(code, 0, "rollback {scope:?} must succeed: {envelope}"); + assert_eq!(envelope["status"], "success", "{envelope}"); + assert!( + warning_codes(&envelope).contains(&"hosted_resolution_orphaned".to_string()), + "{envelope}" + ); + assert_uv_override_retired(tmp.path(), &format!("rollback {scope:?}")); + // Nothing hosted is left: list is clean. + let (code, envelope) = run_cli_json(tmp.path(), &["list"]); + assert_eq!(code, 0, "{envelope}"); + assert!(warning_codes(&envelope).is_empty(), "{envelope}"); + } +} + +#[test] +#[serial] +fn remove_retires_a_uv_override_left_by_relock() { + for target in [SIX_UUID, SIX_PURL] { + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path()); + let (code, envelope) = run_cli_json(tmp.path(), &["remove", target, "--yes"]); + assert_eq!(code, 0, "remove {target} must succeed: {envelope}"); + assert_uv_override_retired(tmp.path(), &format!("remove {target}")); + } +} diff --git a/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs b/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs index c941f0d6d..ce9171e3d 100644 --- a/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs +++ b/crates/socket-patch-cli/tests/vex_e2e_common/uv.rs @@ -1649,6 +1649,17 @@ pub fn run_lane(suite: &str, uv: &Uv, mode: Mode, lane: Lane) { script_sibling_revert(uv, &report, &proj, tmp.path()); } + // ── 5c. the parent dropped after the hosted run (#1445) ─────────── + if mode == Mode::Hosted && lane == Lane::Transitive { + transitive_orphan_unwind( + uv, + &report, + &proj, + tmp.path(), + patch_server.as_deref().unwrap(), + ); + } + // ── 6. the real revert ──────────────────────────────────────────── // Vendored: `vendor --revert` restores every wiring file byte for // byte. Hosted (v5): `rollback` rewrites each pin back to the DEFAULT @@ -1893,6 +1904,98 @@ fn script_sibling_revert(uv: &Uv, report: &Report<'_>, proj: &Path, tmp: &Path) report.row("sibling-revert", "restored, user addition kept"); } +/// #1445: once the parent that pulled `six` in is removed (what `uv remove +/// python-dateutil` does to the metadata, then a relock), uv drops `six` +/// from uv.lock but keeps the hosted `override-dependencies` entry, its +/// `[tool.uv.sources]` URL and the lock's `[manifest] overrides` record. +/// That leftover is Socket's own pin: `list` lists around it and `rollback` +/// retires all three, leaving a lock uv accepts as is. Runs on a copy so +/// the real revert below is unaffected. +fn transitive_orphan_unwind(uv: &Uv, report: &Report<'_>, proj: &Path, tmp: &Path, uri: &str) { + let what = report.what("orphan-unwind"); + let dir = tmp.join("orphan"); + std::fs::create_dir_all(&dir).unwrap(); + for f in ["pyproject.toml", "uv.lock"] { + std::fs::copy(proj.join(f), dir.join(f)).unwrap(); + } + let path = dir.join("pyproject.toml"); + let text = std::fs::read_to_string(&path).unwrap(); + let dropped = text.replace( + "dependencies = [\"python-dateutil==2.9.0.post0\"]", + "dependencies = []", + ); + assert_ne!(dropped, text, "{what}: no python-dateutil to drop:\n{text}"); + std::fs::write(&path, dropped).unwrap(); + let cache = tmp.join("orphan-cache"); + let out = uv.run_py(&dir, &["lock"], &cache); + if !ok(&out) { + report.row("orphan-unwind", "n/a (`uv lock` without the parent failed)"); + println!("{}", dump(&out)); + return; + } + let uuid = Mode::Hosted.uuid(); + let lock = std::fs::read_to_string(dir.join("uv.lock")).unwrap(); + let pyproject = std::fs::read_to_string(&path).unwrap(); + if lock.contains("name = \"six\"\nversion") || !pyproject.contains(uuid) { + report.row("orphan-unwind", "n/a (uv dropped the override itself)"); + return; + } + let server = |args: &[&str]| -> Output { + let mut all = args.to_vec(); + all.extend_from_slice(&[ + "--json", + "--cwd", + dir.to_str().unwrap(), + "--api-url", + uri, + "--api-token", + "fake-token", + "--org", + ORG, + "--patch-server-url", + uri, + ]); + socket_patch(&dir, &all) + }; + let warned = |env: &Value| { + env["warnings"] + .as_array() + .into_iter() + .flatten() + .any(|w| w["code"] == "hosted_resolution_orphaned") + }; + let out = server(&["list"]); + let env = envelope(&out, &what); + assert!( + out.status.code() == Some(0) && warned(&env), + "{what}: list must list around the leftover pin:\n{}", + dump(&out) + ); + let out = server(&["rollback", "--yes"]); + let env = envelope(&out, &what); + assert!( + out.status.code() == Some(0) && warned(&env), + "{what}: rollback must retire the leftover pin:\n{}", + dump(&out) + ); + for f in ["pyproject.toml", "uv.lock"] { + let body = std::fs::read_to_string(dir.join(f)).unwrap(); + assert!( + !body.contains(uuid) && !body.contains("six"), + "{what}: {f} still wires six:\n{body}" + ); + } + let retired = std::fs::read(dir.join("uv.lock")).unwrap(); + let out = uv.run_py(&dir, &["lock"], &cache); + assert!(ok(&out), "{what}: uv lock:\n{}", dump(&out)); + assert_eq!( + String::from_utf8_lossy(&std::fs::read(dir.join("uv.lock")).unwrap()), + String::from_utf8_lossy(&retired), + "{what}: uv rewrote the retired lock" + ); + report.row("orphan-unwind", "retired; uv keeps the lock as is"); +} + // ── production legs ──────────────────────────────────────────────────── /// The uv program a production leg drives: `SOCKET_PATCH_UV_E2E_BIN`, else diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs index d60f68ea6..7d09cc4e3 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/mod.rs @@ -181,8 +181,11 @@ pub struct HostedInventory { pub contested: Vec, /// Leftover hosted wiring no lock installs any more: a yarn berry /// `resolutions` selector `yarn remove` / `yarn up` left behind - /// ([`crate::vex::discover::StaleSelector`], #1203), one pin per - /// package version and patch, wired in the manifest only. Never listed + /// (#1203), or a uv `override-dependencies` pin and its source a relock + /// left behind for a dropped transitive package (#1445) + /// ([`crate::vex::discover::StaleSelector`]), one pin per package + /// version and patch, wired in the manifest (resp. the lock's paired + /// metadata and its `[manifest]` record) only. Never listed /// as a patch; `rollback` and `remove` retire it with a /// `hosted_resolution_orphaned` warning ([`HostedInventory::unwindable`]). pub stale: Vec, @@ -251,7 +254,9 @@ impl HostedInventory { .map(|r| (norm(&r.file), r.uuid.as_str())) .collect(); // A leftover selector's URL (its patch uuid and grant token) is - // retired wiring, not contested wiring, in the manifest naming it. + // retired wiring, not contested wiring, in the manifest naming it — + // and, for a uv lock's leftover override (#1445), in the paired + // metadata that carries the override and its source. let mut stale_excused: BTreeSet<(String, String)> = BTreeSet::new(); let mut stale: BTreeMap<(String, String), BTreeSet> = BTreeMap::new(); for selector in &discovery.stale_selectors { @@ -259,8 +264,12 @@ impl HostedInventory { continue; } let file = norm(&selector.file); + let paired = crate::utils::python_lock::paired_metadata_path(&file); for segment in url_uuid_segments(&selector.url) { - stale_excused.insert((file.clone(), segment)); + stale_excused.insert((file.clone(), segment.clone())); + if let Some(paired) = &paired { + stale_excused.insert((paired.clone(), segment)); + } } stale .entry((selector.purl.clone(), selector.uuid.clone())) diff --git a/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs b/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs index 15a7f5b03..df53c59aa 100644 --- a/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs +++ b/crates/socket-patch-core/src/patch/redirect/upstream/uv.rs @@ -61,7 +61,7 @@ use super::pypi::{ }; use super::{by_uuid, read_or_refuse, refuse_all_in, Ctx, FormatResult, HostedPin, View}; use crate::crawlers::python_crawler::canonicalize_pypi_name; -use crate::utils::python_lock::{paired_metadata_rel, preserve_line_endings, UvSource}; +use crate::utils::python_lock::{paired_metadata_path, preserve_line_endings, UvSource}; use crate::vendor::common::pep508_name; /// One hosted entry of a lock. @@ -111,10 +111,187 @@ pub(crate) async fn restore( let pins = by_uuid(pins); for rel in files { restore_lock(view, rel, &pins, ctx, &mut result).await; + retire_orphaned_overrides(view, rel, &pins, ctx, &mut result).await; } result } +/// Whether a leftover override's `[tool.uv.sources]` entry and the lock's +/// `[manifest]` records of the package all route it to `hit`'s patch. +fn orphan_wiring_names( + doc: &DocumentMut, + meta: &Metadata, + hit: &Hit, + ctx: &Ctx<'_>, +) -> Result<(), String> { + let names_hit = |url: Option<&str>| { + url.and_then(|u| ctx.hosted_uuid(u)) + .is_some_and(|u| u == hit.uuid) + }; + let source = tool_uv(&meta.doc) + .and_then(|uv| uv.get("sources")) + .and_then(Item::as_table_like) + .and_then(|sources| { + sources + .iter() + .find(|(k, _)| canonicalize_pypi_name(k) == hit.name) + .map(|(_, v)| v) + }) + .and_then(Item::as_table_like) + .and_then(|t| t.get("url")) + .and_then(Item::as_str); + if !names_hit(source) { + return Err(format!( + "{}'s `[tool.uv.sources]` entry for {} no longer routes it to this patch", + meta.rel, hit.name + )); + } + let manifest = doc.get("manifest").and_then(Item::as_table_like); + for key in [ + "requirements", + "constraints", + "build-constraints", + "overrides", + ] { + for entry in manifest + .and_then(|m| m.get(key)) + .and_then(Item::as_array) + .into_iter() + .flatten() + .filter_map(Value::as_inline_table) + { + let names_package = entry + .get("name") + .and_then(Value::as_str) + .is_some_and(|n| canonicalize_pypi_name(n) == hit.name); + if names_package + && entry.get("url").is_some() + && !names_hit(entry.get("url").and_then(Value::as_str)) + { + return Err(format!( + "its `[manifest] {key}` record of {} routes it to another artifact", + hit.name + )); + } + } + } + Ok(()) +} + +/// Retire each pin `rel` no longer locks that its paired metadata still +/// wires through a hosted override (#1445): the `override-dependencies` +/// entry the hosted run added for a transitive package, its +/// `[tool.uv.sources]` URL, and the lock's `[manifest]` record of them, +/// which `uv remove ` / `uv lock --upgrade-package ` leave +/// behind (uv never edits either line). Removing all three is what `uv +/// lock` itself writes once the lines are gone, so the lock stays +/// `--locked`-clean; nothing is fetched. A pin with no such override left +/// is not handled here, so it is refused like any unrestorable pin. +async fn retire_orphaned_overrides( + view: &mut View<'_>, + rel: &str, + pins: &BTreeMap<&str, &HostedPin>, + ctx: &Ctx<'_>, + result: &mut FormatResult, +) { + let pending: Vec<&HostedPin> = pins + .values() + .filter(|p| p.files.iter().any(|f| f == rel)) + .filter(|p| !result.handled.contains(&p.uuid) && !result.refused.contains_key(&p.uuid)) + .copied() + .collect(); + if pending.is_empty() { + return; + } + let Ok(Some(text)) = view.read(rel).await else { + return; + }; + let Ok(mut doc) = text.parse::() else { + return; + }; + let (collection, pep751) = crate::utils::python_lock::lock_package_collection(&doc); + if pep751 { + return; + } + let locked: BTreeSet = doc + .get(collection) + .and_then(Item::as_array_of_tables) + .into_iter() + .flat_map(|p| p.iter()) + .filter_map(|p| p.get("name").and_then(Item::as_str)) + .map(canonicalize_pypi_name) + .collect(); + let Some(metadata_rel) = paired_metadata_path(rel) else { + return; + }; + let script = crate::utils::python_lock::is_script_lock_name(rel); + let mut meta = match load_metadata(view, &metadata_rel, script).await { + Ok(Some(meta)) => meta, + Ok(None) => return, + // The metadata is where a leftover override lives: say why it + // cannot be read rather than that no wiring was found. + Err(why) => { + for pin in &pending { + result.refuse(&pin.uuid, format!("{rel}: {why}")); + } + return; + } + }; + let styles = SpecStyle::learn(&doc, Some(&meta)); + let mut retired: BTreeSet = BTreeSet::new(); + for pin in pending { + let Some((name, version)) = pin.name_version() else { + continue; + }; + let hit = Hit { + index: usize::MAX, + uuid: pin.uuid.clone(), + name: canonicalize_pypi_name(&name), + version, + }; + if locked.contains(&hit.name) || pushed_override(&meta, &hit).is_none() { + continue; + } + // Retire the trio only as one: the source and every `[manifest]` + // record of the package must still name this pin's patch, or + // removing the override alone would leave a lock uv rejects. + if let Err(why) = orphan_wiring_names(&doc, &meta, &hit, ctx) { + result.refuse(&hit.uuid, format!("{rel}: {why}")); + continue; + } + let before = doc.clone(); + if let Err(why) = restore_requirements(&mut doc, &hit, Some(&meta), &styles, ctx) { + doc = before; + result.refuse(&hit.uuid, format!("{rel}: {why}")); + continue; + } + restore_metadata(&mut meta, &hit, ctx); + result.warnings.push(( + "hosted_resolution_orphaned", + format!( + "{}: removed the hosted `override-dependencies` entry `{}=={}` and its \ + `[tool.uv.sources]` URL; no {rel} entry installs it any more (the package \ + that depended on it was removed or upgraded)", + meta.rel, hit.name, hit.version + ), + )); + retired.insert(hit.uuid); + } + if retired.is_empty() { + return; + } + let next = preserve_line_endings(&text, doc.to_string()); + if next != text { + view.write(rel, next); + } + if let Some(next) = meta.render() { + if next != meta.text { + view.write(&meta.rel, next); + } + } + result.handled.extend(retired); +} + async fn restore_lock( view: &mut View<'_>, rel: &str, @@ -177,12 +354,7 @@ async fn restore_lock( let metadata_rel = if pep751 { None } else { - paired_metadata_rel(rel.rsplit('/').next().unwrap_or(rel)).map(|leaf| { - match rel.rsplit_once('/') { - Some((dir, _)) => format!("{dir}/{leaf}"), - None => leaf.to_string(), - } - }) + paired_metadata_path(rel) }; let script = crate::utils::python_lock::is_script_lock_name(rel); let mut metadata = match &metadata_rel { @@ -1592,21 +1764,6 @@ fn restore_requirements( // ── paired metadata ────────────────────────────────────────────────────────── -/// Whether the metadata declares `hit` directly (the rewrite adds an -/// override only for a transitive dependency). -fn declares_directly(meta: &Metadata, hit: &Hit) -> bool { - let names = |spec: &&str| canonicalize_pypi_name(pep508_name(spec)) == hit.name; - if meta.script { - return strings(meta.doc.get("dependencies")).iter().any(names); - } - crate::vendor::common::pyproject_dependency_specs(&meta.doc) - .iter() - .any(|(_, spec)| names(spec)) - || strings(tool_uv(&meta.doc).and_then(|u| u.get("dev-dependencies"))) - .iter() - .any(names) -} - /// The index of the `override-dependencies` entry the rewrite added for a /// transitive `hit` (`==` under /// [`HOSTED_OVERRIDE_MARK`](crate::utils::python_script::HOSTED_OVERRIDE_MARK)), @@ -1615,23 +1772,10 @@ fn declares_directly(meta: &Metadata, hit: &Hit) -> bool { /// user's own pin (the rewrite reuses it rather than adding one) and stays, /// along with the lock's `[manifest] overrides` record of it (#411). fn pushed_override(meta: &Metadata, hit: &Hit) -> Option { - if declares_directly(meta, hit) { - return None; - } - let want = format!("{}=={}", hit.name, hit.version); - tool_uv(&meta.doc) - .and_then(|u| u.get("override-dependencies")) - .and_then(Item::as_array)? - .iter() - .position(|value| { - let Some(spec) = value.as_str() else { - return false; - }; - let compact: String = spec.chars().filter(|c| !c.is_whitespace()).collect(); - canonicalize_pypi_name(pep508_name(&compact)) == hit.name - && compact[pep508_name(&compact).len()..] == want[hit.name.len()..] - && crate::utils::python_script::is_hosted_override(value) - }) + crate::utils::python_script::hosted_override_pins(&meta.doc, meta.script) + .into_iter() + .find(|pin| pin.name == hit.name && pin.version == hit.version) + .map(|pin| pin.index) } /// Remove the metadata's hosted `[tool.uv.sources]` entry for `hit` and, diff --git a/crates/socket-patch-core/src/utils/python_lock.rs b/crates/socket-patch-core/src/utils/python_lock.rs index 37d69fb96..d2f3ef080 100644 --- a/crates/socket-patch-core/src/utils/python_lock.rs +++ b/crates/socket-patch-core/src/utils/python_lock.rs @@ -1201,6 +1201,20 @@ pub fn paired_metadata_rel(lock: &str) -> Option<&str> { script_of_lock(lock) } +/// [`paired_metadata_rel`] for a root-relative lock path: the metadata file +/// beside it (`sub/uv.lock` → `sub/pyproject.toml`). +pub fn paired_metadata_path(lock: &str) -> Option { + let (dir, leaf) = match lock.rsplit_once('/') { + Some((dir, leaf)) => (Some(dir), leaf), + None => (None, lock), + }; + let paired = paired_metadata_rel(leaf)?; + Some(match dir { + Some(dir) => format!("{dir}/{paired}"), + None => paired.to_string(), + }) +} + #[cfg(test)] mod script_lock_name_tests { use super::*; @@ -1216,6 +1230,15 @@ mod script_lock_name_tests { assert_eq!(paired_metadata_rel("uv.lock"), Some("pyproject.toml")); assert_eq!(paired_metadata_rel("tool.py.lock"), Some("tool.py")); assert_eq!(paired_metadata_rel("pylock.toml"), None); + assert_eq!( + paired_metadata_path("sub/uv.lock").as_deref(), + Some("sub/pyproject.toml") + ); + assert_eq!( + paired_metadata_path("tool.py.lock").as_deref(), + Some("tool.py") + ); + assert_eq!(paired_metadata_path("package.json"), None); } } diff --git a/crates/socket-patch-core/src/utils/python_script.rs b/crates/socket-patch-core/src/utils/python_script.rs index 88f1fbd93..b5fb19a71 100644 --- a/crates/socket-patch-core/src/utils/python_script.rs +++ b/crates/socket-patch-core/src/utils/python_script.rs @@ -275,6 +275,81 @@ pub(crate) fn is_hosted_override(value: &Value) -> bool { .is_some_and(|prefix| prefix.contains(HOSTED_OVERRIDE_MARK)) } +/// One `override-dependencies` entry the hosted rewrite added for a +/// transitive dependency ([`hosted_override_pins`]). +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct HostedOverridePin { + /// Its index in the `override-dependencies` array. + pub index: usize, + /// PEP 503 canonical name. + pub name: String, + pub version: String, +} + +/// The `==` entries of the metadata's `[tool.uv] +/// override-dependencies` that the hosted rewrite added +/// ([`HOSTED_OVERRIDE_MARK`]) for a package the metadata does not declare +/// directly (`script`: `doc` is a PEP 723 block, whose own requirements are +/// its `dependencies`). The rewrite adds an override only for a transitive +/// dependency, so a marked entry for a direct one is not its own. Hosted +/// mode keeps no ledger: these are the only overrides socket-patch may +/// remove (#411), and the only ones a relock can leave behind as Socket's +/// own wiring (#1445). +pub(crate) fn hosted_override_pins(doc: &DocumentMut, script: bool) -> Vec { + let uv = doc + .get("tool") + .and_then(Item::as_table_like) + .and_then(|t| t.get("uv")) + .and_then(Item::as_table_like); + let Some(overrides) = uv + .and_then(|u| u.get("override-dependencies")) + .and_then(Item::as_array) + else { + return Vec::new(); + }; + let strings = |item: Option<&Item>| -> Vec { + item.and_then(Item::as_array) + .into_iter() + .flatten() + .filter_map(Value::as_str) + .map(|spec| canonicalize_pypi_name(pep508_name(spec))) + .collect() + }; + let direct: Vec = if script { + strings(doc.get("dependencies")) + } else { + pyproject_dependency_specs(doc) + .iter() + .map(|(_, spec)| canonicalize_pypi_name(pep508_name(spec))) + .chain(strings(uv.and_then(|u| u.get("dev-dependencies")))) + .collect() + }; + overrides + .iter() + .enumerate() + .filter(|(_, value)| is_hosted_override(value)) + .filter_map(|(index, value)| { + let compact: String = value + .as_str()? + .chars() + .filter(|c| !c.is_whitespace()) + .collect(); + let raw = pep508_name(&compact); + let version = compact[raw.len()..].strip_prefix("==")?; + let exact = !version.is_empty() + && version + .chars() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '+' | '!' | '-' | '_')); + let name = canonicalize_pypi_name(raw); + (exact && !raw.is_empty() && !direct.contains(&name)).then(|| HostedOverridePin { + index, + name, + version: version.to_string(), + }) + }) + .collect() +} + /// Append `specifier` to `overrides` on its own line, under /// [`HOSTED_OVERRIDE_MARK`]. An array the rewrite just created (empty) is /// laid out multi-line with a trailing comma; an existing array keeps its diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 6ff3ba108..ff3323d42 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -622,8 +622,10 @@ pub struct Discovery { pub withheld: Vec, /// Hosted `resolutions` selectors a yarn berry project's root /// `package.json` still carries although its lock no longer installs - /// them ([`StaleSelector`], #1203). Not refs and not contested wiring: - /// the management commands retire them. Sorted, deduped. + /// them (#1203), and hosted uv `override-dependencies` pins a lock's + /// paired metadata still carries for a package the lock no longer holds + /// (#1445) ([`StaleSelector`]). Not refs and not contested wiring: the + /// management commands retire them. Sorted, deduped. pub stale_selectors: Vec, } @@ -633,13 +635,21 @@ pub struct Discovery { /// `resolutions`, so the selector is left routing a descriptor nothing /// depends on (see /// [`crate::vendor::lock_inventory::yarn::berry_selector_routes_nothing`]). +/// +/// The uv twin (#1445): a hosted `override-dependencies` pin (under the +/// rewrite's ownership comment) plus its `[tool.uv.sources]` URL, for a +/// transitive package `uv remove ` / `uv lock --upgrade-package +/// ` dropped from the lock; uv never edits either line. `file` is +/// then the lock (the restore retires the pair from its paired metadata +/// and the lock's `[manifest]` record), `selector` the canonical package +/// name. #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)] pub struct StaleSelector { - /// Root-relative manifest carrying the selector. + /// Root-relative manifest carrying the selector (uv: the lock). pub file: PathBuf, - /// The `resolutions` key. + /// The `resolutions` key (uv: the canonical package name). pub selector: String, - /// The hosted tarball URL it routes to. + /// The hosted artifact URL it routes to. pub url: String, /// The package version the URL's artifact is (canonical base purl). pub purl: String, diff --git a/crates/socket-patch-core/src/vex/discover/pypi_locks.rs b/crates/socket-patch-core/src/vex/discover/pypi_locks.rs index f6694454d..3c64e4017 100644 --- a/crates/socket-patch-core/src/vex/discover/pypi_locks.rs +++ b/crates/socket-patch-core/src/vex/discover/pypi_locks.rs @@ -96,6 +96,8 @@ //! The native (registry) fixtures are rewired in the tests through the SAME //! rewriter functions the hosted and vendored backends call. +use std::collections::BTreeSet; + use toml_edit::{DocumentMut, Item, Table, TableLike, Value}; use super::{ @@ -344,6 +346,17 @@ async fn extract_python_lock(ctx: &DiscoverCtx<'_>, file: &str, out: &mut Discov // A PEP 723 script lock is scoped to its script's own install: it is not // evidence against the project's locks (see `contest_across_locks`). let script_lock = is_script_lock_name(file); + // The packages a uv / script lock holds (a pylock has no paired + // metadata to leave an override behind in). + let locked: BTreeSet = if pep751 { + BTreeSet::new() + } else { + packages + .iter() + .filter_map(|p| p.get("name").and_then(Item::as_str)) + .map(canonicalize_pypi_name) + .collect() + }; for package in packages.iter() { if pep751 { pylock_candidates(ctx, file, package, out, &mut candidates); @@ -365,6 +378,9 @@ async fn extract_python_lock(ctx: &DiscoverCtx<'_>, file: &str, out: &mut Discov if !pep751 { if let Some(meta) = paired_metadata_rel(file) { ctx.recognize_ignored(meta).await; + if let Some(doc) = quiet_metadata(ctx, meta, script_lock).await { + record_orphaned_overrides(ctx, file, &locked, &doc, script_lock, out); + } } } return; @@ -374,6 +390,9 @@ async fn extract_python_lock(ctx: &DiscoverCtx<'_>, file: &str, out: &mut Discov } else { load_pairing(ctx, file, out).await }; + if let Pairing::Sources { doc, .. } = &pairing { + record_orphaned_overrides(ctx, file, &locked, doc, script_lock, out); + } for c in candidates { match &pairing { Pairing::LockOnly => emit(out, file, c), @@ -411,6 +430,97 @@ async fn extract_python_lock(ctx: &DiscoverCtx<'_>, file: &str, out: &mut Discov } } +/// The paired metadata document of a lock with no Socket entry, read +/// without recording the read or diagnosing a failure (the half-reverted +/// sweep above reads it the same way): only the leftover-override check +/// looks at it. A script's document is its PEP 723 block. +async fn quiet_metadata(ctx: &DiscoverCtx<'_>, rel: &str, script: bool) -> Option { + if ctx.is_ignored(rel) { + return None; + } + let bytes = ctx.view.read_bytes(rel).await.ok()?; + let text = String::from_utf8(bytes).ok()?; + if script { + let (_, block) = crate::utils::python_script::script_metadata(&text).ok()?; + block.parse().ok() + } else { + text.parse().ok() + } +} + +/// Record each hosted override pin the paired metadata still carries for a +/// package `file` no longer locks ([`StaleSelector`], #1445): the hosted +/// rewrite wires a TRANSITIVE package with an `override-dependencies` +/// entry under its ownership comment plus a `[tool.uv.sources]` URL, and +/// once the parent stops depending on it (`uv remove `, `uv lock +/// --upgrade-package `) uv drops its `[[package]]` but never edits +/// either line. That is Socket's own leftover pin, which the management +/// commands retire instead of refusing around it as contested wiring. A +/// user's own (unmarked) override, a package the lock still holds, and a +/// source that is not one unconditional hosted URL naming the pinned +/// release's artifact are left to the contested path. +/// +/// [`StaleSelector`]: super::StaleSelector +fn record_orphaned_overrides( + ctx: &DiscoverCtx<'_>, + file: &str, + locked: &BTreeSet, + doc: &DocumentMut, + script: bool, + out: &mut Discovery, +) { + for pin in crate::utils::python_script::hosted_override_pins(doc, script) { + if locked.contains(&pin.name) { + continue; + } + let Some(source) = unconditional_source(doc, &pin.name) else { + continue; + }; + let Some(url) = str_of(source, "url") else { + continue; + }; + let Some(uuid) = ctx.hosted_uuid(url) else { + continue; + }; + if !artifact_names_release(url, &pin.name, &pin.version) { + continue; + } + let Some(purl) = crate::utils::purl::pypi_purl(&pin.name, &pin.version) else { + continue; + }; + out.stale_selectors.push(super::StaleSelector { + file: file.into(), + selector: pin.name, + url: url.to_string(), + purl: crate::utils::purl_key::canonical_base_purl(&purl), + uuid, + }); + } +} + +/// Whether `url`'s artifact (its last path segment: a wheel +/// `--….whl` or an sdist `-.tar.gz` / `.zip`) +/// is release `version` of canonical package `name`. +fn artifact_names_release(url: &str, name: &str, version: &str) -> bool { + let path = url.split(['?', '#']).next().unwrap_or(url); + let leaf = path.rsplit('/').next().unwrap_or(""); + let stem_version = if let Some(stem) = leaf.strip_suffix(".whl") { + let mut parts = stem.split('-'); + match (parts.next(), parts.next()) { + (Some(n), Some(v)) => Some((n, v)), + _ => None, + } + } else { + [".tar.gz", ".zip"] + .iter() + .find_map(|ext| leaf.strip_suffix(ext)) + .and_then(|stem| stem.rsplit_once('-')) + }; + stem_version.is_some_and(|(n, v)| { + canonicalize_pypi_name(n) == name && (v == version || v == version.replace('-', "_")) + }) +} + /// The `pypi_purl` of a lock table's `name` + `version`. fn package_purl(package: &Table) -> Option { pypi_purl( @@ -643,34 +753,35 @@ fn unusable(file: &str, why: &str) -> Pairing { } } -/// Whether `doc`'s `[tool.uv.sources]` routes canonical package `canon` to -/// exactly `wired`'s patch: a single unconditional `{ url | path }` source -/// (our writers' shape; a marker / extra / group split applies only to some -/// installs, and an array of sources is such a split). -fn sources_confirm(ctx: &DiscoverCtx<'_>, doc: &DocumentMut, canon: &str, wired: &Wired) -> bool { - let Some(sources) = doc +/// `doc`'s `[tool.uv.sources]` entry for canonical package `canon` when it +/// is a single unconditional table (our writers' shape; a marker / extra / +/// group split applies only to some installs, and an array of sources is +/// such a split). +fn unconditional_source<'d>(doc: &'d DocumentMut, canon: &str) -> Option<&'d dyn TableLike> { + let source = doc .get("tool") .and_then(Item::as_table_like) .and_then(|tool| tool.get("uv")) .and_then(Item::as_table_like) .and_then(|uv| uv.get("sources")) - .and_then(Item::as_table_like) - else { - return false; - }; - let Some(source) = sources + .and_then(Item::as_table_like)? .iter() .find(|(key, _)| canonicalize_pypi_name(key) == canon) - .and_then(|(_, item)| item.as_table_like()) - else { - return false; - }; - if ["marker", "extra", "group"] + .and_then(|(_, item)| item.as_table_like())?; + (!["marker", "extra", "group"] .iter() - .any(|key| source.contains_key(key)) - { + .any(|key| source.contains_key(key))) + .then_some(source) +} + +/// Whether `doc`'s `[tool.uv.sources]` routes canonical package `canon` to +/// exactly `wired`'s patch: a single unconditional `{ url | path }` source +/// (our writers' shape; a marker / extra / group split applies only to some +/// installs, and an array of sources is such a split). +fn sources_confirm(ctx: &DiscoverCtx<'_>, doc: &DocumentMut, canon: &str, wired: &Wired) -> bool { + let Some(source) = unconditional_source(doc, canon) else { return false; - } + }; let Some(location) = str_of(source, "url").or_else(|| str_of(source, "path")) else { return false; }; diff --git a/crates/socket-patch-core/tests/hosted_inventory.rs b/crates/socket-patch-core/tests/hosted_inventory.rs index 8ab7e20bd..3a4e85f5f 100644 --- a/crates/socket-patch-core/tests/hosted_inventory.rs +++ b/crates/socket-patch-core/tests/hosted_inventory.rs @@ -581,3 +581,241 @@ async fn restore_retires_a_leftover_berry_selector() { ); } } + +// ── #1445: a uv override pin `uv remove` / `uv lock -P` left behind ────── +// Hosted uv wires a TRANSITIVE package with an `override-dependencies` +// entry (under the rewrite's ownership comment) plus a `[tool.uv.sources]` +// URL, which uv.lock records under `[manifest] overrides`. Once the parent +// stops depending on it (`uv remove `, `uv lock --upgrade-package +// `), uv drops its `[[package]]` but never edits the override or +// the source, so both lines (and the manifest record) are left behind. +// That is Socket's own leftover pin, not contested state. + +const SIX_PATCH: &str = "66666666-6666-4666-8666-666666666666"; + +fn six_hosted_url() -> String { + format!( + "https://patch.socket.dev/patch/pypi/six/1.16.0/{GRANT}/{SIX_PATCH}/six-1.16.0-py2.py3-none-any.whl" + ) +} + +/// The pristine project: `isodate` is the only declared dependency. +const UV_ORPHAN_PYPROJECT_PRISTINE: &str = "[project]\nname = \"demo\"\nversion = \"0.1.0\"\n\ + requires-python = \">=3.9\"\ndependencies = [\"isodate>=0.6\"]\n"; + +/// The uv.lock real uv (0.11) writes for the leftover wiring after `uv lock +/// --upgrade-package isodate` (0.7.2 no longer depends on six), with the +/// `[manifest]` block `manifest` (`None`: no manifest record). +fn uv_orphan_lock(manifest: Option<&str>) -> String { + let manifest = manifest.map(|m| format!("{m}\n\n")).unwrap_or_default(); + format!( + "version = 1\nrevision = 3\nrequires-python = \">=3.9\"\n\n{manifest}\ + [[package]]\nname = \"demo\"\nversion = \"0.1.0\"\nsource = {{ virtual = \".\" }}\n\ + dependencies = [\n {{ name = \"isodate\" }},\n]\n\n\ + [package.metadata]\nrequires-dist = [{{ name = \"isodate\", specifier = \">=0.6\" }}]\n\n\ + [[package]]\nname = \"isodate\"\nversion = \"0.7.2\"\n\ + source = {{ registry = \"https://pypi.org/simple\" }}\n\ + sdist = {{ url = \"https://files.pythonhosted.org/packages/54/4d/e940025e2ce31a8ce1202635910747e5a87cc3a6a6bb2d00973375014749/isodate-0.7.2.tar.gz\", hash = \"sha256:4cd1aa0f43ca76f4a6c6c0292a85f40b35ec2e43e315b59f06e6d32171a953e6\", size = 29705, upload-time = \"2024-10-08T23:04:11.5Z\" }}\n\ + wheels = [\n {{ url = \"https://files.pythonhosted.org/packages/15/aa/0aca39a37d3c7eb941ba736ede56d689e7be91cab5d9ca846bde3999eba6/isodate-0.7.2-py3-none-any.whl\", hash = \"sha256:28009937d8031054830160fce6d409ed342816b543597cece116d966c6d99e15\", size = 22320, upload-time = \"2024-10-08T23:04:09.501Z\" }},\n]\n" + ) +} + +/// A uv project whose hosted six pin (a transitive of isodate 0.6.1) was +/// left behind by `uv lock --upgrade-package isodate`. `marked`: the +/// override carries the hosted rewrite's ownership comment (else it is the +/// user's own pin). +fn write_uv_override_left_by_relock(root: &std::path::Path, marked: bool) { + let mark = if marked { + " # socket-patch hosted: pins a patched transitive dependency; rollback removes it\n" + } else { + "" + }; + std::fs::write( + root.join("pyproject.toml"), + format!( + "{UV_ORPHAN_PYPROJECT_PRISTINE}\n[tool.uv]\noverride-dependencies = [\n{mark} \ + \"six==1.16.0\",\n]\n\n[tool.uv.sources]\nsix = {{ url = \"{}\" }}\n", + six_hosted_url() + ), + ) + .unwrap(); + std::fs::write( + root.join("uv.lock"), + uv_orphan_lock(Some(&format!( + "[manifest]\noverrides = [{{ name = \"six\", url = \"{}\" }}]", + six_hosted_url() + ))), + ) + .unwrap(); +} + +#[tokio::test] +async fn uv_override_left_by_relock_is_stale_not_contested() { + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path(), true); + let inv = inventory(tmp.path()).await; + assert!( + inv.contested.is_empty(), + "a leftover override pin is not contested: {inv:?}" + ); + assert!(inv.contested_refusal().is_none()); + assert!(inv.pins.is_empty(), "{inv:?}"); + assert_eq!(inv.stale.len(), 1, "{inv:?}"); + assert_eq!(inv.stale[0].purl, "pkg:pypi/six@1.16.0"); + assert_eq!(inv.stale[0].uuid, SIX_PATCH); + assert_eq!(inv.stale[0].files, vec!["uv.lock".to_string()]); + assert!(!inv.is_empty(), "the leftover override is hosted state"); +} + +#[tokio::test] +async fn uv_override_left_beside_a_live_hosted_pin_is_stale() { + // Another package is still hosted-pinned in the same lock: that one is a + // pin, the leftover six override is stale, and nothing is contested. + let tmp = tempfile::tempdir().unwrap(); + write_uv_project(tmp.path(), PATCH, None); + let pyproject = std::fs::read_to_string(tmp.path().join("pyproject.toml")).unwrap(); + // The live pin is six 1.17.0; the leftover is `other` 1.0.0. + let other = format!( + "https://patch.socket.dev/patch/pypi/other/1.0.0/{GRANT}/{SIX_PATCH}/other-1.0.0-py3-none-any.whl" + ); + let pyproject = pyproject.replace( + "override-dependencies = [\"six==1.17.0\"]", + "override-dependencies = [\"six==1.17.0\",\n \ + # socket-patch hosted: pins a patched transitive dependency; rollback removes it\n \ + \"other==1.0.0\",\n]", + ) + &format!("other = {{ url = \"{other}\" }}\n"); + std::fs::write(tmp.path().join("pyproject.toml"), pyproject).unwrap(); + let lock = std::fs::read_to_string(tmp.path().join("uv.lock")).unwrap(); + let lock = lock.replacen( + "\n\n[[package]]", + &format!("\n\n[manifest]\noverrides = [{{ name = \"other\", url = \"{other}\" }}]\n\n[[package]]"), + 1, + ); + std::fs::write(tmp.path().join("uv.lock"), lock).unwrap(); + let inv = inventory(tmp.path()).await; + assert!(inv.contested.is_empty(), "{inv:?}"); + let pinned: Vec<&str> = inv.pins.iter().map(|p| p.uuid.as_str()).collect(); + assert_eq!(pinned, vec![PATCH], "{inv:?}"); + assert_eq!(inv.stale.len(), 1, "{inv:?}"); + assert_eq!(inv.stale[0].purl, "pkg:pypi/other@1.0.0"); + assert_eq!(inv.stale[0].uuid, SIX_PATCH); +} + +#[tokio::test] +async fn users_own_uv_override_left_by_relock_stays_contested() { + // Without the hosted rewrite's ownership comment the override is the + // user's own pin: nothing says the wiring is Socket's to retire. + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path(), false); + let inv = inventory(tmp.path()).await; + assert!(inv.stale.is_empty(), "{inv:?}"); + assert!(inv.contested_refusal().is_some(), "{inv:?}"); +} + +#[tokio::test] +async fn uv_override_still_in_the_lock_is_not_stale() { + // The lock still resolves six from the registry while the metadata + // routes it to the hosted wheel: the pair disagrees (half reverted), + // which stays contested. + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path(), true); + let lock = std::fs::read_to_string(tmp.path().join("uv.lock")).unwrap(); + std::fs::write( + tmp.path().join("uv.lock"), + lock + "\n[[package]]\nname = \"six\"\nversion = \"1.16.0\"\n\ + source = { registry = \"https://pypi.org/simple\" }\n", + ) + .unwrap(); + let inv = inventory(tmp.path()).await; + assert!(inv.stale.is_empty(), "{inv:?}"); + assert!(inv.contested_refusal().is_some(), "{inv:?}"); +} + +#[tokio::test] +async fn restore_retires_a_leftover_uv_override() { + use socket_patch_core::patch::redirect::upstream::{ + restore_upstream, PinStatus, RestoreOptions, + }; + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path(), true); + let inv = inventory(tmp.path()).await; + // Offline: retiring a leftover needs no registry lookup. + let opts = RestoreOptions { + offline: true, + ..Default::default() + }; + let outcome = restore_upstream(tmp.path(), &inv.stale, &opts).await; + assert_eq!(outcome.pins.len(), 1); + assert_eq!( + outcome.pins[0].status, + PinStatus::Restored, + "{:?}", + outcome.pins + ); + assert_eq!( + outcome.reverted_files, + vec!["pyproject.toml".to_string(), "uv.lock".to_string()] + ); + assert!( + outcome + .warnings + .iter() + .any(|(code, detail)| *code == "hosted_resolution_orphaned" + && detail.contains("six==1.16.0")), + "{:?}", + outcome.warnings + ); + assert_eq!( + std::fs::read_to_string(tmp.path().join("pyproject.toml")).unwrap(), + UV_ORPHAN_PYPROJECT_PRISTINE, + "the override and the source go, and the emptied [tool.uv] with them" + ); + assert_eq!( + std::fs::read_to_string(tmp.path().join("uv.lock")).unwrap(), + uv_orphan_lock(None), + "the `[manifest] overrides` record goes, as `uv lock` would drop it" + ); + let after = inventory(tmp.path()).await; + assert!(after.is_empty(), "{after:?}"); +} + +#[tokio::test] +async fn leftover_uv_override_is_not_retired_against_another_patchs_record() { + use socket_patch_core::patch::redirect::upstream::{ + restore_upstream, PinStatus, RestoreOptions, + }; + // The lock's `[manifest] overrides` record names another patch than + // the metadata's source: removing the override and source alone would + // leave a lock uv rejects, so the pin is refused and nothing is written. + let tmp = tempfile::tempdir().unwrap(); + write_uv_override_left_by_relock(tmp.path(), true); + let lock = std::fs::read_to_string(tmp.path().join("uv.lock")) + .unwrap() + .replace(SIX_PATCH, OTHER_PATCH); + std::fs::write(tmp.path().join("uv.lock"), &lock).unwrap(); + let pyproject = std::fs::read_to_string(tmp.path().join("pyproject.toml")).unwrap(); + let inv = inventory(tmp.path()).await; + assert_eq!(inv.stale.len(), 1, "{inv:?}"); + assert!( + inv.contested_refusal().is_some(), + "the other patch's record is contested: {inv:?}" + ); + let opts = RestoreOptions { + offline: true, + ..Default::default() + }; + let outcome = restore_upstream(tmp.path(), &inv.stale, &opts).await; + match &outcome.pins[0].status { + PinStatus::Refused(why) => assert!(why.contains("[manifest] overrides"), "{why}"), + other => panic!("must refuse: {other:?}"), + } + assert!(outcome.reverted_files.is_empty(), "{outcome:?}"); + assert_eq!( + std::fs::read_to_string(tmp.path().join("uv.lock")).unwrap(), + lock + ); + assert_eq!( + std::fs::read_to_string(tmp.path().join("pyproject.toml")).unwrap(), + pyproject + ); +}