Fix Poetry rewrite of a source = "pypi" dep (#1282)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #1282

Summary

Poetry lets a dependency name the source it must come from, for example six = {version = "1.16.0", source = "pypi"}. Poetry's docs recommend this when a project also has private sources. Before this PR, hosted and vendored scans rewired such a dependency, reported success, and left a lock that Poetry 1.4–2.2 can't install: every later poetry install failed with Repository "pypi" does not exist. socket-patch no longer writes that lock.

Root cause

Both Poetry lock rewriters only check the lock's own [package.source]:

• hosted: patch/redirect/poetry.rs, on top of utils::poetry_lock
• vendored: vendor/pypi_poetry.rs::check_target_guards

Poetry writes nothing into poetry.lock for a package pinned to PyPI, so the rewriter sees an ordinary registry entry. Poetry 1.4–2.2 then compare the pyproject source with the rewritten url or file lock source and refuse the install. 1.1–1.3 and 2.3+ install it (measured matrix in #1282).

Fix

• utils::poetry_lock::pyproject_source_constraints reads the per-dependency source from the pyproject next to the lock. It covers:

  • the main table, legacy dev-dependencies and every group table
  • table, inline-table, inline-array and [[tool.poetry.dependencies.x]] multiple-constraint forms
  • PEP 503 name normalization

  Vendored classify_dependency now uses the same dependency-table walk (poetry_dependency_entries).

• Lock stamped @generated by Poetry 1.4–2.2: the package is refused before any write.

  • Hosted reports redirect_poetry_lock_unsupported. The uuid isn't confirmed, so there's no VEX attestation.
  • Vendored reports pypi_poetry_source_constraint (failed), before the wheel is built.
  • A lock that an earlier run already rewired is also refused, rather than reported as confirmed or in sync, since its installs already fail.
  • A vendored → hosted takeover is refused in preflight (pypi_takeover.rs), so the package stays vendored instead of being reverted into neither mode.

• Any other writer (1.1–1.3, 2.3+, unstamped): the rewrite goes ahead as before, with a warning that names the 1.4–2.2 range (redirect_poetry_source_constraint / pypi_poetry_source_constraint).

• Lock entry already has a source socket-patch didn't write (a private index, or the user's own path or URL): the original "refusing to replace an existing Poetry source" refusal applies, as before this PR. The source-pin check counts only socket-patch's own .socket/vendor/pypi/ and /patch/pypi/ pins as already wired.

• Remedy text: drop the source key or upgrade to Poetry ≥ 2.3, re-lock, and re-run. Re-locking discards whatever the lock already wires, so:

  • over a vendored pin, the remedy first says to run socket-patch vendor --revert, and that it reverts EVERY vendored package
  • over a hosted pin, it first says to run socket-patch rollback pkg:pypi/<name>@<version>

• Documented in CLI_CONTRACT.md and docs/testing/poetry-compatibility.md.

Why gate on the writer instead of always refusing: the stamp names the Poetry that wrote the lock, which is usually the one that installs it. Refusing every source-pinned dep would stop patching on 2.3+, where it works. Only warning on every writer would keep breaking 1.4–2.2 builds. The issue offers both "refuse" and "at least warn", and this uses each where it fits.

Tests (red → green)

Issue	Test	Without fix	With fix
#1282 hosted	redirect::poetry::…::source_constrained_dependency_is_refused_for_poetry_1_4_to_2_2 (14 lock generations × 6 pyproject forms, sibling lock, already-redirected re-run, remedy order)	FAILED: no warning or refusal	ok
#1282 hosted	redirect::poetry::…::private_index_pin_keeps_the_user_authored_source_refusal	FAILED on 9c56814: wrong remedy	ok
#1282 vendored	vendor::pypi_poetry::tests::source_constrained_dependency_is_refused_for_poetry_1_4_to_2_2 (incl. already-wired lock, revert-first remedy)	FAILED at 0.12.17: no warning	ok
#1282 takeover	pypi_takeover::tests::poetry_source_pinned_package_follows_the_writer (revert-first remedy)	n/a (new path)	ok
#1282 helpers	utils::poetry_lock::tests::pyproject_source_constraints_reads_every_declaration_form, source_constraint_breaks_only_poetry_1_4_to_2_2_writers	n/a	ok
#1282 e2e	e2e_vex_build poetry::poetry_source_pypi_constraint_never_breaks_install: real Poetry, hosted + vendored scan, then a fresh poetry install must succeed, with the patched bytes when rewired	FAILED on Poetry 1.8.5: lock rewritten	ok on 1.8.5 (refusal path) and 2.4.3 (warning path)

CI already runs e2e_vex_build poetry:: on Poetry 1.0.10, 1.1.15, 1.8.5, 2.0.1 and 2.4.3. The new e2e returns early on 1.0, which predates per-dependency sources.

Local runs:

• cargo fmt --all -- --check: clean
• cargo clippy --workspace --all-features -- -D warnings: clean
• cargo test -p socket-patch-core --all-features: every core integration test passes. 4 lib tests fail, all chmod-based and unable to fail as root in this sandbox: copy_tree::relax_loop…, vlt_heal::an_unremovable…, pypi_poetry::wire_write_failure… and pypi_requirements::wire_failure…. Fix gem revert of a commented Gemfile line (#988) #1406, Fix Composer vendor-dir config cascade (#439, #686) #1408 and Fix hosted cargo pin of registry = "crates-io" (#386) #1412 carry the same note.
• SOCKET_PATCH_POETRY_E2E_REQUIRED=1 cargo test -p socket-patch-cli --all-features --test e2e_vex_build -- --ignored poetry::: 4 passed on Poetry 1.8.5 and on 2.4.3
• cargo test -p socket-patch-cli --all-features --test mode_migration_pypi: 58 passed
• No wrapper changes: npm/, pypi/ and gem/ only dispatch to the binary.

PR CI on 64d03c1: ci-ok and every other check green (35 passed, 19 skipped by path filters). Bugbot found no new issues.

Review

/code-review high findings, fixed in df01bb4:

• the takeover preflight gap
• an already-redirected lock (Unchanged/InSync) read as confirmed or in sync
• the [[tool.poetry.dependencies.x]] form
• the lock was rendered once per dep and the pyproject re-parsed per dep
• the duplicated table walk
• the missing contract row
• messages now name the pyproject next to the lock

Bugbot findings:

• the remedy for an already-wired lock now puts revert first (ecb2b34)
• the hosted rollback remedy names only the one package (9c56814)

Final-review finding: a private-index pin keeps its original refusal (64d03c1).

Not fixed:

• A source refusal can mask another refusal for the same package, such as a forked unit or a malformed files. The source message is accurate on its own, and the other reason shows on the next run. Folding the gate into rewrite_poetry_lock_all as a LockStep::Refused would report both, but that touches every PoetryLockDep caller, which is more refactoring than this fix needs.
• The gate sits beside the lock engine instead of inside it. The refusal and advisory text and the 1.4–2.2 rule are shared helpers in utils::poetry_lock, so the two modes can't drift. Moving it into the engine is the same refactor as above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BbSVVjLH4F68h9WmHJAaX1

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[Mikola Lysenko (mikolalysenko)]

Ready for review (burn-down agent)

• Head: 9c56814
• CI: 35/35 non-skipped checks green (19 skipped by path filters); mergeable, no conflicts.
• Bugbot: reviewed 9c56814, no findings; both earlier review threads (pypi_takeover.rs, poetry_lock.rs) are resolved.
• Reviewer focus: hosted and vendored Poetry rewrites now read per-dependency source pins from pyproject and refuse to rewire a source = "pypi" dep for Poetry 1.4–2.2 locks (Hosted and vendored Poetry rewrites break poetry install on Poetry 1.4–2.2 when the patched dependency is declared with source = "pypi" ("Repository \"pypi\" does not exist."), with no warning #1282).

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Final review brief

What it does. A dependency pinned in pyproject with source = "<name>" (e.g. six = {version = "…", source = "pypi"}) leaves no trace in poetry.lock, and Poetry 1.4-2.2 then fail every install once the lock points that package at a URL/file. This PR reads those source pins (main table, legacy dev-dependencies, every group; inline, table and multiple-constraint forms) and, when the lock's @generated by Poetry X.Y stamp is 1.4-2.2, refuses that package before any write in hosted, vendored and vendored→hosted takeover; other writers get the rewrite plus a warning. A lock an earlier run already rewired is refused with a revert/rollback-first remedy.

Risk: low-medium. Every new path only refuses or warns, before any write, and is keyed narrowly on a source pin plus the writer stamp. The medium part is policy: the 1.4-2.2 range is a measured claim, and the stamp names the Poetry that wrote the lock, not necessarily the one that installs it.

Look here

• utils/poetry_lock.rs:227 pyproject_source_constraints, :259 lock_wiring, :351 source_constraint_breaks_writer
• patch/redirect/poetry.rs:76: hosted refusal
• patch/redirect/pypi_takeover.rs:156:`` takeover preflight
• vendor/pypi_poetry.rs:217 and :282: vendored guard, including an already-in-sync lock

Verified

• Read the full diff. Package names are canonicalized on both sides, a dependency missing from the lock still gets entry_not_found, a sibling lock is unaffected (tested), and CRLF is handled when reading the stamp line. The pyproject path is joined like the existing lock path, so there's no new traversal surface. The classify_dependency refactor is behavior-preserving. CHANGELOG.md is untouched, and the CLI_CONTRACT and docs edits match the code.
• cargo test -p socket-patch-core poetry (local, stable toolchain): all 5 new tests pass and assert the refusal/warning, so they would fail without the change. 81/82 passed overall. The one failure is a chmod-0555 test the PR doesn't touch, which can't pass when the sandbox runs as root. The real-Poetry e2e (#[ignore]) wasn't run locally.
• CI on 9c56814: ci-ok and clippy green, no failing checks. Bugbot is clean and all review threads are resolved.

Changes I made: none.

Open questions (non-blocking)

• The check fires for any source name, not only pypi. A package pinned to a private index, which already has a legacy [package.source] in the lock and was refused before as a user-authored source, now gets the "drop the source key" remedy. That's poor advice for a private index. Worth a follow-up to limit the check to PyPI pins, or to entries without a [package.source].
• Small things: the vendored messages hard-code pyproject.toml/the lock name rather than the relative path, and the vendored in-sync path for writers outside 1.4-2.2 doesn't emit the advisory warning that the fresh path does.

Auto-merge is armed, so approving sends it straight to the merge queue.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

[agent] On the final review's open questions:

• Private-index pin: fixed in 64d03c1. lock_wiring now counts only socket-patch's own pins as already wired: a .socket/vendor/pypi/ file, or a /patch/pypi/ URL. A package whose lock entry already has some other [package.source] (a private index, or the user's own path or URL) skips the source-pin check and hits the original "refusing to replace an existing Poetry source" refusal, as it did before this PR. The new test private_index_pin_keeps_the_user_authored_source_refusal failed on 9c56814 with exactly the wrong remedy and passes now. Vendored mode was already correct here: check_target_guards refuses an existing source before it reaches the new check. The real-Poetry e2e still passes on 1.8.5.
• Vendored messages naming pyproject.toml / poetry.lock: the vendored backend always runs at the project root, the same root its other messages use, so those paths are accurate. Left as is.
• No advisory on the vendored in-sync path for writers outside 1.4–2.2: that path doesn't write anything, and the advisory already fired on the run that wired the package. Left as is.

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 64d03c1. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Final review brief (replaces the brief for 9c56814)

What it does. Fixes #1282. Poetry writes nothing to poetry.lock for a dependency pinned with source = "pypi", so the hosted and vendored Poetry rewriters rewired it, leaving a lock that Poetry 1.4–2.2 can't install. The PR reads each dependency's source from the sibling pyproject.toml. When the lock was written by Poetry 1.4–2.2, it refuses before any write, with a remedy suited to the current wiring. That covers the takeover preflight and locks a previous run already rewired. When any other Poetry version wrote the lock, it rewires as before and warns. The newest commit (64d03c1) restores the existing "refusing to replace an existing Poetry source" refusal for private-index pins socket-patch didn't write.

Risk: low to medium. The gate only fires when the pyproject has a source pin, so other projects are unaffected. Behaviour does change on purpose: a 1.4–2.2 lock that was already rewired is now reported as refused, not in sync. The medium part is that lock_wiring recognises socket-patch's own pins by matching URL strings.

Look here

• utils/poetry_lock.rs:227: pyproject_source_constraints (every declaration form)
• utils/poetry_lock.rs:262-283: lock_wiring, the latest change; it matches .socket/vendor/pypi/ and /patch/pypi/
• patch/redirect/poetry.rs:61-108:`` the hosted gate before rewrite_poetry_lock_all
• vendor/pypi_poetry.rs:217-240:``` the vendored guard (in-sync and fresh paths)
• patch/redirect/pypi_takeover.rs:154-198:```` the takeover preflight

Verified. Read the full diff, then the delta since the last brief more closely. Ran the Poetry core lib tests: 127 passed. The 1 failure is a chmod-0555 test that can't fail as root, the same one the PR description names. Mutation checks:

• With the hosted and vendored gates disabled, both source_constrained_dependency_is_refused_for_poetry_1_4_to_2_2 tests fail.
• With poetry_lock.rs reverted to 9c56814, private_index_pin_keeps_the_user_authored_source_refusal fails.

pyproject.toml is in the hosted scan's candidate files, so the gate is reached in real scans. cargo fmt --check is clean. CHANGELOG.md is untouched. CI on 64d03c1: ci-ok and clippy green, 0 failed checks. Bugbot: no issues. All review threads are resolved.

Changes I made. None. I disarmed auto-merge when 64d03c1 landed after the previous brief, and I re-arm it with this brief.

Open questions (non-blocking; they only affect the remedy text, not data safety).

• A vendored package being superseded by a newer patch on a 1.4–2.2 lock gets a remedy that doesn't mention vendor --revert first.
• /patch/pypi/ is a substring match, so a user's own index URL that contains that path would get the hosted-rollback remedy text.

Auto-merge is armed, so approving sends it straight to the merge queue.

---

Generated by Claude Code