Skip to content

CI: hermetic spawns in gem/vendor suites (#823) - #1505

Merged
Mikola Lysenko (mikolalysenko) merged 4 commits into
mainfrom
agent/fix-ci-hermetic-gem-vendor
Oct 14, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 4 commits into
mainfrom
agent/fix-ci-hermetic-gem-vendor

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Refs #823 (slice 14; the issue stays open for the remaining PENDING_RAW_SPAWNS files)

What

Three suites still on PENDING_RAW_SPAWNS started socket-patch with a bare Command::new and their own SOCKET_* sweeps, which had drifted from tests/common/hermetic.rs. Those sweeps let an ambient SOCKET_NO_CONFIG through and never forced the update check off. None of the suites guarded Bundler's environment:

  • e2e_gem.rs: its unscoped scans assert the exact purl set the gem crawler sends upstream. The crawler reads BUNDLE_PATH, BUNDLE_APP_CONFIG, the user's ~/.bundle/config and the gem env homes. With an ambient BUNDLE_PATH pointing at a store that holds a gem, or a HOME whose .bundle/config sets one, scan_discovers_vendored_gems and scan_discovers_gems_with_gemspec both fail on main (left: Some(3), right: Some(2)).
  • vendor/vendor_gem_lockfile_only_e2e.rs and vendor/vendor_rerun_no_network_e2e.rs had the same partial sweeps, kept the real HOME, and scrubbed gem state only partly (BUNDLE_PATH / GEM_HOME by name, no GEM_PATH).

All three now spawn through hermetic::command with Extra::Bundle. Their gem-crawling CLI children also drop GEM_HOME / GEM_PATH and get the stand-in HOME from jvm_env::isolate_cli. e2e_gem and vendor_rerun_no_network_e2e add Extra::Venv. The bundle install child in e2e_gem scrubs SOCKET_* and BUNDLE_* before setting its own BUNDLE_PATH. The two cargo-only spawns in vendor_rerun_no_network_e2e already pin CARGO_HOME, so they only move to hermetic::command. The three files leave PENDING_RAW_SPAWNS, so the spawn_env_hygiene ratchet keeps them hermetic.

This PR doesn't touch common/hermetic.rs, so it doesn't overlap with #1498 / #1502 there. All three PRs remove different lines from PENDING_RAW_SPAWNS, so whichever lands later needs a trivial rebase of that list.

Test-only change; no product code touched.

CI cost

No workflow change. A lean PR run has the same job count (~28 Linux jobs) and job-minutes before and after, because these suites already run in the same jobs.

Test evidence

  • cargo fmt --all -- --check: clean. cargo clippy --workspace --all-features -- -D warnings: clean. Clippy on the e2e_gem / vendor / spawn_env_hygiene test targets reports only existing lints in files this PR doesn't touch (prebuilt_common/mod.rs, e2e_golang_redirect.rs, duplicate-module warnings).
  • Red on main (cargo test -p socket-patch-cli --all-features --test e2e_gem):
    • BUNDLE_PATH=<store holding rogue-9.9.9>: 21 passed, 2 failed (scan_discovers_gems_with_gemspec, scan_discovers_vendored_gems).
    • HOME=<home whose .bundle/config sets BUNDLE_PATH to that store>: the same 2 failed.
  • Green on this branch, under each of: no override; BUNDLE_PATH=<store>; BUNDLE_APP_CONFIG=<config dir>; HOME=<that home>; GEM_HOME/GEM_PATH=<store>; and SOCKET_ECOSYSTEMS=npm SOCKET_NO_CONFIG=0 SOCKET_DRY_RUN=true SOCKET_CLI_API_TOKEN=x VIRTUAL_ENV=<dir>:
    • --test e2e_gem: 23 passed (3 live-API tests ignored, as in CI)
    • --test vendor -- vendor_gem_lockfile_only_e2e vendor_rerun_no_network_e2e: 20/20
    • --test spawn_env_hygiene: 15/15
  • The 3 ignored e2e_gem tests need the live Socket API and Bundler. CI runs them on demand only (ci.yml:1169-1178), and they weren't run here. Their only change is the hermetic env on the CLI child, plus the SOCKET_* / BUNDLE_* scrub on the bundle install child that runs before it sets BUNDLE_PATH.
  • CI on abd04cc: green (run 38470744989), mergeable state clean. Cursor Bugbot reports no issues.

/code-review high

  • Fixed: the vendor suites kept the real HOME, so ~/.bundle/config and ~/.gem could still reach the crawl. GEM_PATH was never removed. The e2e_gem doc comment claimed more isolation than the code gave.
  • Not fixed:
    • Folding e2e_gem's private run / assert_run_ok into common::run_bin_with_env with an extras parameter. That touches every caller of the shared runner, so it belongs in its own change. Earlier slices made the same call.
    • Moving the gem-home and HOME handling into Extra::Bundle. Extra::Bundle also scrubs real bundle children, and those need an ambient GEM_HOME on version-managed Rubies (rvm, chruby). Doing that would also put a third PR on hermetic.rs while CI: hermetic spawns in NuGet/PyPI suites (#823) #1498 and CI: hermetic spawns in Cargo/Composer suites (#823) #1502 edit it. The removal stays on the CLI children.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J5o3ceqEyqmPQbY2WJMEB1


Note

Low Risk
Changes are limited to test harness environment scrubbing; no runtime CLI or production behavior is modified.

Overview
Gem and vendor CLI tests now spawn through the shared hermetic helper instead of bare Command::new plus hand-rolled SOCKET_* loops, so behavior matches tests/common/hermetic.rs (forced opt-outs, no drift on SOCKET_NO_CONFIG / update checks).

In e2e_gem, the CLI run helper uses hermetic::command with Extra::Bundle and Extra::Venv, clears GEM_HOME / GEM_PATH, and stand-in HOME via jvm_env::isolate_cli so unscoped scan assertions on exact PURL sets cannot pick up the developer’s Bundler or gem homes. bundle install scrubs SOCKET_* and BUNDLE_* before pinning BUNDLE_PATH on the fixture.

vendor_gem_lockfile_only_e2e and vendor_rerun_no_network_e2e get the same hermetic spawn for vendor (plus bundle/venv scrub and gem-home isolation where the crawlers read Ruby/Python state). Cargo-only vendor paths in the rerun suite only switch to hermetic::command while still setting CARGO_HOME.

Those three files are removed from PENDING_RAW_SPAWNS in spawn_env_hygiene.rs, tightening the ratchet that forbids new bare binary spawns. Test-only; no product code changes.

Reviewed by Cursor Bugbot for commit abd04cc. Configure here.


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
e2e_gem, vendor_gem_lockfile_only_e2e and vendor_rerun_no_network_e2e
started socket-patch with a bare Command::new and their own SOCKET_*
sweeps, which let an ambient SOCKET_NO_CONFIG through and never forced
the update check off. None of them scrubbed Bundler's environment: an
ambient BUNDLE_PATH, or a ~/.bundle/config that sets one, adds the
developer's gems to the crawl, so both gem scan tests fail on main
(3 gems found instead of 2).

All three now spawn through hermetic::command with Extra::Bundle;
e2e_gem's unscoped scans also scrub an activated venv and use the
stand-in HOME, and its bundle install child scrubs BUNDLE_* before
setting its own BUNDLE_PATH. The files leave PENDING_RAW_SPAWNS, so
the spawn_env_hygiene ratchet keeps them hermetic.

Refs #823

Assisted-by: Claude Code:claude-opus-5-5
The ruby crawler also reads the user's ~/.bundle/config and the
`gem env` homes (GEM_HOME, GEM_PATH, ~/.gem). The two vendor suites kept
the real HOME, and none of the three removed GEM_PATH, so a developer's
global bundler path or gem home could still change which gem store
these tests crawl. Give the vendor suites' CLI children the stand-in
HOME e2e_gem already uses, and remove both gem-home vars from every
gem-crawling CLI child.

Refs #823

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 13, 2026 21:27
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit abd04cc. Configure here.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 13, 2026
…-gem-vendor

# Conflicts:
#	crates/socket-patch-cli/tests/spawn_env_hygiene.rs
Merged via the queue into main with commit 7be10d1 Oct 14, 2026
53 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the agent/fix-ci-hermetic-gem-vendor branch October 14, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants