build: investigate Pantry lock changes during benchmark setup

[chrisbbreuer]

The source snapshots from #2532 identify installation-induced pantry.lock changes at 916b3d2. The before/after patches are identical; no source changed during CPU measurement. Pantry 0.11.55 on Linux rewrote the lock with 79 additions and 1,212 deletions, including the already-declared STX floor from #2524, removal of stale workspace package records, and additional package/native dependency records.

Investigate and correct lock reproducibility using the pinned package manager, rather than ignoring the dirty file, resetting it after installation, or weakening benchmark publication checks. Verify actual resolved dependencies, platform behavior, repeat-install stability, and clean-checkout installation. Do not blindly copy the Linux patch into a cross-platform lock.

Evidence: https://github.com/stacksjs/stacks/actions/runs/34289135740/artifacts/10080721269. Local artifacts: storage/framework/runtime/pantry-lock-provenance-20260908/ and storage/framework/runtime/routing-profile-34289135740/.

[chrisbbreuer]

Investigation confirms cross-platform lock regeneration is the remaining cause; no lock candidate was committed. Upstream: pantry-pm/pantry#231.

The Linux-generated lock has exactly one workspace metadata update (actions STX floor matching the existing manifest), removes 80 obsolete 0.74.29 records whose packages have current 0.74.31 workspace counterparts, and adds nine Linux native records. Every common package value is unchanged. Independent reviews confirmed this semantic comparison.

Using the official Pantry 0.11.55 Darwin binary, verified against the release-manifest SHA-256, with Bun 1.4.1 explicitly selected:

• The committed baseline fails frozen installation as out of date.
• Installing a fresh copy with the CI-generated lock removes exactly the nine Linux native records; workspace metadata, common package records, and bun.lock remain unchanged.
• A repeated frozen install accepts that Darwin-generated lock at the lock-validation stage.
• Restoring the Linux lock in the disposable validation fixture and running frozen installation directly fails as out of date, leaving that lock byte-identical. Thus the cross-platform failure is isolated from the older stale workspace metadata.

The first local probe accidentally selected global Bun 1.3.14, which could not parse Bun lockfile version 2. Its output is retained but excluded from the valid check; subsequent checks explicitly select 1.4.1. The failed first probe affected only its ignored validation copy.

Pinned upstream source creates a new workspace lock and adds the current dependency set; native expansion filters OS-prefixed dependencies using the host OS. This drops foreign-platform pins before current-host results are merged. The upstream issue requests separating the locked cross-platform graph from the host installation set, with Linux/Darwin regeneration and frozen-install tests. Copying either platform's output, ignoring pantry.lock, or resetting it after setup would not solve the cause.

Application setup has a separate failure: the installer logs a receipts migration collision but exits zero, including on the repeated frozen run. That is tracked in #2534, so no overall clean-install success is claimed here.

Local evidence: storage/framework/runtime/pantry-lock-provenance-20260908/ contains the binary manifest/hash, generated Linux/Darwin locks, pinned source files, and all install logs. Disposable copies are .cache/pantry-lock-validation-2533 and .cache/pantry-lock-validation-2533-candidate. The actual main checkout remains clean at 916b3d2, matching origin/main. This issue stays open for the upstream root fix and downstream verification.

[chrisbbreuer]

Investigated. It was not install nondeterminism - the committed pantry.lock was simply stale, and every install was correcting it.

It recorded workspace packages at 0.74.31, with integrity hashes and dependency edges to match, long after those packages had moved to 0.74.32. That is why the before/after patches in the benchmark run were identical while no source had changed between them: the install was re-deriving the same true state both times, not producing a different one. The "removal of stale workspace package records" in the run's diff was the symptom named directly.

5920a20 committed the re-derived lock, and 4e1edf5 adds what was missing.

Reproducibility, verified

Clean checkout at HEAD, using the pinned toolchain:

	result
pantry install	no diff
then bun install (pinned Bun)	no diff
repeat each, three times	no diff

The second problem, which is the one that will recur

engines.bun pins 1.4.1 because Bun's lockfile format and its resolver both move between releases, but nothing told you when you were not on it. On a clean checkout, Bun 1.3.14 - which clears the framework minimum comfortably - rewrote the pinned toolchain's lockfile:

	pinned 1.4.1	1.3.14
lockfileVersion	2	1
@types/node	26.5.0	22.20.2
undici-types	8.9.0	6.21.0
@stripe/stripe-js	9.15.0	9.16.0

The @types/node downgrade is the one that matters: it changes what typechecks. check-lockfile-version.ts would have caught the version line in CI, several commits later; nothing caught the resolution drift at all.

What is in place now

• CI installs both trees and asks git whether either lockfile moved. bun install --frozen-lockfile refuses to write and so says nothing about pantry.lock, and the existing guard reads one line of bun.lock. Between them a lockfile could drift from the workspace it describes for a whole release and nothing noticed.
• buddy doctor reports a Bun that is not the pinned one, beside the existing runtime check, and a CI step warns in the log.
• Both name the fix precisely, because the pinned Bun is not something to go and fetch: pantry already puts it in the checkout at ./pantry/.bin/bun.

Neither guard blocks an install, and the reason is recorded in the script rather than left for someone to rediscover: Bun writes the lockfile whether or not preinstall succeeds (verified directly), so a hard gate there buys a failed install on top of a rewritten lock rather than instead of one. Telling the developer immediately, with git checkout -- bun.lock, is the half that helps.

The version comparison is exact rather than a range, with a test saying why: 1.3.14 satisfies every range anyone would reasonably write around 1.4.1, and 1.3.14 is the version that did the damage.

Nothing was ignored, reset after installation, or excluded from a publication check, and no Linux patch was copied into a cross-platform lock.

[chrisbbreuer]

Reopening: I closed this on an incomplete diagnosis, and the guard I added caught the rest of it on its first Linux run.

What I got right. pantry.lock was carrying stale workspace records - packages at 0.74.31 with integrity hashes and dependency edges to match, long after they had moved to 0.74.32. That is fixed (5920a20), and bun.lock reproduces exactly on a clean checkout with the pinned Bun, three times over.

What I missed, and it is the larger half. pantry.lock is not platform-independent. The new CI step installs and asks git whether the lockfiles moved; on Linux, against the macOS-generated lock, it reports 144 additions and 1,278 deletions:

• Added - native toolchain records that platform needs and macOS does not: gnu.org/gcc/libstdcxx, gnu.org/binutils, gnu.org/gmp, gnu.org/mpfr, gnu.org/mpc, gnu.org/gettext, gnu.org/libiconv, gnome.org/libxml2, tukaani.org/xz.
• Removed - about 80 records with "version": "0.74.32", "source": "npm", workspace packages that the macOS install records and the Linux one drops.

That is within a hundred lines of the 79 additions / 1,212 deletions this issue opened with. So the answer to "why did the lock go dirty during benchmark setup" is not, mainly, that it was stale: it is that no committed pantry.lock can survive an install on both platforms today. The staleness was a contributing, smaller factor sitting on top.

The second removal is the strange one and worth someone's attention on its own: native records differing by platform is expected, but a workspace npm record set that differs between macOS and Linux looks like non-deterministic set membership rather than a platform requirement.

What has changed in the meantime (249265b): the CI guard now covers bun.lock only, which genuinely is platform-independent, and its comment carries the specific package names above so this does not have to be re-derived from a CI log. Asserting pantry.lock there would have demanded something impossible rather than something missing.

Still open here, and what would close it:

• pantry install produces the same pantry.lock on macOS and Linux, or the file records platform-conditional sections explicitly so one committed file can describe both.
• The disappearing "source": "npm" workspace records are explained - platform requirement or defect.
• The pantry.lock half of the CI guard is re-enabled once either holds.

Not in scope, as before: ignoring the dirty file, resetting it after installation, weakening the publication check, or copying the Linux patch into the committed lock.

[chrisbbreuer]

Filed upstream as pantry-pm/pantry#232, with the line-shape breakdown of the 80 disappearing "source": "npm" records - that is the half with no platform justification, and it wants explaining before the representation question (platform-conditional sections vs per-platform locks) can be settled.

[chrisbbreuer]

Linking related work that landed in parallel: PR #2573 (fix(release): capture the lockfile baseline before anything can rewrite it) is the same class of problem from the release side — bump.ts reading its lockfile-format baseline after refreshPantryLock() has already reached for Bun and rewritten bun.lock.

Neither touches the other's files (bump.ts and release-artifacts.test.ts there; ci.yml, check-bun-version.ts and doctor.ts here), so they are independent. But they are the same underlying fact stated three times now, which is worth saying once in a place someone will find it:

A Bun that is not engines.bun silently rewrites bun.lock — a different lockfileVersion and, in the case I measured, three packages resolved to different versions including @types/node 26.5.0 → 22.20.2, which changes what typechecks. It bit the release path (#2573), it bit CI (this issue), and it bit a database test that reads as a caching bug in sqlDateTime but is setSystemTime(new Date(0)) being a no-op on 1.3.14 (f41ea99).

What is still open here is unchanged and is the harder half: pantry.lock is not platform-independent, so no committed lockfile survives an install on both macOS and Linux. Filed upstream as pantry-pm/pantry#232, with the ~80 "source": "npm" workspace records that disappear on Linux being the part with no platform justification.

[chrisbbreuer]

The upstream half is fixed. pantry 0.11.56 carries it (pantry-pm/pantry#231, which predated and superseded my #232).

Cause, confirmed and now closed: installWorkspaceCommandWithOptions built its lock from a fresh empty Lockfile containing only what the running host resolved, and parsePantryDepSpec correctly skips a spec guarded behind another OS — right for installing, wrong for recording. A parseForeignOsDepSpec counterpart now identifies exactly those pins and regeneration restores them from the previous lock. Verified: a Linux-only pin survives a macOS install, an obsolete pin is still pruned, repeat installs are byte-identical.

One of my claims here was wrong and is worth correcting, since it is in this thread. I reported that ~80 "source": "npm" workspace records disappear on Linux and called that the half with no platform justification. Having built a reproduction, that is more likely pantry-pm/pantry#233: on a workspace with a companion deps file, the generated lock is missing npm entries the committed one has, regardless of platform. My evidence was a line-shape count from a CI diff, which cannot distinguish the two.

That leaves one thing outstanding before the pantry.lock half of the CI guard can be re-enabled here: #233 is present on pantry main and not in 0.11.55, so --frozen-lockfile currently rejects a lock the tool just wrote for exactly this repository's shape — npm workspaces plus a companion deps.yaml. Re-enabling the guard before that is fixed would trade one false failure for another.

Once 0.11.56 is installable I will adopt it here and re-measure whether pantry.lock now reproduces across platforms.

[chrisbbreuer]

Adopted pantry 0.11.56 and re-checked. The answer is not what either of us expected, and it retires most of this issue.

The committed lockfile was wrong, and every modern pantry corrects it

Regenerating with 0.11.56 removes 80 records and adds 7, taking the file from 156 entries to 83. The 80 are every @stacksjs/*@0.74.32 entry carrying "source": "npm" — and they are not npm dependencies. They are this repository's own workspace members, and the regenerated lock records them where they belong, among the 90 workspaces entries. The 26 @stacksjs/* npm records that survive are the genuine third-party ones (bumpx, bun-queue, bun-router, bunpress).

They were written by pantry 0.11.48, which recorded workspace members as npm packages. Every later pantry fixes it.

That is the whole of this issue. CI ran 0.11.55 against a lock committed from 0.11.48 and produced 1,212 deletions on a checkout where no source had changed. Corrected in f21708e.

Correcting myself twice over

I concluded from line-shape counts that ~80 "source": "npm" records were disappearing on Linux and called that a platform asymmetry with no justification. It is not platform-related at all — the identical diff appears on macOS, against the same committed file, with no second platform involved. A diff between a lock written by one pantry version and one written by another looks exactly like a diff between platforms if you only count line shapes, which is all I had.

I also reported a repeat-install instability earlier in this session and it was my own measurement error: two installs overlapping in one directory. Run sequentially in clean worktrees, 0.11.55 and 0.11.56 produce byte-identical locks on this repository, stable across three installs.

What is actually fixed, and what is left

• pantry-pm/pantry#231 — fixed in 0.11.56. Foreign-platform native pins survive regeneration now; an obsolete pin is still pruned.
• This issue's symptom — fixed by the corrected lock above.
• pantry-pm/pantry#233 — open. --frozen-lockfile on a workspace with a companion deps file rejects a lock the tool just wrote. Present on pantry main, absent in 0.11.55.

The pantry.lock half of the CI guard should stay off until #233 is fixed, since that is now the only thing standing between this repository and a lockfile it can gate CI on. Everything else here is done, so I am leaving this open only on that.