Repository navigation
CI: required contexts lint and cargo-test are red for every PR (and lint on main itself), so every merge bypasses them #8092
Description
Summary
Two required status contexts, lint and cargo-test, currently fail for every pull request, including on trees that change nothing related to them. lint also fails on a pristine origin/main checkout. While that holds, neither context can distinguish a good PR from a bad one, and every merge necessarily goes through admin bypass.
This is the CLAUDE.md "★ four ways a gate can be unable to fail" pattern in its most expensive form: the jobs run, genuinely report failure, and are required — so the only way to land anything is to ignore them, which trains the team to ignore them for the PR that actually broke something.
lint is red on main itself
Reproduced locally on a clean detached worktree at origin/main = 506f4ab11a5d2a1b7a498e965c79d11e2fa63043, running the exact commands from test.yml:
$ ./scripts/check_file_size.sh
File size limit exceeded (2000 lines).
2010 crates/perry-runtime/src/timer.rs
$ python3 scripts/gc_store_site_inventory.py
GC store-site inventory failed; add nearby GC_STORE_AUDIT markers:
crates/perry-codegen/src/expr/property_set.rs:1475: raw generated heap/global store: blk.store(DOUBLE, &numeric_value, &field_ptr);
$ python3 scripts/raw_handle_debt.py
bare raw-handle reads: 996 (baseline 998)
per-module raw-handle rules: 2 violation(s)
crates/perry-runtime/src/json_tape.rs: 22 bare reads exceeds its ceiling of 21
crates/perry-runtime/src/object/spill.rs: 5 bare reads exceeds its ceiling of 3
All three are independent of any open PR. Note the global raw-handle count (996) is below its baseline (998) — it is the per-module rules that fail, which is exactly the failure mode raw_handle_debt_files.txt was designed to catch, so the gate is working correctly and the debt is real.
cargo-test fails identically on unrelated PRs
The same four tests fail on #8086 (73-file ShapeId change), #8090 (small typed-array fix), and #8087 (CI gates):
inprocess::tests::rs4gc_canonicalizes_construction_time_folds_before_root_liveness
native_emit::tests::native_construction_lowers_precise_roots_before_rs4gc
native_emit::tests::split_native_construction_lowers_precise_roots_before_rs4gc
native_emit::tests::split_native_construction_propagates_shadow_backend_to_workers
Byte-for-byte the same set on all three. These assert that two compilation arms produce byte-identical objects. Decoding the assertion dumps, the arms differ only in an embedded module/source name:
| test | control arm | native arm |
|---|---|---|
rs4gc_canonicalizes_… |
.file "constant_fold_text" |
.file "constant_fold_native" |
native_construction_… |
ELF STT_FILE = perry_llvm_7386555fbfde02f7.ll |
ELF STT_FILE = perry_native_module |
The generated code, stack maps, and PGCM payloads appear identical; the delta is the name each arm records for its translation unit. compile_ll_to_object routes through a scratch perry_llvm_<hash>.ll, while the in-process native arm names its module perry_native_module.
This is consistent with the platform split: both ELF arms of native-roots-rs4gc fail, while macOS/Mach-O and Windows/PE pass — Mach-O and PE do not carry an STT_FILE-style source-name symbol, so the difference is invisible there.
I have not bisected which change introduced this, and I have not confirmed it on a pristine main build (it does not reproduce on macOS, which is the only host I measured on). What is established is that it is not caused by any of the three PRs above.
Also red across the same unrelated PRs
Not diagnosed here, listed so the scope is not understated — every one of these is red on both #8086 and #8090:
gc-stress, ext-link, compiler-output-regression, e2e-scoped, gc-native-roots-complete, gc-root-dominance-statepoints, windows-arm64-build, Warnings (host-compatible), Warnings (product), and both ELF native-roots-rs4gc arms.
The last run of test.yml to conclude success on any branch was 2026-07-24.
Why this is worth fixing before more merges
A required-and-always-red context is strictly worse than no context: it is indistinguishable from a gate that is doing its job, it forces bypass as the normal path, and the bypass is what lands the next real regression. #6925's PERRY_PTR_SHAPE_LOCALS=0 regression landed visibly red and survived three merges for exactly this reason.
Suggested split
- Land the three
lintfixes againstmaindirectly —timer.rsis 10 lines over the cap,property_set.rsneeds oneGC_STORE_AUDITmarker, andjson_tape.rs/object/spill.rseach need one converted pair (which also lowers their ceilings, per the file's own "lower a ceiling whenever you convert a pair" rule). - Decide whether the four byte-identity tests should compare normalized objects (masking the translation-unit name) or whether both arms should name the module identically. A byte-identity assertion whose only observed delta is a filename is testing the filename.
- Triage the remaining red jobs separately; several may share the ELF/
STT_FILEroot cause.
Activity
proggeramlug commentedon Aug 14, 2026
Opened #8093 for the
linthalf — the three violations reproduced above on pristinemain.The other twelve red contexts (
cargo-test,gc-stress,ext-link,compiler-output-regression,e2e-scoped,gc-native-roots-complete,gc-root-dominance-statepoints,windows-arm64-build,Warningsx2, and both ELFnative-roots-rs4gcarms) are untouched and still need owners. Thecargo-testfour are the best-characterised: the arms differ only in the embedded translation-unit name, and the platform split (ELF fails, Mach-O and PE pass) is consistent with anSTT_FILEsymbol that only ELF carries.proggeramlug commentedon Aug 14, 2026
Additional finding:
perry-codegen's integration suites are 11 tests red onmain, and per-PR CI never runs themCLAUDE.md already notes that integration suites under
crates/*/tests/*.rsrun nightly/tag only, so "a regression there can land green and sit red for days" (#5960). Measuring it: on a clean detached worktree atorigin/main=506f4ab11, macOS/arm64:The eleven:
Seven of the eleven cluster on buffer/typed-array numeric-read proofs (
artifact_records_buffer_read_*,explicit_width_guard_proves_wide_buffer_read,proven_buffer_and_typed_array_reads_are_numeric_operands), which looks like one regression rather than seven, and squarely in the representation-selection area the perf campaign cares about.Two practical notes for anyone running this:
cargo test -p perry-codegenwithout--no-fail-faststops at the first failing test target. Without the flag the run aborts inloop_safepoint_purityafter 10 of 27 suites and reports1021 passed; 2 failed, which reads like a much smaller problem than it is. That is how I first saw it.native_emit/inprocessbyte-identity failures described above are a separate set and do not appear here, so the Linux total is likely larger.Worth its own issue if someone picks up the buffer-read cluster; recording it here so the number is written down.
proggeramlug commentedon Aug 14, 2026
The
linthalf is fixed — #8087 merged as514ff7c9aVerified on a clean worktree at the new
origin/main:All three steps that were red on a pristine checkout are green.
timer.rsis 1975 lines, theproperty_set.rsmarker sits against its store, and the raw-handle debt fell 998 → 992 by converting four reads toRuntimeHandle::across_mut, not by raising a ceiling.Also confirmed the merge did not carry the version bump:
Cargo.tomlandCLAUDE.mdstill read 0.5.1510.Still open on this issue — none of these are addressed:
cargo-test: the four ELF-onlynative_emit/inprocessbyte-identity failures, where the arms differ only in the embedded translation-unit name.gc-stress,ext-link,compiler-output-regression,e2e-scoped,gc-native-roots-complete,gc-root-dominance-statepoints,windows-arm64-build,Warnings×2, and both ELFnative-roots-rs4gcarms.perry-codegenintegration failures recorded above.#8087 does contain plausible fixes for two of these — the
ext-linkfeature-group split and the ARM64 PowerShell dispatch fix — so those two are worth re-checking on the next PR that runs them.proggeramlug commentedon Aug 14, 2026
Correction and update: which required contexts are actually unsatisfied
I need to correct a claim in my earlier comments. I wrote that
parityandcompile-smoke"can never report on a PR". They do report — asSKIPPED. Measured on #8090, after all its checks completed:lintmainby #8087cargo-testparitycompile-smokeapi-docs-driftsecurity-auditconformance-smoke-completeSo there are three genuinely failing required contexts, not the "jobs that never report" story I told. That matters, because it changes what has to be fixed:
parityandcompile-smokebeingSKIPPEDis by design (v0.5.1018gated them to tag pushes, and both carrycontinue-on-error: true— they are informational), whereas the other two are real failures that independently explain why every merge needs admin bypass. I cannot isolate whetherSKIPPEDalso blocks, because the three failures are sufficient on their own.New finding:
conformance-smokeis failing 8 of 8 shardsNot just the aggregate — every shard:
CLAUDE.md warns that these shards are flaky and that a red shard should be re-run and A/B'd against a pristine
mainbuild before being believed. Eight of eight is not the flake signature, so this is worth treating as real until someone shows otherwise. It is a required context, so it is a third independent reason nothing can merge cleanly.Where this leaves the required-context list
lint— fixed, green onmainas of fix(ci): restore release readiness gates #8087.cargo-test— the four ELF-onlynative_emit/inprocessbyte-identity failures described above.conformance-smoke-complete— 8/8 shards, undiagnosed.parity,compile-smoke— deliberately skipped on PRs andcontinue-on-error. Whether they should remain in the required list at all is a maintainer decision: a context that is informational by design and never runs pre-merge is not gating anything. Removing them would be a branch-protection change, which I am not making unilaterally.Related: #8101 proposes promoting
gc-ratchet(verified green onmainfor four consecutive runs) and, after #8100,gc-stress. Worth sequencing those against this list so the required set ends up meaning something rather than growing.