Repository navigation
gc-stress: test_gap_repsel_gc_stress fails on every full mark-sweep arm since ~2026-09-03 (suspect: aa8d2ade0 stops marking the remembered set on full traces) #9782
Description
gc-stress matrix (4/4) has been red on main since roughly 2026-09-03. It is a
deterministic failure — byte-identical counters across every run and every SHA I
checked — and the arm split points at a full-collection root-scanning bug, i.e. a
potential collected-live-object class of defect.
The failure
run [8/20] test=test_gap_repsel_gc_stress env=_base arm=gen_gc_off
FAIL test_gap_repsel_gc_stress (output-mismatch cycles=43 evacuated=0 scavenged=0 reclaimed=1327926376)
run [8/20] test=test_gap_repsel_gc_stress env=PERRY_WRITE_BARRIERS_0 arm=wb_off
FAIL test_gap_repsel_gc_stress (output-mismatch cycles=43 evacuated=0 scavenged=0 reclaimed=1327926376)
run [8/20] ... arm=gen_off_verify -> FAIL (same signature)
run [8/20] ... arm=wb_off_force -> FAIL (same signature)
run [8/20] ... arm=all_four -> FAIL (same signature)
repsel_gc_stress PASS PASS PASS PASS PASS PASS FAIL FAIL FAIL FAIL FAIL PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS
summary: PASS=360 UNVER=75 XFAIL=0 FAIL=5
Example run: https://github.com/PerryTS/perry/actions/runs/33829521137 (job 100904513692).
Identical on later runs, e.g. https://github.com/PerryTS/perry/actions/runs/33926006467.
The discriminating evidence: which arms fail
Exactly five arms fail — gen_gc_off, wb_off, gen_off_verify, wb_off_force,
all_four — and those are precisely the arms that disable generational GC or write
barriers, forcing a full / mark-sweep collection instead of a generational minor. All
fifteen other arms pass, including default, evac, force-evacuate and every
representation-selection off-switch.
evacuated=0 scavenged=0 with a large reclaimed=1327926376 is consistent with a
non-moving full sweep reclaiming substantially more than it used to.
Window
| SHA | gc-stress matrix (4/4) |
run |
|---|---|---|
dae0ae34c8cf |
green | 33796583016 (2026-09-03 19:27) |
d239929b4c46 |
red, this signature | 33814875005 (2026-09-03 22:49) |
Every push between those two had its CI run cancelled (superseded) before any job started,
so there is no direct evidence for the intermediate SHAs. That leaves a 17-commit range,
dae0ae34c8cf..d239929b4c46.
Prime suspect — correlation only, NOT a proven bisection
aa8d2ade0 — "fix(gc): stop two passes retaining unreachable objects (#9628, #9629)"
(2026-09-03 22:01) is the only commit in the range that touches GC-cycle root-scanning
semantics. In crates/perry-runtime/src/gc/cycle.rs it makes the remembered set a root
source only for minor collections:
let mark_from_set = self.minor_only; // #9629: only a MINOR needs old->young
// edges as roots; a full trace reaches
// them through the live old objects
// themselves...So on any non-minor (full) collection the remembered set is no longer marked as roots —
which is exactly the code path the five failing arms force. test_gap_repsel_gc_stress is
purpose-built to hold representation-selected locals (Ptr<Shape>-proven objects and
friends) live across escaping churn, which is the shape that would break if such an object's
old-gen reachability depended on the remembered-set path now skipped on full traces.
The very next commit, 51bfe741d ("test(gc): the remembered-set slicing tests drive a
minor, not a full cycle"), rewrote two unit tests to match the new behaviour — so the
reduced retention was intended, but this stress fixture suggests an edge case those unit
tests do not cover.
Stated plainly: this is a code-review correlation, not a bisection. No CI run exists at
aa8d2ade0 itself, so it is not isolated from its 16 neighbours in the range. I did not run
the fixture at aa8d2ade0 and its parent because scripts/gc_repsel_matrix.sh requires a
full compiler build (cargo build --profile <p> of perry plus the runtime/stdlib
staticlibs) before it can compile and run the corpus — two of those plus two matrix runs is
well beyond a cheap check.
Correction: an earlier triage of mine attributed this to the #9644 idle-compaction
commits (3812cc973, 42cd82fdf). That was wrong — both land after d239929b4c46,
which was already red. Please do not chase them.
Paths
- Fixture:
test-files/test_gap_repsel_gc_stress.ts - Harness:
scripts/gc_repsel_matrix.sh(computes the cycles/evacuated/scavenged/reclaimed line and the arm matrix) - Tier driver:
scripts/release_sweep_tiers/tier04_gc_stress.sh - Suspect code:
crates/perry-runtime/src/gc/cycle.rs - Suspect's own tests:
crates/perry-runtime/src/gc/tests/retention_9628_9629.rs
Suggested next step
Build at aa8d2ade0 and at its parent 658007137, run
scripts/gc_repsel_matrix.sh restricted to test_gap_repsel_gc_stress and the
gen_gc_off arm, and compare. That converts the correlation into a bisection in two
builds. If it reproduces, the question is whether a full trace can in fact always reach
old->young targets without the remembered set when the young half was already swept.
Activity
proggeramlug commentedon Sep 5, 2026
Fixed by #9812, landed on
mainvia merge train #9817. The train merged as its own branch, so the close-keyword never fired — closing manually, verified onmainby patch-id.