gc: the budgeted full step is starved by the shape-descriptor prune memmove — +59.5% peak RSS at tenuring threshold 2 #9871
Description
Rewritten 2026-09-06 after profiling the idle window. The original idle-CPU half of this issue is struck: it was measuring a different, already-known defect. What remains is the peak-RSS half, restated around the mechanism the profile found.
Struck: the idle-CPU half
The issue originally claimed that at a tenuring threshold of 2 the idle window's 23–25 s of CPU was deferred page release. That is wrong. Two independent results:
- No copying happens in the idle window at all. Across eight arms on two protocols there is not one
[gc-copy-minor] ranline inside any idle window, and copied bytes in the window are 0 on every arm — including the=2arms that cost 27–65 s. - The
=1and=2idle windows run identical event inventories — same line kinds and the same counts of each (4 idle-reclaim, 3 full collections, 2 old-free, 3 general-reclaim, 3 restore-coverage, 15 young-log, 3 idle-compact, 1 copy-minor), with compaction pauses of 317 ms and 417 ms and comparable live heaps (387 MB, 367 MB) — at 5.5 s versus 64.8 s. Deferral would show as more collections or bigger ones. These are the same collections at the same size.
A 30 s main-thread sample of each idle window says what it is:
=1 |
=2 |
|
|---|---|---|
| collector (by ancestor) | 568 (2.4 %) | 24,271 (100.0 %) |
| mutator | 23,236 (97.6 %) | 0 |
kevent — genuinely parked |
23,212 | 0 |
At =1 the process is idle. At =2 it never idles: every sample is collector work. Inclusive: gc_budgeted_step_work_units_inner_with_progress → GcCycleState::step → prune_dead_owner_side_tables_post_trace (23,426) → prune_dead_shape_keys (23,367). Self: _platform_memmove +22,820, which is 94 % of the whole difference, then scan_shape_table_rekey_mut +340, ShapeTableInner::facts_remove +223, remove_descriptor_indexed_under +150.
That is the same symbol under the same caller as the turn-CPU mode split (__memmove under remove_descriptor_indexed_under ← prune_dead_owner_side_tables_post_trace, identical counters carrying 2× the time) — the O(list) descriptor removal, not tenuring and not release. See #9881 and #9857.
What remains: peak RSS, restated
At =2, peak RSS is +59.5 % against =1 on the quiet host (2208–2244 MB vs 1384–1511 MB, three repeats, disjoint), and the budgeted steps in turns 2–3 run 13 and 2 times against 20 and 16, general-reclaim 0–1 times against 14–17, releasing zero blocks against 39 and 13.
The original explanation was "a full is priced by old-gen yield, so a higher threshold makes it read unproductive". The profile suggests a simpler one: those steps report pct=0% because each budgeted step spends its entire budget memmoving descriptor tails in prune_dead_shape_keys and never reaches the sweep that releases pages. The step is not mis-priced; it is starved — by the same O(list) removal as above. At =2 more objects are still live as owners when the full runs, so there is more to prune, so the starvation is worse.
Falsifier
=2 built with #9881's swap-remove (perf/family-swap-remove-main @ 01313fc2e) should release pages during turns 2–3 and bring peak RSS toward =1, on the mini.
- Pages released in turns 2–3 goes from 0 to non-zero and
[gc-step]pctrises above 0 → the starvation account holds. - Peak RSS does not move while the step cadence and
pctdo → the starvation is real but is not what holds the pages, and this issue is wrong a second time and should be closed rather than reshaped again.
Precondition (unchanged, step 0)
Run =1 and =2 on the host first and confirm peak RSS separates. On the dev box it does not (2298 vs 2230/2790, overlapping), so no arm there can decide this; the mini is the host where it separates. If the separation is absent, stop — every downstream number is undecidable, and that is a host result, not a refutation.
Method note
Both the copying-in-idle hypothesis and the deferred-release hypothesis were refuted from captures that already existed, by counting events rather than reasoning about them. The thing that finally named the cost was a profile, because the cost is per-unit and no counter the collector emits can see it. Third instance today; each time the profile named it, and twice now it named the same symbol.
Activity
[-]gc: a full is priced by old-gen yield, so a higher tenuring threshold stalls page release — +59.5% peak RSS and 24 s of idle CPU[/-][+]gc: the budgeted full step is starved by the shape-descriptor prune memmove — +59.5% peak RSS at tenuring threshold 2[/+]proggeramlug commentedon Sep 6, 2026
Closing: the falsifier fired the way I registered it, and it fired against the issue.
The kill condition in the rewritten body was: if the step cadence and
pctmove but peak RSS does not, this issue is wrong a second time and should be closed rather than reshaped again. On the quiet host (17 runs, load 1.88, every runexited=true) the cadence moved completely and peak RSS did not. So it is closed.The mini's result
Step 0 (precondition) met, at n>1.
=1's own peak is bimodal — 2279 / 1382 / 1495 / 1490 / 1384 — but four of five=1draws separate from all five=2draws, and an independentcc_tenure_candcontrol at=1read 1405. The host can see the phenomenon.Arm C binds on the mini on every repeat — turn-3 steps/general-reclaim/released 17 / 15 / 36–33–36. The dev box's "only B binds, so
GROWTHis the effective knob" was host-specific, and I had already withdrawn it at n=2 for a different reason. C restores the cadence completely and peak RSS reads 2292 / 2298 / 2304 against A's 2231–2242 — +3 %, the wrong direction.B's lower peaks (1753 / 2205 / 1735) are confounded and do not rescue the hypothesis. B runs promote 628–636 MB over 293 minors — 20 % less promotion and 24 % fewer minors than A — while C's collection volume equals A's. The within-B control settles it: the B draw whose volume was A-like (
rm1: cadence bound, 12 blocks released) also had an A-like peak of 2205.So peak RSS tracks collection volume, not blocks released. Releasing pages during the turns is exactly what this issue predicted would lower the peak, and it does not.
Both halves, for the record
=10.5–2.0) and tracks neither cadence nor peak. That is the profile's mechanism, not this issue's: the burn isprune_dead_shape_keys→remove_descriptor_indexed_under→_platform_memmove, present only when the large family exists at the moment a full runs.What was actually being measured
Round-2 family counters name it: the slow mode is one or two minors bulk front-removing a ~512 k-entry family whose only key is
"0", and the fast draws are the ones where a minor pruned that family before it passed 100 k. One burst of ~512 k single-key objects, twice per turn, and when a minor lands relative to that burst decides the prune cost (the turn-CPU mode), the idle burn (the same prune inside the full), and very probably the peak itself: at=1the burst mostly dies in the nursery — low peak — unless the coincidence keeps it alive to promotion (the 2279 outlier), while at=2aging keeps it alive every time (2231–2242, tight).Which means every tenuring measurement taken on this pre-#9857 binary has been measuring that burst. #9857 removes it on claude-code. That is the real reason to close rather than reshape: the workload these arms were run against is not the workload that will ship.
Related: #9881 (the O(list) removal itself), #9857 (removes cc's owners), #9861 (the tenuring branch, whose memory column now waits for a measurement on the landing base).