feat(gc/shape): root and rewrite the keys edge from the ShapeId descriptor, not ObjectHeader.keys_array

Summary

ObjectHeader.keys_array is the last blocker for #8047's header shrink. #8086 made the ShapeId descriptor authoritative for the keys value, but the header word is still the only thing that roots the keys array and the only rewritable location the evacuator can hand to a slot visitor. Removing it unroots every keys array.

This issue is the missing GC protocol. It deliberately does not change ObjectHeader's size; #8047 performs that atomically afterwards.

The state on `main` (`12f758a22`), in the tree's own words

crates/perry-runtime/src/gc/layout_slot_visit.rs:63

// #8067: the header keys slot above is the sole strong edge.

crates/perry-runtime/src/object/shapes.rs:947-950 (scan_shape_table_rekey_mut)

/// Metadata-only forwarding repair for the weak descriptor table and
/// pointer-keyed slot indices. Mark/copy mode does not root anything; live
/// object scans provide descriptor reachability, and post-copy rewrite
/// follows only forwarding records those live edges already created.

crates/perry-runtime/src/object/gc_slots.rs:4 keeps the field deliberately, as a "Compatibility scratch slot", and names #8047 as where it is replaced:

// #8047 can replace this scratch with a descriptor-table rewrite without
// changing the source of the edge.

So today: the descriptor supplies the value; gc_keys_array_slot writes it into the header field; the collector marks and rewrites that word; and scan_shape_table_rekey_mut afterwards repairs the weak table by following forwarding records the header edge already created.

Why the obvious implementation is wrong

Promoting ShapeDescriptor.keys to a GC slot is ruled out in layout_slot_visit.rs:64-68:

// Never enumerate the HashMap bucket as a GC slot: dirty-page work may
// retain enumerated slot addresses across budgeted resumptions, during
// which descriptor insertion can reallocate the table.

The descriptor lives in a rehashing PtrHashMap, and the incremental collector retains enumerated slot addresses across budgeted resumptions.

Required design

Stable-address descriptor storage. A chunked/arena-backed record store whose keys field has a fixed address for the descriptor's lifetime, so the collector can enumerate and rewrite it across budgeted resumptions. The HashMap may remain as an index into that store, but must not own the edge.
An ephemeron / liveness protocol. The table must root keys only while some live object still carries that ShapeId; unconditional rooting makes every keys array ever minted immortal. Note prune_dead_shape_keys currently asks "is the keys array dead?" — that becomes circular the moment the table roots it, and must become "was this id observed on a live object during mark?" (e.g. a mark-time ShapeId bitset, then a fixpoint round because keys arrays contain strings).
Evacuation and verification. PERRY_GC_FORCE_EVACUATE, PERRY_GC_VERIFY_EVACUATION and PERRY_GC_PROTECT_FROMSPACE must all pass with the moving collector demonstrably live — assert the subject ran (copied_objects > 0 or promoted_objects > 0), not merely that nothing threw.
Sibling and agent isolation preserved. shape_drop_does_not_delete_a_potential_siblings_descriptor, shared_sibling_append_clones_before_descriptor_version_changes, a_foreign_agent_id_misses_instead_of_aliasing_same_address and process_global_module_shape_id_installs_with_agent_local_keys must keep passing.

Acceptance

GC mark, rewrite and evacuation recover and rewrite a keys array without reading or writing ObjectHeader.keys_array — the field may still exist, but nothing in the collector consults it.
A keys array reachable only through the descriptor table survives a collection; a keys array whose last carrying object died is reclaimed (both directions tested — an immortality bug is as real as a use-after-free).
The instrument is sabotage-tested: plant a stale descriptor edge and show the canary distinguishes it from a live object, in the manner of quarantine_catches_a_planted_stale_from_space_deref.
ObjectHeader size is unchanged by this issue.

Refs #8047, #8067, #8086, #7154, #7164.

@/tmp/claude-501/-Users-amlug-projects-perry-perry/57b8a088-32db-4195-8398-07486e2e83d0/scratchpad/8112-comment.md

Independently verified the two load-bearing claims. They hold, and together they overturn this issue's own framing — requirement (1), stable-address descriptor storage, is not needed.

1. The keys edge is already rewritten without the header. object/shapes.rs:953 — scan_shape_table_rekey_mut walks inner.descriptors.values_mut(), lifts descriptor.keys into a local usize, passes it to visit_metadata_usize_slot, and writes back:

for descriptor in inner.descriptors.values_mut() {
    let mut addr = descriptor.keys as usize;
    if visitor.visit_metadata_usize_slot(&mut addr) {
        descriptor.keys = addr as u64;
        descriptor_moved = true;
    }
}

So copying evacuation, old-gen defrag and verification already recover the edge from the table. The header word contributes exactly one thing the table does not: a strong mark. That is a much smaller hole than "the sole strong edge" implied to me when I read layout_slot_visit.rs:63 — that comment is true about marking and I over-read it as being about rewriting too.

2. The HashMap-bucket prohibition is protocol-specific, not universal. object/descriptor_state.rs:1145 does precisely what the prohibition appears to forbid — mutates in place inside a rehashing HashMap, through the root-scanner protocol:

for acc in descriptors.values_mut() {
    if acc.get != 0 { visitor.visit_nanbox_u64_slot(&mut acc.get); }
    if acc.set != 0 { visitor.visit_nanbox_u64_slot(&mut acc.set); }
}

So "Never enumerate the HashMap bucket as a GC slot" is a constraint on the layout-slot protocol, whose buffering consumer is DirtyHeaderSlotScan holding addresses across budgeted resumptions. Taking the keys edge out of that protocol is a second exit, and it needs no stable addresses at all.

That distinction is the crux, and it is why I wanted to check rather than accept: my own #8047 read treated the prohibition as global.

What this changes for #8047

The blocker is smaller than the census concluded. Two further findings sharpen it: the class-keys and compile-time-literal populations are already immortal (shape_cache_overflow has no production removal site), and the benchmarks carrying the prize — retain, retain_wide, tree — are object-literal workloads that need no ephemeron whatsoever. So a first cut can be scoped to the runtime-minted tail.

Also worth weighting: this removes collector work rather than adding it. Three object_shape_descriptor lookups per layout visit of a GC_TYPE_OBJECT today, each TLS + RefCell::borrow + HashMap::get, plus a store into (*obj).keys_array on every walk including read-only ones. Mutator cost of the proposed design is zero, which matters given the standing directive that GC work must never cost the mutator hot path.

The unsettled item is the right one to settle next

#descriptors in a real workload is unmeasured, and it is what decides whether the per-cycle pass is free. That is the measurement to take before any implementation — it is cheap next to the design, and it is the number that could invalidate the approach.

Two traps from the writeup worth keeping visible: cycle.rs is 1991 lines against the 2000 cap, so a new subphase body cannot live there; and post-#8047 a wrongly pruned descriptor is worse than a lost keys array, because gc_field_slot_range takes the traced field count from it once field_count is gone — hence "never remove descriptors, only unroot their keys".

fix(gc/shape): root and rewrite the keys edge from the ShapeId descri…

Implemented in #8221 (ready for review). Two results worth recording against this issue's framing:

The census came first. Peak descriptor population over the 14 gc_ratchet probes plus a 1 MB JSON-parse kernel is ~4 250 descriptors naming ~1 150 distinct keys arrays, dominated by a fixed bootstrap cohort (steady-state workload contribution 1–229 per cycle). So no exotic data structure is warranted, and requirement (1) is satisfied by simply boxing the record.

Requirement (1) is necessary, and requirement (2) is necessary for a reason the issue did not state. The lift/write-back scan_shape_table_rekey_mut already performs covers the rewrite only, so a stable address is still needed for a strong, rewritable edge. And the tempting shortcut — emit the descriptor edge once per TRACED receiver and let per-object liveness be the ephemeron relation, with no table state at all — is wrong for a generational minor. PERRY_GC_VERIFY_EVACUATION aborted on it:

old-young-edge-verifier failed: missing_edges=1
  parent=… type=object old_arena=true marked=false
  slot=… child=… child_type=array slot_page_ever_dirty=false

A minor never enumerates old carriers, and the record is SHARED, so one sibling's rewrite creates an old→young edge for a parent the minor never visits and which no per-parent remembered-set page can describe. The gate landed as per-descriptor old_carrier state, armed by the slot visitor for any non-nursery receiver and recomputed by every full trace — not the mark-time ShapeId bitset plus fixpoint the issue sketched, which would have had the same blind spot.

Acceptance: mark, rewrite and evacuation all recover a keys array with the header mirror suppressed (the #8047 rehearsal); a keys array reachable only through the descriptor survives; one whose last carrier died is still reclaimed; the instrument is sabotage-tested by suppressing the descriptor edge as well; ObjectHeader size unchanged.

fix(gc/shape): root and rewrite the keys edge from the ShapeId descri…

perf(runtime): store shape descriptors in an id-indexed slab and reti…

perf(gc): borrow the shape record in the slot visitor, stop moving th…

Navigation Menu

Repository navigation

feat(gc/shape): root and rewrite the keys edge from the ShapeId descriptor, not ObjectHeader.keys_array #8112

Summary

The state on `main` (`12f758a22`), in the tree's own words

Why the obvious implementation is wrong

Required design

Acceptance

What this changes for #8047

The unsettled item is the right one to settle next

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

feat(gc/shape): root and rewrite the keys edge from the ShapeId descriptor, not ObjectHeader.keys_array #8112

Description

Summary

The state on main (12f758a22), in the tree's own words

Why the obvious implementation is wrong

Required design

Acceptance

Activity

proggeramlug commented on Aug 14, 2026

proggeramlug commented on Aug 14, 2026

What this changes for #8047

The unsettled item is the right one to settle next

proggeramlug commented on Aug 16, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

The state on `main` (`12f758a22`), in the tree's own words