Skip to content

GC: enumerate the unrooted runtime-side caches — the population no static checker can see #7231

Closed

Description

@proggeramlug

Summary

#7226 fixed three instances of a bug class nothing static can find: a runtime-side table that holds a GC pointer and is not a registered root. This issue is the enumeration nobody had done — a sweep of every thread_local! / static / OnceLock / Cell / RefCell holder of a GC pointer across all of crates/perry-runtime/src/, cross-referenced against the complete gc_register_mutable_root_scanner* registry.

The class matters because it is strictly worse than the #7154 stale-register class, and is detected by the opposite signal:

unrooted register (#7154) unrooted runtime cache
goes bad only if a collection lands in a few-instruction window at collection #0, permanently
reproduces intermittently deterministically
found by gc_root_dominance_check.py nothing static — the checker reads emitted IR and a runtime table is not in it

gc_root_dominance_check.py will never report any of these. That is not a gap in the tool; it is outside its subject.

Independently verified (I read the declaration, the allocator, the dereference, and grepped gc/ for a scanner)

1. process/env_misc.rs:1075 — CACHED_ENV, the process.env object. Highest impact in the list.

thread_local! { static CACHED_ENV: std::cell::Cell<f64> = const { std::cell::Cell::new(0.0) }; }

Populated in js_process_env_impl with crate::object::js_object_alloc(0, alloc_limit) — nursery, no _longlived. Dereferenced and written through on every process.env.X = v (env_misc.rs:934-944):

let obj = crate::value::js_nanbox_get_pointer(cached) as *mut crate::ObjectHeader;
… js_object_set_field_by_name(obj, key, val_f64);

grep -rn "CACHED_ENV" crates/perry-runtime/src/gc/ → nothing. process.env is touched by nearly every real Node program. The sibling PROCESS_FINALIZATION_OBJECT (process.rs:302) uses the same materialize-once-cache idiom and is rooted (scan_process_finalization_roots_mut, gc/mod.rs:614) — so this is an omission, not a design.

2. object/global_this/populate.rs:822 — ERROR_CONSTRUCTOR_PTR.

pub(crate) static ERROR_CONSTRUCTOR_PTR: std::cell::Cell<usize> = …;
ERROR_CONSTRUCTOR_PTR.with(|c| c.set(ctor as usize));   // ctor from js_closure_alloc — nursery

Dereferenced on the .stack compute path at error.rs:1342. No scanner. The canonical Error closure inside globalThis is rewritten via globalThis's structural trace; this raw-usize duplicate lives outside that graph.

3. tui/input.rs:38 — INPUT_HANDLER.

static INPUT_HANDLER: AtomicI64 = AtomicI64::new(0);

Holds the user's useInput((input, key) => …) closure — the only reference to it — and drain_input calls through it on every keystroke. No scanner. The idiomatic Ink-style inline-arrow pattern dies at the first GC.

Reported by the sweep, not yet individually re-verified by me

Listed so they are not lost; each needs the same four-step confirmation before being treated as fact.

  • process/permission.rs:195 CACHED_PERMISSION, process/report.rs:111 CACHED_REPORT — same shape as (1).
  • tty.rs:58 RESIZE_CALLBACK — process.stdout.on('resize', cb) stored in a native slot that bypasses the (rooted) EventEmitter listener array.
  • frame.rs:30 FRAME_CALLBACKS — onFrame(cb); rooted only transiently by a RuntimeHandleScope during registration.
  • object/class_registry/construct.rs:4 CURRENT_NEW_TARGET — an unrooted duplicate of the value NEW_TARGET (object/this_binding.rs:28) already tracks correctly via scan_implicit_this_roots_mut.
  • object/field_get_set/accessors.rs:199 ACCESSOR_RECEIVER_OVERRIDE — armed across a prototype walk that allocates.
  • object/collection_proto_thunks.rs:19 BUILTIN_MAP_SET_VALUE_BITS / BUILTIN_SET_ADD_VALUE_BITS; object/native_module/callable_exports.rs:1710 BUILTIN_CLOSURE_LENGTH / _NON_CONSTRUCTABLE — address-identity keys; stale after a move, and wrong after address reuse.
  • json/mod.rs:99 SHAPE_CACHE — its safety comment says "within one top-level stringify call no GC runs over the user object graph", which toJSON() falsifies.
  • Scanner gaps (the table is rooted, one field is not): timer.rs IntervalTimer.args — setInterval(fn, d, freshObject); the adjacent CALLBACK_TIMERS and MOCK_TIMERS blocks both loop args, INTERVAL_TIMERS does not. promise/rejection.rs internally_handled. fs/dir_glob_watch/watch.rs GLOB_ITERATORS. fs/stream.rs Utf8StreamState.mode_value.
  • Narrower/latent: node_repl.rs:14 RECOVERABLE_ERRORS, module_require.rs:240 MODULE_PATH_REGISTRY (may be independently rooted via a codegen module global — unresolved), child_process/v8_serde.rs:1048 id_table across streaming calls, dgram.rs:260 DGRAM_REGISTRY.bound (test-mode only), static_plugins.rs:16 (currently unreachable), event_target.rs:439 DOM_EXCEPTION_ERRORS, node_submodules/diagnostics.rs:282 symbol-keyed channels.

Negative results worth keeping

The sweep confirmed ~70 registered scanners by reading the scanner body, not by name. Notable:

  • object/alloc.rs:10 CLASS_KEYS_BY_ID is safe because every write site uses a _longlived allocator — the same fact that makes GC: the remaining unrooted-alloca hazards after #7207 — class-keys pointer caches, interleaved staging arrays, inlined-callee param slots #7210's 64 perry_class_keys alloca reports false positives.
  • object/prop_plan.rs caches are epoch-invalidated on every collection rather than scanned — a legitimate third strategy.
  • bun_ffi/types.rs: FFI_TYPE_OBJECT_CACHE is this exact shape, already correctly fixed — the pattern to copy.
  • gc/roots/scanner_shims.rs:69-78 overflow_fields_mutable_root_scanner is defined, re-exported, and never called (coverage is real via the structural per-object trace). Dead code with a doc comment asserting it is needed — the shape CLAUDE.md's kill-policy warns about.

Suggested shape of the fix

Not one PR. The three verified ones are independent and each is a scan_*_roots_mut + one gc_register_mutable_root_scanner line, following scan_process_finalization_roots_mut exactly. The scanner gaps are cheaper still (one loop each) and should go first, because a partially-correct scanner is more dangerous than an absent one: it reads as covered.

Each fix needs a witness of the kind #7226 established — PERRY_GC_ZEAL=1 PERRY_GC_PROTECT_FROMSPACE=1, red at base, faulting at the stale dereference rather than cycles later.

Refs #7226, #7210, #7154, #7196, #6910.

Activity

  1. proggeramlug commented on Aug 2, 2026

    @proggeramlug
    ContributorAuthor

    The sweep, completed — every candidate taken to a verdict

    Method for every row: read the declaration, the allocator, the dereference, and grep the identifier across crates/perry-runtime/src/gc/ and the holding module for a registration. Registry side: every gc_register_*root_scanner* call site enumerated, and every budgeted (FULL, STEP) pair diffed by reading both bodies rather than trusting the names.

    Fixes in #7239. Checker half in #7235. New issue from the sweep: #7236.

    REAL — fixed in #7239

    holder mechanism hazard
    process/env_misc.rs:1076 CACHED_ENV nursery js_object_alloc; the cell is the entire reference graph (process.env is a js_process_env() call, not a field of process) sweep and move. SIGBUS 10/10 at base under PERRY_GC_MOVING_LOOP_POLLS=1
    process/permission.rs CACHED_PERMISSION same shape. Its runtime_write_barrier_root_nanbox is an incremental mark barrier, not a root registration — it does nothing for the sweep or the evacuation rewrite sweep + move
    process/report.rs CACHED_REPORT same shape, no barrier at all sweep + move
    object/global_this/populate.rs:822 ERROR_CONSTRUCTOR_PTR raw usize of a js_closure_alloc closure. The canonical Error closure IS a globalThis field, so the structural trace keeps it alive and rewrites that reference; this duplicate is outside the graph move only
    tui/input.rs:38 INPUT_HANDLER the useInput arrow, inline in idiomatic ink style ⇒ no other reference once the FFI returns sweep + move
    tty.rs:58 RESIZE_CALLBACK native slot that bypasses the (rooted) EventEmitter listener array sweep + move
    frame.rs:30 FRAME_CALLBACKS rooted only transiently by a RuntimeHandleScope during registration. Its unsafe impl Send SAFETY comment asserted the closures "point to global compiled code / GC-rooted data" — the premise was false sweep + move
    object/class_registry/construct.rs:5 CURRENT_NEW_TARGET a second copy of a value scan_implicit_this_roots_mut tracks, held across a whole constructor body move
    object/field_get_set/accessors.rs:209 ACCESSOR_RECEIVER_OVERRIDE armed across a prototype walk that can reach a Proxy get trap sweep + move
    object/global_fetch.rs:15 PENDING_FETCH_SIGNAL the in-flight AbortSignal, sole reference across the fetch call's own argument lowering sweep + move
    perry-stdlib/worker_threads.rs MESSAGE_CALLBACK, CLOSE_CALLBACK ★ scanner gap — scan_parent_port_event_roots_mut walked MESSAGE_EVENT_CALLBACKS and neither sibling. Three slots, same thread_local! block, same raw ClosureHeader* sweep + move

    REAL — verified, deliberately NOT fixed here (each with the reason)

    holder why it is real why not here
    promise/rejection.rs:68 internally_handled HashSet<usize> of raw promise addresses consulted by identity; scan_unhandled_rejection_roots_mut visits unhandled/reported/pending_handled and not it. A stream's closed promise loses its marking after a move (spurious unhandledRejection); an address reused later inherits it (a real one swallowed) the fix is a rekey, not a root — visit_metadata_usize_slot + the remove/insert loop scan_map_iterator_array_roots_mut already uses. Wants its own witness. Not a step-twin gap: it is registered non-budgeted
    CURRENT_NEW_TARGET / ACCESSOR_RECEIVER_OVERRIDE save/restore each parks the displaced value in a bare Rust local across the window and republishes it. Runtime frames are not covered by the precise scan ⇒ #7226's prev_this defect, in Rust rooting the cell closes the ARMED value only; the saved one needs RuntimeHandleScope plumbing at every save site
    module_require.rs:240 MODULE_PATH_REGISTRY holds module.exports values; the codegen module global that also holds them is a registered root, so this is a stale-address (move) hazard, not a sweep one it is a process-global RwLock while arenas are per-thread — a naive scanner would be unsound across perry/thread agents
    json/mod.rs:99 SHAPE_CACHE its comment ("within one top-level stringify call no GC runs over the user object graph") is falsified by toJSON() consequence is a wrong shape template; fix is invalidation, not rooting

    REFUTED — the negative results

    • ★ The step-twin gap shape is NOT present anywhere else. All 8 budgeted (FULL, STEP) pairs diffed field-by-field by reading both bodies and every helper they call: runtime_handles, temp_roots, promise (13 tables), timer (5 tables), class side-table (10 tables), symbol (5), tui hooks, tui state. No drift. fix(gc): root the callback and every staged argument of the timer family #7230's IntervalTimer.args fix landed in both twins (timer.rs:1667 and timer/gc_scan.rs:183-189, same commit). Premise re-confirmed first: gc/cycle.rs:375-390 runs only the step fn when budgeted_scanner is Some, so a step-only omission really would be invisible to gc()-driven tests.
      Two symmetric oddities, therefore not drift, noted and left: CALLBACK_TIMERS.args is visited twice on both paths (timer.rs:1638 + :1642, mirrored at gc_scan.rs:109 + :134) — a merge artifact from f252989a53 landing a second loop beside the pre-existing one; harmless but it doubles that table's incremental root budget. And MOCK_TIMERS skips the crate::agent::owns filter its three siblings apply, contradicting the [thread] Cross-heap unsoundness in perry/thread: global queue drains, timer queues, module globals — design decision needed #6185 rule stated at timer.rs:1613.
    • buffer/header.rs's nine address registries are not root gaps. GC_TYPE_BUFFER and GC_TYPE_TYPED_ARRAY both carry movable: false in gc/types.rs, so the addresses cannot go stale, and the registries are identity sets rather than liveness references. What they do share with the pre-fix SYMBOL_POINTERS is address-reuse aliasing after death — that wants pruning (prune_dead_symbol_pointers's shape), not rooting. Same verdict for regex.rs's REGEX_POINTERS/REGEX_SOURCE_TABLE, node_repl.rs's RECOVERABLE_ERRORS, diagnostics.rs's five ERROR_MESSAGE_* maps, and BUILTIN_MAP_SET_VALUE_BITS / BUILTIN_CLOSURE_LENGTH.
    • static_plugins.rs:16 STATIC_PLUGINS is unreachable. perry_register_static_plugin has no caller anywhere in the workspace. Latent, not live — and by CLAUDE.md's kill-policy it should get a caller or be deleted rather than acquire a scanner nobody exercises.
    • event_target.rs's DOM_EXCEPTION_ERRORS already has per-dead-error cleanup; child_process/v8_serde.rs's id_table runs under an explicit relocation-suppression window (documented at :1041).
    • gc/roots/scanner_shims.rs:72-78 overflow_fields_* and :91 shadow_stack_root_scanner are defined, exported and never registered — dead code carrying doc comments that assert they are needed. Unchanged here; flagged as the shape the kill-policy warns about.

    One thing the sweep found that is NOT this class

    --unrooted-allocas --moving-only went 98 → 2 once #7235 split the predicate, and the 2 residuals are real: Type::Symbol is listed in collectors/pointer_locals.rs's is_definitely_non_pointer_type, while alloc_symbol is a gc_malloc of a movable GC object whose own comment concedes fresh symbols are kept alive "not at all". A Symbol-typed local therefore gets no shadow slot. Filed as #7236; it is the only thing between gc-root-dominance and promotion to a required context (#7198).

  2. proggeramlug commented on Aug 2, 2026

    @proggeramlug
    ContributorAuthor

    Partially fixed on main = d4ac65f21 — the three independently-verified caches are closed, the unverified tail is not. Re-verified in the sweep #7262. Leaving open for the tail.

    Closed by #7239

    All three of the "I read the declaration, the allocator, the dereference, and grepped gc/ for a scanner" items are now rooted, plus seven more:

    • CACHED_ENV — scan_process_env_cache_roots_mut registered at gc/mod.rs:639, with your reasoning quoted at :636. Runtime-verified in this sweep: test_gap_gc_process_env_cache_rooting 10/10 byte-exact on the allocation-point arm at 5 copying minors / 16 750 objects copied.
    • ERROR_CONSTRUCTOR_PTR — scan_error_constructor_root_mut, global_this/populate.rs:873.
    • INPUT_HANDLER — rooted.

    Also CACHED_PERMISSION, CACHED_REPORT, RESIZE_CALLBACK, FRAME_CALLBACKS, CURRENT_NEW_TARGET, ACCESSOR_RECEIVER_OVERRIDE, PENDING_FETCH_SIGNAL, and the worker_threads.rs scanner gap (MESSAGE_CALLBACK / CLOSE_CALLBACK — three slots in one thread_local! block, one visited).

    Still open — no gc/ reference for any of these

    SHAPE_CACHE                  (json/mod.rs:99)
    BUILTIN_MAP_SET_VALUE_BITS   (object/collection_proto_thunks.rs:19)
    MODULE_PATH_REGISTRY         (module_require.rs:240)
    RECOVERABLE_ERRORS           (node_repl.rs:14)
    GLOB_ITERATORS               (fs/dir_glob_watch/watch.rs)
    internally_handled           (promise/rejection.rs)
    

    #7239 states why for three of them rather than leaving them silent, which is worth preserving: internally_handled needs a rekey rather than a root; MODULE_PATH_REGISTRY is process-global while arenas are per-thread, so a naive scanner would be unsound; and the save/restore idiom in CURRENT_NEW_TARGET / ACCESSOR_RECEIVER_OVERRIDE still parks the displaced value in a bare Rust local even though the cells themselves are now rooted.

    SHAPE_CACHE looks like the sharpest remaining one on this list, for the reason given in the original sweep: its safety comment asserts "within one top-level stringify call no GC runs over the user object graph", and toJSON() falsifies that.

    Refuted, and worth not re-deriving

    #7239 diffed all 8 budgeted (FULL, STEP) scanner pairs field-by-field and found no drift. buffer/header.rs's nine address registries are not root gaps — GC_TYPE_BUFFER/GC_TYPE_TYPED_ARRAY are movable: false and these are identity sets, not liveness references. static_plugins.rs is unreachable (perry_register_static_plugin has no caller in the workspace).

  3. proggeramlug commented on Aug 3, 2026

    @proggeramlug
    ContributorAuthor

    Consolidated plan reference — docs/src/internals/rfc-rooting-by-construction.md ("Where this sits in the one GC correctness plan", landed in #7292).

    This issue is layer 3: hand-written Rust in perry-runtime — a *mut ObjectHeader in a Rust local or a cache, held across an allocation.

    Unlike the codegen-side rooting issues, this class is NOT subsumed by the planned statepoint work. LLVM never sees this code, so stack maps cannot help. It is also the class the static checker is structurally blind to — scripts/gc_root_dominance_check.py reads emitted LLVM IR and read 0 violations on both sides of #7249, a real bug. See the blind-spot list added in #7289.

    ⇒ Work here is safe to start and will not be thrown away. The long-term direction is making RuntimeHandleScope non-optional rather than opt-in — today every bug in this class is a site where someone simply didn't reach for it.

    Verification note: a 10/10-deterministic failure is diagnostic of this class rather than a stale register; the instruments that see it are PERRY_GC_ZEAL=1 PERRY_GC_PROTECT_FROMSPACE=1 PERRY_GC_PROTECT_FROMSPACE_DEPTH=800 on a dependency-scale workload. #7280 records 25 curated corpus files passing while 20 lines of stock zod fail.

  4. proggeramlug commented on Aug 9, 2026

    @proggeramlug
    ContributorAuthor

    The tail is now enumerated mechanically rather than by hand — PR #7695.

    scripts/gc_runtime_root_holders.py (runs from lint, already a required context) enumerates every static / thread_local! in perry-runtime + perry-stdlib whose type can hold a GC pointer, computes which of them a registered scanner actually reaches, and requires a written verdict for the rest in scripts/gc_runtime_root_holders.json. An unclassified holder fails; an entry that no longer matches also fails, so a fix must delete its own exemption.

    Coverage is computed, not name-matched: registered scanners are read from every gc_register_*root_scanner*(...) call site, a call graph is walked from them, and a holder counts as covered when its identifier appears in a reachable function defined in the same file. The graph walk is what catches the accessor-indirection cases this issue's sweep flagged (cp_live_lock(), get_closure_props(), buffer_props()); the same-file rule is what stops REGISTRY / SLOTS / ROOTS / STATES / CACHED — each of which names several different holders here — from certifying the wrong one.

    Current state: 81 holders, 47 reached by a registered scanner, 30 classified.

    Against this issue's "still open" list:

    Two holders the sweep had not named, now tracked as open_gap:

    • json/mod.rs PARSE_KEY_RING — a hot-key mirror of the rooted PARSE_KEY_CACHE holding the same *const StringHeader values. A move rewrites one copy and not the other, so a ring hit hands out a pre-move address. Narrow: the keys are js_string_from_bytes_longlived, i.e. old-gen, so only old-gen defrag can move them — not the copying minor.
    • perf_hooks.rs PERF_ENTRY_KEYS_ARRAY — a nursery keys_array address compared by identity at perf_hooks.rs:119 and never rewritten. Stale ⇒ a silent slow path, or a match against a newly-allocated array recycled into the address. The neighbouring PERF_ENTRIES is covered by scan_perf_entries_roots_mut; this one slot is not.

    One unverified entry is left deliberately, named and dated rather than silent: diagnostics.rs DIAG_CHANNEL_BY_KEY, whose DiagChannelKey::Symbol variant carries a symbol address — whether that can be a gc_malloc'd fresh symbol rather than a Box::leak'd registered one has not been established.

    Three bugs found while building it, all in the checker, all of the "green because it matched nothing" shape — recording them because this is the failure mode the repo keeps paying for:

    1. string literals were not stripped, so brace counting swallowed scan_raw_json_key_root_mut and reported RAW_JSON_KEY — which that scanner visits three lines below its own declaration — as uncovered;
    2. the registration regex captured only the first argument, so gc_register_mutable_root_scanner_named("stdlib:worker_threads:workers", scan_worker_roots_mut) registered nothing and six worker_threads holders read as uncovered;
    3. rule B keyed on the file rather than the function and reported 544 holders, four fifths of them counters and ids.

    --self-test plants a covered holder, one reachable only through an accessor, one uncovered per rule, and a same-named decoy in another file; then asserts the verdict machinery itself can go red. Live sabotage: planting an unrooted Cell<*mut ObjectHeader> into regex.rs fails the real scan.

    Also from this cluster, in case they help scope what is left: #7274 is fixed (PR #7693), #7246 is fixed (PR #7697), and #7247 is closed — re-verified green 10/10 on main with copying minors confirmed live.

  5. proggeramlug commented on Aug 9, 2026

    @proggeramlug
    ContributorAuthor

    Closed by #7695 (v0.5.1402).

    scripts/gc_runtime_root_holders.py + scripts/gc_runtime_root_holders.json now enumerate the population this issue was about, and — the part that makes it a gate rather than a list — coverage is computed, not declared: registered scanners → call-graph walk → holder named in a reachable function in the same file.

    Current state on main: 80 holder declarations scanned, 47 reached by a registered scanner, 29 classified in the inventory, 122 registered scanners. Wired into lint.

    It is shrink-only and proved it on its first real run: the SHAPE_CACHE entry carried its own expiry note ("Fixed in PR #7694 — DELETE THIS ENTRY when that lands, the gate will tell you"), #7694 landed, and both the gate and its --self-test went red on exactly that entry until it was deleted (30 → 29).

    Sabotage-verified independently at merge: a planted static AUDIT_PLANT_HOLDER: Mutex<Option<*mut ArrayHeader>> in json/mod.rs exits 1 naming the holder, its line, and the rule that caught it.

    The two previously-untracked open_gaps it surfaced — PARSE_KEY_RING (an unrewritten mirror of the rooted PARSE_KEY_CACHE) and PERF_ENTRY_KEYS_ARRAY — remain tracked in the inventory as the follow-on work.

    Also worth recording: five separate ways the checker could report coverage it did not have were found and fixed before it shipped (unstripped string literals breaking brace-counting; a registration regex that read only the first argument so ..._named("…", scanner) registered nothing; rule B keyed on file rather than function, reporting 544 holders; bare-name module collisions able to certify the wrong module's holder; and inventory entries accepted with no verdict). A coverage checker that over-reports coverage is worse than none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions