Linux-only debug-profile SIGSEGV in cargo-test from #10619 (shadow-stack TLS hot-cache), mechanism unconfirmed #10709

Summary

PR #10619 (perf/try-entry, "route the shadow-stack TLS through the
hot-cache fast path") caused CI's cargo-test job (Linux, debug profile,
ubuntu-latest) to fail with a SIGSEGV:

error: test failed, to rerun pass `-p perry-runtime --lib`
Caused by:
  process didn't exit successfully: `.../target/debug/build/perry-runtime/.../perry_runtime-...` (signal: 11, SIGSEGV: invalid memory reference)

Run: https://github.com/PerryTS/perry/actions/runs/35374727647/job/105594641738

The job's stdout is block-buffered, so no FAILED line and no panic message
survived; the last visible line was
test async_hooks::test_support::tests::before_after_restore_execution_ids ... ok.

The PR's own change is small: crates/perry-runtime/src/gc/roots/shadow_stack.rs
swaps the SHADOW: UnsafeCell<ShadowStackState> thread-local from a raw
thread_local! to crate::perry_thread_local! (the tls_hot.rs mechanism
that caches a resolved TLS address in a per-thread slot table), for an ~8%
win on try/catch entry (_tlv_get_addr was a real per-entry leaf cost on
Darwin). It was confirmed clean on cargo-test runs for #10644, #10647,
#10650, #10651 against the same base — so this is #10619's own failure, not
pre-existing flake.

What was tried

Reproduction, debug profile, RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib:

• macOS (arm64), Darwin perry_thread_local! path (as shipped): clean.
  4013 passed, 0 failed, 4 ignored, ~167s. The test immediately after the
  CI log's last visible line
  (async_hooks::test_support::tests::native_async_resource_accepts_string_and_symbol_expandos)
  ran fine, back to back with no gap.
• macOS (arm64) with the Darwin pthread-TSD path forced off (all 11
  target_vendor = "apple" cfg predicates in tls_hot.rs flipped to an
  unsatisfiable vendor, so hot() resolves through the generic
  hot_via_tls() that every non-Darwin-aarch64 target already uses): also
  clean. 4010 passed, 0 failed (same 4010/4013 shape as the release
  numbers below, matching the compiled-out debug_assert! count for a debug
  build vs. what's expected — no crash). This weakens the theory that the
  generic (non-Darwin) hot_via_tls() code path is inherently broken for
  SHADOW: on this host it isn't.
• Linux x86_64, real target (qemu-emulated Ubuntu VM, matching
  ubuntu-latest's triple and the CI-pinned nightly-2026-08-20 toolchain),
  same debug command: ran clean through at least 1968 of ~4013 tests in
  alphabetical run order (comfortably past the async_hooks region where CI's
  log cuts off — no crash there or anywhere in between). The run stalled
  (not crashed — process stayed alive, just made no forward progress for
  10+ minutes) at
  gc::tests::runtime_roots::perex_replace::perex_replace_output_is_not_capped_by_the_scratch_limit,
  which is unrelated to shadow-stack/TLS (a regex-replace scratch-buffer
  stress test) and runs in well under a second natively on macOS — almost
  certainly a qemu-emulation-specific slowdown for that one test, not a hang
  tied to this PR. A second run skipping that one test
  (-- --skip perex_replace_output_is_not_capped_by_the_scratch_limit) was
  in progress at the time of writing; this issue will be updated with its
  result.

So: three independent environments (Darwin fast path, Darwin with the
fast path forced off, and Linux/x86_64 on the actual target triple) all ran
clean, and none reproduced the SIGSEGV.

What's still unexplained

The CI SIGSEGV is real (three separate PRs' clean cargo-test runs against
the same base rule out pre-existing flake) but was never pinned to a
specific test, a specific line, or a backtrace. Two hypotheses were floated
during the investigation and neither is confirmed or ruled out:

1. Re-entrancy in tls_hot.rs's fill(). fill() (tls_hot.rs:~296)
   resolves the 17 named HotTls fields and writes temp_roots last,
   with a comment that this ordering stops "a re-entrant call from inside
   one of the providers above" from observing a half-filled cache as ready.
   That ordering prevents using a half-filled cache — it does not prevent
   recursion: a re-entrant caller mid-fill() still sees
   temp_roots.is_null() and calls fill() again. Whether anything on the
   resolution path (HotKey::resolve_and_cache → hot() →
   hot_via_tls()/hot_uncached() → fill()) can actually re-enter itself
   for SHADOW specifically was not confirmed either way.
2. Debug-profile stack depth under CI's default thread, since this repo
   has hit exactly this signature before (a deep recursive-descent test,
   green locally under --release, SIGSEGV in CI's debug cargo-test, same
   missing FAILED line from stdout buffering). Not confirmed here either.

Given the reproduction environments above are the closest available proxies
for CI's actual runner and all came back clean, closing this without a
confirmed mechanism is reasonable if a future maintainer wants to; it's
filed instead so the SIGSEGV doesn't evaporate silently. If it recurs, the
fill() ordering note above is where to look first, and a real
ubuntu-latest GitHub-hosted runner (not qemu-emulated) would be worth
trying before a from-scratch re-investigation.

Resolution shipped alongside this issue

Independent of the crash's cause, tls_hot.rs's own module docs already say
the published-cache shortcut only pays for itself on Darwin aarch64 — on
every other target "resolving a thread-local is already a fixed offset and
the extra cache indirection has no demonstrated benefit." So routing
SHADOW through perry_thread_local! unconditionally was strictly more
work off that platform even before the SIGSEGV: one extra thread-local
resolution (HOT) plus a slot-array indirection, replacing the single
direct TLS access a raw thread_local! already was. The try-entry win was
only ever measured on Darwin.

crates/perry-runtime/src/gc/roots/shadow_stack.rs's SHADOW declaration
is now cfg-split: perry_thread_local! only under
all(target_vendor = "apple", target_arch = "aarch64", target_pointer_width = "64"),
a plain thread_local! everywhere else (which is what shipped before
#10619). This keeps the measured win where it was measured and removes the
pessimization — and, as a side effect, removes CI's only exposure to
whatever caused this SIGSEGV, since the generic hot_via_tls() path no
longer touches SHADOW on Linux at all. That is a mitigation, not a
diagnosis: the mechanism is still not understood, which is the reason this
issue stays open.