Skip to content

aarch64-linux: the fast stack-map walker and the unwinder disagree by 96 bytes on the same root (PERRY_STACKMAP_WALKER=verify) #7984

Description

@proggeramlug

Summary

On aarch64 Linux (ELF), PERRY_STACKMAP_WALKER=verify catches the fast
frame-pointer-chain walker and the unwinder resolving the same root to two
different addresses. The gate is working; the code is wrong.

This is the ubuntu-24.04-arm arm of gc-native-roots (#7970). It is a root
misidentification on a supported target
, not a harness problem — unlike the
macos-14 arm of the same run, which was a missing PERRY_GC_DIAG=1 and is
fixed separately.

Evidence

Run 31405431962, job native-roots-rs4gc (ubuntu-24.04-arm, aarch64, ELF),
sha bf8af0e0f, probe 01_nursery_churn:

thread '<unnamed>' (20234) panicked at crates/perry-runtime/src/gc/roots/stack_maps.rs:507:5:
assertion `left == right` failed: PERRY_STACKMAP_WALKER=verify: fast walk visited 1 unique slots, unwinder visited 1
  left:  [281474742909688]
  right: [281474742909592]

Both walkers found one slot, so this is not a missed or invented frame. They
disagree on where that slot is:

281474742909688 - 281474742909592 = 96 bytes

verify_visit (gc/roots/stack_maps.rs) runs fp_chain::visit non-mutating,
then unwind::visit for the real visitation, and asserts the address sets are
equal. A 96-byte constant offset between them is the signature of a
frame-base/CFA disagreement, not of a parse failure.

Why this matters more than a red CI arm

fp_chain is the fast walker — the one that runs when verify is off. If it
resolves a root 96 bytes away from the real slot, then on this target the
collector marks and rewrites the wrong stack word. That is precisely CLAUDE.md's
"root-store dominance" failure class: nothing is visibly wrong at collection
time, and it surfaces cycles later as TypeError: value is not a function.

verify mode exists to catch exactly this, and it did — the first time anyone
ran it end-to-end on aarch64 ELF. The workflow's own header predicted the shape:

What no arm has ever exercised is the end-to-end chain on this target — real
LLVM asm output, real ELF linking, real runtime walking — where the failure
mode is not a parse error but a wrong answer.

Reproduction

# on aarch64 Linux
export RUSTFLAGS="-C force-frame-pointers=yes -C force-unwind-tables=yes"
cargo build --profile perry-dev -p perry -p perry-runtime-static -p perry-stdlib-static
PERRY_RS4GC=1 ./target/perry-dev/perry benchmarks/gc_ratchet/probes/01_nursery_churn.ts \
  -o /tmp/rs4gc-01
PERRY_GC_TRACE=1 PERRY_GC_DIAG=1 PERRY_STACKMAP_WALKER=verify \
PERRY_RS4GC=1 PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1 \
PERRY_GC_HEAP_LIMIT=8 PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off \
  /tmp/rs4gc-01

x86-64 ELF (ubuntu-latest) passes the same step, so this is aarch64-specific.

Open questions for whoever picks this up

  1. Which walker is right? The unwinder is the reference implementation, so
    the presumption is fp_chain is 96 bytes off — but that should be confirmed
    against the emitted stack map rather than assumed.
  2. Is 96 constant? One probe, one slot is a thin sample. Worth running the
    whole probe matrix under verify on this target and checking whether the
    delta is always 96 (a fixed frame-layout term) or varies (an offset-scaling
    bug, e.g. the .word-vs-.long field-width hazard word_width_for warns
    about, which is aarch64-ELF-specific).
  3. Does the fast walker run in production on this target? If yes, this is a
    live correctness bug on aarch64 Linux, not only a CI failure.

Found while repairing #7970. Sibling: the macOS arm of the same run was a gate
defect (missing PERRY_GC_DIAG=1), and the Windows arm is a separate link
failure.

Activity

  1. proggeramlug commented on Aug 12, 2026

    @proggeramlug
    ContributorAuthor

    Answered, and fixed in #7997

    The fp-chain walker is the wrong one — the bad case, since it is what runs when verify is off.

    Reproduced end to end on aarch64 Linux (Ubuntu 24.04 arm64 container, LLVM 22.1.8 from apt.llvm.org, this issue's own RUSTFLAGS). The trigger is the tuning, not the distro. Perry builds a host binary with -mcpu=native; on a Neoverse-class core that enables SVE, and SVE changes the prologue LLVM emits:

    PERRY_TARGET_CPU=neoverse-n2   -> diverges
    PERRY_TARGET_CPU=neoverse-n1   -> passes
    

    The container's own -mcpu=native (an Apple core, no SVE) passes all 14 probes under all three walkers, which is why this was never visible off that runner — and no macOS arm could ever have seen it.

    The frame

    main, the module body, 100 stack-map records, built -mcpu=neoverse-n2, read with objdump -d:

    12479c: 910083fd  add   x29, sp, #0x20      <- fp established; decoder reads 32
    1247a0: a9036ffc  stp   x28, x27, [sp, #48] <- NOT a `sub sp`: the run ENDED here
    ...     four more callee-save pairs
    1247b4: d10143ff  sub   sp, sp, #0x50       <- 80 bytes, DROPPED
    1247b8: 043f57df  addvl sp, sp, #-2         <- 2 x VL more, DROPPED
    

    Two independent blind spots in fp_to_sp_offset, in one prologue:

    1. A callee-save store ended the accumulation run. It does not move sp, so it says nothing about whether the stack adjustments are finished — but the rule was "the first instruction that is not a sub sp ends the run", and LLVM interleaves those stores with the frame-pointer setup whenever SVE is on.
    2. addvl sp, sp, #-N was not decoded at all. Its multiplier is in the instruction; its unit is the runtime vector length, which is not.

    Q1, which walker is right — established arithmetically, not by presumption

    Under qemu -cpu max (VL = 64 B), from the report #7997 adds to verify:

    fp-chain minus unwinder = [208, 208] byte(s)
      slot 0x400000800af0 = base 0x400000800ab0 +64 | ip … (fn 0xaaaaaabc4790 + 0x22c)
        fp_to_sp_offset(fn) = Some(32), prologue words: fc180fea 6d0123e9 a9027bfd
          910083fd a9036ffc a90467fa a9055ff8 a90657f6 a9074ff4 d10143ff
    

    Same function, same ip, same record, same offsets — a base disagreement:

    caller_fp        = fast base + decoded = 0x…ab0 + 32 = 0x…ad0
    true x29-body_sp = 32 + 0x50 + 2*64                  = 240
    true body_sp     = 0x…ad0 - 240                      = 0x…9e0   <- the unwinder's base
    

    The unwinder's base is the frame's real body SP, derived independently from the prologue the fast walker misread.

    Q2, is 96 constant — no

    It is that frame's missed tail and it scales with the vector length:

    host delta =
    ubuntu-24.04-arm (VL = 16 B) 96 0x50 + 1 x 16
    qemu -cpu max (VL = 64 B) 208 0x50 + 2 x 64

    A fix keyed on 96 would have been wrong on every other vector length. (Not the .word-vs-.long hazard the issue suspected: the compact map decodes fine, and a parse error would move both walkers identically and produce no divergence at all.)

    Q3, does the fast walker run in production on this target — yes

    rs4gc_enabled() defaults to NATIVE_ROOTS_TARGET_OK, which is true on aarch64 Linux, and WalkerMode::Fast is the default. So this was a live correctness bug on aarch64 Linux with SVE, not only a CI failure: the collector marked and rewrote a stack word 96 bytes away from the root.

    The fix

    Stores through sp with no writeback are transparent to the accumulation run (enumerated by opcode, so an unrecognised instruction still ends it). addvl/addpl writing sp is decoded — multiplier from imm6, unit from prctl(PR_SVE_GET_VL), cached; where the length cannot be read the whole decode fails and the frame goes to the platform unwinder.

    Failing closed only was the first cut and was not enough: on an SVE host that frame is the module body, so the fast walker would decline at every collection and verify turns a decline into a panic — this arm would have stayed red on its own fix.

    Validated at two vector lengths: 28 verify runs (14 probes x VL 16 B and 64 B), all byte-exact, with fp_walks 12 / fallback_walks 0 / locations_visited 2678 proving the fast walker is the one doing the work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions