KRN-1117: Stackdepot Redesign Kernel Port by caleb-kan · Pull Request #4 · caleb-kan/linux

caleb-kan · 2026-07-08T11:32:17Z

Summary

This ports the KRN-1117 stackdepot redesign to Linux 6.18.

DOG KASAN hosts can exhaust stackdepot pools because stackdepot currently deduplicates only byte-identical full stacks. The RFC and userspace prototype showed that these stacks still share many frame sequences, even when the whole stack is unique.

This adds an opt-in path-compressed trie backend for persistent stack saves. The trie stores shared frame runs once and compresses kernel text frames when the arch helper can reconstruct them safely.

stackdepot.trie_enabled defaults off. The next step is DOG one-off testing and validation.

Copilot

Pull request overview

This PR ports a redesigned stack depot implementation into the kernel, adding an optional trie-based backend for persistent (non-refcounted) stacks, extending stack depot flags, and updating in-tree users to a backend-independent fetch API.

Changes:

Add trie-backed storage (boot-param controlled) and a new stack_depot_fetch_into() API for backend-independent stack retrieval.
Introduce STACK_DEPOT_FLAG_COUNTABLE and tighten semantics around hash-backed vs trie-backed handles.
Update multiple MM/DRM call sites and add KUnit coverage for the public stack depot APIs and arch frame compression hooks.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
mm/slub.c	Switch stack retrieval to `stack_depot_fetch_into()` with caller-owned buffers.
mm/page_owner.c	Save stacks with `stack_depot_save_flags()` using the new `STACK_DEPOT_FLAG_COUNTABLE`.
mm/kmsan/report.c	Convert origin printing to `stack_depot_fetch_into()` and reuse a fixed buffer for chained stacks.
mm/kmsan/kmsan_test.c	Update test to use `stack_depot_fetch_into()` into a fixed buffer.
mm/kmemleak.c	Switch kmemleak trace printing to `stack_depot_fetch_into()` with fixed-size storage.
lib/tests/stackdepot_kunit.c	Add new KUnit suite covering fetch-into behavior, flags, snprint, and arch frame compression.
lib/tests/Makefile	Wire up the new `stackdepot_kunit.o` test object.
lib/stackdepot.c	Implement dual backends (hash + optional trie), new flags behavior, and `stack_depot_fetch_into()`.
lib/Kconfig.debug	Add `STACKDEPOT_KUNIT_TEST` Kconfig option.
include/linux/stackdepot.h	Extend flags, document new semantics, and add `stack_depot_fetch_into()` declaration.
include/asm-generic/stackdepot.h	Add generic arch hooks for frame compression/decompression.
include/asm-generic/Kbuild	Export the new asm-generic stackdepot header.
drivers/gpu/drm/drm_modeset_lock.c	Use `stack_depot_snprint()` to avoid direct hash-backend fetch.
arch/x86/include/asm/stackdepot.h	Provide x86_64 frame compression/decompression implementation.
arch/um/include/asm/Kbuild	Map `stackdepot.h` from asm-generic for UML builds.
arch/arm64/include/asm/stackdepot.h	Provide arm64 frame compression/decompression implementation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

commit f223d27 upstream. In binder_free_transaction(), the t->to_proc is read under the t->lock. However, once the t->lock is dropped, the to_proc can die in parallel. This leads to a use-after-free error when we attempt to acquire its inner lock right afterwards: ================================================================== BUG: KASAN: slab-use-after-free in _raw_spin_lock+0xe4/0x1a0 Write of size 4 at addr ffff00001125da70 by task B/672 CPU: 20 UID: 0 PID: 672 Comm: B Not tainted 7.1.0-rc6-00284-g8e65320d91cd #4 PREEMPT Hardware name: linux,dummy-virt (DT) Call trace: _raw_spin_lock+0xe4/0x1a0 binder_free_transaction+0x8c/0x320 binder_send_failed_reply+0x21c/0x2f8 binder_thread_release+0x488/0x7e0 binder_ioctl+0x12c0/0x29a0 [...] Allocated by task 675: __kmalloc_cache_noprof+0x174/0x444 binder_open+0x118/0xb70 do_dentry_open+0x374/0x1040 vfs_open+0x58/0x3bc [...] Freed by task 212: __kasan_slab_free+0x58/0x80 kfree+0x1a0/0x4a4 binder_proc_dec_tmpref+0x32c/0x5e0 binder_deferred_func+0xc48/0x104c process_one_work+0x53c/0xbc0 [...] ================================================================== To prevent this, pin the target thread (t->to_thread) to guarantee the target process remains alive. Undelivered transactions without a target thread are already safe, as the target process can only be the current context in those paths. Cc: stable <stable@kernel.org> Reported-by: Alice Ryhl <aliceryhl@google.com> Closes: https://lore.kernel.org/all/aikJKVuny_eOivwN@google.com/ Fixes: a370003 ("binder: fix possible UAF when freeing buffer") Signed-off-by: Carlos Llamas <cmllamas@google.com> Reviewed-by: Alice Ryhl <aliceryhl@google.com> Link: https://patch.msgid.link/20260619185233.2194678-2-cmllamas@google.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Add an opt-in path-compressed trie backend for persistent stackdepot saves so related stack traces can share frame runs instead of storing every saved stack as a flat hash record. Keep refcounted and countable users on the existing hash backend, add backend-neutral fetch helpers for persistent callers, and wire the trie through stackdepot pool-backed storage with RCU-safe publication. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Keep the GET-only stack_depot_put() misuse warning visually separate from the corrupt-handle check so the final port diff is easier to review. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Do not let stack_depot_fetch_into() silently accept caller-owned output buffer misuse for a valid handle. A NULL destination should fault naturally, and a zero-sized destination is now reported as an invalid caller contract instead of being treated as a normal missing stack. Keep the existing no-stack and undersized-buffer semantics intact, but warn when new trie code is asked to encode an impossible leaf ID or materializes an impossible zero-length stack. Update the KUnit coverage and API comment to match the tightened contract. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Reduce the trie port's state and failure surface while retaining the pool-backed reuse and live tail append required for its memory savings. Type topology pointers as RCU data, combine leaf ID and slot preparation, remove duplicate frame-run work and scratch storage, and return leaf IDs directly from insertion. Simplify retired-object quarantine to one FIFO and overlay mutually exclusive header state. Preserve base hash behavior outside the COUNTABLE distinction, reserve trie handle space on large-page kernels, and derive side-table geometry from PAGE_SIZE. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Use synthetic addresses that fit in unsigned long on 32-bit builds so the new public API tests compile on every supported architecture. Declare the suite's three-frame configuration requirement instead of allowing valid one-frame or two-frame configurations to run tests whose fixtures would be truncated. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Keep the single KMSAN stack buffer required for bounded report-path stack usage, but remove the redundant capacity alias and zero-result branch from the caller conversion. Restore the existing explicit void-pointer cast so the port does not carry unrelated cleanup in the KMSAN test. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

The GDB helper can only materialize contiguous hash-backed stack records. Detect the trie handle range explicitly instead of reporting a valid trie handle as an out-of-bounds hash pool index. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Use the configured stack_depot_max_pools value as the immutable hash/trie handle split. If no encoded values remain for trie IDs, let optional trie initialization fail instead of silently reducing hash pool capacity. Use the full pool-index field for trie IDs. Allocate the side-table root at its exact size and natural alignment because only directories and chunks require page-sized storage. Remove the redundant fetch capacity aliases. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Mark the pure leaf limit computation inline to match the neighboring stackdepot size and index helpers and make its role explicit. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Separate backend classification from leaf ID decoding so malformed trie handles remain in the trie namespace and warn instead of being interpreted as hash handles. Clarify the recycler, handle encoding, and RCU publication comments. Rename size helpers by their units and remove redundant child-array and side-table index helpers to reduce the number of trie helper functions. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Route constrained trie saves through a single lookup before insertion so the allocation path can assume complete preallocated resources. Replace speculative side-table topology checks with a cached directory and chunk pair, and defer leaf selection until insertion. Use explicit recycler payload storage and loud allocator invariant checks while keeping expected fanout exhaustion on the normal failure path. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Side-table directories and chunks are always exactly one page, so carrying byte and order helpers obscures their fixed allocation contract. Use PAGE_SIZE directly for early memblock storage and pair runtime get_zeroed_page() allocations with free_page(). Normalize nested GFP flags once and remove the redundant size, order, allocation, and free wrappers. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

The trie port does not need to change punctuation in the existing __stack_depot_get_stack_record() kernel-doc. Restore the original wording. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

The generic workspace obscures how append, split, and promotion use its storage and initializes allocation results before insertion selects an operation. Replace it with writer-owned insertion allocation state, use role-specific names, and initialize only requested outputs at the carve boundary. Derive chain-array counts from node counts and fold split sizing into the selected path so allocation roles remain explicit without changing recycler, pool, or RCU publication behavior. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

The append allocation helper only sizes a new chain and forwards the derived node and child-array counts to trie_pool_carve(), while both callers are adjacent branches of the insertion path. Inline the allocation recipe where each append operation is selected so its size and carve requirements remain visible without another helper layer. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Trie handle, pool, and child-array sizes are bounded by their encoding and allocation invariants. Remove overflow and state checks that cannot fire so the implementation does not imply unsupported recovery paths. Inline the single-use child-array capacity calculation and keep expected resource failures and corrupt-handle diagnostics unchanged. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Exercise divergent splits, internal-node promotion, descendant appends, child-array growth, tail append, and old-handle materialization through the public stackdepot APIs. Add mixed compressed and raw frame roundtrip coverage, while keeping every test vector within the configured three-frame minimum. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Stackdepot owns one process-lifetime trie, but lookup and insertion helpers still accept a root argument that is always stack_depot_trie_root. Remove the unused generality so the singleton ownership is explicit at each operation. Document why the recycler scans raw bitmap words instead of using the instrumented multi-word bitmap helper. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Refcounted stack records can be recycled after their RCU grace period while retaining the same encoded handle. A post-put fetch can therefore resolve to an unrelated replacement stack instead of reliably returning zero. Describe post-put handles as invalid with undefined results and keep the defined zero-return cases limited to valid API outcomes. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Trie-specific public API tests can otherwise pass after optional trie initialization falls back to hash storage. Add a test-only expected pool split and verify saved handles use the trie namespace. Skip trie-specific cases when no split is supplied so ordinary hash runs do not claim trie topology or compressed-storage coverage. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

The DRM modeset-lock caller only needs to switch from direct fetch to the backend-neutral stack_depot_snprint() API. Restore its existing literal frame-array size and remove the one-use macro and comment so the port carries no unrelated caller abstraction. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Compression output is valid only when the architecture hook returns true. Requiring a failed compression attempt to preserve the caller-provided payload adds a contract that no stackdepot caller needs. Keep the raw-fallback result check while allowing architecture hooks to clobber the output payload on failure. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Replace the variable-size trie recycler and its static size-class tables with fixed 16-byte slots tracked by one bitmap per trie pool. Keep retired slots reserved until their RCU grace period completes, then make them available for first-fit reuse. This removes best-fit lists, tail splitting, fragment bookkeeping, and ten allocator helpers while retaining pool accounting and local unwind for unpublished insertions. Production boots used 69-72 pools compared with 69-70 for the recycler and 90-92 for hash storage. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Inline the static-key check and side-table root sizing because each helper only hides one expression used at two call sites. Keep the public API comments backend-neutral now that handles may refer to either hash records or trie leaves. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Fold countable fetch, direct-record, and namespace-isolation assertions into one public behavior test. Inline one-use no-allocation and flag wrappers so the suite exposes the API calls directly. Preserve every distinct assertion while reducing the suite from eleven cases to nine. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Remove derived side-table limit state, redundant assignments, and over-general storage parameters. Use the authoritative handle limit and RCU publication primitive directly so the implementation carries fewer states and wrappers without changing trie behavior. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Exercise same-capacity interior insertion and splitting a node with existing descendants, then verify that every earlier handle still materializes. Keep the fixtures valid at the KUnit suite's three-frame minimum and move x86 synthetic frames away from real IRQ text ranges. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Use the trie writer lock as the sole owner of side-table topology and leaf IDs, while retaining a separate lock only for the preallocation cache. This removes redundant nested locking without adding lockless state access. Restore the base one-time initialization guard and return child-slot match state directly instead of carrying a second output parameter. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Extend public save-flags coverage to verify that an allocation-constrained trie save returns zero for a missing stack without inserting it. Keep hash mode unchanged and use the existing backend parameter to scope the assertion. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Split trie insertion into path, split, and promotion helpers so the writer loop only selects an operation or descends through a matching node. Derive topology under the writer lock, consolidate leaf publication, and remove one-use split and append output. Inline preallocation into the save path and make its bounded two-attempt policy explicit. A failed insertion can register the first of the two pools needed by the largest supported stack, so the second attempt is required. Use the existing lockless pool hint while preserving cleanup for partial preallocation and failed reservations. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

UML can define CONFIG_X86_64 while using the generic stackdepot frame codec. The generic compressor always returns false, so the x86-specific compression assertions do not describe the code under test. Exclude the x86 codec case under CONFIG_UML while retaining generic raw fallback coverage. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Use children containers that own nodes[] and consistently distinguish entries, positions, and bitmap slots. Keep capacity immutable so release size remains a property of the allocation. Remove the generic insertion allocator. Let path insertion, splitting, and promotion reserve and unwind exactly the resources they own, without clearing or scanning the maximum workspace. Reject impossible fanout before reservation and keep split sizing and construction outside pool_lock. Inline the one-use tail publisher and keep three bounded save attempts. One attempt can lose the racy pool hint, while a maximum insertion can need two newly registered pools. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Add a shorter-stack split where the input ends inside an existing run. Re-save every topology fixture after all mutations and require its original handle, so the test covers lookup and deduplication as well as materialization. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Replace the maximum-sized writer workspace and separate sizing, reservation, construction, and rollback passes with trie_path_alloc(). Build each frame run once and unwind an unpublished path through parent links, removing init-time workspace allocation and bulk cleanup state. Use stack ID, path, prefix, and suffix terminology consistently. Inline one-use publication and replacement helpers, derive publication IDs from nodes, and fold pass-through initialization wrappers while preserving allocator-specific memblock and runtime paths. Retain bounded retry, complete pool-size guards, and tail append. A stalled-RCU fanout probe used 622 pools with always-COW children versus four with tail append, so the simpler topology still preserves bounded pool behavior. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Place a maximum-depth roundtrip first and alternate compressed and raw frames on arm64 and x86_64. Require a trie handle, exact fetch, and stable deduplication; with 256 frames this covers one node per frame and the two-pool insertion and retry path. Order the remaining functions and cases by public save, print, countable, fetch, topology, storage, fallback, and architecture codec behavior so definitions match execution order. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Mark the spare pool pointer as poisoned when registering the final allowed pool so later saves do not allocate and retain an unusable order-2 page after capacity is reached. Document why bitmap allocation scans every physical slot and clarify that side-table mappings precede topology publication for new and remapped nodes. Signed-off-by: Caleb Kan <ckan@cloudflare.com>

Copilot AI review requested due to automatic review settings July 8, 2026 11:32

Copilot started reviewing on behalf of caleb-kan July 8, 2026 11:32 View session

caleb-kan changed the title ~~Caleb/krn 1117 kernel port~~ KRN-1117: Stackdepot Redesign Kernel Port Jul 8, 2026

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread lib/stackdepot.c

Comment thread lib/tests/stackdepot_kunit.c

Comment thread lib/stackdepot.c

Comment thread include/asm-generic/stackdepot.h

caleb-kan self-assigned this Jul 8, 2026

caleb-kan force-pushed the caleb/KRN-1117-kernel-port branch from 5a044da to b553fb5 Compare July 13, 2026 15:22