Repository navigation
store: shrink root_map mem-usage - #12476
Open
kprotty-fd wants to merge 2 commits into
Open
kprotty-fd wants to merge 2 commits into
kprotty-fd wants to merge 2 commits into
Conversation
┌─ ⚡ PERF · e70f855 vs main@3a633f8 ─────────────────────────────────
│ SUITE BASELINE NEW Δ
│ replay tps, mainnet 12,927 tps 12,787 tps · -1.09%
│ bench tps, localnet 669,411 tps 670,314 tps · +0.13%
│ snapshot load, testnet 4.93 s 4.94 s · +0.17%
+│ mem total, mainnet 150.35 GiB 148.35 GiB ▼ -1.33%
+│ mem total, testnet 86.66 GiB 85.66 GiB ▼ -1.15%
│ clean compile, firedancer 3.17 s 3.14 s · -0.86%
│ binary size, firedancer 57.92 MB 57.89 MB · -0.04%
├─────────────────────────────────────────────────────────────────────
@@ 0 REGRESSIONS · 0 WARNINGS · 2 IMPROVED · 5 NOISE @@
└─────────────────────────────────────────────────────────────────────history · 4 pushes ┌─ HISTORY · Δ vs main, per push, newest first ──────────────────────────────────
│ HEAD TPS BENCH SNAP MEM·M MEM·T COMPILE BINARY
+│ e70f855 -1.09% +0.13% +0.17% -1.33% -1.15% -0.86% -0.04%
+│ 309bc85 +0.20% +0.10% -0.12% -1.33% -1.15% +0.42% -0.04%
+│ dd3e08d +0.33% -0.48% -5.52% -1.33% -1.15% -0.17% -0.04%
+│ d8e3670 +0.03% +0.23% -19.98% 0.00% 0.00% +0.66% 0.00%
└──────────────────────────────────────────────────────────────────────────────── |
kprotty-fd
force-pushed
the
king/store-shrink-rootmap
branch
from
October 23, 2026 18:03
d8e3670 to
dd3e08d
Compare
kprotty-fd
marked this pull request as ready for review
October 23, 2026 18:11
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Generation tags repeat on consecutive cell reuse at supported large store sizes, undermining concurrent-read validation.
1 open issue
What changed in this PR
Reduces the disk store’s index memory overhead by consolidating metadata and replacing concurrent hash maps with lock-protected chain maps.
Changes:
- Combines tags and map entries into a shared ring pool with smaller generations and bucket arrays.
- Reworks insertion, eviction, and queries around a shared read/write lock.
- Updates footprint assertions and adds collision, aliasing, and slot-hint tests.
| File | Description |
|---|---|
src/disco/store/test_store.c |
Updates memory expectations and exercises ring and map edge cases. |
src/disco/store/fd_store.h |
Defines the compact layout, map types, and updated API documentation. |
src/disco/store/fd_store.c |
Implements shared-pool allocation and lock-protected disk operations. |
Review effort: Balanced
Folds the store's tags, shred_pool, and root_map into the same ring_pool. Combined with smaller ticket seq, use of fd_map_chain (over fd_map_chain_para) guarded under an rwlock, and smaller bucket counts reduces disk ring index memory overhead.
guards against early ticket wrap-around for shred rings configured above (2^30)/1280 = ~1.3TiB by using padding bits within the ring-cell entries as additional ticket bit storage.
kprotty-fd
force-pushed
the
king/store-shrink-rootmap
branch
from
October 23, 2026 22:26
309bc85 to
e70f855
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Folds the store's tags, shred_pool, and root_map into the same ring_pool. Combined with smaller ticket seq, use of fd_map_chain (over fd_map_chain_para) guarded under a rwlock (rserve only caller), and smaller bucket counts reduces store memory overhead.