Skip to content

perf: reuse incremental Merkle trie snapshots for state roots - #282

Open
arjanz wants to merge 1 commit into
mainfrom
az-incremental-trie
Open

arjanz wants to merge 1 commit into
mainfrom
az-incremental-trie

Conversation

@arjanz

@arjanz arjanz commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Summary

State-root calculation rebuilds the entire trie even when only a few entries change. This PR retains an immutable, incremental Merkle trie and updates only the affected paths.

  • Add immutable leaf and branch nodes with cached hashes and structural sharing.
  • Build the initial trie in bulk; apply subsequent insertions, updates and deletions incrementally.
  • Reuse the existing bounded leaf and branch hash caches.
  • Retain one materialized snapshot per StateStorage, shared by state_root() and state_root_at().
  • Freeze mutable buffers and preserve the original builder as a fallback for custom encoders and unsupported inputs.

Each root calculation still materializes the selected state and compares its contents. This preserves detection of external database writes, fork switches, rollback, finalization and independent trace pre-state reloads.

State materialization and comparison remain O(number of entries); the optimization reduces trie reconstruction and hashing work.

Validation

  • 129 focused tests and 11 subtests passed.
  • Replayed 1,000 traces across eight suites.
  • Verified 2,000 roots against the original full trie builder.
  • Added randomized update tests and coverage for structural sharing, shared prefixes, empty values, deletions, mutable buffers, external writes, forks, historical roots and transaction lifecycle operations.
  • The broader local suite reports the same 33 failures when using the baseline implementation.

Performance

Local median total trace execution time from three alternating runs per implementation:

Trace suite Improvement
fallback 2.2%
safrole 2.6%
storage 2.6%
storage_light 3.3%
preimages 14.1%
preimages_light 9.7%
fuzzy 5.7%
fuzzy_light 10.6%

Results depend on state size and how many entries change between root calculations. These are local measurements with run-to-run variation.

Retain one materialized trie per StateStorage and update only changed paths using immutable nodes. Preserve state-view validation and fallback behavior for custom encoders and unsupported inputs.

Add coverage for fork isolation, external writes, mutable buffers, rollback, finalization and historical roots.

Validation: 129 focused tests and 11 subtests passed; 1,000 traces passed with 2,000 root comparisons against the existing builder.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant