Skip to content

Improve loadtxoutset performance - #36603

Open
ajtowns wants to merge 12 commits into
bitcoin:masterfrom
ajtowns:202610-loadtxoutset-speedy
Open

ajtowns wants to merge 12 commits into
bitcoin:masterfrom
ajtowns:202610-loadtxoutset-speedy

Conversation

@ajtowns

@ajtowns ajtowns commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Significantly improves loadtxoutset performance via a number of changes:

  • parallel threads for decoding, hashing, serializing and writing to leveldb
  • no fsync() gating writes
  • no redundant loading into CoinsView

Also adds a subcommand to bitcoin-util for getting a snapshot's hash_serialized_3, muhash, coin count and total amount values without importing to leveldb first.

On my systems, performance ends up bottlenecked by leveldb CPU usage.

@DrahtBot

DrahtBot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

External sites

Reviews

See the guideline and AI policy for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #36739 (refactor: pass BlockManager to ChainstateManager by pzafonte)
  • #36601 (test: avoid reusing AssumeUTXO coin insertion method in test fixtures by l0rinc)
  • #36543 (validation: check the most recent AssumeUTXO commitment during IBD by l0rinc)
  • #36542 (AssumeUTXO: allow retrying loadtxoutset after an interrupted load by l0rinc)
  • #36500 (AssumeUTXO: reject duplicate coins and empty snapshot groups by l0rinc)
  • #36346 (validation: don't accept txs to the mempool while loading a snapshot by 0xShadowX)
  • #36326 (kernel: Use typed errors for fatal and flush error notifications by arejula27)
  • #36167 ([RFC] Enable -Wunused by fanquake)
  • #36074 (scripted-diff: [test] Add util/check.h includes for assertions by maflcko)
  • #36000 (validation: prefetch blocks while connecting by l0rinc)
  • #35788 (bitcoin-util: Add evalscript subcommand by ajtowns)
  • #35511 (RFC: consensus: Make CAmount a class by hodlinator)
  • #34864 (coins: tighten cache entry state invariants by l0rinc)
  • #33117 (Interfaces: Expose UTXO Snapshot Loading and Add Progress Notifications by D33r-Gee)
  • #31260 (scripted-diff: Type-safe settings retrieval by ryanofsky)
  • #29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible places where named args for integral literals may be used (e.g. func(x, /*named_arg=*/0) in C++, and func(x, named_arg=0) in Python):

  • src/test/validation_chainstatemanager_tests.cpp: cache.AddCoin(item.outpoint, Coin{item.coin}, false)

2026-10-28 02:47:34

@ajtowns

ajtowns commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

The performance change I see on a Ryzen 9 3900X system with RAID HDD storage is ~11minutes with to ~1m35s for a load, consisting of ~1m15s of CPU time followed by ~20s of disk flushing; with SSD and a slightly faster CPU, total time is reduced from ~5m30s to ~50s (no significant flush delay after the CPU work is completed here). These numbers are with the block 965000 utxo set.

@DrahtBot

Copy link
Copy Markdown
Contributor

🚧 At least one of the CI tasks failed.
Task tidy: https://github.com/bitcoin/bitcoin/actions/runs/39762952091/job/119771350806
LLM reason (✨ experimental): CI failed because clang-tidy flagged a mismatched argument comment (moveout instead of moveto) in test/coinsviewoverlay_tests.cpp; warnings are treated as errors.

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

@ajtowns
ajtowns force-pushed the 202610-loadtxoutset-speedy branch 3 times, most recently from 51d7d9e to 32575f0 Compare October 23, 2026 06:04
@l0rinc

l0rinc commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Can you please give us more context? Why do you consider an 11-minute load too slow? Is there a reason to think this is what's preventing people from using it - enough to justify yet more complexity in the most "load-bearing" parts of the code? Or is this just a demo to explore the possibilities?

Over the past years we've discussed how relying on someone else's UTXO set makes us uncomfortable (see #32377 and #36598).
This change adds even more AssumeUTXO-specific complexity, which is exactly why we're considering removing it completely. I wouldn't object to adding it back later through IPC and the Kernel if there's still demand and doesn't add complexity - but there's no indication of that, especially since #36476 (possibly combined with SwiftSync) would already speed up actual UTXO set construction by an order of magnitude on resource-constrained nodes. Quickly loading someone else's UTXO set could still be a useful academic pursuit, but many of us are not comfortable polluting the code with multiple chainstates, which make consensus-critical paths harder to reason about.

@ajtowns
ajtowns force-pushed the 202610-loadtxoutset-speedy branch from 32575f0 to e81eeb8 Compare October 23, 2026 11:19
@ajtowns

ajtowns commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Can you please give us more context?

In bips#2137 Sjors suggested splitting up the snapshot by coin to improve performance, which didn't seem sensible to me.

Why do you consider an 11-minute load too slow?

I don't consider it too slow; though it's possible it could be on slower systems than the ones I test on. But the reason it is that slow is because the database gets compacted multiple times, and it also seems to use excessive amounts of memory while loading, and there isn't really a good reason to do that. The difference between checking whether better performance is possible and actually making it available to users isn't that large.

This change adds even more AssumeUTXO-specific complexity, which is exactly why we're considering removing it completely.

As I've said on the list, I think the assumeutxo approach is a much better approach to resolving the "fast startup of a new node" challenge than trying to get a few extra percent out of IBD performance (which introduces its own large sets of complexities). If it's been decided that that's not one of this project's goals, that's fine I guess, but I haven't seen a project statement along those lines.

Making assumeutxo a reasonable default, rather than something for experts, does require significantly more complexity (ie, the various stuff for the BIP above, cf #35054). Likewise if we want to do FEC encoding so pruned nodes can help share utxo snapshots or support IBD.

@ajtowns
ajtowns force-pushed the 202610-loadtxoutset-speedy branch 2 times, most recently from 4840412 to 92c6b60 Compare October 25, 2026 05:07
@DrahtBot

Copy link
Copy Markdown
Contributor

🚧 At least one of the CI tasks failed.
Task iwyu: https://github.com/bitcoin/bitcoin/actions/runs/40163823354/job/121080686219
LLM reason (✨ experimental): CI failed because the IWYU check modified includes in src/node/utxo_snapshot.cpp, leaving a diff that caused the check to exit with code 1.

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

The snapshot file format is not node state: reading a snapshot needs no
chainstate, so move it out. This prepares the way for adding snapshot
parsing to bitcoin-util in a future commit.

Mostly move-only. The commit can be reviewed with the
--color-moved=dimmed-zebra option for an easier review.
SnapshotMetadata took the expected network in its constructor and compared
it while parsing, throwing on mismatch. Split that check out as a separate
`CheckNetwork` method, so that callers have the option to read snapshots
for arbitrary networks.

Also replace the per-object `const std::set` for checking supported
versions with an inline function.
Normally, we want updates to the database to be fsynced to disk so that
the database is in a consistent state even after a crash. This adds an
option to drop that guarantee, in favour of faster bulk writes, relying
on the caller to make other arrangements to ensure the resulting database
is flushed to disk.
ajtowns and others added 8 commits October 28, 2026 12:45
This populates a chainstate leveldb (as a CDBWrapper) directly from a
batch (generated from Coin objects). This avoids the overhead of going
through a CoinsView map, and allows splitting the batch creation and
the batch write/flush across different threads.
Multi-producer, multi-consumer thread-safe queue with maximum capacity
for handing out jobs to threads.
Allows importing snapshots directly into leveldb (dbwrapper), without
passing through Chainstate or CoinsView first. Supports multithreaded
processing, with one thread reading, one thread hashing, one thread
serializing for the database, and one thread populating the database.

Adds an additional check that the coins in the snapshot are sorted in
txid order.

Note that because v2 dump follow the database key order (which encodes
output indices as VARINTs), but the UTXO commitment requires outputs
to be ordered numerically by output index, a sorting step is required
(eg VARINT(16512) sorts less than VARINT(16511). We bound the total
serialized transaction outputs by the consensus block weight limit to
bound the amount of date being sorted in case of corrupt snapshot files.

Nothing calls it yet, so behaviour is unchanged.

Co-Authored-By: Sjors Provoost <sjors@sprovoost.nl>
ActivateSnapshot and, therefore, the loadtxoutset RPC now use the new
utxo_snapshot.cpp pipeline.

The dump is no longer staged in a CCoinsViewCache and then hashed and
flushed out of it, so `-dbcache` worth of cached coins during a mainnet
load are gone along with the flush cycles. In addition, the hashing
of the dump takes place in parallel with the import, rather than after
it completes.

The old loader is left in place until the next commit removes it;
nothing calls it any more.
After the previous two commits, this code is no longer used, so remove it.

This also removes the circular dependency between node/utxo_snapshot
and validation.
Adds a subcommand for quickly generating the summary of a serialized
utxo set for verification purposes (including various hashes, coin count
and value).
@DrahtBot

DrahtBot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

🐙 This pull request conflicts with the target branch and needs rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants