Repository navigation
Conversation
Start a dedicated import worker when a chunk list is accepted. Feed it the verified, contiguous chunk prefix, decoding coins and filling an inactive chainstate while later chunks download. Out-of-order chunks remain on disk. Serialize snapshot imports and interrupt/join the worker before shutdown destroys its dependencies. Remove partial downloads on cancellation. Use version 3's strict global outpoint order to compute HASH_SERIALIZED during insertion, removing the database scan at the end of both downloaded and file-based imports. Activation still requires the complete snapshot and the existing assumeutxo commitment to match. Expand the existing snapshot tests to cover processing before completion, out-of-order and duplicate chunks, cancellation, restart and agreement with an independent database hash. Register snapshot request messages in the functional test peer to exercise a paused download. An offline optimized benchmark of the 165,734,792-coin mainnet snapshot at height 965000 took 190.0 seconds with the database hashing pass and 155.7 seconds with incremental hashing. Final flush/verification took 42.4 and 0.41 seconds respectively. Both matched Core's mainnet commitment. These are single warm-cache runs with a tmpfs chainstate and a 450 MiB coin cache; they exclude P2P transfer and download-file writes. Validation: 158 native suites; 454 functional cases across the extended run and corrected P2P recheck (7 environment skips; pruning and crash stress tests excluded); snapshot fuzz replays; Clang thread-safety checks on the changed import code and unit test; container lint.
Bypass the coins hash-map cache when importing a fresh, private snapshot chainstate. Serialize validated coins into bounded database batches and publish the best block only after the complete snapshot hash matches. Extend the existing flush test to populate its database using many small snapshot batches, check every coin, then exercise ordinary cache writes.
Keep two reusable database batches: the importer decodes, validates and hashes coins into one while a worker writes the other. Backpressure at each batch boundary bounds buffering independently of snapshot size. Wait for the final database write before activation and propagate worker errors to the importer. On cancellation, join any outstanding write and discard the incomplete batch before the private database is destroyed. Extend the existing flush test to exercise an abandoned import.
Measure a version 3 snapshot import into a fresh on-disk database using the bounded snapshot writer. Verify the full coin count, ordering and UTXO commitment to detect regressions in the complete import path. Include database close in the measured iteration and document SSD invocation. Keep the height-965000 fixture confined to the benchmark; production snapshot acceptance is unchanged. Skip the import unless BITCOIN_BENCH_SNAPSHOT is set.
Author
|
It looks like v3 offers no benefit on top of @ajtowns's recent effort. So I'm going to close this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This (vibe coded) branch adds three performance enhancements on top of bitcoin#35054:
(1) and (2) have the biggest impact