Repository navigation
fix(dash-spv): speed up the rescan for derived scripts - #1368
Conversation
Two rescan costs remained once blocks that already saw the new scripts were skipped. - `WalletManager::process_block_for_wallets` checked each transaction once, so a transaction paying an address that a later transaction of the same block derives was missed, and the rescan applied the block again for every round of derivation. A testnet sync applied one CoinJoin block six times in two seconds this way. The wallets that derived addresses now check the whole block again until none derives more. A transaction found again is listed and recorded once, and a record corrected by a later pass is reported as inserted. `WalletInterface::checks_derived_addresses` says whether an implementation does this. `WalletManager` returns `true`, so `BlockMatchTracker` no longer re-applies the deriving block. The default is `false`, which keeps re-applying it. - Each `BlockProcessed` with new scripts made the next tick, and every commit attempt, rescan all up to 16 scanned batches, so the lowest batch's `BlocksNeeded` waited behind the others while the commit frontier waited on exactly those blocks. New scripts are now deferred to each scanned batch that has not matched them, with the pass that derived them. The lowest batch is rescanned while the `BlockProcessed` is handled, and each tick rescans the lowest batch that still has deferred scripts. A batch matches its deferred scripts before it commits, and reconciliation still tests whatever else the wallet watches.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 11 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #1368 +/- ##
==========================================
+ Coverage 85.46% 85.52% +0.06%
==========================================
Files 268 268
Lines 69420 69593 +173
==========================================
+ Hits 59329 59519 +190
+ Misses 10091 10074 -17
|
The helper that derives CoinJoin account 0 External addresses beyond the wallet's watch window moved to `key_wallet_manager::test_utils::coinjoin_external_addresses`, and both the `process_block` tests and the dash-spv CoinJoin gap discovery tests now use it instead of their own copies.
|
Waiting for bot review — coderabbitai not yet. Wait for the missing reviews, or a writer can post |
|
/self-reviewed |
|
Policy satisfied — this can merge. |
…passes Since #1368, process_block_for_wallets checks a block again until no wallet derives more, so a block that derives its own addresses is credited by one call and the filter rescan no longer delivers it again. The rescan still delivers an earlier block again when a later block derives an address it pays, which is the path #1326 fixed. Both tests now live in process_block.rs and use that cross-block shape on BIP44 account 0; dev's part of the credits test keeps the shared CoinJoin helper from test_utils: - test_process_block_credits_addresses_the_block_derives also delivers a block whose far output only the next block derives, applies a ChainLock, and delivers it again: the coins, the balance and the last emitted record must own every output. - test_rescan_after_an_account_reload_keeps_late_outputs_spent spends both outputs in the block after, prunes observed spends at a checkpoint, reloads the account through serde, and delivers the blocks again: both coins must end spent. rescan_tests.rs, its rescan helper and MAX_PASSES are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(key-wallet-manager): cover rescans over chainlocked blocks #1326 fixed crediting outputs that a rescan newly matches on a known or finalized transaction, with tests at the `check_core_transaction` level. These tests drive `WalletManager::process_block_for_wallets` the way the filter rescan does: the block is processed again while a pass derives new scripts, with the ChainLock applied before the first pass, between passes, or not at all. For a transaction paying a run of `gap + gap/2` addresses, and for a transaction whose far output only a later transaction's gap extension reaches, they check the coins held, the balance, and the last record the `BlockProcessed` events carried, which is what a store built from the events keeps. Another set checks that outputs spent in the next block end spent after the rescan, including after a checkpoint prunes the observed spends and after the account is reloaded through serde. The `ManagedAccountRefMut::confirm_transaction` doc said it returns a record only when the confirmation status changes. It also returns one that gains outputs derived since it was recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): reuse the shared block and tx helpers The rescan tests built their blocks with a local `block_at`, a near copy of `Block::dummy`. They now use `Block::dummy` and `TX_AMOUNT`, and their multi-input, multi-output `transaction` builder moves to `test_helpers` as `create_tx`, which `create_tx_paying_to` now builds on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): trim the rescan tests to two The rescan tests ran ten cases over three ChainLock timings. Under mutants of the #1326 code, the case where the ChainLock lands before the first pass caught exactly what the case where it lands between passes caught, the gap-limit run and the spent-output cases caught nothing the others did not, and the unchainlocked path is caught by #1326's own wallet_checker tests. Two tests remain: a later payment's derived output with the ChainLock between passes, and the spend after an account reload. One `rescan` helper now delivers the blocks with their heights, then re-delivers them until a pass derives nothing, with a hook after the first pass. The payments come from `create_tx_paying`, which takes a list of addresses and which `create_tx_paying_to` now builds on; the spender is the one local builder left. The `confirm_transaction` doc is shortened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): consolidate rescan coverage on dev's block passes Since #1368, process_block_for_wallets checks a block again until no wallet derives more, so a block that derives its own addresses is credited by one call and the filter rescan no longer delivers it again. The rescan still delivers an earlier block again when a later block derives an address it pays, which is the path #1326 fixed. Both tests now live in process_block.rs and use that cross-block shape on BIP44 account 0; dev's part of the credits test keeps the shared CoinJoin helper from test_utils: - test_process_block_credits_addresses_the_block_derives also delivers a block whose far output only the next block derives, applies a ChainLock, and delivers it again: the coins, the balance and the last emitted record must own every output. - test_rescan_after_an_account_reload_keeps_late_outputs_spent spends both outputs in the block after, prunes observed spends at a checkpoint, reloads the account through serde, and delivers the blocks again: both coins must end spent. rescan_tests.rs, its rescan helper and MAX_PASSES are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): assert the reloaded rescan credits the late coin With keep-finalized-transactions the reloaded account still holds the spender's record, so the parent's re-delivery credits nothing. Under the default features finalized records are pruned, and the re-delivery must credit the far output before the spender's re-delivery spends it again. Checking that keeps the reload test from passing when the crediting itself stops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): share the standalone external address helper The rescan tests carried a private copy of the shared CoinJoin helper for BIP44 account 0. The shared helper now takes the account type, as `external_addresses`, and `coinjoin_external_addresses` calls it for CoinJoin account 0, so its callers are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): read the BIP44 gap limit from the account late_and_extending only needs the external gap limit, so take it from `external_gap_limit()` and drop the `external_pool` clone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(key-wallet-manager): build spend_first_output_of on spending Both helpers built a version 2 spender with default inputs; let spend_first_output_of start from `spending` and set the output it needs. The transactions it builds are byte-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
After #1344, two costs still slowed the rescan for newly derived scripts.
The wallet manager checked each transaction of a block once. A transaction paying an address that a later transaction of the same block derives was missed, so the rescan applied that block again for every round of derivation. A testnet sync applied one CoinJoin block six times in two seconds.
WalletManager::process_block_for_walletsnow checks the whole block again for the wallets that derived addresses, until none derives more. A transaction found again is recorded once. The newWalletInterface::checks_derived_addressestellsBlockMatchTrackerthat the deriving block does not need to be applied again. It defaults tofalse, which keeps the old behaviour for other implementations.Every block that derived scripts also made the next tick rescan all scanned batches, up to 16 of them. The lowest batch, which the commit frontier waits on, queued its block requests behind the others. New scripts are now deferred to each scanned batch that has not matched them. The lowest batch is rescanned right away, each tick rescans the lowest batch that still has deferred scripts, and a batch matches its deferred scripts before it commits.
PR Hygiene ·
00d20ffWhen every merge requirement is met, the
PR Hygienecheck passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.