Repository navigation
Description
Problem
#1326 (f2dd27c2f) lets a rescan credit outputs a finalized transaction owns only through addresses derived after it was recorded. Its finalized re-delivery path, ManagedCoreFundsAccount::adopt_finalized_outputs, credits such an output unless a spend guard in take_in_output refuses it. For an output spent before the account derived it, the guards that can know of the spend are:
- the account's
spent_outpoints, which holds the inputs of recorded transactions; - the wallet's observed spends, which finality prunes.
When neither holds the spend, the re-delivery credits a coin that is already spent. Further gaps arise when a managed account is reloaded through serde, and in the record rebuilt from a summary. All line numbers below are at dev 0e9668b01.
The gaps:
- A spent late output is credited as confirmed until its spender's block replays. This happens without any reload, when the spender was never recorded on the account and the host rewinds the checkpoint, or after a reload.
- After a reload, the re-delivered spender removes the coin, but no record carries the spend.
- A snapshot written before fix(key-wallet): re-emit a record a rescan finds owning more outputs #1326 (no summaries) misses a late output whose address another payment has used.
- The bincode layout of managed-account snapshots changes in both directions.
- A record rebuilt from a summary has
fee: Noneand an empty label.
Who is affected
- Gap 1: every host that re-delivers a finalized funding block after finality has pruned the observed spend, whether or not it serializes managed accounts. The late output's spender must also hold no spent mark on the account. One way is to rewind a live wallet's checkpoint to rescan. Another is to reload a managed account. Platform-wallet rewinds the checkpoint in memory, with no reload, in two places (dashpay/platform
8411b4e1d5):spv_rescan_filters_blocking, atpackages/rs-platform-wallet/src/manager/accessors.rs:851-868, exported asplatform_wallet_manager_spv_rescan_filters(packages/rs-platform-wallet-ffi/src/spv.rs:630-639);- DashPay contact reconciliation, at
packages/rs-platform-wallet/src/wallet/identity/network/payments.rs:179, called frommanager/dashpay_sync.rs:491.
- Gaps 2–4: need a reload. Gaps 2 and 3 affect hosts that serialize managed accounts (
ManagedCoreFundsAccount, or aManagedWalletInfoholding them) withfinalized_txidsand load them back. Gap 4 affects hosts that do so with bincode. Hosts that rebuild wallet state fromWalletEvents, or from records and UTXOs stored from events, never loadfinalized_txidsor summaries, so gaps 2–4 can't reach them. Platform's SQLite store is one (packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs:280,322-326). - Gap 5: needs no reload. It affects any host that replaces a stored record with an
updatedrecord fromBlockProcessedorTransactionCheckResult.
1. A spent late output is credited until its spender replays
Where:
- the finalized summary path takes the output in at
key-wallet/src/managed_account/managed_core_funds_account.rs:1147; take_in_mined_outputsinserts it withis_confirmed = trueat:1188-1191;- every guard
take_in_outputchecks (:299) misses it. A non-coinbase, unlocked UTXO is spendable (key-wallet/src/utxo.rs:86).
The output stays credited until the spender's block replays.
Why no guard holds the spend. update_utxos puts a transaction's inputs into spent_outpoints only when it records the transaction (managed_core_funds_account.rs:522-523). The late output's spender may never have been recorded on the account:
- It was irrelevant on first sight. It spent only the late output, which the account had not derived yet, and paid nothing the account watches.
check_core_transactionnotes its input as an observed spend (key-wallet/src/transaction_checking/wallet_checker.rs:297;key-wallet/src/wallet/managed_wallet_info/mod.rs:546). It then returns before recording, because the transaction is not relevant (wallet_checker.rs:322-331). - Its block was never delivered, because its filter matched nothing the wallet watched at the time.
Either way, once a ChainLock and the synced height pass the spender's height, prune_finalized_observed_spends drops the observed spend (mod.rs:576-583). A serde reload loses the mark even for a spender that was recorded: rebuild_spent_outpoints (managed_core_funds_account.rs:1685-1691) reads only kept records, and finality pruned the spender's.
Reproduction A, no reload, at the WalletManager level (gap limit 30, VALUE per payment). Run as a manager-level test on dev, this sequence gives the balances in steps 7 and 8.
- A ChainLock at height 10000 is applied.
- Block 100 holds
A, which pays external indices 0 and 40. OnlyA:0is in the window.Ais finalized, with a summary owning output 0. - Block 5101 holds
S, which spendsA:1to a foreign script, and a payment to index 1, so the block is delivered.Sis irrelevant and not recorded. Its input is observed. - The synced height reaches 10000, which prunes the observed spend.
- Block 10001 pays index 20. That extends the window over index 40. The active-batch rescan doesn't revisit heights below the committed checkpoint.
- The host rewinds the checkpoint with
update_synced_height(99)on the liveManagedWalletInfo, asspv_rescan_filters_blockingdoes. dash-spv's filterticksees the wallet behind its committed height and rescans from it (dash-spv/src/sync/filters/sync_manager.rs:220-251). - Block 100 replays.
A:1is adopted and credited as confirmed. The balance goes from 3·VALUEto 4·VALUE, and the block'sBlockProcessedcarriesA's updated record owning{0, 1}. - Block 5101 replays.
Sis now relevant, so it is recorded as new, andupdate_utxosspendsA:1. The balance is back to 3·VALUE.
Steps 7 and 8 can be far apart. Filter batches are 5000 blocks (dash-spv/src/sync/filters/manager.rs:62), and between those blocks the coin is in the balance and coin selection can pick it. The rewound checkpoint lives only in memory (accessors.rs:844-849 at platform 8411b4e1d5). If the host stops between the two heights, a store that applied block 100's updated record keeps A:1 unspent, because block 5101 isn't re-delivered after a restart.
Reproduction B, after a reload, using #1326's test_redelivered_finalized_spender_spends_a_late_output_a_reload_forgot (wallet_checker.rs:4499-4522):
earlypays addresses 1 and2·gap, and is recorded in a chainlocked block. Its record is pruned to a summary owning output 0 only.spendspendsearly:1(far) and is recorded and finalized.note_chain_lock_height(101)andupdate_synced_height(101)prune the observed spend offar.- The account is reloaded through
serde_json(:4511-4513).rebuild_spent_outpointsfinds no kept record that spendsfar. laterpays addressgap, which extends the pool over2·gap.earlyis re-delivered (:4517). The summary does not own output 1, sofaris adopted and inserted as a confirmed coin.- Only the re-delivery of
spend(:4518) removes it, through the re-mark loop atmanaged_core_funds_account.rs:1071-1074. The assertion at:4520runs after both calls.
In both, the window is visible from outside, because the funding and spending transactions are in different blocks:
WalletManager::finalize_block_advancerefreshes balances (key-wallet-manager/src/process_block.rs:620-630) and emitsBlockProcessedafter each block (:666);- dash-spv releases the wallet lock after each
process_block_for_walletscall (dash-spv/src/sync/blocks/manager.rs:136-137).
The comment at managed_wallet_info/mod.rs:568-571 notes the temporary credit. But it says "the re-delivery of its finalized spender" re-marks the coin. In reproduction A the spender was never finalized on the account: it is recorded fresh when its block replays.
The final state is right in both, and a rescan in Dash Core also credits the coin at the funding block and debits it at the spender's.
2. The re-delivered spender removes the coin, but no record carries the spend
Where: managed_core_funds_account.rs:1071-1074, together with the summary path at :1087-1122.
Reproduction B of gap 1:
- At step 6 of reproduction B,
earlyis emitted asupdated, rebuilt from its summary and owningfar. - At step 7,
spendis re-delivered. The loop removesfarand setscoins_changed, so the result reportsstate_modified. spend's summary already owns its output, so nothing is adopted and no record is returned. The rebuiltinput_detailswould come from the summary anyway, which has no entry forfar.attribute_late_inputs(wallet_checker.rs:56) walks only kept records (funds.transactions(),:73-75), so a pruned spender never has the input attributed.
The in-memory state ends right. A store that applies BlockProcessed records gets far from early's updated record and no record that spends it, so it keeps a spent coin as unspent. The trait doc at wallet_checker.rs:35-40 says rebuilt records have only the inputs known when they were pruned. It does not say that a spend can go unreported.
Without a reload (reproduction A of gap 1), the spender is not finalized on the account. It is recorded as new when its block replays, with the late input in its input_details, so a store sees the spend. Gap 2 needs the reload.
3. A snapshot without summaries misses a late output whose address was used
Where: managed_core_funds_account.rs:1133-1144, especially the filter !info.is_used() at :1138.
A snapshot from before #1326 loads with an empty finalized_summaries (serde(default), key-wallet/src/managed_account/managed_core_keys_account.rs:80-82). For its finalized txids, an output counts as new only while its address is unused. That does not tell whether this outpoint was ever credited.
Reproduction: combine the legacy reload variant of test_reloaded_account_rebuilds_the_pruned_record_a_rescan_corrects (wallet_checker.rs:4355-4359) with the intervening payment of test_redelivered_finalized_transaction_credits_a_late_output_another_payment_used (:4394-4415):
earlypays addresses 1 and2·gapand is finalized, owning output 0. The account is reloaded withfinalized_summariesremoved.- A payment to address
gapextends the pool over2·gap. - A payment to address
2·gapmarks that address used. earlyis re-delivered. Output 1 is filtered out at:1138. Further passes can't change the used flag, so the coin is never credited.
The existing used-address test passes because it has a summary. The existing legacy variant passes because nothing pays the late address first.
4. The bincode layout changes in both directions
Where: managed_core_keys_account.rs:80-82.
finalized_summaries sits between finalized_txids and the skipped monitor_revision. ManagedCoreKeysAccount is the first field of the funds account's deserialize Helper (managed_core_funds_account.rs:1700-1709). serde(default) supplies a missing named field in JSON. bincode is positional, so it does nothing there:
- old snapshot, new reader: the reader takes the next bytes, the start of
balance, as the map length. It fails, or misreads when that varint is 0. - new snapshot, old reader: the reader takes the map bytes as
balance.
Still fine:
- JSON loads both ways. An old reader ignores the key and drops the summaries, which leaves those txids on the gap-3 path.
keep-finalized-transactionsremoves the field, so this layout already depends on that feature.- No in-tree code bincode-encodes managed accounts:
key-wallet-manager/src/lib.rs:382encodes aWalletwith native bincode, and that type is unchanged. The bincode case in the reload test (wallet_checker.rs:4349-4353) only round-trips new to new.
5. A rebuilt record has no fee and an empty label
Where: managed_core_funds_account.rs:1109-1119 builds the rebuilt copy with TransactionRecord::new (key-wallet/src/managed_account/transaction_record.rs:186-208), which sets fee: None and label: String::new(). FinalizedSummary keeps the owned output indices and input details, but not the fee or the label.
Any re-delivery that finds a pruned record owning more outputs emits this copy as updated. That includes a block born chainlocked with no reload. A store that replaces the whole stored record with it loses the label or fee it held for that txid.
Possible directions
These come from review; none is prescribed here.
- Gap 1:
- keep enough spent evidence across a reload, finality and a checkpoint rewind until the replay is done, for example the spent outpoints of pruned records, or observed spends of outpoints the wallet has not credited;
- or keep coins credited during a replay out of balance and selection until the spender replays;
- on the host side: hold sends while a rewind's rescan runs, or persist the rewound checkpoint, so an interrupted rescan resumes instead of leaving the coin stored as unspent.
- Gap 2: when the re-mark removes a held coin, rebuild and emit the spender's record with that input, and add the input to its summary.
- Gap 3:
- an explicit rebuild or rescan step for finalized state that has no summaries;
- or stop treating address use as proof that an outpoint was processed.
- Gap 4: document the break, and require regeneration or migration of binary managed-account snapshots (or version them).
- Gap 5:
- keep the fee and label in the summary;
- or document that a rebuilt copy lacks them, so stores merge instead of replacing.
🤖 Generated with Claude Code
Activity