Skip to content

key-wallet: finalized re-delivery can credit spent outputs after a checkpoint rewind or a serde reload #1361

Open

Description

@PastaPastaPasta

Problem

#1326 (f2dd27c2f) lets a rescan credit outputs a finalized transaction owns only through addresses derived after it was recorded. Its finalized re-delivery path, ManagedCoreFundsAccount::adopt_finalized_outputs, credits such an output unless a spend guard in take_in_output refuses it. For an output spent before the account derived it, the guards that can know of the spend are:

  • the account's spent_outpoints, which holds the inputs of recorded transactions;
  • the wallet's observed spends, which finality prunes.

When neither holds the spend, the re-delivery credits a coin that is already spent. Further gaps arise when a managed account is reloaded through serde, and in the record rebuilt from a summary. All line numbers below are at dev 0e9668b01.

The gaps:

  1. A spent late output is credited as confirmed until its spender's block replays. This happens without any reload, when the spender was never recorded on the account and the host rewinds the checkpoint, or after a reload.
  2. After a reload, the re-delivered spender removes the coin, but no record carries the spend.
  3. A snapshot written before fix(key-wallet): re-emit a record a rescan finds owning more outputs #1326 (no summaries) misses a late output whose address another payment has used.
  4. The bincode layout of managed-account snapshots changes in both directions.
  5. A record rebuilt from a summary has fee: None and an empty label.

Who is affected

  • Gap 1: every host that re-delivers a finalized funding block after finality has pruned the observed spend, whether or not it serializes managed accounts. The late output's spender must also hold no spent mark on the account. One way is to rewind a live wallet's checkpoint to rescan. Another is to reload a managed account. Platform-wallet rewinds the checkpoint in memory, with no reload, in two places (dashpay/platform 8411b4e1d5):
    • spv_rescan_filters_blocking, at packages/rs-platform-wallet/src/manager/accessors.rs:851-868, exported as platform_wallet_manager_spv_rescan_filters (packages/rs-platform-wallet-ffi/src/spv.rs:630-639);
    • DashPay contact reconciliation, at packages/rs-platform-wallet/src/wallet/identity/network/payments.rs:179, called from manager/dashpay_sync.rs:491.
  • Gaps 2–4: need a reload. Gaps 2 and 3 affect hosts that serialize managed accounts (ManagedCoreFundsAccount, or a ManagedWalletInfo holding them) with finalized_txids and load them back. Gap 4 affects hosts that do so with bincode. Hosts that rebuild wallet state from WalletEvents, or from records and UTXOs stored from events, never load finalized_txids or summaries, so gaps 2–4 can't reach them. Platform's SQLite store is one (packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs:280,322-326).
  • Gap 5: needs no reload. It affects any host that replaces a stored record with an updated record from BlockProcessed or TransactionCheckResult.

1. A spent late output is credited until its spender replays

Where:

  • the finalized summary path takes the output in at key-wallet/src/managed_account/managed_core_funds_account.rs:1147;
  • take_in_mined_outputs inserts it with is_confirmed = true at :1188-1191;
  • every guard take_in_output checks (:299) misses it. A non-coinbase, unlocked UTXO is spendable (key-wallet/src/utxo.rs:86).

The output stays credited until the spender's block replays.

Why no guard holds the spend. update_utxos puts a transaction's inputs into spent_outpoints only when it records the transaction (managed_core_funds_account.rs:522-523). The late output's spender may never have been recorded on the account:

  • It was irrelevant on first sight. It spent only the late output, which the account had not derived yet, and paid nothing the account watches. check_core_transaction notes its input as an observed spend (key-wallet/src/transaction_checking/wallet_checker.rs:297; key-wallet/src/wallet/managed_wallet_info/mod.rs:546). It then returns before recording, because the transaction is not relevant (wallet_checker.rs:322-331).
  • Its block was never delivered, because its filter matched nothing the wallet watched at the time.

Either way, once a ChainLock and the synced height pass the spender's height, prune_finalized_observed_spends drops the observed spend (mod.rs:576-583). A serde reload loses the mark even for a spender that was recorded: rebuild_spent_outpoints (managed_core_funds_account.rs:1685-1691) reads only kept records, and finality pruned the spender's.

Reproduction A, no reload, at the WalletManager level (gap limit 30, VALUE per payment). Run as a manager-level test on dev, this sequence gives the balances in steps 7 and 8.

  1. A ChainLock at height 10000 is applied.
  2. Block 100 holds A, which pays external indices 0 and 40. Only A:0 is in the window. A is finalized, with a summary owning output 0.
  3. Block 5101 holds S, which spends A:1 to a foreign script, and a payment to index 1, so the block is delivered. S is irrelevant and not recorded. Its input is observed.
  4. The synced height reaches 10000, which prunes the observed spend.
  5. Block 10001 pays index 20. That extends the window over index 40. The active-batch rescan doesn't revisit heights below the committed checkpoint.
  6. The host rewinds the checkpoint with update_synced_height(99) on the live ManagedWalletInfo, as spv_rescan_filters_blocking does. dash-spv's filter tick sees the wallet behind its committed height and rescans from it (dash-spv/src/sync/filters/sync_manager.rs:220-251).
  7. Block 100 replays. A:1 is adopted and credited as confirmed. The balance goes from 3·VALUE to 4·VALUE, and the block's BlockProcessed carries A's updated record owning {0, 1}.
  8. Block 5101 replays. S is now relevant, so it is recorded as new, and update_utxos spends A:1. The balance is back to 3·VALUE.

Steps 7 and 8 can be far apart. Filter batches are 5000 blocks (dash-spv/src/sync/filters/manager.rs:62), and between those blocks the coin is in the balance and coin selection can pick it. The rewound checkpoint lives only in memory (accessors.rs:844-849 at platform 8411b4e1d5). If the host stops between the two heights, a store that applied block 100's updated record keeps A:1 unspent, because block 5101 isn't re-delivered after a restart.

Reproduction B, after a reload, using #1326's test_redelivered_finalized_spender_spends_a_late_output_a_reload_forgot (wallet_checker.rs:4499-4522):

  1. early pays addresses 1 and 2·gap, and is recorded in a chainlocked block. Its record is pruned to a summary owning output 0 only.
  2. spend spends early:1 (far) and is recorded and finalized.
  3. note_chain_lock_height(101) and update_synced_height(101) prune the observed spend of far.
  4. The account is reloaded through serde_json (:4511-4513). rebuild_spent_outpoints finds no kept record that spends far.
  5. later pays address gap, which extends the pool over 2·gap.
  6. early is re-delivered (:4517). The summary does not own output 1, so far is adopted and inserted as a confirmed coin.
  7. Only the re-delivery of spend (:4518) removes it, through the re-mark loop at managed_core_funds_account.rs:1071-1074. The assertion at :4520 runs after both calls.

In both, the window is visible from outside, because the funding and spending transactions are in different blocks:

  • WalletManager::finalize_block_advance refreshes balances (key-wallet-manager/src/process_block.rs:620-630) and emits BlockProcessed after each block (:666);
  • dash-spv releases the wallet lock after each process_block_for_wallets call (dash-spv/src/sync/blocks/manager.rs:136-137).

The comment at managed_wallet_info/mod.rs:568-571 notes the temporary credit. But it says "the re-delivery of its finalized spender" re-marks the coin. In reproduction A the spender was never finalized on the account: it is recorded fresh when its block replays.

The final state is right in both, and a rescan in Dash Core also credits the coin at the funding block and debits it at the spender's.

2. The re-delivered spender removes the coin, but no record carries the spend

Where: managed_core_funds_account.rs:1071-1074, together with the summary path at :1087-1122.

Reproduction B of gap 1:

  1. At step 6 of reproduction B, early is emitted as updated, rebuilt from its summary and owning far.
  2. At step 7, spend is re-delivered. The loop removes far and sets coins_changed, so the result reports state_modified.
  3. spend's summary already owns its output, so nothing is adopted and no record is returned. The rebuilt input_details would come from the summary anyway, which has no entry for far.
  4. attribute_late_inputs (wallet_checker.rs:56) walks only kept records (funds.transactions(), :73-75), so a pruned spender never has the input attributed.

The in-memory state ends right. A store that applies BlockProcessed records gets far from early's updated record and no record that spends it, so it keeps a spent coin as unspent. The trait doc at wallet_checker.rs:35-40 says rebuilt records have only the inputs known when they were pruned. It does not say that a spend can go unreported.

Without a reload (reproduction A of gap 1), the spender is not finalized on the account. It is recorded as new when its block replays, with the late input in its input_details, so a store sees the spend. Gap 2 needs the reload.

3. A snapshot without summaries misses a late output whose address was used

Where: managed_core_funds_account.rs:1133-1144, especially the filter !info.is_used() at :1138.

A snapshot from before #1326 loads with an empty finalized_summaries (serde(default), key-wallet/src/managed_account/managed_core_keys_account.rs:80-82). For its finalized txids, an output counts as new only while its address is unused. That does not tell whether this outpoint was ever credited.

Reproduction: combine the legacy reload variant of test_reloaded_account_rebuilds_the_pruned_record_a_rescan_corrects (wallet_checker.rs:4355-4359) with the intervening payment of test_redelivered_finalized_transaction_credits_a_late_output_another_payment_used (:4394-4415):

  1. early pays addresses 1 and 2·gap and is finalized, owning output 0. The account is reloaded with finalized_summaries removed.
  2. A payment to address gap extends the pool over 2·gap.
  3. A payment to address 2·gap marks that address used.
  4. early is re-delivered. Output 1 is filtered out at :1138. Further passes can't change the used flag, so the coin is never credited.

The existing used-address test passes because it has a summary. The existing legacy variant passes because nothing pays the late address first.

4. The bincode layout changes in both directions

Where: managed_core_keys_account.rs:80-82.

finalized_summaries sits between finalized_txids and the skipped monitor_revision. ManagedCoreKeysAccount is the first field of the funds account's deserialize Helper (managed_core_funds_account.rs:1700-1709). serde(default) supplies a missing named field in JSON. bincode is positional, so it does nothing there:

  • old snapshot, new reader: the reader takes the next bytes, the start of balance, as the map length. It fails, or misreads when that varint is 0.
  • new snapshot, old reader: the reader takes the map bytes as balance.

Still fine:

  • JSON loads both ways. An old reader ignores the key and drops the summaries, which leaves those txids on the gap-3 path.
  • keep-finalized-transactions removes the field, so this layout already depends on that feature.
  • No in-tree code bincode-encodes managed accounts: key-wallet-manager/src/lib.rs:382 encodes a Wallet with native bincode, and that type is unchanged. The bincode case in the reload test (wallet_checker.rs:4349-4353) only round-trips new to new.

5. A rebuilt record has no fee and an empty label

Where: managed_core_funds_account.rs:1109-1119 builds the rebuilt copy with TransactionRecord::new (key-wallet/src/managed_account/transaction_record.rs:186-208), which sets fee: None and label: String::new(). FinalizedSummary keeps the owned output indices and input details, but not the fee or the label.

Any re-delivery that finds a pruned record owning more outputs emits this copy as updated. That includes a block born chainlocked with no reload. A store that replaces the whole stored record with it loses the label or fee it held for that txid.

Possible directions

These come from review; none is prescribed here.

  • Gap 1:
    • keep enough spent evidence across a reload, finality and a checkpoint rewind until the replay is done, for example the spent outpoints of pruned records, or observed spends of outpoints the wallet has not credited;
    • or keep coins credited during a replay out of balance and selection until the spender replays;
    • on the host side: hold sends while a rewind's rescan runs, or persist the rewound checkpoint, so an interrupted rescan resumes instead of leaving the coin stored as unspent.
  • Gap 2: when the re-mark removes a held coin, rebuild and emit the spender's record with that input, and add the input to its summary.
  • Gap 3:
    • an explicit rebuild or rescan step for finalized state that has no summaries;
    • or stop treating address use as proof that an outpoint was processed.
  • Gap 4: document the break, and require regeneration or migration of binary managed-account snapshots (or version them).
  • Gap 5:
    • keep the fee and label in the summary;
    • or document that a rebuilt copy lacks them, so stores merge instead of replacing.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions