Repository navigation
Description
Registered in the platform-wallet e2e test plan as Found-034 (pinned by
the found_coinjoin_gap_limit_sync::sim_tests simulation suite).
Summary
FiltersManager's new-script rescan (added in #820) re-applies matched
blocks only within active batches. Once a batch commits, it is removed
from active_batches and its per-wallet processed records are pruned, and
the per-wallet synced_height advances past its range. Scripts derived
later by wallet gap-limit maintenance are therefore never re-tested against
that range — the re-scan gate is effectively keyed by (wallet,
block/commit progress) instead of (wallet, address/script). Outputs that
pay gap-window addresses whose scripts did not exist yet when their block's
batch committed are silently and permanently missed: a fresh re-sync
from genesis walks into the same wall deterministically, so the funds are
invisible until the user manually pre-derives addresses.
This is the residual of the CoinJoin gap-limit sync defect found by the
rs-platform-wallet e2e suite (dashpay/platform#3549,
found_coinjoin_gap_limit_sync.rs): the dense same-batch shape (empirical
stall at highest_used = 59) was fixed by #820; the cross-commit-boundary
shape is still live. The e2e ground-truth data for the affected testnet
wallet shows real index↔height inversions (first at index ~1767), so this
is not hypothetical for deep CoinJoin usage; a plain BIP-44 wallet paid
out of index order across a batch boundary (5000 blocks) hits it too.
Affected code (dev @ 647fa98)
| Site | Role |
|---|---|
dash-spv/src/sync/filters/manager.rs:623-632 |
rescan_batch bails for any batch_start not in active_batches — committed ranges unreachable |
dash-spv/src/sync/filters/manager.rs:479-512 |
commit-time rescan covers only the committing batch + later scanned active batches |
dash-spv/src/sync/filters/manager.rs: 519, 535 |
commit removes the batch and prunes tracker records at/below the committed height |
dash-spv/src/sync/filters/manager.rs:648-666 |
rescan skips heights at/below the wallet's own synced_height |
dash-spv/src/sync/filters/block_match_tracker.rs:72-117 |
gate keyed by (wallet, block); track_for_new_scripts (#820) bypasses it only for blocks the rescan can still reach |
dash-spv/src/sync/filters/sync_manager.rs:164 |
wallet recorded done for a block on BlockProcessed |
key-wallet/src/transaction_checking/account_checker.rs:644 |
check_transaction_for_match recognises only already-generated addresses — correct, but it makes the SPV-side re-test the only recovery path |
Root cause
Discovery suppression is keyed by scan progress (which blocks/batches this
wallet has already had applied/committed), not by which scripts the block
was actually tested against. #820 fixed the in-flight half by letting
rescan_batch re-queue already-processed blocks of active batches. The
committed half still loses information: after commit there is no record of
which scripts a block was matched against, no way to requeue it, and the
wallet's advanced synced_height filters the heights out of any later
match anyway. Newly derived gap-limit scripts consequently have zero
effective backward reach across a commit boundary.
Reproduction (deterministic, crate-level, no network)
Crate-level repro: dash-spv/src/sync/filters/coinjoin_gap_discovery_tests.rs (committed on
branch repro/pr3549-rdc) drives the real pipeline — FiltersManager +
BlockMatchTracker + a real WalletManager<ManagedWalletInfo> with a
CoinJoin account — through the production event loop (try_process_batch →
BlocksNeeded → process_block_for_wallets → BlockProcessed →
commit-time rescan) with synthetic blocks and real BIP-158 filters:
coinjoin_gap_limit_dense_same_batch_recovers— the original e2e
stall-at-59 shape (block funds CoinJoin indices 0..=51, next block
52..=139, one batch). PASSES (regression guard for fix(dash-spv): reprocess blocks on rescan against newly derived scripts #820).coinjoin_gap_limit_inversion_within_batch_recovers— indices 40..=51
funded at height 10, indices 0..=29 at height 20, same batch.
PASSES (fix(dash-spv): reprocess blocks on rescan against newly derived scripts #820's fixpoint recovers the earlier block).coinjoin_gap_limit_stall_across_committed_batch— identical shape,
but height 10 sits in batch 0..=99 and the in-window block at height 110
in batch 100..=199. FAILS (RED):
assertion `left == right` failed: CoinJoin External indices 40..=51 were funded at
height 10 in a batch that committed before their scripts were derived, and the
new-script rescan never looks below the committed boundary [...]
highest_used stalls at Some(29), used_count=30
left: Some(29)
right: Some(51)
Run: cargo test -p dash-spv --lib coinjoin_gap
Expected vs actual
- Expected: any output paying an address within the gap-limit recovery
contract (index ≤ highest_used + 1 + gap at fixpoint) is discovered
regardless of which batch/commit its block landed in — the guarantee
BIP-44 recovery semantics give and that dash-core/Electrum-style
per-address history scans provide. In the repro:highest_used = Some(51),
all 42 funded indices credited. - Actual:
highest_used = Some(29); indices 40..=51 (watched by then —
highest_generated = Some(59)) are never re-tested against the committed
block; their UTXOs and balance are missing. Re-syncing from scratch
reproduces the loss deterministically.
Severity (OWASP-normalized floats — label derived by pipeline)
risk: 0.4 — needs an index↔height inversion (or restart-shaped
equivalent) crossing a 5000-block batch boundary; ground-truth e2e data
shows such inversions on a real CoinJoin wallet; deterministic once the
shape exists, and completely silent (no error, no log).impact: 0.7 — silent integrity failure of wallet state: confirmed
funds invisible to balance/UTXO views, unrecoverable by resync; user may
act on a wrong balance.scope: 0.4 — dash-spv historical filter sync; wallets with
out-of-index-order funding across batch boundaries (deep CoinJoin usage
is the known concrete population).
Likelihood caveat: the repro compresses the batch boundary to ~100
blocks so the inversion is deterministic to construct; production batches
are 5000 blocks, so a genuine index↔height inversion spanning that wider
boundary is rarer in practice than the repro's density implies. The risk: 0.4 estimate sits at the upper edge of what that lower real-world
frequency would justify — it is defensible mainly because the e2e
ground-truth data already shows a real inversion on a live testnet wallet
(first at index ~1767), not because the condition is common. Severity
MEDIUM stands regardless: impact and scope are unaffected by this
caveat.
overall = (0.4 + 0.7 + 0.4) / 3 ≈ 0.5 → MEDIUM
Fix direction
Any of (roughly in order of fidelity):
- Key match suppression by script coverage: have
BlockMatchTracker(or
filter storage) record the script-set revision a block/batch was
matched against, and re-open committed ranges for a wallet when its
script set grows (re-runcheck_compact_filters_for_script_pubkeys
over stored filters belowcommitted_heightwith only the new
scripts — cheap, filters are already on disk). - On new-script derivation, lower the owning wallet's effective rescan
floor (thetickcatch-up path already knows how to restart a wallet
whosesynced_heightfell behind — reuse it withsynced_height = min(synced_height, birth-of-new-scripts-uncertainty)or simply the
batch start of the oldest still-stored filter). - Minimum: iterate commit-time rescans over stored (not just active)
filter ranges for the wallets named innew_scripts.
The GREEN tests (1)–(2) above pin the already-fixed behaviour so a fix for
the committed-range hole can't regress the #820 fixpoint; test (3) flips to
GREEN when the residual is fixed.
Cross-references
- Origin: test(platform-wallet): e2e framework + full test suite — triage pins, Found-*/PA-* guards, fail-closed persist, Stage-2 merge platform#3549 e2e
found_coinjoin_gap_limit_sync.rs
(diagnose-only reproduction + block-atomic simulation of the old
stall-at-59; itsSPV_BACKWARD_RESCAN_BLOCKS = 0constant models the
pre-fix(dash-spv): reprocess blocks on rescan against newly derived scripts #820 behaviour and should be revisited after this issue). - Partial fix: fix(dash-spv): reprocess blocks on rescan against newly derived scripts #820 (
1b534581) — reprocess blocks on rescan against newly
derived scripts (active batches only). - Related account-resolution gap for CoinJoin funding: Found-012:
wait_for_proofandrecover_asset_lock_blockinghard-code BIP-44 — asset-lock proof flow misses CoinJoin / legacy BIP-32 funding platform#3642.
🤖 Co-authored by Claudius the Magnificent AI Agent
Activity
fix(dash-spv): rescan committed filter ranges for newly derived scrip…
bfoss765 commentedon Jul 12, 2026
Real-world confirmation of this defect from Android integration testing (dashj→Kotlin-SDK migration work), plus one datum on gap-limit sizing:
Setup: a real, long-lived testnet wallet — 12.08713251 DASH, 1,047 transactions, 104 UTXOs, 4,517 used CoinJoin keys (heavy mixer), birth height 0. Restored from seed side-by-side: dashj 22.0.3 as oracle, and a fresh SDK wallet (dash-spv via rs-platform-wallet/unified-sdk JNI) with
DEFAULT_COINJOIN_GAP_LIMITalready widened to 100 (dashj lookahead parity, per the #868 proposal).Result:
synced_heightat tip, no fault raised — the funds silently invisible, deterministically, exactly the Found-034 shape.Two takeaways for this issue:
A fix PR for this issue is in progress (re-keying the re-scan reach so scripts derived after a batch commits can still be tested against the committed range), targeting the red repro test here (
coinjoin_gap_limit_stall_across_committed_batch).docs(dash-spv): update gap-discovery test narrative — committed-batch…
fix(dash-spv): rescan committed filter ranges for newly derived scrip…
test(key-wallet): pin born-fully-spent recovered transactions stay in…
test(key-wallet): pin born-fully-spent recovered transactions stay in…
fix(key-wallet): out-of-order UTXO spend recorded nowhere — phantom u…
fix(dash-spv): rescan committed filter ranges for newly derived scrip…
fix(dash-spv): durable pending-sweep set — replay interrupted script …
fix(dash-spv): durable pending-sweep set — replay interrupted script …
fix(dash-spv): coalesce committed-range filter rescans across batch c…
fix(dash-spv): rescan committed filter ranges for newly derived scrip…
fix(dash-spv): coalesce committed-range filter rescans across batch c…
refactor(dash-spv): drop the committed-range sweep and the collected-…
refactor(dash-spv): drop the committed-range sweep and the collected-…
refactor(dash-spv): drop the committed-range sweep and the collected-…
refactor(dash-spv): drop the committed-range sweep and the collected-…
refactor(dash-spv): drop the committed-range sweep and the collected-…