Repository navigation
fix(scanner): stop at an unscannable oracle block instead of silently skipping it - #20
Conversation
… messages Adds an in-process seam (BlockScanDataStream) between scan_block_range and the scan loop, with no behaviour change, and tests that drive a real Scanner through it. They fail on this commit: an empty-hash block mid-range is treated as an empty block (payments in it are lost and the scan reports success), height gaps and short streams are accepted, and any error status panics the process via unwrap(). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ipping it An oracle answers a height it has not indexed with an OK message carrying an empty block hash and no data. scan_block_range took that as a block with no payments: it warned, advanced the scan height past it and reported success, so any payment in that block was never found (proven by the testkit mutation check unindexed-ok-empty: 80000 sats reported instead of 120600). Separately, every stream message was unwrap()ed, so an error status (NOT_FOUND, UNAVAILABLE) panicked the process, as did failing to open the stream. Now each message must be the next expected height and carry a valid 32-byte, non-zero block hash; otherwise, and on any error status or a stream that ends before the requested end, the scan returns an error naming the height to rescan from. The scan height stays at the last fully processed block, so watch_chain retries from there on its next poll (every 10 s, no spin) and blindbit-cli exits non-zero after saving that height. The example is updated the same way. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s below it; report stalls A wallet whose start height lies below the oracle's first indexed block (the hosted oracle has nothing below 800000 on mainnet and 100000 on signet) stopped at its first height on every poll, forever, with only an error-level log line. Since #20 refuses to scan past an unindexed height, that is the only thing such a wallet ever did. - When `watch_chain`'s scan stops at the wallet's start height because the oracle has not indexed it, the scanner locates the oracle's lowest indexed height with GetBlockHashByHeight lookups and starts there, logs a warning, persists the adjustment (`oracle_floor_start`, optional in the state file) and publishes it in the scan health. - Only heights below the oracle's floor are skipped. A height above the first scanned block, or one with an indexed height below it (a gap in the oracle's index), keeps #20's stop-and-retry. The search is a binary search verified by exponential back-off probes below the boundary, so a gap above the floor does not move the floor up. NOT_FOUND and an empty or zero hash both mean "not indexed". - A scan that cannot get past a height (or cannot reach the oracle) is published as a stall: height, reason, and the time of the first failed attempt at that height. It clears when the scan makes progress. - Stream stops are a typed `ScanStopped` error (same message text) so the loop can tell "not indexed" from other failures. - Health lives in the wallet's Electrum index, which status readers lock without waiting on the scan task. friglet's GetStatus reports it as `StatusInfo::scan_health` (serde default, so old trays and daemons interoperate), and the tray's status window shows a stall line and a note for an adjusted start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd-labels #20 lands before this PR, so this branch carries it to merge cleanly after it. changeset.rs: keep both imports (#20's OracleFloorStart and this branch's OwnedOutputRecord); the serde_hex module #20 still had is the one this branch replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n attribute) The status window's error, stall and scan-notes blocks use class="col", whose display: flex beats the user-agent [hidden] rule, so an empty "Last error" label was always shown and the new stall/notes blocks would be too. Seen in a screenshot against a live daemon. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd-labels Picks up the tray CSS fix for hidden status blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fix/reorg-rollback Keeps this branch a superset of the stack below it. - changeset.rs / scanner.rs: keep both sides' new fields (scanned_block_hashes; format_version, oracle_floor_start). - scanning.rs: keep this branch's scan_range_from / BlockStreamSource; OracleStreams::open now reports a refused stream as #20's typed ScanStopped. watch_chain combines both: the no-new-block hash check from here, and #20's floor start (only for a real range) and stall reporting, with the ReorgTooDeep log kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Picks up #20's tray CSS fix for hidden status blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ood/preview Brings in the #17/#20/#19/#21/#31 stack. One conflict, in friglet/src/electrum/mod.rs, resolved keeping both sides: - #26's ChainSource (p2p/network/oracle) and pending-broadcast store, - #21's reorg handling: checkpoints behind a mutex, reorg_rx feeding forget_blocks_above; its test now builds the server state with #26's test helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the fix stack (#17, #20, #19, #21) so the T0 harness runs against the receive path that records owned outputs. scanning.rs: both sides extracted apply_matched_block; kept #21's sync_owned_outputs() call after it in the scan loop and its doc line, and kept both test modules (owned_outputs_tests from #21, testkit_t0 from this branch). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The T0 harness replayed each block through a hand copy of the scan loop's per-block steps, and that copy had already fallen behind: the loop on the fix stack records owned outputs (sync_owned_outputs) after apply_matched_block and the copy did not, so nine spends were carried as KNOWN_SPEND_GAP. process_block now drives Scanner::scan_block_stream, the per-block loop scan_block_range and watch_chain run, with the oracle's message streamed in-process and the full block served in place of the P2P fetch (the stream-safety test hooks from #20/#21). There is no copy left to drift: a step added to the loop is exercised here. State files the loop now saves are removed when a scenario ends. KNOWN_SPEND_GAP and its assertions are gone; all nine scenarios must pass with their spends. Dropping the sync_owned_outputs() call from the loop turns exactly those nine spends red again (k-gap, labelled-multi, spent-outputs, dust-boundary, rescan). Fixture provenance now names testkit 68f9f1c, the merge of testkit PR #25 into master; the fixtures are byte-identical to the export at b1f962e. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since ScanStopped became a struct (22c50bf), blindbit-cli printed a stopped scan as `Error: ScanStopped { height: 104, reason: "...", not_indexed: true }`, the Debug form main uses for a returned error. The message the type renders for the user, "scan stopped at height 104: ...; nothing from height 104 on was scanned, rescan from height 104", no longer appeared. The testkit's mutation checks match that message and went red for stale-view, truncated-stream, close-mid-stream and unindexed-ok-empty. main now runs the command and prints the error with Display. The exit status of a failed command is still 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the blindbit-cli error display fix from #20. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the blindbit-cli error display fix from #20. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the blindbit-cli error display fix from #20. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Added d989e53 Problem. Since How it was found. The testkit's mutation checks match on that message. With the pin moved to #31's previous head d3200b8, four of them went red: stale-view, truncated-stream, close-mid-stream and unindexed-ok-empty (testkit run 36878809374). Fix. Propagation. Merged up the stack (merges only):
Verification. Built |
Important
Deploy order (re-checked against the oracle PRs' diffs):
NOT_FOUNDat an unindexed height) deploys last, only once clients with this PR are out. Older clientsunwrap()every stream message; withpanic = "abort"in the release profile the whole friglet process dies, and any wallet whose start height is below the oracle's first block hits that on its first poll.NOT_FOUND) have no ordering constraint.NOT_FOUNDfromGetBlockHashByHeightis read as "not indexed".Bug: silent fund loss
block_hash, an empty compute index and no spent outputs.scan_block_rangehandled that message inblindbit-lib/src/scanner/scanning.rsatorigin/integration:scan_short_block_dataloggedWARN block has malformed block_hash; spent-output notifications will be skipped(:627-631) and skipped only the spent-output check.unindexed-ok-emptygets 80000 sats instead of 120600, so two outputs are lost silently. friglet runs the same scan loop.A second bug is in the same loop.
stream.message().await.unwrap()(:95) and the stream-opening.await.unwrap()(:70) panic the process on any error status, such asNOT_FOUNDorUNAVAILABLE. Because of this, the oracle could not start returning errors either.Fix (
blindbit-lib/src/scanner/scanning.rs)Every block message must now be the next expected height and carry a valid block hash, meaning 32 bytes and not all zero. The scan stops with an error, instead of skipping, in each of these cases:
endThe error names the height to rescan from, for example:
last_scanned_block_heightis never moved past the last fully processed block. The old guard that "advanced progress and moved on" for a malformed hash is removed. There is no retry inside the stream, so the scan stops at the bad height and the caller resumes there:watch_chain)scan_block_range failed, will retry on next pollat error level, sleeps 10 s, and rescans fromlast_scanned + 1. No spin and no crash loop; the task no longer dies from a panic.examples/stream_oracle.rs?instead ofunwrap(), and an empty hash is rejected.To make this testable in-process, the scan loop now reads messages through a small
BlockScanDataStreamtrait, implemented fortonic::Streaming.scan_block_rangeopens the stream and callsscan_block_stream. The loop body is unchanged apart from the checks.Tests (
blindbit-lib/src/scanner/stream_safety_tests.rs)Each test drives a real
Scannerthroughscan_block_stream, using in-process messages and real silent-payment outputs. Only the P2P block fetch is replaced, by a#[cfg(test)]hook that serves the full block.empty_hash_block_mid_range_stops_scan_and_next_scan_finds_its_outputslast_scanned + 1then finds all 3.malformed_block_hash_stops_scanskipped_height_stops_scan/stream_starting_at_wrong_height_stops_scanerror_status_mid_stream_is_a_clean_errorNOT_FOUNDandUNAVAILABLEmid-stream: no panic, the error names the height and the code, and the height stays at the last full block.stream_ending_before_requested_end_is_an_errorunreachable_oracle_is_a_clean_errorRed without the fix: commit ad80ea4 adds the seam and the tests with the old behaviour. On that commit all 7 tests fail. The three panic cases hit
called Result::unwrap() on an Err value: Status { code: NotFound, message: "height 1501 is not indexed" }(and the equivalent forUnavailable). The rest fail witha block without a valid hash must stop the scan: (), meaning the old code returnedOk. With the fix all 7 pass.Wallet start below the oracle's first block; stall reporting (commit 22c50bf)
The hosted oracle has no data below 800 000 on mainnet and 100 000 on signet (both probed:
GetBlockHashByHeight(99 999)on signet answers an empty hash, 100 000 a real one). friglet's config accepts any start height ≥ 1, so with the stop-and-retry above, a wallet starting lower stopped at its first height on every poll, forever, with only a log line.watch_chain's scan stops at the wallet's start height because the oracle has not indexed it, the scanner finds the oracle's lowest indexed height withGetBlockHashByHeightlookups (find_oracle_floor,scanner/health.rs) and starts there. It logs a warning, persists the adjustment asoracle_floor_startin the state file (optional field, omitted when unset; older binaries ignore it), and publishes it in the scan health.sp_start_height, the configured start). Once a block was scanned, an unindexed height after it keeps the stop-and-retry. A start height with any indexed height below it (a gap in the index) is not moved either.GetStatuscarries it asStatusInfo::scan_health(stall,start_adjusted; serde defaults, so old trays and old daemons still talk to each other). The tray's status window shows a "Scan stalled" line and a note for an adjusted start.ScanStoppederror (message text unchanged), so the loop can tell "not indexed" from other failures. The health is kept in the wallet's Electrum index, which status readers lock without waiting on the scan task (the scan task holds the scanner lock).Tests (
blindbit-lib/src/scanner/oracle_floor_tests.rs, plusfrigletstatus_reports_a_stalled_scan_and_an_adjusted_start):start_below_a_contiguous_index_finds_its_first_blockoracle_missing_its_own_start_height_starts_one_abovegap_above_the_floor_does_not_move_the_floor_upstart_inside_a_gap_above_the_floor_is_never_skipped/randomised_gappy_indexeswallet_start_below_the_floor_starts_at_the_floor_and_says_sounindexed_height_after_the_wallet_start_is_never_skipped/only_a_not_indexed_stop_at_the_start_moves_the_starta_stall_keeps_its_start_time_until_the_height_changes_and_clears_on_progressRed checks (each reverted): without the back-off verification, the gap test and the randomised test fail. Without the below-start probes, the gap-start tests fail. Without the wallet-start condition,
unindexed_height_after_the_wallet_start_is_never_skippedfails.Verification
cargo test --workspaceat 22c50bf: all suitesok. blindbit-lib 30 passed, friglet 26, friglet-ipc 8+1, friglet-tray 26+9.cargo clippy --no-deps -p friglet -p friglet-ipc -p friglet-tray --all-targets -- -D warnings: clean.cargo clippy -p blindbit-lib --all-targets --features serde: 10 warnings, the same count as before this PR, none on changed lines.cargo build -p blindbit-lib --examplesbuilds.cargo fmt -p friglet -p friglet-ipc -p friglet-tray --checkis clean; the new blindbit-lib files were formatted on their own (AGENTS.md: no workspace-wide fmt).integration+ test: BIP-352 vectors through the live short receive path (SNB-541) #17.requested_height=99000 oracle_floor=100000 skipped_blocks=1000, scanning went on from 100 000, andGetStatusshowedstart_adjusted.NOT_FOUND(324 470, above the wallet's start) was never skipped. Each poll stopped there,GetStatusshowedstall {height: 324470, since_unix: 1790863242}with the reason, and it cleared on the first poll after the fault was removed..colblocks ignoredhidden, which also left an empty "Last error" label on screen; 5492765 fixes it with.col[hidden] { display: none; }.End to end with blindbit-testkit (
testkit mutate, master at the #18 merge, release build of blindbit-cli from this branch):unindexed-ok-emptyis no longer a silentMISSING [big-k1 dust-k0]. blindbit-cli now exits 1 withscan stopped at height 102: the oracle sent no valid block hash ..., so the run is unjudgeable, not wrong.stale-viewandclose-mid-streamused to pin the panic (signal: aborted ... panicked at ... Status { code: NotFound|Unavailable). They now end in a cleanexit status 1naming the code.truncated-streamused to be caught only by the harness's tip check. It is now refused by the client itself.Merge note
Merges cleanly after #17. #19 is stacked on this PR (it contains #17 and #20), and #21 on #19, so the planned order is #17 → #20 → #19 → #21.
🤖 Generated with Claude Code