Repository navigation
blindbit-lib, friglet: share the Electrum-index and oracle-hash code (SNB-671) - #76
Merged
Merged
Conversation
…e build (SNB-671) Nothing compared the Electrum index a restarted daemon rebuilds from its state file with the one the live scan writes, and the existing tests only check that an entry exists. The new test scans receives at 101-103 (the 103 output spent in its own block) and a later spend at 104, then asserts the exact (tx, height) list of every scripthash, the SP history and the served txids, live and after save, restore and rebuild. Same-height entries are compared by txid after a restore, which walks the wallet graph in hash order. It passes on the code as it is, ahead of the refactor that shares this code between the two paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(SNB-671) The per-transaction Electrum index update was written twice in blindbit-lib, once in the live scan and once in the restore from the wallet graph, and the history insert a third time in friglet's broadcast handler. It now lives in WalletElectrumIndex: - `add_history` (was `upsert_history_entry`): lists a tx in a scripthash's history, sorted by height; a listed tx keeps its entry unless an unconfirmed one confirms. friglet's broadcast handler calls it at height 0, where it is exactly the old push-if-absent and sort. - `index_confirmed_tx`: receive and spend entries plus the raw tx, for the live scan's pass over a matched block and for the restore. The txid is formatted only for wallet transactions, as before. - `add_sp_history`: the SP history entry, for both. - `OwnedScripts::of`: the owned-outpoint set and outpoint-to-script map both paths built identically, still built outside the index lock and after the block was applied. The restore used push-if-absent with one sort at the end; it now uses the same insert-and-sort as the live scan. On the fresh index a restore runs on, the result is identical: a stable sort after every insert leaves the same list as one stable sort at the end, and an unconfirmed entry to promote cannot exist yet. The live scan keeps its three passes over the block in the same order. friglet uses blindbit-lib's `convert_network` (now public) instead of its identical `wallet_network`, and its oracle block-hash decoding calls the new `blindbit_lib::scanner::oracle_block_hash` (length check and byte order), keeping its own wrong-length message. blindbit-lib's `indexed_block_hash` is that function plus its "all zeros means not indexed" rule, unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndex Brings in the SNB-662 and SNB-668 review follow-ups (a live vout test, the opt-level comment, the README note on http_addr). No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sition (SNB-671) Review follow-ups. - index_confirmed_tx also adds the SP history entry, so the live scan and the restore no longer each look up the tweak; WalletView (was OwnedScripts) carries the indexer's tweak map. The live scan loses its separate SP pass and one compute_txid per transaction of a matched block. The two lists are independent, so their contents and order are unchanged. - add_history and the SP history insert a new entry where pushing it and stably sorting would put it (after every entry at the same or a lower height) instead of pushing and re-sorting the whole list. - The restore feeds transactions in height order (stable, so same-height transactions keep the graph's order, as before), so each entry is appended. Re-sorting the wallet-wide SP history after each insert, in the graph's hash order, made a 3,000-payment restore 128 ms instead of 78 ms; it now takes 73 ms. - The characterisation test also checks the SP tweaks and the served raw bytes (it passes on the code before the refactor too). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review nit: the outpoints and tweaks come from the SP index, only the scripts from the graph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from
snb-662-remove-full-receive-path
to
integration
October 30, 2026 06:33
setavenger
added a commit
that referenced
this pull request
Oct 30, 2026
Brings #18 up to date with master after the Friglet Deslop cleanup (#67-#76). Conflicts resolved as described in #75: - blindbit-lib/Cargo.toml, friglet/Cargo.toml: keep #18's bdk-sp rev pin and comments on top of the reduced manifest lines from SNB-670. - scanning.rs: BIP352_K_MAX loses its #[expect(dead_code)] (the compile-time assert against bdk_sp::K_MAX now uses it); its doc no longer mentions the removed scan_transaction_full, which stays deleted. - bip352_vectors.rs: the full-path K_max boundary test stays deleted; the live K_max test takes the label set and imports BIP352_K_MAX; its doc no longer links the deleted test. - Cargo.toml: the secp256k1-sys comment no longer says the live K_max test is ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setavenger
added a commit
that referenced
this pull request
Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #75 (SNB-662), which sits on #74 (SNB-668) and #72 (
snb-661-stack-06-docs, top of the Friglet cleanup stack #67–#72); review only the commits after #75 (the merge commit only brings #75/#74 review follow-ups in). Retargets tointegrationonce #67–#72 land.Implements SNB-671. Non-test code: net −80 lines (180 added, 260 removed); tests: +1 characterisation test of the Electrum index and +1 unit test of the shared insert.
What was duplicated
The per-transaction update of the wallet's Electrum index (what Sparrow reads through
blockchain.scripthash.get_history,blockchain.transaction.getandsilentpayments.subscribe) was written twice in blindbit-lib and a third time, in part, in friglet:scanning.rs, matched block)rebuild_electrum_index_from_graph)friglet also had its own copy of
convert_network(wallet_network) and of the oracle's block-hash decoding.What it is now
WalletElectrumIndexowns the logic:add_history(scripthash, tx_hash, height) -> bool(was the free fnupsert_history_entry): lists a tx in a scripthash's history, sorted by height with unconfirmed entries first. A listed tx keeps its entry, except that an unconfirmed one is promoted when it confirms. Returns whether it added an entry. friglet's broadcast handler calls it at height 0, where it is exactly the old push-if-absent + sort, and counts thetrues as before.index_confirmed_tx(tx, txid, height, &WalletView): receive entries, spend entries, raw bytes if the tx touched the wallet, and the SP-history entry if it paid the wallet. The live scan calls it once per transaction of a matched block, the restore once per confirmed graph transaction. The txid is formatted only for wallet transactions, as before.WalletView::of(&indexer): what an update needs to know about the wallet (owned outpoints, their scripts, the SP tweak map), built outside the index lock and afterapply_matched_block/sync_owned_outputs, so an output received and spent in the same block counts.friglet calls blindbit-lib's
convert_network(now public) instead of its identicalwallet_network, and its oracle block-hash decoding calls the newblindbit_lib::scanner::oracle_block_hash(length check + byte order). The two decoders were not identical: blindbit-lib'sindexed_block_hashtreats an all-zero hash as "not indexed", friglet's returns it and has its own wrong-length error text that reaches Electrum JSON-RPC errors. Both keep exactly that:indexed_block_hashisoracle_block_hashplus the zero rule, and friglet'sblock_hash_from_oraclekeeps its signature and message.Why behaviour is unchanged
scanning.rs.add_history(.., 0)is the old loop body;convert_networkis the same match; the decoder keeps the same results and messages.Measurements
Electrum index before/after on identical state files (SNB-671's verify step). A throwaway harness (not committed) restored each state file on base 08bf11d and on this branch and dumped every scripthash history, its Electrum status, the SP history (with tweaks), the served txids with a hash of the raw bytes, and the tip:
state_before_owned_outputs.json, format 0, 2 receives at one height)Restore time (
rebuild_electrum_index_from_graph, best of 16, 3 interleaved rounds): 3,000-payment wallet base 79–83 ms, this PR 72–73 ms; 200-payment wallet 3.8 ms → 3.2 ms. (The first version of this PR re-sorted the wallet-wide SP history after every insert and took 128 ms; review caught it, and the height-ordered restore fixed it.)Scan cost, with PR #52's method: release blindbit-cli, signet blocks 274010..=276010, scan key 1 / spend key G, state deleted before each run, 6 runs per build in rotating order,
/usr/bin/time -v:Re-run at the end on the combined branch
friglet-deslop-live-test-2(ce0d417, all four PRs) against base, 8 runs each, interleaved: base 3.23 ± 0.07 s CPU / 3.45 s median wall, combined 3.24 ± 0.04 s CPU / 3.46 s median wall. All runs, 34 in total, end in the same state file (sha2568c7aec9071b0…, as in #52). The VM is shared (load average about 2 from other work during the runs), which is why the runs were interleaved. These keys never match a block, so this measures the common no-match path; the changed code runs only for matched blocks and at startup, which the restore timing and the testkit runs below cover.BlindBit testkit, baseline 08bf11d vs
friglet-deslop-live-test-2(all four PRs):rescan × cut-through-dustfails its judgement identically on both sides (a known scenario/client mismatch the testkit's own tests skip).testkit_t0, applied locally on both sides): 10/10 pass on both.testkit mutateagainst the candidate blindbit-cli: 28 of 28 mutants caught, all 28 weakened checks go red (same as baseline).input-typesis lib-only by design and refused at T2 (it passed at T0).Not changed on purpose
WalletElectrumIndex's fields staypub: friglet reads and its tests seed them about 100 times; filed as SNB-783.Reviews
review: ship with fixes. Applied: the restore cost (see above), the test now also checks the SP tweaks and the raw bytes. Kept: the test borrows the in-process oracle fromreorg_tests(widening seven test items topub(super)) rather than moving it totest_support.thermo-nuclear-review: ship, no findings; a focused re-review of the follow-up commit (insert position, height-ordered restore, SP entry folded intoindex_confirmed_tx): ship.codebase-design: ship. Applied its option (a): the SP entry moved intoindex_confirmed_tx(one call per transaction for both paths, onecompute_txidless per transaction of a matched block). Deferred: moving the pending-promotion pass behind a method and making the fields private (SNB-783).Tests
live_scan_and_restore_build_the_same_electrum_index(first commit; passes on the code before the refactor, including the stronger checks): receives at 101–103 (the 103 output spent in its own block) and a spend at 104; asserts every scripthash's exact(tx, height)list, the SP history with tweaks and the served raw transactions, after the live scan and after save → restore → rebuild. A mutant that drops the spend entries fails it.add_history_keeps_one_entry_per_tx_and_promotes_unconfirmed_ones: sort order, repeated tx, promotion, never back to unconfirmed, return value.Verification
cargo test --workspace: 254 passed, 1 ignored, 0 failed (blindbit-lib: remove the unused full-transaction receive path (SNB-662) #75: 252, +2), 12.5 s.-D warningson friglet/friglet-ipc/friglet-tray: clean. fmt check: clean.cargo check --workspace --all-targets --all-features: 0 warnings.