Repository navigation
feat(electrum): add ElectrumClient.populate_anchor_cache - #1151
Conversation
|
Hey @thunderbiscuit, I've made a sample app to test different sync configurations related to #1150 and #1151 ranging from 2x factor and up to 70x factor after the first sync on 10 rounds. Here are the results on a Pixel 5 (GrapheneOS) under LTE and Wi-Fi: |
|
This is super cool thanks for the detailed examples @ethicnology. I'll dig into this one tomorrow. As is it probably solves a lot of the issues that #1012 eventually solves, but from a completely different angle. I think work on 1012 is still valuable because we need flexibility in what we sync, but if this cuts your sync times by 70x... well I'm very happy with that kind of improvement let's say! |
There was a problem hiding this comment.
Looks good to me, I have just a few nits.
I think your comment "Callers should only pass direct anchors (ChainPosition::Confirmed { transitively: None, .. })" should be added to the API docs, just because adding those anchors doesn't do anything but that's not obvious if you don't know. One design decision we could do is to pass the whole wallet to the method instead, and the method could pull the anchors directly, but after looking into it I think keeping the two separate is easier, at the cost of the new type TxAnchor, which I'm fine with.
My last comment is that it'd be good to have at least an example somewhere of how to build these TxAnchors. It's not obvious at first glance how to do it. Maybe a Rust test or just a little codeblock in the API docs?
|
@ethicnology would you like me to make these changes for you to get this PR ready to merge? Let me know, otherwise I'd want a rebase plus fix of the small comments. Thanks! |
|
I will do it by the end of the day 👍 |
a0cbfe1 to
87849bc
Compare
|
amended |
|
I changed my mind (sorry!) about the tests. The Rust tests were adding too much noise and it wasn't clear that their purpose was to show how (and why) to build the TxAnchor objects. I added a smaller and simpler Kotlin test instead. Thanks for this! |
cbfe706 to
09c9274
Compare
thunderbiscuit
left a comment
There was a problem hiding this comment.
Thanks for the work! Just an FYI, we require commits be PGP-signed before merging, and I think your agent doesn't have that setup so those commits were not signed (doesn't matter as I squashed and fixed the test so now the commit is, but just for future reference).
Expose bdk_electrum's populate_anchor_cache so a caller can seed a new ElectrumClient with the confirmation anchors it already holds, as TxAnchor records built from Wallet.transactions. A sync or full scan then requests a Merkle proof only for a transaction whose confirmation is new, instead of one per known confirmed transaction. Together with populate_tx_cache this makes a fresh client per sync cheap, without keeping one alive across reorgs.
09c9274 to
b1ad3ea
Compare


Description
Exposes
BdkElectrumClient::populate_anchor_cacheonElectrumClient, taking a list ofTxAnchorrecords (a txid and one of itsConfirmationBlockTimeanchors). The anchors come fromWallet.transactions(): each confirmedCanonicalTxwhose chain position is not transitive carries its own anchor.This complements #1150 (
populate_tx_cache). With the transaction cache alone, a scan on a fresh client still requests one Merkle proof per known confirmed transaction (onebatch_transaction_get_merkleper keychain on a full scan, one per sync); with the anchor cache, it only requests a proof for a transaction whose confirmation is new. Both together let a wallet create a fresh client per sync, prefilled from its own data, instead of keeping one client alive across reorgs.Request counts on a loopback Electrum server for the same wallet, second scan after one new block (each
get_merkleentry is one proof in the batch):transaction.gettransaction.get_merkleIn a Flutter wallet using both methods through bdk-dart, with a new client per sync prefilled with the wallet's transactions, the parents of their inputs and their anchors, a sync of a 275-transaction wallet went from 59–82 s to about 2 s over Wi-Fi and LTE, and from about 225 s to 5–6 s over Tor (Android).
Notes to the reviewers
populate_anchor_cachein bdk_electrum takes(Txid, impl IntoIterator<Item = ConfirmationBlockTime>)pairs; uniffi cannot expose tuples or generics, so theTxAnchorrecord flattens one pair to one anchor, following the naming and wrapping ADRs. A caller with several anchors for one txid passes several records.The cache is keyed by
(txid, block hash)and only consulted for the block hash the server reports at that height, so a stale anchor after a reorg is never used: the proof is fetched and validated as usual (batch_fetch_anchors). Callers should only pass direct anchors (ChainPosition::Confirmed { transitively: None, .. }): a transitive anchor is a descendant's block, not this transaction's.The change applies with or without #1150 (different hunks of the same file). No binding-level test, as for
fetch_txand the other thinElectrumClientwrappers, since the client connects in its constructor; I can add one against a loopback server if wanted.Documentation
BdkElectrumClient::populate_anchor_cache,ChainPosition::ConfirmedChangelog
I don't have permission to add labels; this should be
changelog: added.Checklists
All Submissions:
cargo fmtandcargo clippybefore committingchangelog:*labelNew Features: