Skip to content

Add custom sync methods - #1012

Merged
thunderbiscuit merged 2 commits into
bitcoindevkit:masterfrom
thunderbiscuit:feat/custom-sync
Nov 11, 2026
Merged

thunderbiscuit merged 2 commits into
bitcoindevkit:masterfrom
thunderbiscuit:feat/custom-sync

Conversation

@thunderbiscuit

@thunderbiscuit thunderbiscuit commented May 26, 2026 •

Copy link
Copy Markdown
Collaborator

This PR introduces two new methods:

  • Wallet::start_sync_with_spk_indices(external_indices, internal_indices)
  • start_sync_with_last_revealed_spks(external_count, internal_count)

Note that when using these methods, invalid inputs can be provided, and so the methods actually return a Result (for example if you give an index you haven't revealed yet or if you ask for indices on a keychain the wallet doesn't have). This required a new error that doesn't exist upstream: SpkIndexError.

Why adding methods to the wallet instead of exposing the whole builder?

I looked at exposing SyncRequest::builder() to the bindings instead because that could potentially be the most flexible option, but it turned out to be super clumsy:

  • chain_tip takes a CheckPoint, which the bindings do not expose. A request built without a tip produces no chain update, so confirmed transactions cannot be connected to the local chain and show up as unconfirmed. Exposing the CheckPoint is not impossible, but it's a linked list of unknown size so it ends up being a lot more than we probably want, especially that you don't actually need it, you just need to pass it to your builder!
  • expected_spk_txids is weird to compute. It's a reimplementation of internal bookkeeping that every caller would write, and keep in step with upstream if the definition of "relevant spk" changes. Getting it wrong, or skipping it, fails silently, and a dropped or replaced transaction just stays pending forever. No error, no missing-argument complaint. Also, the wallet already has the answer, so having each app recompute it is wasted effort.

Keeping the entry point on Wallet means both are always filled in. The cost is that we own two methods upstream doesn't have.

Notes on API shape

There is a bit of bikeshedding to be done if you want to around certain choices I made:

  1. Right now the arguments are all optional and default to None, so you can call a sync for the last 3 addresses on your external keychain like so:
wallet.startSyncWithLastRevealedSpks(externalCount = 3)
  1. Both being optional means you might also mistakenly call
wallet.startSyncWithLastRevealedSpks()

But for that you'll get an error (can't construct an empty SyncRequest)
3. None is a bit different than an empty list, and if you try to sync an empty list on an internal keychain but you have a single-keychain wallet you get an error.



Documentation

There are no direct upstream equivalent to these methods.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing
  • I've added exactly one changelog:* label

New Features:

  • I've added tests for the new feature
  • I've added docs for the new feature

Comment thread bdk-ffi/src/wallet.rs Resolved
Comment thread bdk-ffi/src/wallet.rs Resolved
@thunderbiscuit thunderbiscuit added this to the 3.2.0 milestone Sep 11, 2026
@thunderbiscuit thunderbiscuit changed the title Exploration: add custom sync methods Exploration: custom sync methods Sep 15, 2026
@thunderbiscuit
thunderbiscuit force-pushed the feat/custom-sync branch 2 times, most recently from ceefaac to 2e9dc1a Compare September 15, 2026 19:42
@thunderbiscuit thunderbiscuit modified the milestones: 3.1.1, 3.2.0 Sep 17, 2026
@thunderbiscuit
thunderbiscuit requested a review from a team as a code owner October 13, 2026 19:49
@thunderbiscuit
thunderbiscuit force-pushed the feat/custom-sync branch 2 times, most recently from ebff0bf to 5d88586 Compare October 14, 2026 19:12
Comment thread bdk-ffi/src/wallet.rs Outdated
if spk_index.get_descriptor(keychain).is_none() {
return Err(SpkIndexError::KeychainNotFound { keychain });
}
if spk_index.last_revealed_index(keychain).is_none() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we only error when the combined selection is empty? Example: externalCount = 5, internalCount = 5 currently fails when external addresses are revealed but no change addresses are.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And maybe we could cover this with tests where only external addresses are revealed, and vice versa?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per comment above I'm removing this method entirely.

fetchPrevTxouts = true
)

assertEquals(7, counter.inspected, "Only the 5 external and 2 internal scripts should be synced")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also assert which scripts were synced because checking only the count would also pass if we accidentally selected the first scripts instead of the last I think

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good idea. I added a custom SyncScriptInspector that records the scripts as they are being synced by the client, so we can then check.

Comment thread bdk-ffi/src/wallet.rs
BdkSyncRequest::builder()
.chain_tip(chain_tip)
.spks_with_indexes(spks)
.expected_spk_txids(expected_spk_txids)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to add a rust test checking that the custom request preserves the wallets chain tip and expected transaction IDs for the selected scripts to possibly would help catch regressions in the metadata needed for eviction tracking? Non-blocking

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one at the Rust layer (which doesn't use/see regtest) requires building custom transactions and whatnot, and I propose we leave it out. Let me know if you'd rather I add it.

Comment thread bdk-ffi/src/wallet.rs Outdated
Comment on lines +946 to +947
/// an unrevealed index would be misleading: the wallet's indexer does not track it, so the
/// outputs of any transaction found there would not be recognized as the wallet's own.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the “every index must have been revealed” requirement and drop the explanation about outputs not being recognized?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks

Comment thread bdk-ffi/src/wallet.rs Outdated
Comment on lines +956 to +957
external_indices: Option<Vec<u32>>,
internal_indices: Option<Vec<u32>>,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non blocking: what do you think about these using plain lists defaulting to empty and the count arguments default to zero, just since None has the same meaning, or is there a better reason we need the optional types?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nah I agree with you, let's just use plain lists, and they can be made default empty lists, so that's even simpler to use!

Comment thread bdk-ffi/src/wallet.rs Outdated
continue;
}
if spk_index.get_descriptor(keychain).is_none() {
return Err(SpkIndexError::KeychainNotFound { keychain });

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is rejecting the internal keychain intentional for single keychain wallets? Upstream methods like reveal_next_address map Internal to External in that case so this behaves differently is all.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this helper method from the PR. We can easily add it later, and the first one (start_sync_with_spk_indices) is the one that really counts and provides strong value for users. This is more like syntactic sugar, and I think we need to cut the code surface here.

@reez

reez commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator

concept looking good, added some comments/questions.

in original description we can update 3. None is a bit different than an empty list, and if you try to sync an empty list on an internal keychain but you have a single-keychain wallet you get an error. because None and an empty list now behave the same (including when the wallet has no internal keychain)? Not that update needed before merge but partially just to make sure I'm understanding everything properly.

@thunderbiscuit thunderbiscuit changed the title Exploration: custom sync methods Add custom sync methods Oct 17, 2026
@thunderbiscuit
thunderbiscuit force-pushed the feat/custom-sync branch 2 times, most recently from b16fb8e to 52311fe Compare November 11, 2026 18:16

@reez reez left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 52311fe

@thunderbiscuit
thunderbiscuit merged commit f0d4935 into bitcoindevkit:master Nov 11, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants