Skip to content

feat(backend): add frigate backend for ephemeral key scanning - #166

Open
sdmg15 wants to merge 4 commits into
cygnet3:masterfrom
sdmg15:feat/frigate-backend
Open

sdmg15 wants to merge 4 commits into
cygnet3:masterfrom
sdmg15:feat/frigate-backend

Conversation

@sdmg15

@sdmg15 sdmg15 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

This PR adds frigate as an alternative backend support.

Changes

TBD

Notes to reviewer

TBD

Closes #147

@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from de09f27 to cc5fecb Compare September 11, 2026 13:38
@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from cc5fecb to 4cd8b66 Compare September 30, 2026 18:06
@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from 4cd8b66 to fde25c2 Compare October 6, 2026 19:16
@cygnet3

cygnet3 commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Regarding the scanner being consumed/moved when calling scan_blocks:

I think either we need to consume the scanner, or we need to take a mutable reference. This is because the scanner keeps track of the outpoints to scan for, and therefore needs to mutate internal data. I think this is true for any scanner, regardless of the implementation specifics.

If using a mutable reference, I believe we basically need to make the scanner have a 'static lifetime in order to use tokio::task::spawn, or we need to wrap the owned_outputs in an Arc<Mutex<_>>. I prefer to just consume the scanner instead of those two options, but if you have a different take then let me know.

@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from df56ebe to 57b8502 Compare October 14, 2026 09:12
@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from 57b8502 to 89c8048 Compare October 16, 2026 11:31
@sdmg15
sdmg15 marked this pull request as ready for review October 19, 2026 06:59

#[derive(Debug)]
pub enum FrigateError {
Connection(String),

@cygnet3 cygnet3 Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of switching to the thiserror crate, this might be a good time to introduce that to get rid of some of the boilerplate code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes as we proposed as we discussed I'll work on that but after this PR. I'll have a follow up where I get rid of anyhow in the whole project

Comment thread scanners/remote/frigate/src/client.rs Resolved
Comment thread scanners/remote/frigate/src/lib.rs Resolved
@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from f8a1aec to 9a62033 Compare October 27, 2026 11:58
… factored out the outputs processing functions
@sdmg15
sdmg15 force-pushed the feat/frigate-backend branch from 9a62033 to c2d3435 Compare October 27, 2026 12:13
use spdk_wallet::client::{SpClient, SpendKey};

const NETWORK: Network = Network::Signet;
const FRIGATE_BACKEND_URL: &str = "100.81.213.115:50102";

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could replace these with the 2140 frigate server (frigate.2140.dev), alternatively we could host our own signet frigate server. I think for an example file, using the 2140.dev server is probably the most useful.

client_name: "spdk".into(),
protocol_version: request::SupportedVersion::Range([
"1.3.2".into(),
"1.4.1".into(),

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have 1.4.1 be the last supported version? The 'main' public frigate server (frigate.2140.dev) runs version 1.6.0, I think it makes sense to target that

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we allow version more than 1.4.1, the negotiation will happen but we will send invalid requests.
There are few differences that make v1.4.1 and latest in the payload returned payloads incompatible.

We're using electrum_streaming_client to make requests to the frigate server, which for now is capped at v1.4.1
we could easily extend that but I suggest to keep the PR scope small let's allow this here and in subsequent one we will support more recent version.

let subscribe_params = request::SpSubscribe {
scan_priv_key: self.b_scan,
spend_pub_key: self.sp_receiver.receiving_code().m_pubkey(),
start_height: Some(range.start().to_consensus_u32()),

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned in the other comment, Dana is currently responsible for checking the latest block_height and gives this information to the scanner in the form of a block_range. However, we could change it so that we instead only give a start_height, much like what the SpSubscribe api expects.

The only negative about this is that we can no longer request to scan for specific block ranges. We had some ideas in the past to do scanning in a different order (for example, first scan block range [1000-2000], and then scan block range [0-1000]. But we decided the extra complexity wasn't really worth it, so I think we can make this change.

@sdmg15 sdmg15 Nov 5, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is also possible with latest version of Frigate, please see https://github.com/sparrowwallet/frigate#blockchainsilentpaymentssubscribe. You can pass "FROM-TO". So we can keep the initial idea you had.
The reason we only have startHeight is because prior version didn't support scanning in range, but once we upgraded this is possible.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had some ideas in the past to do scanning in a different order (for example, first scan block range [1000-2000], and then scan block range [0-1000].

Fwiw I think our idea was more relevant to local scanning, the whole point of remote scanning like frigate is that you shouldn't need that kind of optimisation to make it tolerable. What I mean is that I'm ok with remote scanner not allowing closed scan range, we always scan to the tip.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way we can support both, the endHeight would be optional if not passed we just scan till the chain tip.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll see for version 1.6 then

let end = Height::from_consensus(SCAN_END_HEIGHT)?;

// It's important to note that `scan_blocks` will not monitor scripts outputs.
// Call `scan_blocks_with_spent_outputs` instead if you want to monitor spent outputs as well.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scan_blocks api currently expects that we're also scanning the inputs for spent scripts within this block range. If this is hard to do for the Frigate scanner, we could reconsider the Scanner api.

Currently the scan_blocks function is intended to be something a wallet calls in order to synchronize the changes that occur within a given block range. After all blocks are processed, the wallet is considered 'up to date' until this latest block height. Afterwards, the job of the scanner is completed. It does not monitor the blockchain any longer. We could call this a 'one-and-done' approach.

Electrum (and by extension, Frigate) takes a 'monitoring' approach, where we subscribe to a set of scripts, and continuously monitor the blockchain for changes.

Just to make the difference more explicit, this is how Dana does it:

  1. Dana comes online, for example after the user opens the app, or a background process wakes up the app
  2. Dana checks the current block_height from the chain backend api, and checks the stored last_height (the last known block height)
  3. if last_height < block_height, Dana creates a scanner object and initiates a scan for range last_height+1..=block_height.
  4. After the scan is complete, the scanner's job is finished and is dropped.
  5. If Dana is in the background, it goes back to sleep. If the app is currently open, Dana will fetch the current block_height every 10 seconds. Once Dana reads that a new block has been confirmed (block_height is increased), go back to step (3)

Most importantly, it is not the Scanner that monitors the blockchain for new blocks. The wallet (Dana) monitors the blockchain, and when it detects a new block, it kick-starts the scanning process.

We could change this, so that the Scanner becomes responsible for monitoring when new blocks arrive. I suspect this change would be much more compatible with the Electrum/Frigate approach, as the Scanner will now be the one to continuously monitor the blockchain. This also means the scanner can optionally notify the wallet of unconfirmed incoming payments.

So with this new approach, how Dana would work:

  1. Dana comes online, either by the user opening the app or the background process wakes up
  2. Dana reads last_height from storage, and makes a new Scanner with begin_height = last_scan+1
  3. The scanner now checks the block_height. If we are behind on block_height, Scanner initiates a scan of these blocks, and feeds Dana the updates (through the Receiver channels).
  4. If the scanner has reached the chain tip, one of things can occur:
    • (a) If the wallet is currently open, keep monitoring the scripts for changes (very similar to the native Electrum approach)
    • (b) If the wallet is currently operating as a background process, we have to terminate the scanner and go back to sleep. We tell the Scanner to unsubscribe from the scripts. The next time the app wakes up, start again from (1)

From the perspective of Dana as a background process, the 'one-and-done' approach is easier to reason about. This is because Dana is expected to wake up every hour or so, do the scanning, and then terminate the scanner and go back to sleep. However I'm sure it's possible, and allowing for continuous monitoring has other benefits as well, so I'm not opposed to making this change.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed explanation. What you said here clearly summarizes the difference in approaches. It now makes much more sense on to why the scanner was taken by value.

(b) If the wallet is currently operating as a background process, we have to terminate the scanner and go back to sleep. We tell the Scanner to unsubscribe from the scripts. The next time the app wakes up, start again from (1)

I think here that on waking up there will need to be subscribe requests to the scripts known, at least for the Frigate/Electrum approach else you won't be able to know whether something is spent or not.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because Dana is expected to wake up every hour or so, do the scanning, and then terminate the scanner and go back to sleep.

I'm pretty sure that foreground service would allow us to keep the subscription to frigate open and allow for the continuous monitoring approach. We had to use that polling approach for blindbit because of the way the v1 works, not because of android foreground services.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add alternative backend support

3 participants