Repository navigation
feat(backend): add frigate backend for ephemeral key scanning #166
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
fde25c2
89c8048
dcbbcb6
c2d3435
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| /target |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| [package] | ||
| name = "frigate" | ||
| version = "0.1.0" | ||
| edition.workspace = true | ||
| repository.workspace = true | ||
| authors.workspace = true | ||
| license.workspace = true | ||
|
|
||
| [dependencies] | ||
| electrum_streaming_client.workspace = true | ||
| spdk-core.workspace = true | ||
| silentpayments.workspace = true | ||
|
|
||
| bitcoin.workspace = true | ||
| futures.workspace = true | ||
| tokio = { version = "1", features = ["macros", "rt-multi-thread", "net", "time"] } | ||
| log.workspace = true | ||
| anyhow.workspace = true | ||
|
|
||
| [dev-dependencies] | ||
| spdk-wallet.workspace = true | ||
| env_logger = "0.11" | ||
|
|
||
| [lints] | ||
| workspace = true |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| use std::collections::{HashMap, HashSet}; | ||
|
|
||
| use bitcoin::absolute::Height; | ||
| use bitcoin::secp256k1::SecretKey; | ||
| use bitcoin::{BlockHash, Network, OutPoint}; | ||
| use frigate::Scanner; | ||
| use spdk_core::scanner::{DiscoveredOutput, Scanner as _}; | ||
| use spdk_wallet::client::{SpClient, SpendKey}; | ||
|
|
||
| const NETWORK: Network = Network::Signet; | ||
| const FRIGATE_BACKEND_URL: &str = "100.81.213.115:50102"; | ||
| // scan range settings | ||
| const SCAN_START_HEIGHT: u32 = 326_000; | ||
| const SCAN_END_HEIGHT: u32 = 326_595; | ||
|
|
||
| // scan & spend key bytes, these should be randomly generated, | ||
| // but for this example we use simple byte arrays | ||
| const SCAN_SK_BYTES: [u8; 32] = [0x01; 32]; | ||
| const SPEND_SK_BYTES: [u8; 32] = [0x02; 32]; | ||
|
|
||
| #[derive(Debug)] | ||
| pub struct UpdateResult { | ||
| pub blkheight: Height, | ||
| pub blkhash: BlockHash, | ||
| pub discovered_inputs: HashSet<OutPoint>, | ||
| pub discovered_outputs: HashMap<OutPoint, DiscoveredOutput>, | ||
| } | ||
|
|
||
| #[tokio::main] | ||
| async fn main() -> anyhow::Result<()> { | ||
| env_logger::init(); | ||
| let scan_sk = SecretKey::from_slice(&SCAN_SK_BYTES)?; | ||
| let spend_sk = SecretKey::from_slice(&SPEND_SK_BYTES)?; | ||
|
|
||
| let sp_client = SpClient::new(scan_sk, SpendKey::Secret(spend_sk), NETWORK)?; | ||
| let sp_receiver = sp_client.receiver(); | ||
|
|
||
| let frigate_scanner = Scanner::new( | ||
| FRIGATE_BACKEND_URL.to_owned(), | ||
| scan_sk, | ||
| sp_receiver, | ||
| None, | ||
| HashSet::new(), | ||
| ) | ||
| .await?; | ||
|
|
||
| println!("Receiving code for this key pair + network:"); | ||
| println!("{}", sp_client.receiving_code()); | ||
|
|
||
| let start = Height::from_consensus(SCAN_START_HEIGHT)?; | ||
| let end = Height::from_consensus(SCAN_END_HEIGHT)?; | ||
|
|
||
| // It's important to note that `scan_blocks` will not monitor scripts outputs. | ||
| // Call `scan_blocks_with_spent_outputs` instead if you want to monitor spent outputs as well. | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Currently the Electrum (and by extension, Frigate) takes a 'monitoring' approach, where we subscribe to a set of scripts, and continuously monitor the blockchain for changes. Just to make the difference more explicit, this is how Dana does it:
Most importantly, it is not the Scanner that monitors the blockchain for new blocks. The wallet (Dana) monitors the blockchain, and when it detects a new block, it kick-starts the scanning process. We could change this, so that the Scanner becomes responsible for monitoring when new blocks arrive. I suspect this change would be much more compatible with the Electrum/Frigate approach, as the Scanner will now be the one to continuously monitor the blockchain. This also means the scanner can optionally notify the wallet of unconfirmed incoming payments. So with this new approach, how Dana would work:
From the perspective of Dana as a background process, the 'one-and-done' approach is easier to reason about. This is because Dana is expected to wake up every hour or so, do the scanning, and then terminate the scanner and go back to sleep. However I'm sure it's possible, and allowing for continuous monitoring has other benefits as well, so I'm not opposed to making this change.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the detailed explanation. What you said here clearly summarizes the difference in approaches. It now makes much more sense on to why the scanner was taken by value.
I think here that on waking up there will need to be subscribe requests to the scripts known, at least for the Frigate/Electrum approach else you won't be able to know whether something is spent or not.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure that foreground service would allow us to keep the subscription to frigate open and allow for the continuous monitoring approach. We had to use that polling approach for blindbit because of the way the v1 works, not because of android foreground services. |
||
| let mut rx = frigate_scanner.scan_blocks(start..=end); | ||
|
|
||
| while let Ok(update) = rx.recv().await { | ||
| // print all received updates | ||
| println!("{update:#?}"); | ||
| } | ||
|
|
||
| Ok(()) | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,190 @@ | ||
| use std::time::Duration; | ||
|
|
||
| use bitcoin::Txid; | ||
| use electrum_streaming_client::response::{FullTx, HeaderResp}; | ||
| use electrum_streaming_client::{ | ||
| AsyncClient, ElectrumScriptHash, ElectrumScriptStatus, Event, request, | ||
| }; | ||
| use futures::channel::mpsc::UnboundedReceiver; | ||
| use log::{debug, info}; | ||
| use tokio::net::TcpStream; | ||
| use tokio::task::JoinHandle; | ||
| use tokio::time::timeout; | ||
|
|
||
| #[expect(unused, reason = "worker is used to keep the connection alive")] | ||
| pub struct FrigateClient { | ||
| pub client: AsyncClient, | ||
| pub events: UnboundedReceiver<Event>, | ||
| pub worker: JoinHandle<Result<(), std::io::Error>>, | ||
| pub request_timeout: Duration, | ||
| protocol_version: Option<String>, | ||
| } | ||
|
|
||
| #[derive(Debug)] | ||
| pub enum FrigateError { | ||
| Connection(String), | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was thinking of switching to the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes as we proposed as we discussed I'll work on that but after this PR. I'll have a follow up where I get rid of anyhow in the whole project |
||
| Timeout(String), | ||
| Request(String), | ||
| } | ||
|
|
||
| type Result<T, E = FrigateError> = std::result::Result<T, E>; | ||
|
|
||
| impl std::fmt::Display for FrigateError { | ||
| fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | ||
| match self { | ||
| Self::Connection(msg) => write!(f, "connection error: {msg}"), | ||
| Self::Timeout(msg) => write!(f, "request timeout: {msg}"), | ||
| Self::Request(msg) => write!(f, "request error: {msg}"), | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl std::error::Error for FrigateError {} | ||
|
|
||
| impl FrigateClient { | ||
| pub async fn connect(host_url: &str) -> Result<Self> { | ||
| let stream = TcpStream::connect(host_url).await.map_err(|err| { | ||
| FrigateError::Connection(format!("can't connect to socket '{host_url}': {err}")) | ||
| })?; | ||
|
|
||
| let (reader, writer) = stream.into_split(); | ||
| let (client, events, worker) = AsyncClient::new_tokio(reader, writer); | ||
|
|
||
| let worker = tokio::spawn(worker); | ||
|
|
||
| Ok(Self { | ||
| client, | ||
| events, | ||
| worker, | ||
| request_timeout: Duration::from_secs(10), | ||
| protocol_version: None, | ||
| }) | ||
| } | ||
|
|
||
| /// Sets a custom request timeout for this client. | ||
| #[expect(unused, reason = "tbd")] | ||
| pub const fn with_timeout(mut self, timeout: Duration) -> Self { | ||
| self.request_timeout = timeout; | ||
| self | ||
| } | ||
|
|
||
| pub async fn get_block_header(&mut self, height: u32) -> Result<HeaderResp> { | ||
| self.version().await?; | ||
|
|
||
| let res = timeout( | ||
| self.request_timeout, | ||
| self.client.send_request(request::Header { height }), | ||
| ) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("Header request timed out".to_owned()))? | ||
| .map_err(|e| FrigateError::Request(format!("Header request failed: {e}")))?; | ||
| Ok(res) | ||
| } | ||
|
|
||
| pub async fn get_transaction(&mut self, txid: Txid) -> Result<FullTx> { | ||
| self.version().await?; | ||
|
|
||
| let res = timeout( | ||
| self.request_timeout, | ||
| self.client.send_request(request::GetTx { txid }), | ||
| ) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("GetTx request timed out".to_owned()))? | ||
| .map_err(|e| FrigateError::Request(format!("GetTx request failed: {e}")))?; | ||
| Ok(res) | ||
| } | ||
|
|
||
| /// Negotiate the Frigate protocol version once and cache the result. | ||
| /// | ||
| /// The negotiation is sent as the first request on the connection. Later calls return the | ||
| /// cached version without sending another request. | ||
| pub async fn version(&mut self) -> Result<String> { | ||
| if let Some(protocol_version) = &self.protocol_version { | ||
| return Ok(protocol_version.clone()); | ||
| } | ||
|
|
||
| debug!("Sending version request to Frigate server..."); | ||
| let res = timeout( | ||
| self.request_timeout, | ||
| self.client.send_request(request::ServerVersion { | ||
| client_name: "spdk".into(), | ||
| protocol_version: request::SupportedVersion::Range([ | ||
| "1.3.2".into(), | ||
| "1.4.1".into(), | ||
|
Owner
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why have 1.4.1 be the last supported version? The 'main' public frigate server (frigate.2140.dev) runs version 1.6.0, I think it makes sense to target that
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we allow version more than 1.4.1, the negotiation will happen but we will send invalid requests. We're using electrum_streaming_client to make requests to the frigate server, which for now is capped at v1.4.1 |
||
| ]), | ||
| }), | ||
| ) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("Version request timed out".to_owned()))? | ||
| .map_err(|e| FrigateError::Request(format!("Version request failed: {e}")))?; | ||
| self.protocol_version = Some(res.protocol_version.clone()); | ||
| Ok(res.protocol_version) | ||
| } | ||
|
|
||
| /// Make a request to the Frigate electrum server to subscribe to the outputs beloging to the | ||
| /// given silent payment address Once the server receives the request notification will be | ||
| /// sent to the client everytime an ouput is found. | ||
| /// | ||
| /// See: <https://github.com/sparrowwallet/frigate#blockchainsilentpaymentssubscribe> | ||
| pub async fn subscribe(&mut self, subscribe_req: request::SpSubscribe) -> Result<String> { | ||
| self.version().await?; | ||
| debug!("Sending subscribe event request..."); | ||
|
|
||
| let res = timeout( | ||
| self.request_timeout, | ||
| self.client.send_request(subscribe_req), | ||
| ) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("Subscribe request timed out".to_owned()))? | ||
| .map_err(|e| FrigateError::Request(format!("Subscribe request failed: {e}")))?; | ||
|
|
||
| info!("Subscribed to silent payment address: {res}"); | ||
| Ok(res) | ||
| } | ||
|
|
||
| #[expect(unused, reason = "tbd")] | ||
| pub async fn unsubscribe(&mut self, unsub_req: request::SpUnsubscribe) -> Result<()> { | ||
| self.version().await?; | ||
|
|
||
| let res = timeout(self.request_timeout, self.client.send_request(unsub_req)) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("Unsubscribe request timed out".to_owned()))? | ||
| .map_err(|e| FrigateError::Request(format!("Unsubscribe request failed: {e}")))?; | ||
|
|
||
| info!("Unsubscribed to silent payment address: {res:?}"); | ||
| Ok(()) | ||
| } | ||
|
|
||
| pub async fn scripthash_subscribe( | ||
| &mut self, | ||
| sub_req: request::ScriptHashSubscribe, | ||
| ) -> Result<Option<ElectrumScriptStatus>> { | ||
| self.version().await?; | ||
| let res = timeout(self.request_timeout, self.client.send_request(sub_req)) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("Script hash subscription timed out".to_owned()))? | ||
| .map_err(|e| { | ||
| FrigateError::Request(format!("Script hash subscription request failed: {e}")) | ||
| })?; | ||
|
|
||
| info!("Subscribed to script: {res:?}"); | ||
| Ok(res) | ||
| } | ||
|
|
||
| pub async fn get_history( | ||
| &mut self, | ||
| script_hash: ElectrumScriptHash, | ||
| ) -> Result<Vec<electrum_streaming_client::response::Tx>> { | ||
| self.version().await?; | ||
|
|
||
| let res = timeout( | ||
| self.request_timeout, | ||
| self.client | ||
| .send_request(request::GetHistory { script_hash }), | ||
| ) | ||
| .await | ||
| .map_err(|_| FrigateError::Timeout("Script history request timed out".to_owned()))? | ||
| .map_err(|e| FrigateError::Request(format!("Script history request failed: {e}")))?; | ||
| Ok(res) | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| mod client; | ||
| mod scanner; | ||
| pub use scanner::FrigateScanner as Scanner; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could replace these with the 2140 frigate server (frigate.2140.dev), alternatively we could host our own signet frigate server. I think for an example file, using the 2140.dev server is probably the most useful.