Repository navigation
Conversation
de09f27 to
cc5fecb
Compare
cc5fecb to
4cd8b66
Compare
4cd8b66 to
fde25c2
Compare
|
Regarding the scanner being consumed/moved when calling I think either we need to consume the scanner, or we need to take a mutable reference. This is because the scanner keeps track of the outpoints to scan for, and therefore needs to mutate internal data. I think this is true for any scanner, regardless of the implementation specifics. If using a mutable reference, I believe we basically need to make the scanner have a |
df56ebe to
57b8502
Compare
57b8502 to
89c8048
Compare
|
|
||
| #[derive(Debug)] | ||
| pub enum FrigateError { | ||
| Connection(String), |
There was a problem hiding this comment.
I was thinking of switching to the thiserror crate, this might be a good time to introduce that to get rid of some of the boilerplate code.
There was a problem hiding this comment.
Yes as we proposed as we discussed I'll work on that but after this PR. I'll have a follow up where I get rid of anyhow in the whole project
f8a1aec to
9a62033
Compare
… factored out the outputs processing functions
9a62033 to
c2d3435
Compare
| use spdk_wallet::client::{SpClient, SpendKey}; | ||
|
|
||
| const NETWORK: Network = Network::Signet; | ||
| const FRIGATE_BACKEND_URL: &str = "100.81.213.115:50102"; |
There was a problem hiding this comment.
We could replace these with the 2140 frigate server (frigate.2140.dev), alternatively we could host our own signet frigate server. I think for an example file, using the 2140.dev server is probably the most useful.
| client_name: "spdk".into(), | ||
| protocol_version: request::SupportedVersion::Range([ | ||
| "1.3.2".into(), | ||
| "1.4.1".into(), |
There was a problem hiding this comment.
Why have 1.4.1 be the last supported version? The 'main' public frigate server (frigate.2140.dev) runs version 1.6.0, I think it makes sense to target that
There was a problem hiding this comment.
If we allow version more than 1.4.1, the negotiation will happen but we will send invalid requests.
There are few differences that make v1.4.1 and latest in the payload returned payloads incompatible.
We're using electrum_streaming_client to make requests to the frigate server, which for now is capped at v1.4.1
we could easily extend that but I suggest to keep the PR scope small let's allow this here and in subsequent one we will support more recent version.
| let subscribe_params = request::SpSubscribe { | ||
| scan_priv_key: self.b_scan, | ||
| spend_pub_key: self.sp_receiver.receiving_code().m_pubkey(), | ||
| start_height: Some(range.start().to_consensus_u32()), |
There was a problem hiding this comment.
As mentioned in the other comment, Dana is currently responsible for checking the latest block_height and gives this information to the scanner in the form of a block_range. However, we could change it so that we instead only give a start_height, much like what the SpSubscribe api expects.
The only negative about this is that we can no longer request to scan for specific block ranges. We had some ideas in the past to do scanning in a different order (for example, first scan block range [1000-2000], and then scan block range [0-1000]. But we decided the extra complexity wasn't really worth it, so I think we can make this change.
There was a problem hiding this comment.
Yes this is also possible with latest version of Frigate, please see https://github.com/sparrowwallet/frigate#blockchainsilentpaymentssubscribe. You can pass "FROM-TO". So we can keep the initial idea you had.
The reason we only have startHeight is because prior version didn't support scanning in range, but once we upgraded this is possible.
There was a problem hiding this comment.
We had some ideas in the past to do scanning in a different order (for example, first scan block range [1000-2000], and then scan block range [0-1000].
Fwiw I think our idea was more relevant to local scanning, the whole point of remote scanning like frigate is that you shouldn't need that kind of optimisation to make it tolerable. What I mean is that I'm ok with remote scanner not allowing closed scan range, we always scan to the tip.
There was a problem hiding this comment.
Either way we can support both, the endHeight would be optional if not passed we just scan till the chain tip.
There was a problem hiding this comment.
We'll see for version 1.6 then
| let end = Height::from_consensus(SCAN_END_HEIGHT)?; | ||
|
|
||
| // It's important to note that `scan_blocks` will not monitor scripts outputs. | ||
| // Call `scan_blocks_with_spent_outputs` instead if you want to monitor spent outputs as well. |
There was a problem hiding this comment.
The scan_blocks api currently expects that we're also scanning the inputs for spent scripts within this block range. If this is hard to do for the Frigate scanner, we could reconsider the Scanner api.
Currently the scan_blocks function is intended to be something a wallet calls in order to synchronize the changes that occur within a given block range. After all blocks are processed, the wallet is considered 'up to date' until this latest block height. Afterwards, the job of the scanner is completed. It does not monitor the blockchain any longer. We could call this a 'one-and-done' approach.
Electrum (and by extension, Frigate) takes a 'monitoring' approach, where we subscribe to a set of scripts, and continuously monitor the blockchain for changes.
Just to make the difference more explicit, this is how Dana does it:
- Dana comes online, for example after the user opens the app, or a background process wakes up the app
- Dana checks the current
block_heightfrom the chain backend api, and checks the storedlast_height(the last known block height) - if
last_height<block_height, Dana creates a scanner object and initiates a scan for rangelast_height+1..=block_height. - After the scan is complete, the scanner's job is finished and is dropped.
- If Dana is in the background, it goes back to sleep. If the app is currently open, Dana will fetch the current
block_heightevery 10 seconds. Once Dana reads that a new block has been confirmed (block_heightis increased), go back to step (3)
Most importantly, it is not the Scanner that monitors the blockchain for new blocks. The wallet (Dana) monitors the blockchain, and when it detects a new block, it kick-starts the scanning process.
We could change this, so that the Scanner becomes responsible for monitoring when new blocks arrive. I suspect this change would be much more compatible with the Electrum/Frigate approach, as the Scanner will now be the one to continuously monitor the blockchain. This also means the scanner can optionally notify the wallet of unconfirmed incoming payments.
So with this new approach, how Dana would work:
- Dana comes online, either by the user opening the app or the background process wakes up
- Dana reads
last_heightfrom storage, and makes a new Scanner withbegin_height = last_scan+1 - The scanner now checks the
block_height. If we are behind onblock_height, Scanner initiates a scan of these blocks, and feeds Dana the updates (through the Receiver channels). - If the scanner has reached the chain tip, one of things can occur:
- (a) If the wallet is currently open, keep monitoring the scripts for changes (very similar to the native Electrum approach)
- (b) If the wallet is currently operating as a background process, we have to terminate the scanner and go back to sleep. We tell the Scanner to unsubscribe from the scripts. The next time the app wakes up, start again from (1)
From the perspective of Dana as a background process, the 'one-and-done' approach is easier to reason about. This is because Dana is expected to wake up every hour or so, do the scanning, and then terminate the scanner and go back to sleep. However I'm sure it's possible, and allowing for continuous monitoring has other benefits as well, so I'm not opposed to making this change.
There was a problem hiding this comment.
Thanks for the detailed explanation. What you said here clearly summarizes the difference in approaches. It now makes much more sense on to why the scanner was taken by value.
(b) If the wallet is currently operating as a background process, we have to terminate the scanner and go back to sleep. We tell the Scanner to unsubscribe from the scripts. The next time the app wakes up, start again from (1)
I think here that on waking up there will need to be subscribe requests to the scripts known, at least for the Frigate/Electrum approach else you won't be able to know whether something is spent or not.
There was a problem hiding this comment.
This is because Dana is expected to wake up every hour or so, do the scanning, and then terminate the scanner and go back to sleep.
I'm pretty sure that foreground service would allow us to keep the subscription to frigate open and allow for the continuous monitoring approach. We had to use that polling approach for blindbit because of the way the v1 works, not because of android foreground services.
This PR adds frigate as an alternative backend support.
Changes
TBD
Notes to reviewer
TBD
Closes #147