Skip to content

Apply dustlimit and cut_through in StreamComputeIndex - #61

Open
bitsagarob wants to merge 2 commits into
setavenger:masterfrom
bitsagarob:feat/compute-index-filters-clean
Open

bitsagarob wants to merge 2 commits into
setavenger:masterfrom
bitsagarob:feat/compute-index-filters-clean

Conversation

@bitsagarob

@bitsagarob bitsagarob commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

GRPC.md marks both fields reserved. This implements them, on StreamComputeIndex only.

Both filters apply per transaction, never per output. A tx is kept when at least one of its outputs is unspent at the pinned tip and at or above dustlimit, and a kept tx carries all of its output prefixes.

dustlimit: 0 with cut_through: false returns the stored index unchanged, by an early return.

Running here on a full mainnet index. Block 900000: 39677 bytes unfiltered, 7588 with cut_through.

Caveat: cut_through costs a spend lookup per output. Measured end to end over 200 blocks at height 900000, 327 ms/block against 5 ms unfiltered. Fine for a batch client, too slow to serve live. A materialised index is the real fix, presumably what tweaks_cut_through_with_dust_filter was for. Happy to do that instead.

@bitsagarob
bitsagarob force-pushed the feat/compute-index-filters-clean branch from acb48d5 to 442121c Compare September 21, 2026 07:19
@setavenger

Copy link
Copy Markdown
Owner

Thanks for the PR. A quick pre-check:

Can you clear up one thing for me. It seems like outputs are individually dropped within a transaction and not once all taproot outputs within the transaction are spent. My agents seem to confirm this. Cutting outputs pre-maturely will lead to undetectable unspent outputs within the transaction.

The following example applies:

  • Alice has received two or more outputs within the same transaction to the same address/label
  • Alice spends the k=0 output, leaving her with k=1..n
  • Alice recovers on a new device and scans
  • Alice checks the transaction but cannot match on k=0 and skips to the next transaction
  • k=1 is "lost"

Regarding the architecture decision with an index I will do a deep-dive. I was already experimenting with a form of cut-through for outputs which received twice and noticed that also requires some additional indexes for speeding things up. Perhaps the approaches can be combined. In the meantime could you please double check that the lost fund scenario does not apply.

Cheers!

@bitsagarob
bitsagarob force-pushed the feat/compute-index-filters-clean branch from cd951e2 to f6cfcbc Compare September 21, 2026 13:23
dropping single outputs stops a BIP-352 scanner at the first missing k
@bitsagarob
bitsagarob force-pushed the feat/compute-index-filters-clean branch from f6cfcbc to d85f5f0 Compare September 21, 2026 13:23
@bitsagarob

Copy link
Copy Markdown
Contributor Author

Confirmed against the BIP-352 reference implementation: a tx carrying k=0 and k=1 to the same recipient returns 2 outputs when both are served, and 0 when only the spent k=0 is withheld. The scan stops at the first k it can't match as you describe.

Fixed in d85f5f0, now all or nothing per tx.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants