Skip to content

Full Silent Payment Implementation - #145

Open
notTanveer wants to merge 24 commits into
diybitcoinhardware:masterfrom
notTanveer:feat/silent-payments-V2
Open

notTanveer wants to merge 24 commits into
diybitcoinhardware:masterfrom
notTanveer:feat/silent-payments-V2

Conversation

@notTanveer

@notTanveer notTanveer commented Jul 9, 2026 •

Copy link
Copy Markdown
Collaborator

This PR contains the complete Silent Payments implementation with a clean, linear commit history to make the review process easier.. covers PRs #64, #87, #95, #96, #125, #126, #130, #133,

also includes a number of fixes by @odudex , squashed into their corresponding feat: commits to make the review process easier.

how the flow works

two flows here: send to an sp1.. address (BIP-352/375, proofs via BIP-374) and spend a received SP utxo (BIP-376). there is no scanning in embit, the wallet finds the utxos and hands over sp_tweak.

send, who does what

sequenceDiagram
    participant W as Coordinator (Sparrow)
    participant P as SilentPaymentsPSBT
    participant S as Signer (Krux, root key)
    W->>P: PSBTv2, outputs carry PSBT_OUT_SP_V0_INFO (scan, spend), no script
    S->>P: sp_output_scripts(root)
    P-->>S: {out idx: P2TR script}, nothing written, shown for review
    S->>P: sign_with(root)
    P->>P: derive outputs, write ECDH share + DLEQ + PSBT_OUT_SCRIPT
    P->>P: clear inputs/outputs modifiable flags
    P->>P: sign every input (SIGHASH_ALL only)
    P-->>W: signed PSBT, W verifies DLEQ and finalizes
Loading

@notTanveer
notTanveer requested review from miketlk and odudex as code owners July 9, 2026 14:17
@notTanveer notTanveer changed the title Feat/silent payments v2 [FEAT] Full Silent Payment Implementation Jul 9, 2026
@notTanveer notTanveer changed the title [FEAT] Full Silent Payment Implementation Full Silent Payment Implementation Jul 9, 2026
@odudex

odudex commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

It's great to make it more digestible for review!
One detail, I spent some days rebasing scattered PRs and did tens of bugfixes. None of this work is present in this PR?

@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch from 08a5d40 to 2d1da69 Compare July 13, 2026 14:34
@notTanveer

notTanveer commented Jul 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

One detail, I spent some days rebasing scattered PRs and did tens of bugfixes. None of this work is present in this PR?

I did include all of your fixes. a few might have been dropped during the cleanup since I removed a number of functions we no longer needed and moved helper functions that were only used by tests into their respective test files. if you notice anything specific that's missing, let me know and I'll take another look.

(i squashed the fix: commits into the corresponding `feat' commit)

@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch 2 times, most recently from 043a824 to 3105a96 Compare July 13, 2026 20:40
Comment thread src/embit/silent_payments/psbt.py Fixed
Comment thread src/embit/silent_payments/psbt.py Fixed
Comment thread src/embit/silent_payments/psbt.py Fixed
Comment thread src/embit/silent_payments/psbt.py Fixed
Comment thread src/embit/silent_payments/psbt.py Fixed
Comment thread src/embit/psbt.py Outdated
try:
start_pos = stream.tell()
while True:
k = read_string(stream)
Comment thread src/embit/psbt.py Fixed
Comment thread src/embit/silent_payments/sp.py Fixed
Comment thread src/embit/silent_payments/psbt.py Fixed
@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch 2 times, most recently from f20926b to 9f26ec7 Compare July 20, 2026 14:03
@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch 3 times, most recently from 6443846 to dcd028b Compare August 4, 2026 18:40

@macgyver13 macgyver13 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of great work here! The single signer flow appears to be nearing completion.

This review focuses on the silent-payments specific code. Rather than scatter comments across the diff, I pushed a review branch:
notTanveer/embit@feat/silent-payments-V2...macgyver13:embit:review/silent-payments-V2

Two commits address issues that surfaced during a Claude-assisted review, makes sense to me but you may want to dig deeper:

  • harden resolve BIP32 derivation match with our UTXOs - _resolve_bip32_privkey didn't bind the derived key to the input's scriptPubKey, so a PSBT claiming a path we own for an input we don't control would fold that key into a_sum. Comparison now includes parity.
  • raise SPValidationError for invalid Silent Payment spend inputs in sign_with() - sign_input_with_sp_tweak returned 0 for a bad tweak. BIP-376 requires signer to fail.

The bulk of the branch is a larger change moving validation logic under the psbt.sign_with() public API, so downstream consumers get the validation rather than re-implementing it. Per-input share test vectors are explicitly disabled until that work is ready.
Doing the same for the BIP-352 vectors would likely turn up more. Happy to take a pass at that if it's useful. Reuse as much or as little of the branch as you like.

Comment thread src/embit/silent_payments/sp.py Resolved
Comment thread src/embit/silent_payments/sp.py Resolved
Comment thread src/embit/silent_payments/sp.py Resolved
Comment thread src/embit/silent_payments/sp.py Resolved
Comment thread src/embit/silent_payments/sp.py Resolved
Comment thread src/embit/silent_payments/sp.py Resolved
Comment thread src/embit/psbt.py Resolved

@notTanveer notTanveer left a comment •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review findings fixed in, 59908f5:

SilentPaymentsPSBT.sign_with() was crashing when it hit a key that couldn't sign. _resolve_sp_privkeys() was dropping the can_sign flag returned by resolve_signing_root(), so a public-only descriptor key ended up in resolve_input_privkey(), where hasattr(root, "secret") raises ArgumentError for a descriptor Key instead of just returning False.

PSBT.sign_with() should return 0 in this case. There was also a worse case with multi-key descriptors: if an earlier private key had already written ECDH shares, DLEQ proofs, and output scripts, hitting the public key would throw and leave the PSBT half-derived.

Also cleaned up a few related things:

  • Moved the duplicated _update_tx_modifiable logic from PSBT and PSBTView into a shared next_tx_modifiable().
  • Removed the dead tx_modifiable_flags block in PSBTView.sign_with(), which could only run after nothing was signed.
  • Removed the unused NETWORKS import.

@odudex

odudex commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Branch needs to be rebased, as a PSBTv2 related hardening was committed to master

@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch from 137a493 to 839c5ab Compare August 19, 2026 14:31
@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch from 52817fd to 533cd85 Compare August 28, 2026 12:20
kdmukai and others added 19 commits September 27, 2026 14:57
- replaced None with error messages.
- made Bech32DecodeError an EmbitError.
- Subclass EmbitError so existing except EmbitError callers still catch
  malformed-address errors (e.g. script.address_to_scriptpubkey).
- Restore the BIP-141 2-40 byte witness program limit in decode() and
  drop the sp/tsp special-casing; SP addresses are not witness programs.
- Replace the 118 length cap with BIP-352's recommended 1023 so valid
  higher-version silent payment addresses decode; cite BIP-173/BIP-352.
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: macgyver <4712150+macgyver@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
SilentPaymentsPSBT.sign_with() aborted with ArgumentError on a public-only
descriptor key. _resolve_sp_privkeys() discarded the can_sign flag from
resolve_signing_root(), so such a key reached resolve_input_privkey(), whose
hasattr(root, "secret") probe raises on a descriptor Key instead of returning
False; PSBT.sign_with() returns 0 in the same situation. With a multi-key
descriptor a private key earlier in root.keys had already written ECDH shares,
DLEQ proofs and output scripts before a later public key threw, leaving the
PSBT half-derived. Honour can_sign before the SP send path runs.

Also:

- Extract next_tx_modifiable() in psbt.py. PSBT and PSBTView each carried a
  byte-identical copy of the BIP-370 sighash -> flag rules, so a later
  correction to one would silently leave the constrained-device path wrong.
  Going through a pure function also removes a None &= -2 TypeError on a
  PSBTView built with version=2 and no global_kvs, whose flags setter no-ops.

- Drop the trailing tx_modifiable_flags block in PSBTView.sign_with(). Every
  path in sign_input() that adds a signature already updates the flags, so the
  block could only fire after signing nothing, stamping TX_MODIFIABLE=0 onto a
  PSBT this pass never touched. PSBT.sign_with() does not do this.

- Remove the unused NETWORKS import in silent_payments/sp.py.
… rebase

Rebasing feat/silent-payments-V2 onto upstream/master's PSBTv2 hardening
series (eb6104f) merged in upstream's declarative _validate_global_key,
V2_FIELDS/_validate_key scope checks, and _read_global_value eager-shape
validation, but left _validate_global_fields (canonical count-field
encoding, fixed-length checks, required-field presence) unwired in both
PSBT.read_from and PSBTView.view. Malformed PSBT_GLOBAL_INPUT_COUNT /
PSBT_GLOBAL_OUTPUT_COUNT values crashed with RuntimeError/ValueError
instead of PSBTError, non-canonical compact encodings went unrejected,
and PSBTView accepted a v2 PSBT missing PSBT_GLOBAL_TX_VERSION until
something later touched the lazy .tx_version property.

Wire _validate_global_fields into both parse paths, replacing the
now-duplicate manual v0/v2 field-exclusion checks it subsumes. All 25
of upstream's tests/tests/test_psbt_versions.py cases now pass.
BIP-352 makes H the single exception to "the sending wallet MUST have
access to the private key corresponding to the taproot output key", so a
NUMS internal key is the one PSBT field that can drop an input from
a_sum. It was believed on faith, and believing it is silent: an excluded
input never reaches _resolve_sp_privkeys to be missed, yet the signer
still key-path signs it, so the recipient - who scores eligibility from
the on-chain witness - derives a shared secret we never used and never
finds the payment.

The claim now counts only once it reproduces the scriptPubKey, which is
all the UTXO vouches for - the doctrine _p2sh_redeem_script and the
bip32 hash160 binding already apply. Unproven, the input stays eligible:
a false claim about an input we control then resolves normally, and a
true one has no key to resolve and aborts the send. A BIP-341 tweaked
NUMS lands there too, which is correct - such a transaction cannot be a
silent payment at all.

Also in this pass:

- Generate every DLEQ proof before writing any of them. A proof failure
  used to leave shares without proofs, a state validate_sp() rejects
  forever, so the PSBT could never be signed again.
- Restore the clear_metadata contract: KEEP_ALL is a no-op, and sp_data
  is dropped only once script_pubkey exists. Before the send it is the
  output's whole definition, and dropping it produced a PSBT that could
  no longer be parsed.
- Reject malformed and PSBTv0 global SP fields instead of retaining them
  as unknowns and re-serializing them verbatim, matching the input and
  output paths.
- Refuse derive_sp_outputs_from_keys() on a PSBT with no SP outputs
  instead of clearing the global shares and TX_MODIFIABLE and returning
  as if it had succeeded.
- Range-check the silent payment address version, which bech32_encode
  indexes CHARSET with: 32 raised a bare IndexError and -1 wrapped to
31.
- Drop the DLEQ verification claim from the validate_sp() docstring; the
  check needs A_sum, which the PSBT does not carry.
_verify_declared_sp()
  does it on the send path.
…signed byte helpers

Parsing tx_version unsigned closes the same
Transaction(version=tx_version) overflow the helpers were added for.
A descriptor with two private keys re-ran the whole derivation for each
one, so the second key aborted on a half-signed PSBT. It is derived once
for the root now, and each input tries every descriptor key before it
counts as foreign.

Also in this pass:

- Validate keydata on 0x09/0x0a/0x20 and lengths on PSBT_IN_SP_ECDH_SHARE
  and PSBT_IN_SP_DLEQ. A key with trailing bytes fell through to unknown,
  which skipped every BIP-375 check and re-serialized it verbatim.
- Refuse derive_sp_outputs_from_keys() outside PSBTv2; on v0 it derived
  into global fields that serialize() then dropped.
- Pin the type in __eq__, guard the empty key before k[0], and call the
  _v2_output_has_amount hook instead of hardcoding out.value.
- Share the taproot key-path signing with sign_input_with_tapkey, and
  serialize the sort key once per output instead of once per comparison.
PSET.blinded_tx used self.locktime, the fallback, and ignored the
required height/time locktimes that PSBT.tx already resolves. with
fallback_locktime=7 and required_height_locktime=800000, tx.locktime was
800000 and blinded_tx.locktime was 7, so the sighash and the final tx
disagreed and the signatures would not validate. it uses
determine_locktime() now.

LInputScope.vin and blinded_vin also did `self.sequence or 0xFFFFFFFF`,
which turns an explicit sequence of 0 into 0xFFFFFFFF. InputScope.vin
was fixed for this already, the liquid copies were missed.

also in this pass:

- liquid.addresses.address() returned None for a script that is not a
  witness program (OP_RETURN). bech32.encode raises now, so address()
  raised too. it catches that and returns None again.
- docs/api/bech32.md still said decode returns (None, None). it raises.
the v2 vout prescan used stream.tell(), which MicroPython's BytesIO does
not have. the AttributeError fallback then turned the compress OOM
protection off without saying so, so on the device an input map with
0x00 before 0x0f stored the whole previous tx in non_witness_utxo. the
position comes from seek(0, 1) now, and the scan is its own function,
_scan_vout.

PSBTView.view() kept every non-fixed global key for v0 too, so RAM grew
with untrusted global keys where it used to be constant. 3000 unknown
globals took the peak from 1 KB to 530 KB. v0 only skips them now. v2
needs them to rebuild the global scope on write, but it keeps an
(offset, length) pair and copies the value over from the source stream
in write_to, so one big unknown global no longer stays in RAM. the
output is byte-identical.

v0 PSBT.read_from built temp_inputs and temp_outputs while the
placeholder scopes were still alive, so the peak held about 2N scopes
(+39% on 200 inputs). the placeholders are replaced in place now.

also in this pass:

- InputScope and OutputScope share one _read_map. PSBTScope.read_from
  was dead code.
- _raw_input_count_from_global and _raw_output_count_from_global were
  parse-time state kept on every PSBT and re-synced in add_input and
  add_output. they are plain loop bounds in read_from now, and the
  "missing count/version" checks after parsing, which could not fire,
  are gone.
- PSBTView.sign_input calls resolve_signing_root and derive_hdkey
  instead of its own copies of them.
- PSBT.tx drops its v0/v2 locktime branch, determine_locktime() gives
  the same answer for v0. add_input calls choose_locktime directly
  instead of a one-line wrapper.
- new _v2_output_has_script hook next to _v2_output_has_amount, so a
  subclass that only relaxes the script check does not have to copy
  _validate_v2_output.
SilentPaymentDescriptor had no write_to, so serialize(), ==, hash() and
parse() all failed. SPScanKey had no is_private, so
sign_with(sp_descriptor) raised AttributeError in resolve_signing_root.
read_from also expected the caller to have eaten "sp(" already, which is
why Descriptor.read_from had to rewind and check the ")" itself, and a
truncated "sp(" gave a base58 ValueError. read_from owns the sp(...)
framing now, Descriptor.read_from hands over to it, and an empty key
expression raises DescriptorError.

the bech32m framing (version value, 8 to 5 bit conversion, checksum
and padding checks) was written twice, once for sp addresses and once
for spscan keys. bech32m_encode_versioned and bech32m_decode_versioned
in bech32.py do it now. each caller keeps its own hrp, version and
length rules.

also in this pass:

- derive_sp_outputs multiplies the scan key once into one buffer and
  reuses it for the shared secret, instead of parsing and serializing
  the point twice. _tweak_mul is gone.
- the single-arg sp() error offered spspend keys, which are not
  supported. the message says spscan only.
- fix a "comaprison" typo, em-dash out of the sp.py docstring.
sign_with filtered the keys through resolve_signing_root, and then
derive_sp_outputs resolved the same keys again inside
_resolve_sp_privkeys. _signers() does it once and both paths use the
result, so a public-only key is dropped the same way in each.

also in this pass:

- SilentPaymentsPSBT._validate_v2_output was a copy of the base method
  with one condition changed. it overrides the new _v2_output_has_script
  hook and calls super() now.
- __eq__ compared the shares and proofs on top of super().__eq__, but
  both are part of the serialization already.
- error paths use hexlify(...).decode() instead of bytes.hex(), like the
  rest of embit.
- compute_dleq_proof and _resolve_bip32_privkey call sec() once, it
  derives the public key from the private key on every call.
- em-dashes out of the docstrings this PR added.
@notTanveer
notTanveer force-pushed the feat/silent-payments-V2 branch from f18d23b to 3fb1227 Compare September 27, 2026 09:30
- Map every network to its HRP explicitly in SP_HRPS, regtest shares tsp as BIP-352 has no regtest HRP
- Map spscan HRPs per network, regtest shares tspscan as BIP-392 has no regtest HRP
- Raise on unknown or non-string networks instead of falling back to a testnet HRP
- Add tests for the HRP of each network and the rejected network values
- Reduce the challenge e modulo the curve order when generating and verifying proofs
- Reject proofs whose e is not below the curve order, same as s
- Add the upstream verify vectors for e and s equal to the curve order
- Replace the BIP-352 send vectors with upstream, adding the zero intermediate sum and K_max cases
- Expand recipient objects by their count now that upstream lists recipients as objects
- Treat a failed send as no outputs so the K_max exceeded vector passes
- Add the BIP-375 invalid vector for a PSBT_OUT_SP_V0_LABEL that is not 4 bytes
- Add SilentPaymentsPSBT.sp_output_scripts to derive the taproot output scripts without writing to the PSBT
- Move the checked derivation out of derive_sp_outputs_from_keys into a helper both paths share
- Skip DLEQ proof generation on the preview path so a later sign_with does not pay for it twice
- Add tests for matching sign_with, leaving the PSBT untouched and rejecting a foreign input
@rademenezbtc

rademenezbtc commented Nov 16, 2026 •

Copy link
Copy Markdown

Hi @notTanveer @odudex, I’m building an independent test suite that runs BIP 375/376 PSBTs (the official vectors plus malicious-host cases) against signers. I started with this PR, pinned at e89d799. Before publishing results, here’s what it found:

  1. BIP 375 vectors I10/I12/I14 (bad per-input share or DLEQ proof): these are skipped in test_bip375.py, and embit replaces the per-input fields with a correct global share. BIP 375 only requires checking proofs for inputs the signer has no key for, so this may be intended. Is “repair” the behaviour you want, or should the signer refuse?

  2. A witness UTXO that disagrees with the non-witness UTXO (claims P2WSH, really P2WPKH): sign_with() trusts the witness UTXO, drops the input from the BIP 352 input set and derives an output the recipient can’t find. In this exact case the signatures are also invalid against the real UTXOs, so the transaction couldn’t be broadcast. The BIP 375 reference validator accepts the returned PSBT too, so this may also need a new invalid test vector upstream. Should validate_sp() cross-check the two?

  3. A forged non_witness_utxo (txid mismatch) is accepted because sign_with() doesn’t call verify(). In this case the forged data wasn’t used, so outputs and signatures were still correct. Is verify() meant to be the caller’s job?

  4. Also checked, all correct: BIP 376 wrong or borrowed tweak, a script that doesn’t match the sp1 address, non-SIGHASH_ALL inputs, odd-Y tweaked keys, and mixed SP and ordinary inputs. Multi-party cases are skipped, since embit doesn’t support them.

Case definitions and the scorecard: https://github.com/rademenezbtc/silentwitness. Happy to turn any of these into tests in this PR if useful.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants