Repository navigation
Full Silent Payment Implementation - #145
notTanveer wants to merge 24 commits into
Conversation
|
It's great to make it more digestible for review! |
08a5d40 to
2d1da69
Compare
I did include all of your fixes. a few might have been dropped during the cleanup since I removed a number of functions we no longer needed and moved helper functions that were only used by tests into their respective test files. if you notice anything specific that's missing, let me know and I'll take another look. (i squashed the |
043a824 to
3105a96
Compare
| try: | ||
| start_pos = stream.tell() | ||
| while True: | ||
| k = read_string(stream) |
3105a96 to
c9f4918
Compare
f20926b to
9f26ec7
Compare
6443846 to
dcd028b
Compare
macgyver13
left a comment
There was a problem hiding this comment.
Lots of great work here! The single signer flow appears to be nearing completion.
This review focuses on the silent-payments specific code. Rather than scatter comments across the diff, I pushed a review branch:
notTanveer/embit@feat/silent-payments-V2...macgyver13:embit:review/silent-payments-V2
Two commits address issues that surfaced during a Claude-assisted review, makes sense to me but you may want to dig deeper:
- harden resolve BIP32 derivation match with our UTXOs - _resolve_bip32_privkey didn't bind the derived key to the input's scriptPubKey, so a PSBT claiming a path we own for an input we don't control would fold that key into a_sum. Comparison now includes parity.
- raise SPValidationError for invalid Silent Payment spend inputs in sign_with() - sign_input_with_sp_tweak returned 0 for a bad tweak. BIP-376 requires signer to fail.
The bulk of the branch is a larger change moving validation logic under the psbt.sign_with() public API, so downstream consumers get the validation rather than re-implementing it. Per-input share test vectors are explicitly disabled until that work is ready.
Doing the same for the BIP-352 vectors would likely turn up more. Happy to take a pass at that if it's useful. Reuse as much or as little of the branch as you like.
dcd028b to
e101ce5
Compare
e101ce5 to
0efca34
Compare
There was a problem hiding this comment.
AI review findings fixed in, 59908f5:
SilentPaymentsPSBT.sign_with() was crashing when it hit a key that couldn't sign. _resolve_sp_privkeys() was dropping the can_sign flag returned by resolve_signing_root(), so a public-only descriptor key ended up in resolve_input_privkey(), where hasattr(root, "secret") raises ArgumentError for a descriptor Key instead of just returning False.
PSBT.sign_with() should return 0 in this case. There was also a worse case with multi-key descriptors: if an earlier private key had already written ECDH shares, DLEQ proofs, and output scripts, hitting the public key would throw and leave the PSBT half-derived.
Also cleaned up a few related things:
- Moved the duplicated
_update_tx_modifiablelogic fromPSBTandPSBTViewinto a sharednext_tx_modifiable(). - Removed the dead
tx_modifiable_flagsblock inPSBTView.sign_with(), which could only run after nothing was signed. - Removed the unused
NETWORKSimport.
|
Branch needs to be rebased, as a PSBTv2 related hardening was committed to master |
137a493 to
839c5ab
Compare
52817fd to
533cd85
Compare
- replaced None with error messages. - made Bech32DecodeError an EmbitError. - Subclass EmbitError so existing except EmbitError callers still catch malformed-address errors (e.g. script.address_to_scriptpubkey). - Restore the BIP-141 2-40 byte witness program limit in decode() and drop the sp/tsp special-casing; SP addresses are not witness programs. - Replace the 118 length cap with BIP-352's recommended 1023 so valid higher-version silent payment addresses decode; cite BIP-173/BIP-352.
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com> Co-authored-by: macgyver <4712150+macgyver@users.noreply.github.com>
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
SilentPaymentsPSBT.sign_with() aborted with ArgumentError on a public-only descriptor key. _resolve_sp_privkeys() discarded the can_sign flag from resolve_signing_root(), so such a key reached resolve_input_privkey(), whose hasattr(root, "secret") probe raises on a descriptor Key instead of returning False; PSBT.sign_with() returns 0 in the same situation. With a multi-key descriptor a private key earlier in root.keys had already written ECDH shares, DLEQ proofs and output scripts before a later public key threw, leaving the PSBT half-derived. Honour can_sign before the SP send path runs. Also: - Extract next_tx_modifiable() in psbt.py. PSBT and PSBTView each carried a byte-identical copy of the BIP-370 sighash -> flag rules, so a later correction to one would silently leave the constrained-device path wrong. Going through a pure function also removes a None &= -2 TypeError on a PSBTView built with version=2 and no global_kvs, whose flags setter no-ops. - Drop the trailing tx_modifiable_flags block in PSBTView.sign_with(). Every path in sign_input() that adds a signature already updates the flags, so the block could only fire after signing nothing, stamping TX_MODIFIABLE=0 onto a PSBT this pass never touched. PSBT.sign_with() does not do this. - Remove the unused NETWORKS import in silent_payments/sp.py.
… rebase Rebasing feat/silent-payments-V2 onto upstream/master's PSBTv2 hardening series (eb6104f) merged in upstream's declarative _validate_global_key, V2_FIELDS/_validate_key scope checks, and _read_global_value eager-shape validation, but left _validate_global_fields (canonical count-field encoding, fixed-length checks, required-field presence) unwired in both PSBT.read_from and PSBTView.view. Malformed PSBT_GLOBAL_INPUT_COUNT / PSBT_GLOBAL_OUTPUT_COUNT values crashed with RuntimeError/ValueError instead of PSBTError, non-canonical compact encodings went unrejected, and PSBTView accepted a v2 PSBT missing PSBT_GLOBAL_TX_VERSION until something later touched the lazy .tx_version property. Wire _validate_global_fields into both parse paths, replacing the now-duplicate manual v0/v2 field-exclusion checks it subsumes. All 25 of upstream's tests/tests/test_psbt_versions.py cases now pass.
BIP-352 makes H the single exception to "the sending wallet MUST have access to the private key corresponding to the taproot output key", so a NUMS internal key is the one PSBT field that can drop an input from a_sum. It was believed on faith, and believing it is silent: an excluded input never reaches _resolve_sp_privkeys to be missed, yet the signer still key-path signs it, so the recipient - who scores eligibility from the on-chain witness - derives a shared secret we never used and never finds the payment. The claim now counts only once it reproduces the scriptPubKey, which is all the UTXO vouches for - the doctrine _p2sh_redeem_script and the bip32 hash160 binding already apply. Unproven, the input stays eligible: a false claim about an input we control then resolves normally, and a true one has no key to resolve and aborts the send. A BIP-341 tweaked NUMS lands there too, which is correct - such a transaction cannot be a silent payment at all. Also in this pass: - Generate every DLEQ proof before writing any of them. A proof failure used to leave shares without proofs, a state validate_sp() rejects forever, so the PSBT could never be signed again. - Restore the clear_metadata contract: KEEP_ALL is a no-op, and sp_data is dropped only once script_pubkey exists. Before the send it is the output's whole definition, and dropping it produced a PSBT that could no longer be parsed. - Reject malformed and PSBTv0 global SP fields instead of retaining them as unknowns and re-serializing them verbatim, matching the input and output paths. - Refuse derive_sp_outputs_from_keys() on a PSBT with no SP outputs instead of clearing the global shares and TX_MODIFIABLE and returning as if it had succeeded. - Range-check the silent payment address version, which bech32_encode indexes CHARSET with: 32 raised a bare IndexError and -1 wrapped to 31. - Drop the DLEQ verification claim from the validate_sp() docstring; the check needs A_sum, which the PSBT does not carry. _verify_declared_sp() does it on the send path.
…signed byte helpers Parsing tx_version unsigned closes the same Transaction(version=tx_version) overflow the helpers were added for.
A descriptor with two private keys re-ran the whole derivation for each one, so the second key aborted on a half-signed PSBT. It is derived once for the root now, and each input tries every descriptor key before it counts as foreign. Also in this pass: - Validate keydata on 0x09/0x0a/0x20 and lengths on PSBT_IN_SP_ECDH_SHARE and PSBT_IN_SP_DLEQ. A key with trailing bytes fell through to unknown, which skipped every BIP-375 check and re-serialized it verbatim. - Refuse derive_sp_outputs_from_keys() outside PSBTv2; on v0 it derived into global fields that serialize() then dropped. - Pin the type in __eq__, guard the empty key before k[0], and call the _v2_output_has_amount hook instead of hardcoding out.value. - Share the taproot key-path signing with sign_input_with_tapkey, and serialize the sort key once per output instead of once per comparison.
PSET.blinded_tx used self.locktime, the fallback, and ignored the required height/time locktimes that PSBT.tx already resolves. with fallback_locktime=7 and required_height_locktime=800000, tx.locktime was 800000 and blinded_tx.locktime was 7, so the sighash and the final tx disagreed and the signatures would not validate. it uses determine_locktime() now. LInputScope.vin and blinded_vin also did `self.sequence or 0xFFFFFFFF`, which turns an explicit sequence of 0 into 0xFFFFFFFF. InputScope.vin was fixed for this already, the liquid copies were missed. also in this pass: - liquid.addresses.address() returned None for a script that is not a witness program (OP_RETURN). bech32.encode raises now, so address() raised too. it catches that and returns None again. - docs/api/bech32.md still said decode returns (None, None). it raises.
the v2 vout prescan used stream.tell(), which MicroPython's BytesIO does not have. the AttributeError fallback then turned the compress OOM protection off without saying so, so on the device an input map with 0x00 before 0x0f stored the whole previous tx in non_witness_utxo. the position comes from seek(0, 1) now, and the scan is its own function, _scan_vout. PSBTView.view() kept every non-fixed global key for v0 too, so RAM grew with untrusted global keys where it used to be constant. 3000 unknown globals took the peak from 1 KB to 530 KB. v0 only skips them now. v2 needs them to rebuild the global scope on write, but it keeps an (offset, length) pair and copies the value over from the source stream in write_to, so one big unknown global no longer stays in RAM. the output is byte-identical. v0 PSBT.read_from built temp_inputs and temp_outputs while the placeholder scopes were still alive, so the peak held about 2N scopes (+39% on 200 inputs). the placeholders are replaced in place now. also in this pass: - InputScope and OutputScope share one _read_map. PSBTScope.read_from was dead code. - _raw_input_count_from_global and _raw_output_count_from_global were parse-time state kept on every PSBT and re-synced in add_input and add_output. they are plain loop bounds in read_from now, and the "missing count/version" checks after parsing, which could not fire, are gone. - PSBTView.sign_input calls resolve_signing_root and derive_hdkey instead of its own copies of them. - PSBT.tx drops its v0/v2 locktime branch, determine_locktime() gives the same answer for v0. add_input calls choose_locktime directly instead of a one-line wrapper. - new _v2_output_has_script hook next to _v2_output_has_amount, so a subclass that only relaxes the script check does not have to copy _validate_v2_output.
SilentPaymentDescriptor had no write_to, so serialize(), ==, hash() and
parse() all failed. SPScanKey had no is_private, so
sign_with(sp_descriptor) raised AttributeError in resolve_signing_root.
read_from also expected the caller to have eaten "sp(" already, which is
why Descriptor.read_from had to rewind and check the ")" itself, and a
truncated "sp(" gave a base58 ValueError. read_from owns the sp(...)
framing now, Descriptor.read_from hands over to it, and an empty key
expression raises DescriptorError.
the bech32m framing (version value, 8 to 5 bit conversion, checksum
and padding checks) was written twice, once for sp addresses and once
for spscan keys. bech32m_encode_versioned and bech32m_decode_versioned
in bech32.py do it now. each caller keeps its own hrp, version and
length rules.
also in this pass:
- derive_sp_outputs multiplies the scan key once into one buffer and
reuses it for the shared secret, instead of parsing and serializing
the point twice. _tweak_mul is gone.
- the single-arg sp() error offered spspend keys, which are not
supported. the message says spscan only.
- fix a "comaprison" typo, em-dash out of the sp.py docstring.
sign_with filtered the keys through resolve_signing_root, and then derive_sp_outputs resolved the same keys again inside _resolve_sp_privkeys. _signers() does it once and both paths use the result, so a public-only key is dropped the same way in each. also in this pass: - SilentPaymentsPSBT._validate_v2_output was a copy of the base method with one condition changed. it overrides the new _v2_output_has_script hook and calls super() now. - __eq__ compared the shares and proofs on top of super().__eq__, but both are part of the serialization already. - error paths use hexlify(...).decode() instead of bytes.hex(), like the rest of embit. - compute_dleq_proof and _resolve_bip32_privkey call sec() once, it derives the public key from the private key on every call. - em-dashes out of the docstrings this PR added.
f18d23b to
3fb1227
Compare
- Map every network to its HRP explicitly in SP_HRPS, regtest shares tsp as BIP-352 has no regtest HRP - Map spscan HRPs per network, regtest shares tspscan as BIP-392 has no regtest HRP - Raise on unknown or non-string networks instead of falling back to a testnet HRP - Add tests for the HRP of each network and the rejected network values
- Reduce the challenge e modulo the curve order when generating and verifying proofs - Reject proofs whose e is not below the curve order, same as s - Add the upstream verify vectors for e and s equal to the curve order
- Replace the BIP-352 send vectors with upstream, adding the zero intermediate sum and K_max cases - Expand recipient objects by their count now that upstream lists recipients as objects - Treat a failed send as no outputs so the K_max exceeded vector passes - Add the BIP-375 invalid vector for a PSBT_OUT_SP_V0_LABEL that is not 4 bytes
- Add SilentPaymentsPSBT.sp_output_scripts to derive the taproot output scripts without writing to the PSBT - Move the checked derivation out of derive_sp_outputs_from_keys into a helper both paths share - Skip DLEQ proof generation on the preview path so a later sign_with does not pay for it twice - Add tests for matching sign_with, leaving the PSBT untouched and rejecting a foreign input
|
Hi @notTanveer @odudex, I’m building an independent test suite that runs BIP 375/376 PSBTs (the official vectors plus malicious-host cases) against signers. I started with this PR, pinned at e89d799. Before publishing results, here’s what it found:
Case definitions and the scorecard: https://github.com/rademenezbtc/silentwitness. Happy to turn any of these into tests in this PR if useful. |
This PR contains the complete Silent Payments implementation with a clean, linear commit history to make the review process easier.. covers PRs #64, #87, #95, #96, #125, #126, #130, #133,
also includes a number of fixes by @odudex , squashed into their corresponding
feat:commits to make the review process easier.how the flow works
two flows here: send to an
sp1..address (BIP-352/375, proofs via BIP-374) and spend a received SP utxo (BIP-376). there is no scanning in embit, the wallet finds the utxos and hands oversp_tweak.send, who does what
sequenceDiagram participant W as Coordinator (Sparrow) participant P as SilentPaymentsPSBT participant S as Signer (Krux, root key) W->>P: PSBTv2, outputs carry PSBT_OUT_SP_V0_INFO (scan, spend), no script S->>P: sp_output_scripts(root) P-->>S: {out idx: P2TR script}, nothing written, shown for review S->>P: sign_with(root) P->>P: derive outputs, write ECDH share + DLEQ + PSBT_OUT_SCRIPT P->>P: clear inputs/outputs modifiable flags P->>P: sign every input (SIGHASH_ALL only) P-->>W: signed PSBT, W verifies DLEQ and finalizes