Skip to content

PSBTv2 (BIP-370) support and PSBT hardening - #152

Open
odudex wants to merge 44 commits into
diybitcoinhardware:masterfrom
odudex:feat/psbt-v2
Open

odudex wants to merge 44 commits into
diybitcoinhardware:masterfrom
odudex:feat/psbt-v2

Conversation

@odudex

@odudex odudex commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

This PR carves the PSBT work out of #145 (Full Silent Payment Implementation) into a smaller, self-contained branch so it can be reviewed and merged first. It adds PSBTv2 support to PSBT and PSBTView, then fixes a series of correctness and hardening gaps found while reviewing PSBT handling end to end. Silent payments (BIP-352/374/375/376/392) will be rebased on top of it as a follow-up PR.

Nothing in this PR is silent-payments specific. The only forward-looking piece is PSBT._sign_taproot_keypath, a small extraction the SP signer subclasses.

What is in it

PSBTv2 (BIP-370)

  • Parsing, validation and serialization of the v2 global, input and output fields in both PSBT and PSBTView; version-scoped field checks (v2 fields rejected in v0 and vice versa), canonical count fields, required-field checks.
  • Unsigned transaction reconstruction, BIP-370 locktime determination from per-input requirements, PSBT_GLOBAL_TX_MODIFIABLE updates on signing, Creator/Constructor helpers (PSBT.create_v2, add_input, add_output).
  • All BIP-370 test vectors run through both PSBT and PSBTView; valid vectors must round-trip byte for byte through both.

Correctness fixes

Hardening

  • Every malformed PSBT raises PSBTError from both parsers: truncated input, non-canonical or truncated compact sizes (compact.read_from now matches Bitcoin Core), corrupted nested transactions, scripts and keys, absurd lengths, trailing bytes, out-of-range values on serialization. Tests truncate and bit-flip sample PSBTs at every byte.
  • PSBT_IN_NON_WITNESS_UTXO is held to its declared length so a wrong length cannot shift the map framing.
  • PSBTView.view() walks all maps once and rejects trailing data and count/map mismatches, like PSBT.parse.
  • Inputs without a utxo, PSET outputs without an asset, and duplicate keys anywhere in a map are rejected with PSBTError.

PSBTView performance

  • Prevouts, sequences and outputs are hashed in one pass; taproot utxo data is cached per view. Signing 40 inputs went from 33,580 scope skips to 2,380 (p2pkh v2) and from 1,680 input-scope parses to 80 (taproot).

Behaviour changes to be aware of

  • PSBTView.locktime now means the same as PSBT.locktime: the raw fallback field for v2 (None when absent). Callers that displayed the derived transaction locktime from a view should use determine_locktime().
  • PSBTView.view() rejects a stream with data after the PSBT, as PSBT.parse always did.
  • compact.read_from rejects non-canonical encodings, as Bitcoin Core does.
  • PSBT.parse raises PSBTError (still an EmbitError) instead of a bare EmbitError on trailing bytes.

Issues covered

Authorship

PRs covered or replaced

@odudex
odudex requested a review from miketlk as a code owner September 2, 2026 16:54
Copilot AI lite review requested due to automatic review settings September 2, 2026 16:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A confirmed version-propagation bug in PSBTScope.read_from() can prevent correct parsing/validation of versioned fields for any scope class using the base implementation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds full PSBTv2 (BIP-370) support across PSBT and streaming PSBTView, and hardens parsing/signing to consistently reject malformed PSBTs with PSBTError while improving PSBTView signing performance via single-pass hashing and caching.

Changes:

  • Implement PSBTv2 global/input/output parsing, validation, serialization, locktime derivation, Creator/Constructor helpers, and BIP-370 PSBT_GLOBAL_TX_MODIFIABLE updates on signing.
  • Fix Taproot sighash correctness (ANYONECANPAY outpoint hashing; SIGHASH_SINGLE sha_single_output hashing) and align PSBT/PSBTView output bytes + locktime APIs.
  • Harden compact-int parsing and PSBT framing (canonical compact sizes, bounded non-witness-utxo length, duplicate-key rejection, trailing-bytes rejection) and add/extend regression + vector tests.
File summaries
File Description
tests/tests/test_psbtview.py Strengthens PSBTView round-trip assertion to byte-for-byte equality.
tests/tests/test_psbt.py Refactors signing test vectors into constants for reuse/clarity.
tests/tests/test_psbt_versions.py Adds PSBTv2 negative-int64 amount rejection coverage; minor formatting.
tests/tests/test_liquid.py Adds PSETv2 signing/locktime/sequence/asset-required and corruption tests; refactors fixtures.
src/embit/transaction.py Adds segwit-aware parsing for unsigned txs and fixes Taproot sighash edge cases.
src/embit/psbtview.py Implements PSBTv2 global handling, strict streaming validation, locktime derivation, caching, and tx_modifiable updates.
src/embit/psbt.py Implements PSBTv2 core support, hardened parsing helpers, locktime determination, tx_modifiable semantics, and signing refactors.
src/embit/liquid/transaction.py Mirrors Transaction.read_from(segwit=...) signature for Liquid transactions.
src/embit/liquid/psetview.py Aligns parsing hardening with PSBTView and ensures derived locktime is used in sighash.
src/embit/liquid/pset.py Fixes v2 sequence defaulting, ensures derived locktime in blinded tx, and adds v2 output validation (asset/commitment).
src/embit/compact.py Hardens compact-size parsing to reject truncated and non-canonical encodings.
CHANGELOG.md Documents PSBTv2, hardening, behavior changes, and correctness fixes.
Review details
  • Files reviewed: 13/13 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/embit/psbt.py Resolved
Copilot AI review requested due to automatic review settings September 2, 2026 17:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Locktime determination can incorrectly accept mixed (height+time) locktime requirements (including the case where an input specifies both), which should be rejected to avoid deriving an invalid signing locktime.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/embit/psbt.py:212

  • choose_locktime() can return a height-based locktime even when at least one time-based requirement is present (e.g., when an input sets both PSBT_IN_REQUIRED_HEIGHT_LOCKTIME and PSBT_IN_REQUIRED_TIME_LOCKTIME). In that case there is no single nLockTime that can satisfy both requirement domains, so this should be rejected instead of silently preferring height.
  • Files reviewed: 13/13 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@odudex

odudex commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

Locktime determination can incorrectly accept mixed (height+time) locktime requirements (including the case where an input specifies both), which should be rejected to avoid deriving an invalid signing locktime.

Rejecting the "both" case would fail the first two BIP vectors and diverge from Bitcoin Core, whose ComputeTimeLock treats an input carrying both fields as satisfiable by either type.

Comment thread src/embit/psbt.py Resolved
Comment thread src/embit/psbt.py Resolved
Copilot AI review requested due to automatic review settings September 3, 2026 21:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

PSBTView’s PSBTv2 global-scope reconstruction currently buffers deferred unknown global values into memory, enabling a potential memory-exhaustion DoS via oversized unknown global fields.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/embit/psbtview.py Resolved
Copilot AI review requested due to automatic review settings September 8, 2026 11:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

PSBTView’s tx_modifiable_flags setter can currently raise ValueError on invalid input, violating the PR’s stated “raise PSBTError” hardening guarantee.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/embit/psbtview.py:285

  • tx_modifiable_flags setter can raise ValueError (bytes([value])) for out-of-range or non-int inputs, which leaks a non-PSBTError exception path. Since this is part of the PSBTv2 hardening surface, validate the type/range and raise PSBTError with a clear message instead.
    src/embit/psbtview.py:427
  • The _scan_scope_values() docstring claims that scope_cls._validate_key() performs “the same checks the full scope parser applies”, but _validate_key() only checks key shape (e.g., PSBTv2 keyed-field rule) and does not validate per-key value formats like read_from()/read_value() do. This is misleading for callers relying on it for full validation.
  • Files reviewed: 14/14 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@notTanveer

Copy link
Copy Markdown
Collaborator

tested this at 12deedc and found a few things, fixes are here as separate commits on top of your branch: https://github.com/odudex/embit/compare/feat/psbt-v2...notTanveer:embit:pr-152?expand=1

each commit has its own tests, so cherry-pick whatever you like.

notTanveer and others added 12 commits October 16, 2026 08:40
Co-authored-by: odudex <5049478+odudex@users.noreply.github.com>
…SBTView

Extract next_tx_modifiable() in psbt.py. PSBT and PSBTView each carried a
byte-identical copy of the BIP-370 sighash -> flag rules, so a later
correction to one would silently leave the constrained-device path wrong.
Going through a pure function also removes a None &= -2 TypeError on a
PSBTView built with version=2 and no global_kvs, whose flags setter no-ops.

Drop the trailing tx_modifiable_flags block in PSBTView.sign_with(). Every
path in sign_input() that adds a signature already updates the flags, so the
block could only fire after signing nothing, stamping TX_MODIFIABLE=0 onto a
PSBT this pass never touched. PSBT.sign_with() does not do this.

(PSBT-only half of 59908f5 on feat/silent-payments-V2.)
… rebase

Rebasing feat/silent-payments-V2 onto upstream/master's PSBTv2 hardening
series (eb6104f) merged in upstream's declarative _validate_global_key,
V2_FIELDS/_validate_key scope checks, and _read_global_value eager-shape
validation, but left _validate_global_fields (canonical count-field
encoding, fixed-length checks, required-field presence) unwired in both
PSBT.read_from and PSBTView.view. Malformed PSBT_GLOBAL_INPUT_COUNT /
PSBT_GLOBAL_OUTPUT_COUNT values crashed with RuntimeError/ValueError
instead of PSBTError, non-canonical compact encodings went unrejected,
and PSBTView accepted a v2 PSBT missing PSBT_GLOBAL_TX_VERSION until
something later touched the lazy .tx_version property.

Wire _validate_global_fields into both parse paths, replacing the
now-duplicate manual v0/v2 field-exclusion checks it subsumes. All 25
of upstream's tests/tests/test_psbt_versions.py cases now pass.
…signed byte helpers

Parsing tx_version unsigned closes the same
Transaction(version=tx_version) overflow the helpers were added for.
…oot_keypath

Subclasses that tweak the key differently (BIP-376 silent payment spends)
encode and store the BIP-341 key-path signature identically, so share
that step instead of duplicating it.

(PSBT-only half of 533cd85 on feat/silent-payments-V2.)
…like PSBT

PSBTView.vin()/vout() looked up PSBT_IN_PREVIOUS_TXID, OUTPUT_INDEX, SEQUENCE,
PSBT_OUT_AMOUNT and SCRIPT with seek_to_value(), which matches by key prefix
and returns the first hit. A keyless BIP-370 field carrying keydata
(0x0e||xx) or a duplicate of the field was therefore accepted, and its value
fed hashPrevouts/hashSequence/hashOutputs. PSBT.parse rejects both. On a
two-input PSBT with the bogus key in input 1, PSBTView.sign_with() computed
input 0's sighash over the wrong prevout and wrote its signature to the sig
stream before input 1 was ever parsed.

vin()/vout() now go through _scan_scope_values(), which matches keys
exactly, rejects duplicates of the requested keys and runs the scope class's
_validate_key() on every key it skips over, so a malformed map is rejected
while hashing, before any signature is emitted. The same walk serves
_determine_locktime_v2().

input()/output() now run the PSBT_CLS._validate_v2_input/_output hooks
(PSET overrides them), so a v2 map missing a required field fails in
input(i) instead of only in vin(i). tx_version and the fallback locktime
are read from the materialised _global_kvs instead of re-walking the
global map with a prefix match.
…ence

PSET.blinded_tx, which PSET.sighash_segwit/sighash_legacy hash against,
still built the transaction with the raw fallback locktime while PSBT.tx
went through determine_locktime(). A PSETv2 with a required height or time
locktime therefore signed one transaction and extracted another, and
PSET.sign_with() disagreed with PSETView.sign_with(), which already used
the derived value.

LInputScope.vin/blinded_vin also rewrote sequence == 0 to 0xffffffff via
`self.sequence or 0xFFFFFFFF`, the bug the base InputScope.vin fixed on
this branch, so the same zero-sequence input hashed and serialized
differently depending on which scope class held it.
Truncated or corrupted input reached callers as RuntimeError from
compact.read_from, ValueError/TransactionError/ECError/HDError from the
nested transaction, script and key parsers, IndexError from a one-byte
PSBT_GLOBAL_XPUB key, OverflowError from an absurd length reaching
BytesIO.read/seek, and the base-class EmbitError for trailing bytes. Only
read_string() wrapped its length read; the prescan added on this branch,
PSBTView's deferred-global and scope walks, and GlobalTransactionView all
called compact.read_from directly.

- compact.read_from now fails on a truncated multi-byte prefix instead of
  returning int.from_bytes(b"") == 0, and rejects non-canonical encodings
  the way Bitcoin Core's ReadCompactSize does. A separator written as
  fd0000 was accepted as an empty key by both parsers, and PSBTView's
  offset bookkeeping assumes canonical lengths.
- read_compact/read_exact/skip_exact wrap the low-level reads; every scope
  and global read goes through them and through _read_value_checked(),
  which reports nested parser errors as PSBTError naming the key.
- PSBT.parse raises PSBTError on trailing bytes; PSBT_GLOBAL_XPUB keys are
  length-checked in the shared _validate_global_key; PSBTView bounds the
  global tx input/output counts by the declared tx length.
- Serialization of out-of-range v2 fields (tx_modifiable_flags=300,
  locktime=2**32, negative sequence, 31-byte txid, amount >= 2**63)
  raises PSBTError instead of OverflowError/ValueError.

Tests truncate and bit-flip three sample PSBTs at every byte and assert
both parsers only ever raise PSBTError.
…wing

PSBT.parse fails on anything after the last output map (as Bitcoin Core's
DecodeRawPSBT does) and on a PSBT_GLOBAL_INPUT_COUNT larger than the maps
present, but PSBTView accepted both silently: it only ever walked as far as
the scope being asked for, so an extra map, trailing garbage or a missing
map surfaced late, in whatever accessor happened to reach it, or not at all.

PSBTView.view() now walks all the input and output maps once, reading only
the key and value lengths, and fails if the maps run out or bytes remain.
Every proper prefix of a PSBT now fails in view() instead of in a later
lazy read.
…igs and errors

Several places where the two implementations disagreed on the same PSBT:

- PSBTView.locktime returned the BIP-370 derived locktime while PSBT.locktime
  is the raw PSBT_GLOBAL_FALLBACK_LOCKTIME field, so test_psbtview's equality
  assertion only held for vectors without per-input requirements. PSBTView
  now mirrors PSBT: .locktime is the raw field and determine_locktime() is
  the transaction locktime, which is what every sighash uses.
- PSBTView wrote the v2 global map in sorted key order, PSBT in xpubs /
  BIP-370 fields / unknowns order, so a PSBT with an unknown global field
  round-tripped to different bytes through each. PSBTView now uses PSBT's
  order and keeps the source order of unknowns with an OrderedDict
  (MicroPython dicts are unordered).
- PSBTView's taproot key-path signer filled only final_scriptwitness while
  PSBT also stores PSBT_IN_TAP_KEY_SIG, so the signed PSBTs differed. Both
  now fill both fields and the sig stream carries 0x13 as well.
- PSBTScope(unknown={...}) could not carry PSBTv2 fields because
  parse_unknowns() never passed a version, and any unknown key raised
  "Duplicated key" because the entry was never popped before read_value()
  re-added it. Scopes take version= and consume the dict as they parse.
- An input without a utxo raised AttributeError from is_taproot when
  signing; it is now PSBTError("Missing previous utxo on input N") from
  sighash() in both PSBT and PSBTView, matching PSBT.fee().
- Documented that PSBTView.sign_with() keeps the TX_MODIFIABLE update on the
  view, not in the sig stream, so the same view must write the result.

Parity tests sign taproot key-path and script-path PSBTs, in v0 and v2 and
with KEEP_ALL and PARTIAL compression, through PSBT and PSBTView and compare
the resulting bytes.
A declared length that did not match the transaction used to shift the key/value framing of the rest of the input map, so PSBT.parse and PSBTView disagreed on the same bytes. The value is already parsed through _BoundedReader; add regression coverage for a short and a long length in both the streaming (compress) and the full parse.
jdlcdl
jdlcdl previously approved these changes Nov 5, 2026
@jdlcdl

jdlcdl commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

ACK at 9b33a67

tl;dr: I believe that this should be merged into embit "master". It has so many important fixes and is mostly backwards compatible for krux and seedsigner with no changes at all -- after much testing.

The size and number of commits in this PR was daunting to review. I visually read code changes many times before resorting to looking one commit at a time (which was best for me). I also read top-down all of the new tests (also a good place to start).

On devices, I have tested this PR on:

  • Krux using Odudex's chore/bump-embit-psbt-fixes (krux "develop" as of Nov 1, 2026)
  • seedsigner using "dev" as of Nov 1 2026 (last pr merged 1047)

I used testqrs.com (keith's seedsigner psbt testing site) to randomly test dozens of normal PSBTs that are expected to be parsed and signed (legacy single/multi, native segwit single/multi, nested segwit single/multi, taproot).

Testqrs also has many test-cases for particular seedsigner PRs around hardening PSBTs since Aug 2026, and the majority of these test-cases are problematic (psbt is: broken, inconsistent, has extra/missing data, or is an attempted attack, etc). For the most part, krux behaves well (correctly classifying what will be signed), and seedsigner behaves EXACTLY as expected in all cases (except for one: 6th test in pr-995 "output index doesn't exist (legacy)") for recent seedsigner PRs: 995, 1013, 1032, 1041, 1044, 1046, 1047. (1040 and 1060 not tested with this version of embit, but not expecting problems).

@newtonick

Copy link
Copy Markdown

tACK 9b33a67 (partial review)

First, I appreciate all the work put into this PR and Embit as a whole over the last many months.

were-not-worthy-waynes-world gif

I plan to keep reviewing more thoroughly, but please don't wait on me to merge. So far I've spent a few hours reading BIP-370, the PR description and comments, and the code, and running tests. I focused on the library changes (very little time on the unit test code) and on how the PR affects SeedSigner, good and bad. I'm looking forward to getting these changes into a SeedSigner future release.

I used Claude Opus 5.5 to build a harness that runs PSBTs through SeedSigner's signing path (PSBTParser → sign_with → trim) on embit 0.8.0, master and this PR. The PSBTs come from multiple sources including Bitcoin Core's rpc_psbt.json, btc-datagen and more. It then used Bitcoin Core v32.1 to check the resulting signatures.

  • For well-formed PSBTs, nothing SeedSigner displays changes: fees, amounts, addresses and change are identical. The behavior all appear to be good fixes.
  • No signature SeedSigner produced failed Core's check.
  • A couple of malformed PSBTs now fail in PSBT.parse instead of reaching SeedSigner's own error screens. One is pr-995's out-of-range output index, which @jdlcdl also noted. That's ours to handle in SeedSigner, not a problem with this PR.

I have no concerns with merging. I did not test on MicroPython, Liquid/PSET or PSBTView.

@1ma 1ma mentioned this pull request Nov 6, 2026
5 of 12 tasks

@qlrd qlrd left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK 9b33a67

Made integration tests agains core 32.1 and 32.0 with krux pinned to this PR. Shared the report itself for @jdlcdl and @notTanveer to check the test code itself before share here, since the conditions that i made was little "smashed". But basically did the following flow: create v2, parse, 16 malformed cases, sighash refusal, sign, broadcast.

Congratulations on everyone that worked on that!

Comment thread src/embit/psbtview.py
Comment on lines +1202 to +1204
if inp.taproot_key_sig:
ser_string(sig_stream, b"\x13")
ser_string(sig_stream, inp.taproot_key_sig)

@miketlk miketlk Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid duplicating existing Taproot key-path signatures

When an input already contains taproot_key_sig and sign_with() receives a descriptor with multiple private leaf keys, each sign_input() call reloads the original input and appends its existing 0x13 field to the same signature map. Consequently, write_to(..., extra_input_streams=[sig_stream]) raises PSBTError: Duplicated key, even though the generated leaf signatures are valid.

Reproduced with no initial final witness or leaf signatures, on both PSBTv0 and PSBTv2 across all three compression modes. The same inputs merge successfully on master; removing the existing key-path signature or using only one signing key also succeeds.

Please emit 0x13 only when this invocation actually generates a key-path signature, preserving the existing signature through the merge. Checking the total signature count would not suffice because leaf signatures also increment it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6b7b281. Reproduced: with an existing taproot_key_sig and a multi-key descriptor, each sign_input() call re-emitted 0x13 and write_to() failed with Duplicated key.

The same thing happened with three other fields, and those fail on master too: an existing final_scriptwitness (0x08), taproot leaf signatures (0x14) and partial signatures (0x02). So instead of special-casing 0x13, sign_input() now writes only the signatures the call itself makes, for all four fields. Existing signatures still reach the output through the merge in write_to(), so the written PSBT is unchanged.

One visible change: the signature stream no longer repeats signatures the input already had. Noted in the changelog and the sign_with() docstring.

Tests cover each existing-signature state with a multi-key descriptor on PSBTv0 and PSBTv2 in all three compression modes, plus a 2-of-3 multisig with one cosigner already signed.

Comment thread tests/tests/test_psbtV2.py Outdated
if delta > 0:
# pad so the value itself is still present, the framing is off
bad += b"\x00" * delta
raw = raw.replace(good, b"\x01" + bad, 1)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserve the non-witness UTXO key in the fixture

good includes both the key-length byte (01) and the non-witness UTXO type byte (00), but the replacement restores only 01. The parser then treats the modified value length as the field type. These cases pass because of unrelated parsing errors. All tested PSBTView cases fail before reaching input(0).

Use b"\x01\x00" + bad to preserve the key. For the padded positive-delta cases, checking for Trailing bytes in PSBT value would also verify that the test reaches the intended length validation.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with test: keep the non-witness UTXO key in the length mismatch fixture

Comment thread src/embit/psbt.py
Comment on lines +387 to +390
if other._utxo is not None:
self._utxo = other._utxo
self._txhash = other._txhash
self._utxo_vout = other._utxo_vout

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clear the stale full transaction when adopting a new cache

Updating a full scope for transaction A from a compressed scope for transaction B copies B’s prevout and cache but leaves non_witness_utxo pointing to A. With no witness_utxo, or with one matching B’s output, verify() succeeds using B’s cached hash. Serialization then writes A, so the resulting PSBT fails verification after reparsing.

When adopting the incoming cache, clear any retained full transaction whose TXID differs from the cached hash. A regression test that updates a full scope from a compressed scope, verifies it, then serializes and reparses it would cover this gap.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fix(psbt): drop a stale non_witness_utxo when update() adopts another cache

Comment thread src/embit/liquid/pset.py Outdated
Comment on lines +518 to +521
if out.script_pubkey is None:
raise PSBTError(
"PSBTv2 output %d missing required PSBT_OUT_SCRIPT (0x04)" % i
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allow omitted scripts on PSET fee outputs

The PSET specification explicitly allows L-BTC fee outputs to omit PSBT_OUT_SCRIPT. Such outputs have script_pubkey = None, so this check rejects them during parsing and through PSETView.output().

Removing only the empty script field from the existing fee output fixture reproduces the rejection. An earlier revision parses and reserializes the same bytes successfully. An explicitly empty script still passes, but producers using the permitted omission can no longer round-trip their PSETs.

Please account for this exception in PSET validation and add a regression test for an omitted fee output script.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fix(pset): accept PSETv2 fee outputs that omit PSBT_OUT_SCRIPT

Comment thread src/embit/psbt.py Outdated
try:
# BIP-174: the unsigned tx is serialized without witness data,
# so a leading zero is an empty input list, not a segwit marker
tx_for_v0 = cls.TX_CLS.parse(tx_bytes, segwit=False)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global transaction parsing aborts the MicroPython suite

With segwit=False, the existing bit-flip test reaches an unbounded script read after a mutation changes the input count from one to zero. On the local MicroPython 1.12 build, cd tests && micropython ./run_tests.py aborts at test_parsing.py:448 with MemoryError, requesting 4,125,086,467 bytes. Neither _PARSE_ERRORS nor the test catches it.

The corpus passes immediately before e5e6996 and fails afterward, so this parsing change exposes the older allocation weakness. Adding the test did not cause the failure.

Please bound nested reads by the available transaction bytes, so this malformed input raises PSBTError before attempting the allocation, while preserving parsing without witnesses.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fix: bound nested reads of in-memory PSBT values by the value length

MicroPython allocates the requested size before reading, so a script or witness item length from corrupted bytes raised MemoryError instead of PSBTError. The global transaction, witness UTXO and final scriptwitness now parse through _BoundedReader.
The PSET spec makes the script optional for L-BTC fee outputs. An explicit output without a script now parses, serializes back without the field, and gets the empty script in the transaction, so PSET and PSETView sign it like an explicitly empty one.
… cache

Updating a full input scope from a compressed scope for a different previous transaction kept the old full transaction next to the new cached hash, so verify() passed on the cache while serialization wrote the stale transaction.
The replacement dropped the key type byte, so every case failed on unrelated framing errors and the view never reached input(0). The expected error is now pinned for each case.
sign_with() calls sign_input() once per private key of a descriptor, and each call wrote the input's existing key-path signature, final witness, leaf signatures and partial signatures again, so write_to() failed with a duplicated key. The stream now holds what the call made; existing signatures still come through the merge in write_to().
Copilot AI lite review requested due to automatic review settings November 11, 2026 13:35
@odudex

odudex commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

Thank you @miketlk for reviewing!
Issues were all confirmed and should be addressed now. Please validate fixes when possible.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because the two PSBT serialization paths handle an explicit version-zero field differently, creating a moderate round-trip and byte-consistency risk.

No open issues 🎉

1 resolved since last review

Review effort: Lite

@jdlcdl

jdlcdl commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

ACK at 6b7b281 with intention to merge all 44 commits into "hardened" (based on master at 2b375a3)

@kkdao

kkdao commented Nov 14, 2026

Copy link
Copy Markdown

small memory note on the PSBTv0 branch of PSBT.read_from, seen on hardened (d8b18e2, src/embit/psbt.py around line 1444). it now collects the parsed scopes in temp_inputs and temp_outputs while the placeholder scopes built from the global transaction are still alive. master replaces them one at a time (psbt.inputs[i] = ...), so there every input and output exists once, here twice until the loop ends.

peak memory during read_from, measured with tracemalloc on CPython: a 50 input p2wpkh PSBT goes from 191 kB on eb6104f (what Krux 26.09.0 ships) to 227 kB, and a 150 input one from 562 kB to 670 kB. assigning in place again gives 178 kB and 520 kB, and the five psbt test files still pass (334 tests).

not a blocker: a 50 input PSBT still loads and signs on a TZT with this branch. it only costs headroom on the K210, where the heap is 1.5 MB.

         else:
-            temp_inputs = []
             for i in range(len(psbt.inputs)):
-                temp_inputs.append(
-                    cls.PSBTIN_CLS.read_from(
-                        stream,
-                        compress=compress,
-                        vin=psbt.inputs[i].vin,
-                        version=version,
-                    )
+                psbt.inputs[i] = cls.PSBTIN_CLS.read_from(
+                    stream,
+                    compress=compress,
+                    vin=psbt.inputs[i].vin,
+                    version=version,
                 )
-            psbt.inputs = temp_inputs

and the same for the outputs loop below it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

8 participants