Repository navigation
Parse the OP_RETURN push opcode instead of assuming OP_PUSHDATA1 - #1042
Conversation
a0723c6 to
1b2aaab
Compare
1b2aaab to
38db4a4
Compare
9479bbe to
1ea95f1
Compare
|
@Chaitanya-Keyal PR 995 merge created conflicts. Once you resolve I’ll review and hopefully merge this PR. |
a41ec7e to
0359343
Compare
|
@newtonick I've rebased onto dev (again) and resolved the conflicts. The commits from my earlier replies are now 79e22d2 (was 5f9cc52) and 6403723 (was a41ec7e). |
The payload was read at a fixed 3-byte offset, which only holds when the script
uses OP_PUSHDATA1. Opcodes 0x01-0x4b are themselves the byte count, and that is
the minimal encoding Bitcoin Core emits, so every payload of 75 bytes or fewer
lost its first byte:
scriptPubKey 6a 28 "Chancellor on the brink of third bailout"
parsed b'hancellor on the brink of third bailout'
PSBTOpReturnView is a pre-signing review screen, so the user approved a
transaction whose OP_RETURN content was displayed wrong. When the payload is not
human-readable it renders as hex, where a one byte shift is invisible.
Read the push opcode and slice to the declared length, in a static helper so the
loop stays readable. OP_PUSHDATA2 and OP_PUSHDATA4 are handled too: Bitcoin Core
v30 raised the default -datacarriersize to 100,000 bytes, and consensus never
limited OP_RETURN data at all, so a payload past what OP_PUSHDATA1 can describe
is no longer exotic.
Several pushes after OP_RETURN are legal and are concatenated; the user is being
shown the data the transaction commits to, and the push boundaries carry no
consensus meaning. Anything that is not a data push, or a push that overruns the
script, ends the walk and the remaining bytes are surfaced verbatim rather than
dropped.
Also reset op_return_data in _parse_outputs, which only ever set it in
__init__, and compare the first byte by slice so an empty scriptPubKey does not
raise IndexError.
Fixes SeedSigner#963
Co-authored-by: ruipereira1 <159703278+ruipereira1@users.noreply.github.com>
Co-authored-by: kwsantiago <44934418+kwsantiago@users.noreply.github.com>
Every existing OP_RETURN vector hand-built OP_PUSHDATA1, which is exactly the one encoding the parser assumed, so the mis-slice never showed up. The regtest psbt behind test_parse_op_return_content carries 2b 6a 4c 28 43 68 61 6e ... and the screenshot generator concatenated OP_PUSHDATA1 by hand. Add fixture helpers that build an OP_RETURN scriptPubKey with the minimal push by default, and that can force a specific OP_PUSHDATA* so the non-minimal encodings a coordinator is free to produce are testable too. Note in the helper that the scriptPubKey and value have to be set on the OutputScope: PSBT.tx rebuilds the transaction from the scopes on every access, so assigning through psbt.tx.vout[i] is silently discarded and leaves a test asserting against the fixture it meant to replace. The new tests cover the direct-push/OP_PUSHDATA1 boundary at 75/76, OP_PUSHDATA2 and OP_PUSHDATA4, a non-minimal encoding, several pushes in one script, a bare OP_RETURN, an empty scriptPubKey, a truncated length field, an over-long declared length, and a leading opcode that is not a data push. Co-authored-by: ruipereira1 <159703278+ruipereira1@users.noreply.github.com> Co-authored-by: kwsantiago <44934418+kwsantiago@users.noreply.github.com>
The generator concatenated OP_PUSHDATA1 unconditionally, so its 49-byte payload was encoded a way Bitcoin Core would never produce. Push directly at 75 bytes or fewer and via OP_PUSHDATA1 above that, matching the parser. The rendered screenshots are unchanged; the vector behind them is no longer wrong. Co-authored-by: kwsantiago <44934418+kwsantiago@users.noreply.github.com>
`op_return_script`'s `push_opcode` and `create_op_return_psbt`'s `change_amount` and `fee_amount` were never used by any test. The non-minimal encoding cases build their bytes by hand and call `_parse_op_return_payload` directly.
Bitcoin Core treats an OP_RETURN output as nonstandard unless everything after the opcode is a push. The parser used to display whatever was left when a push ran past the end of the script, or when it met an opcode that is not a push. It now raises PSBTMalformedOpReturnError instead, and the user gets the same plain "Transaction Problem" warning used for other malformed psbts before discarding it. OP_0, OP_1NEGATE, OP_RESERVED, and OP_1 through OP_16 still count as pushes to Core, so they are kept and the walk carries on past them rather than stopping there.
0359343 to
6403723
Compare
|
ACK 6403723 One minor note (not a blocker): my clankers keep reporting this to me as a “finding,” but I disagree. In the rare case where a transaction uses an OP_RETURN script with special instructions like OP_0 or OP1–OP16, the parser can report the instruction byte as the payload ... e.g., 6a00 (OP_0, which pushes nothing) comes back as 00 (to be displayed as data). This would be the same as a script that pushes a real 0x00 byte. Since normal OP_RETURNs are parsed correctly, and this doesn’t affect funds or where they go ... I don’t think it’s worth addressing here. |
|
I agree this would be a (very) rare edge case. If you think it's worth it, I can do a small follow-up that shows what OP_0 and OP_1–OP_16 actually push (empty for OP_0, 01–10 for OP_1–OP_16) instead of the opcode byte |
I don't think it's worth it personally. |
|
post merge ACK I thought that this PR had what looks like a more strict interpretation of compact-size integers (when pushing op_return data), but looking again it's not the case. Maybe related, reading compact size integers in embit was too tolerant in the past (tolerating values that could have been more compact). The enormous embit PR-152, which has been merged into embit "hardened" does address stricter interpretation of compact-size integers. |
Closes #963 and supersedes #965 and #1003. Both fix the same bug and can be closed once this lands. @ruipereira1 and @kwsantiago have been added as co-authors on the relevant commits.
Description
Problem or Issue being addressed
PSBTParser._parse_outputs()read the OP_RETURN payload at a fixed offset:That comment is only true for payloads of 76 bytes and up. Script push opcodes are variable width:
0x01-0x4b)OP_PUSHDATA1OP_PUSHDATA2OP_PUSHDATA4Bitcoin Core emits the shortest encoding that fits, so most real OP_RETURNs are a direct push and
[3:]eats their first byte:Why the test suite did not catch it
Every existing fixture hard-coded
OP_PUSHDATA1, the one encoding the parser assumed:tests/screenshot_generator/generator.py:85-89concatenated it unconditionally. Its text payload is 50 bytes, which Core would push directly, so the screenshot was right for the wrong reason. Its other payload is 80 bytes, whereOP_PUSHDATA1really is minimal. No fixture ever produced a direct push.test_parse_op_return_contentwas built the same way:2b 6a 4c 28 43 68 61 6e ....Solution
Read the push opcode and slice to the length it declares, in a static
PSBTParser._parse_op_return_payload()helper instead of inline in the loop. All four encodings are handled.Two judgement calls, both explained in the docstring:
PSBTMalformedOpReturnError. The user gets the plain "Transaction Problem" warning used for other malformed psbts, and the psbt is discarded. OP_0, OP_1NEGATE and OP_1 to OP_16 still count as pushes, so they still parse.Also fixed on the same lines:
op_return_datais reset in_parse_outputs. It was only set in__init__, so re-parsing an instance kept a stale payload.b""instead of raisingIndexError.Additional Information
Left for a follow-up PR (#1043), to keep this one to a single change. None of these depend on push encoding; they are separate defects in the same area:
op_return_datais onebytesthe loop overwrites. Core v30 dropped the one-per-transaction relay limit.inputs = spend + change + feedoes not balance when sats are burned on one.PSBTOpReturnScreendraws the whole payload into one unboundedTextArea. Not just cosmetic: it reflows the entire payload before drawing anything, at quadratic cost. 1 KB takes 83ms, 4 KB takes 1.2s, 16 KB takes 15.8s on a dev machine, and the device is much slower. At Core's new default carrier size the screen is effectively unreachable.New Screenshots
The existing OP_RETURN screenshot also receives the correct bytes now.
This pull request is categorized as a:
Checklist
I ran
pytestlocallyI included screenshots of any new or modified screens
I added or updated tests
I tested this PR hands-on on the following platform(s):
Tested with my fork of Keith's testqrs.com (kdmukAI-bot/btc-datagen#2)
I have reviewed these notes: