Repository navigation
Show every OP_RETURN output, what it burns, and oversized payloads - #1043
Chaitanya-Keyal wants to merge 4 commits into
Conversation
2b37b23 to
5ad487f
Compare
5ad487f to
54605d0
Compare
a771b27 to
a103504
Compare
68075b7 to
0c42eaa
Compare
2393c45 to
f0b5a92
Compare
|
@Chaitanya-Keyal PR #1047 was merged and created merge conflicts. Can you rebase when you have time. |
f0b5a92 to
4ed5547
Compare
|
I don't really love the really small text under the OP_RETURN label. I would love to get @easyuxd's perspective if he has the time. An Op_RETURN that burns sats looks like this. Really hard to read on a 240x240 display. Another possible solution would be a Caution screen like is used for High Fees. Here is the High Fee caution screen: I'm thinking for any OP_RETURN that burns sats a dedicated "Caution" ❗10,000 Sats and a message like " OP_RETURN( The burning fees stands out to me since it has monetary impact more than the # of bytes displayed or truncated. |
|
I agree that it'd be nice to increase the text size, especially for grey text which will be especially hard to read. Will the subheads fit on 2 lines so it can support the larger font, like: 300 bytes I like Nick's idea to offload it to a messaging screen, especially for a warning. The trade-off is more clicks. |
4ed5547 to
6abe705
Compare
|
Sorry for the force-push noise here. I had a small rework of this PR going locally, but my last push somehow only had the rebase, not the rework. It's all pushed now. What changed since the last push (PR description also has updated screenshots):
I put the Caution screen right before the output that burns, as @newtonick described. If it reads better next to the high fee and no change warnings after the overview, I can move it there and make it one warning for the total. About the small subhead text: The size and page number only show for payloads that don't fit on one screen, and those are rare imo. A bigger two-line subhead, as @easyuxd suggested, does fit, but it costs a line of payload on every page, so I feel like it might be better to leave it as is. I agree though that burned sats are different because they lose funds, and deserve the dedicated warning screen. |
|
Overview flow diagram: more rows than the chart can hold On a 240x240 screen, the flow diagram's right-hand column fits about six rows. At seven, the rows touch and the bottom row is clipped. At eight or more, the first and last rows are cut off and the labels overlap. The column's rows:
So for the same transaction, this PR draws up to two more rows than
And 1 recipient, 2 self-transfers, 3 OP_RETURN outputs (the second one burns sats), and 1 change output:
Suggested fix: show all the OP_RETURN outputs in one "OP_RETURN" row, with the warning mark if any of them burns sats.
@staticmethod
def op_return_row(op_return_amounts: list) -> str:
"""
Represent OP_RETURN outputs as a single row. Even if there is more than one
in the transaction, we have to prioritize conserving vertical space.
Note: an option to display a plural form ("OP_RETURNs") creates more
translation complications than it's worth.
"""
# TRANSLATOR_NOTE: Technical term, should probably NOT be translated in most languages
row = _("OP_RETURN")
for amount in op_return_amounts:
if amount > 0:
row += PSBTOverviewScreen.WARNING_MARK
break
return rowAt the call site in if self.op_return_amounts:
destination_column.append(PSBTOverviewScreen.op_return_row(self.op_return_amounts))The "suggested" screens also include the column width change from the separate comment, which lets the addresses widen to match "OP_RETURN (!)". With this change applied, the two tests in Out of scope here: change rows are never collapsed, so |
The OP_RETURN screenshot fixture simplifies the multisig psbt down to its last output. Dropping the other outputs turned their ~9.3 BTC into fee, which has tripped the high fee warning since SeedSigner#722, so the overview screenshot showed a "fee (!)" mark that has nothing to do with OP_RETURN. Fold the dropped value into the kept output instead, so the fee stays what the base psbt paid.
The payload was drawn into one unbounded TextArea. TextArea reflows the whole string before drawing any of it and only logs a warning when the result does not fit, so a large payload ran off the bottom of the screen with nothing to say so, and the reflow cost is quadratic: 1 KB takes 83ms, 4 KB 1.2s and 16 KB 15.8s on a development machine, with the device slower again. Since Bitcoin Core v30 raised the default -datacarriersize to 100,000 bytes, and consensus never limited OP_RETURN data at all, that is a size a signer can now be handed. The payload is paged instead, at a fixed number of characters or bytes per page, and the screen always reports the payload's real size so a truncated one cannot be mistaken for the whole of it. The page size is fixed rather than measured from the rendered screen because the View decides how many pages there are and has to reach the same answer under the FlowTests, where the renderer is mocked out. Whether the payload is human-readable is decided once for the whole payload rather than per page, so that slicing cannot cut a multi-byte character in half and flip a page in the middle of readable text over to hex. A text page ends after its last space or line break, so no word is split across two pages. A payload that isn't valid UTF-8 is shown as hex, as before. A payload that needs more than one page gets a line with its size and the page number. On a truncated payload that line is in the warning color, and the last page says how many bytes were not shown instead, because otherwise the end of a truncated payload looks exactly like the end of a complete one. It is its own short line, like "raw hex data". Hex fills the page from the top in small lines, so a hex page gives up a line of hex to it; text is drawn larger and centered in the space left, which has room for it. Unit tests for what fits on a page, where truncation starts, word breaks, text versus hex, and the bytes-not-shown count. Screenshots for a paged payload and one too large to page through in full. The screenshot generator now builds its OP_RETURN script with op_return_script, since its own push encoding stopped at OP_PUSHDATA1.
Only the last OP_RETURN survived the parse. op_return_data was a single `bytes` that the output loop overwrote, so a transaction carrying more than one showed only the final payload. Several have always been consensus-valid; the one-per-transaction limit was Bitcoin Core relay policy and v30 dropped it. It is now a list, one entry per output in output order. Each output gets its own screens, titled "OP_RETURN (#2)" and so on when there is more than one. "Next" moves through the pages of one output and then on to the next output. The overview keeps its single OP_RETURN row, which now stands for every OP_RETURN output. The routing into those screens tested the payload's truthiness rather than whether the output existed, so a bare OP_RETURN, one that pushes nothing, was skipped entirely. That matters more than it sounds, since a bare OP_RETURN can still carry value. It now gets its own screen, which says "(no data)" rather than leaving a blank panel that reads as the screen having failed. A transaction whose only outputs are OP_RETURNs never reached them either: with no recipient, the math screen went straight to the change screen, which then crashed on the change output that wasn't there. It now goes to the OP_RETURN screens. The parser tests now expect the list, and FlowTests cover the routing, which unit tests cannot reach: every page of every OP_RETURN output, a bare OP_RETURN, and a transaction with nothing but an OP_RETURN. Screenshot for a bare OP_RETURN.
Sats sent to an OP_RETURN were counted nowhere. They belong to neither the spend nor the change, and embit's fee() is inputs minus ALL outputs, so the fee stayed correct while `inputs = spend + change + fee` quietly stopped holding and the burn never appeared on screen at all. op_return_amounts now runs parallel to op_return_data the way destination_amounts runs parallel to destination_addresses, with op_return_amount as their total, and the math screen gains a "burned" line so the arithmetic on screen reconciles against the inputs again. That line shows only when something was actually burned: nearly every OP_RETURN carries no value, and a row reading "0 burned" is noise on the one screen whose job is to make the arithmetic legible. A burn is also a warning in the parser's warnings list, shown by PSBTWarningView ahead of the high fee warning: one Caution screen with the total burned as its headline. Value destroyed by an OP_RETURN is marked the way an excessive fee is: the dire warning color and a "(!)" on the overview's OP_RETURN row and on the math screen's burned line, and a "burns N sats" line above the payload on every page of that output. A hex page carrying that line has one line less for the payload, so it is paged against a smaller budget; text has room for it. The overview still draws a single OP_RETURN row however many outputs there are, since the flow diagram has no room for more. "OP_RETURN (!)" is wider than the column the addresses were sized for, so the fixed labels are now measured first and the addresses widen to match the widest of them. Tests for the accounting: several OP_RETURN outputs, each reported as its own entry with its own value, with the burned sats outside spend and change so the totals still add up to the inputs. Also which output, page and amount each screen is given, when the overview row is marked, and a FlowTest that the burn warning comes before the high fee warning. Screenshots for a transaction with three OP_RETURN outputs where only the second burns sats: the burn warning, that output's page, the math screen with its burned line, and the overview with the marked row. The screenshot generator's OP_RETURN outputs can now carry a value, funded out of an existing output so that the outputs cannot exceed the inputs.
6abe705 to
3bf880c
Compare
|
I've rebased this PR onto latest Other than Keith's suggestions, the following changes were also made:
Also updated kdmukAI-bot/btc-datagen#2 for testqrs.com. |
|
ACK and reviewed 3bf880c. Hands on testing complete. |
btc-datagen PR has been merged. |
|
ACK at 3bf880c on a pi0 via raspi-dev build. Looks and feels great for different types of data, no problems going back halfway thru to review, alerted when burning sats. Nice work. Great tests too! made it a breeze. |
|
Tested 3bf880c independently. I had written a fix help from Mythos 5.2 for the same three issues before finding this PR (per-output list, burn accounting, the OP_RETURN-only crash), and the routing here matches what I arrived at.
One non-blocking observation: when a PSBT's only non-change output is a burn, the overview headline still follows the self-transfer rule, so it shows the change amount, or "0 tSats" when there is no change (screenshot). The (!) row and the Caution screen make the burn hard to miss, so this may be fine as is or a follow-up.
|
|
Thanks for the reviews and testing, everyone! @knorrium, the overview headline never includes burned sats, in any transaction:
So a burn-only transaction with no change shows 0. The fee is never in it either. Putting burned sats in the headline would change what that number means, from "sent to recipients" to "leaving this wallet", and then we'd also have to decide whether the fee belongs in it. That feels like its own discussion, so I'd leave it out of this PR, but I'm happy to follow up in whichever direction is preferred, and I'd like to hear others' opinions on it too. |











Description
Problem or Issue being addressed
#1042 fixed reading an OP_RETURN payload. This fixes what we do with it afterwards.
Only the last OP_RETURN is kept.
op_return_datais a singlebytesvalue and the output loop overwrites it, so a transaction with more than one OP_RETURN only ever shows the last payload. Multiple OP_RETURNs have always been valid by consensus, and Bitcoin Core v30 dropped the relay rule that allowed only one per transaction.Sats burned on an OP_RETURN are never shown. They are added to neither the spend total nor the change total. embit's
fee()is inputs minus all outputs, so the fee is still correct, butinputs = spend + change + feeno longer adds up and the burned amount appears nowhere on screen.The payload is drawn with no size limit.
TextAreaworks out the line breaks for the whole payload before drawing any of it, and only logs a warning if the result doesn't fit, so a large payload runs off the bottom of the screen with nothing to say so. Finding those line breaks is also slow on long text, sincereflow_text_for_widthbinary-searches every break and re-measures a growing run of words on each probe:That's on a laptop; the Pi is slower. Bitcoin Core v30 raised the default
-datacarriersizeto 100,000 bytes, and consensus never capped OP_RETURN data at all, so payloads this large can now arrive.Two more turned up while testing:
OP_RETURNgets a row in the overview's flow diagram, but the routing then skips its screen, because it checks whether the payload is non-empty rather than whether the output exists. A bare OP_RETURN can still burn sats.Solution
op_return_databecomes a list with one entry per output, in output order.op_return_amountsruns alongside it andop_return_amountis their total, the same shapedestination_addresses,destination_amountsandspend_amountalready have.PSBTOpReturnViewsteps through the outputs the wayPSBTAddressDetailsViewsteps through recipients, titledOP_RETURN (#2)and so on when there is more than one, and pages through each payload with the usual "Next".OP_RETURNrow for all of them, so the flow diagram has the same number of rows as ondev.burnedline so the numbers on screen add up to the inputs again. It only appears when sats were actually burned.warningslist ([security] Check change derivation paths against the inputs #1073), shown byPSBTWarningViewahead of the high fee warning: one Caution screen with the total burned as the headline. They're also marked the way an excessive fee is in [Feature] Warning Screen for High Tx Fees #722, with the dire warning color and a(!)on the overview's OP_RETURN row and the math screen's burned line, plus aburns N satsline on every page of an output that burns.OP_RETURN (!)is wider than the column the addresses were sized for, so the overview now measures its fixed labels first and lets the addresses widen to match.(no data)instead of leaving the screen blank, which reads as a rendering bug.New Screenshots
Note
This PR also fixes the existing
PSBTOverviewView_op_returnscreenshot, which picked up a strayfee (!)mark after #722 because its fixture was paying a ~9.3 BTC fee.Also, I'm not sure all seven new screenshots need to stay in the generator. If some aren't worth keeping, I'm happy to drop them.
This pull request is categorized as a:
Checklist
I ran
pytestlocallyI included screenshots of any new or modified screens
I added or updated tests
I tested this PR hands-on on the following platform(s):
Tested with my fork of Keith's testqrs.com (kdmukAI-bot/btc-datagen#2)
pi02w-devbuild)I have reviewed these notes: