Repository navigation
backup: restore reissued grouped assets from the backup file - #2298
Conversation
86aa2ad to
e5d99f9
Compare
e5d99f9 to
57a0cec
Compare
|
Pushed a new round addressing the review.
|
jtobin
left a comment
There was a problem hiding this comment.
LLMs flagged that import checks only that the wallet's description of a group is internally consistent, but doesn't verify any proofs before writing the group into durable storage. I don't think this is a huge deal because the backup is considered trusted, but Opus pointed out the following:
It doesn't need an attacker. A truncated file, a half-written one, bit rot on a NAS, a file from a different wallet restored by mistake, a file from an older buggy tapd — none of those involve intent, so "trusted" doesn't cover them. And in exactly those cases the import can write group rows and then import nothing at all. Wipe and retry doesn't undo it; the rows outlive the act that created them, which is not how the rest of the import behaves — every other bad entry is skipped and leaves nothing behind.
The second point is that the contamination isn't confined to what you asked for. The witness is kept on a first-writer-wins basis, so a bad file can pre-empt the anchor witness of a group you legitimately care about and would otherwise have learned correctly from the universe later. The later correct data silently loses to the earlier wrong data, and nothing in tapd repairs it. Then your own re-exports carry it onward. That's the part I'd call a real defect rather than a consequence of trusting the file: you can be worse off than if you'd never restored.
Opus describes a fix as follows:
Accept the file's claim provisionally, commit it only once something corroborates it.
The in-memory whitelist already does the first half, and it's the part that makes the feature work — a group the file describes is accepted for the duration of that import, which is exactly enough to let a reissued leaf's proof verify. Leave it alone.
The second half is to move the database write out of the up-front pass and behind a successful per-entry import: persist a group once a leaf of it has actually verified and landed. Nothing about the format, the derivation, or the tests changes; a restore that imports nothing then leaves nothing behind, and a bad witness can no longer outrun a good one.
Probably worth making the fix, but I think not a blocker.
A group is stored from whatever proof first mentions it. A reissuance proof knows only the tweaked key and a witness, so the row ends up with the tweaked key in place of the raw key and without version or roots, and the upsert never touched those fields again. Callers that know the raw key now replace the whole row, so the group re-derives from the database regardless of the order the proofs arrived in. Two ingestion paths did not pass on what the reveal tells them. The universe ingestion kept only the raw key and tapscript root of a V1 reveal, so the group read back as V0 and no longer derived the tweaked key. The wallet-side proof import ignored the reveal altogether, so a wallet that received the group anchor, or restored it from a backup, stored an incomplete row. The genesis reported for a group was the one with the oldest witness row, so a reissuance stored before the anchor passed for the group's genesis. When the raw key is known, the anchor is now picked by re-deriving the tweaked key for each genesis minted into the group.
A group key reveal only exists on the genesis proof of the asset that created a group. Every tranche minted into the group afterwards carries the group key without a reveal, by protocol rule. A wallet that only ever received such reissued leaves therefore holds no proof a restore could learn the group from, and the import skips every one of them with "group key not known". Record the group on the backup entry instead: the genesis of the group anchor together with the version, raw key, tapscript root, custom subtree root and anchor witness of the group key. That is exactly what the wallet stores in asset_groups for a group it learned through a universe sync, and it is enough to re-derive the tweaked group key. The writer only records a group whose record actually derives the group key, a group first seen through a reissuance proof has the tweaked key stored as its raw key and is left out with a warning. The info is an odd TLV type on the entry, so older readers skip it, and it is resolved once per distinct group while collecting a backup.
Before any proof is verified, the import now reads the group info of every grouped entry, re-derives the tweaked group key from the recorded raw key and anchor genesis and only accepts a record that derives the key the entry claims. A tampered record cannot whitelist a foreign group. Accepted groups are added to the set of known group keys for the duration of the import. They are written to the wallet database, with the anchor genesis and witness, through the new optional GroupRegistrar only once a leaf of the group has verified and been imported. The file's description of a group is corroborated by that leaf's proof, whose group witness was signed by the group key. A truncated, foreign or otherwise bad file therefore leaves nothing behind, and a bad record cannot pre-empt the anchor witness of a group the wallet would learn correctly later. The restored wallet's own backup file describes the group again, so a restore does not degrade the next backup. Groups that neither the proof data nor the entries describe are now logged at warn level once, naming the group and the reason, instead of only surfacing as a verifier error per skipped asset.
…file The existing backup itests only ever restore group anchors, whose genesis proof carries a group key reveal. Cover the failing case: Bob receives a tranche minted into an existing group, is wiped and restored on a node that has no universe server to learn the group from. Without the previous two commits the import reports success with the leaf skipped and "group key not known" in the log. The test also restores a second wiped node from the restored node's own backup file and spends the restored leaf, so the fix holds across restore cycles and yields a spendable asset, not just a listed one.
The reissuance restore test minted a V0 group with a wallet key. V1 is what any group with an external key uses, and its version and custom subtree root must survive the receiver's database and the backup entry for the import to re-derive the group key. Mint the same anchor and tranche with a chantools external key and run the identical restore flow, asserting the recorded group version along the way. Without the universe ingestion fix the receiver stores the group as V0, the writer cannot re-derive it and leaves the entry without group info. A second test restores the group anchor itself from an entry without group info, the shape of a file written before groups were recorded. The import learns the group from the reveal in the anchor's proof, and storing that proof must keep the reveal's details so the restored node's own file describes the group again and a further restore from it works. Without the wallet-side import fix the second file has no group info.
57a0cec to
8178da1
Compare
|
Done. Groups are now only whitelisted for the import and written to the database after a leaf of them actually imported, so a bad file leaves nothing behind. While testing this I found the group anchor was resolved as the oldest witness row, so a reissuance stored first took its place. Fixed in the tapdb commit by picking the genesis that derives the key. Covered by TestImportBackupBadFileLeavesNoGroups, TestAssetGroupKeyRepair and a corrupted file stage in the reissuance itest. |
a661ce9
Restoring from
assets.backupskipped every reissued grouped asset withgroup key not known, while the import still reported success. A group key reveal only exists on the genesis proof of the asset that created the group. Later tranches carry the group key without a reveal, so a wallet that only holds reissued leaves has nothing in its proofs the restore can learn the group from. On a long lived wallet that is nearly every leaf, and one unknown group takes out all of its leaves.Follow up to #2277 and c98e3ad, which fixed the extraction ordering but could not add a reveal that was never in the proof. The first commit fixes two pre-existing tapdb issues this surfaced: group rows created from a reissuance were never completed, and V1 reveal details were dropped on ingestion.
asset_groupsalready holds and it re-derives the tweaked key. Odd TLV type, so older readers skip it and the format version stays the same. The writer only records a group whose record actually derives the key.The new itest
backup_file_reissuancerestores a wallet holding only a reissued leaf onto a node without a universe server in its federation, then restores a second node from that node's own file and spends the leaf. Without the fix commits it fails withNumSkipped == 1. Two more itests do the same for a V1 group minted with an external key, and restore the group anchor from an entry without group info.The import still returns success with a non-zero
NumSkipped. Whether a partial restore should be an error is left for a separate discussion.