Repository navigation
backup: keep a live encrypted asset wallet backup file on disk - #2277
Conversation
58cbdc2 to
1dc5835
Compare
There was a problem hiding this comment.
LGTM. 👍 👍
I ran this through reviews w/both Opus (multi-agent) and Sol. They each reported a bunch of stuff, but all practical concerns evaporated when I dug into points individually.
(EDIT: there's obviously some issue, presumably minor, keeping the postgres itests job from passing.)
2485cd4 to
ef60c54
Compare
jtobin
left a comment
There was a problem hiding this comment.
Came back to this one and noticed it had grown a bit. A quick pass w/Sol pointed out the following that seem worth looking into:
[High] Key-lookup failures can create permanently incomplete backups. backup/backup.go:181 discards errors from FetchInternalKeyLocator and emits a zero locator. The updater then considers the entry successfully built and only rebuilds it when its block hash changes (backup/updater.go:565). Restoration persists that incorrect locator (backup/import.go:530), potentially leaving the restored anchor unspendable. Key-lookup failures should keep the entry dirty and be retried.
[High] A failed final backup during shutdown is reported as success. backup/updater.go:327 logs any Sync failure but always returns nil. Consequently, the server’s error handling at server.go:924 is unreachable. Disk-full, permission, timeout, or proof failures can therefore leave the backup stale while a graceful shutdown appears successful.
(One nit as well: the commit series could probably be compressed somewhat without losing much information.)
Split the per-asset collection out of ExportBackup so callers that keep their own container, like the on-disk backup file updater, can build entries without encoding a full blob. The archive parameter only needs FetchProof, so narrow it to proof.Exporter.
Wrap an encoded wallet backup in a container that is encrypted with the same key lnd uses for channel.backup: XChaCha20-Poly1305 keyed by the hash of the public key at KeyFamilyBaseEncryption index 0, via lnencrypt. The key is derived through lnd's WalletKit, so a tapd connected to an lnd restored from the same seed can decrypt the file. The container carries its own magic bytes so it can be told apart from plaintext exports.
Persist the packed backup by writing a temporary sibling file, syncing it, renaming it over the main file and syncing the directory, so a crash can never leave a half written backup behind. A symlink at the path is followed so the operator's link stays and the file behind it is replaced. The parent directory is created if needed. Directory handles cannot be synced on Windows, so that step is skipped there.
The Updater keeps an encrypted compact backup of the wallet on disk and follows the wallet state, mirroring lnd's chanbackup.SubSwapper. It holds the entry set in memory keyed by asset ID, script key and anchor outpoint, subscribes to the asset store proof notifications plus the planter and chain porter events (a mint writes its proof file before the asset row, and a full value send without change imports no local proof, so the proof notifications alone miss both), and on every change reconciles against the database after a short debounce: new and re-anchored leaves get a fresh entry, spent leaves are dropped, entries only found on disk are kept unless the database knows them as spent. Unconfirmed leaves are left out until they confirm. The initial write happens right after startup and a final sync runs on shutdown, whose failure Stop returns so a stale file does not look like a clean shutdown. Transient failures are retried, and a new change pulls a pending retry forward. A file at the path that cannot be decrypted with the wallet key, or that is an optimistic (v3) export without proof data, is a startup error.
ImportAssetsFromBackup detects the encrypted container header and decrypts with the wallet key before decoding, so the on-disk backup file can be handed to the import RPC as is. Plaintext exports are handled exactly as before. Three gaps found while testing restores are fixed alongside: The import skipped an asset as already present when its proof could be fetched from the multi archiver, which falls through to the proof files on disk. A node whose database was wiped while its proofs directory survived recovered nothing. The existence check now consults the database backed store only. The backup entry carried the script key and its tweak but not the type, and the import guessed the type from the tweak. Unique Pedersen keys, as used for V2 address receives, carry a tweak yet are spent like BIP-86 keys, so every such leaf was registered as an external script path key and dropped from the default filters of ListAssets, ListBalances and coin selection. The type is now recorded as an optional TLV, and entries without it are classified from the key material: tombstones by the NUMS key, burns from the first witness, BIP-86 and unique Pedersen keys by re-deriving them, other tweaked keys as external script path. A decoded type outside the known range reads as unknown. An anchor key lookup failure used to produce an entry with a zero locator, which a restored anchor cannot be spent from. Unknown keys are recorded with the public key only and not registered on import, any other lookup failure fails the entry so it is retried. Entries missing their public keys are skipped as data errors.
Wire the backup updater into the daemon. The file lives at <datadir>/<network>/assets.backup by default, configurable with --backup.filepath and disabled with --backup.disable. A path naming a directory is rejected at config validation. Channel funding leaves are left out of the file, they belong to lnd's channel state and cannot be used by a tapd restored on its own. The updater starts after all other subsystems and stops right after them so its final sync sees the complete wallet state.
Cover the backup file end to end: written empty on startup, gains the minted leaf, swaps it for the change leaf after a send, gains the received leaf on the receiver together with a grouped asset received on a V2 address (a unique Pedersen script key), cannot be imported by a node with a different wallet key, restores a fresh tapd on the same lnd, and restores again after wiping only the database while the proofs directory stays. Both restored leaves must be visible under the default filters with the right script key types and balances, and both are spent back at the end. Adds a harness option to restart a tapd on an existing data directory.
Add the operator guide docs/backup-file.md (what the file covers and does not, where it lives, how it is protected, how to keep a copy and verify it, how to restore), a mechanics section with the container format, the update flow and its cost in docs/backup.md, link both from the safety guidelines, and add release notes.
|
Both points addressed, thanks.
Also squashed the series down to 8 commits and rebased on main. |
ef60c54 to
c45f1e7
Compare
This adds an lnd
channel.backupstyle file for the asset wallet. tapd nowkeeps
<datadir>/<network>/assets.backupin sync with the wallet stateinstead of relying on operators to run the export RPC by hand.
What the file is
ID, script key and anchor outpoint.
channel.backup. Any tapdconnected to an lnd restored from the same seed can read it.
through temp file, fsync, rename, so a copy taken at any time is complete.
How it stays up to date
backup.Updaterholds the entries in memory and subscribes to the assetstore proof notifications and the planter mint events.
new and re-anchored leaves get a fresh entry, spent leaves are dropped,
entries only found on disk are kept.
with the wallet key is a startup error, since it most likely belongs to
another seed.
Restore and compatibility
ImportAssetsFromBackupaccepts the encrypted file directly and detectsthe format from the content.
tapcli assets backup importworksunchanged.
proto change.
--backup.filepathand--backup.disable.them was still on disk. A node whose database was wiped but whose
proofs directory survived recovered nothing. The existence check now
consults the wallet database only.
from the tweak, so unique Pedersen keys (V2 address receives) came back
as external script path keys, hidden from
ListAssets,ListBalancesand coin selection. Entries now carry the type as an optional TLV, and
older backups are classified by re-deriving BIP-86 and Pedersen keys.
proof, so the spent leaf stayed in the file. The chain porter send
events are now a trigger next to the proof notifications and the mint
events.
disk-only entries the database knows as spent are pruned, type-less
tombstone and burn entries from older backups are classified correctly, a
decoded type outside the known range falls back to classification, an
optimistic (v3) export at the backup path is refused, a symlinked path is
followed, and
backup.filepathnaming a directory is a config error.Testing
add, remove, reorg, debounce, disk only retention, retries after disk and
DB failures, startup failures, sync and stop.
backup_file_updates: startup, mint, send, receive, wrong keyrejection, a grouped asset received on a V2 address, restore of a fresh
tapd on the same lnd, restore after wiping only the database, both
restored leaves visible with default filters and spent back.
disable flag, the foreign file at path case, and a full seed recovery
(second lnd restored from the same mnemonic, fresh tapd, import, spend).
Docs
docs/backup-file.md: operator guide (what it covers, how to keep a copy,how to restore).
docs/backup.md: mechanics and container format.docs/safety.md: now points at the file.Not in this PR
Writing the key family markers from #2225 into the file. Once that lands
the updater needs one
ListAccountscall per write. The two format changesare independent.