Skip to content

backup: restore LND key family indexes - #2225

Open
GeorgeTsagk wants to merge 5 commits into
lightninglabs:mainfrom
GeorgeTsagk:codex/fix-seed-restore-key-index
Open

GeorgeTsagk wants to merge 5 commits into
lightninglabs:mainfrom
GeorgeTsagk:codex/fix-seed-restore-key-index

Conversation

@GeorgeTsagk

Copy link
Copy Markdown
Member

Description

After restoring LND from aezeed, LND has no record of the derivation indexes previously consumed by tapd’s key family. This can cause newly generated Taproot Asset addresses to reuse historical script or internal keys.

This PR adds an optional key-family high-water marker to asset wallet backups. During backup import, tapd verifies that each marker belongs to the connected LND seed and advances LND’s key-family counter through the recorded index using DeriveNextKey.

The marker is encoded as a trailing, checksummed extension, preserving compatibility:

  • New tapd versions can read backups with or without the marker.
  • Legacy backups fall back to the highest local key locator represented by their active assets.
  • Older readers continue to decode the existing backup payload.
  • Raw, compact, and optimistic backup modes use the same extension.

Recovery is monotonic and never rewinds LND’s counter. Markers belonging to another seed are ignored, and advancement is bounded to prevent excessive derivation work.

Tests

Tests cover:

  • Marker round trips for raw, compact, and optimistic backup versions.
  • Decoding legacy backups where the marker field is absent.
  • Legacy asset-locator fallback.
  • No counter rewind.
  • Wrong-seed markers.
  • Maximum advancement limits.
  • Full aezeed recovery, backup import, fresh address derivation, subsequent spending, and proof validation.
  • Marker export/import across all three backup modes.

Other

Closes #2221

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@hieblmi

hieblmi commented Jul 29, 2026

Copy link
Copy Markdown

Description

After restoring LND from aezeed, LND has no record of the derivation indexes previously consumed by tapd’s key family. This can cause newly generated Taproot Asset addresses to reuse historical script or internal keys.

Just a drive-by comment:

We face the same problem for static addresses in loop as well.

Instead of modifying the backup file we will go for a hd-wallet style recovery with index gap limits.
Constantly modifying a backup file and setting a new watermark can be tricky since you always also have to sync the
secure storage where the backup file lies.

While this functionality is not in place yet we provide a single command line tool to recover specific on-chain outputs that the client identifies manually.

@hieblmi

hieblmi commented Jul 29, 2026

Copy link
Copy Markdown

Idea: Maybe a lnd supported tapscript wallet recovery feature would be nice that scans for generic tapscript outputs on-chain during recovery, additionally to the normal wallet outputs...

@jtobin
jtobin self-requested a review July 30, 2026 17:50
@jtobin jtobin added the backup label Jul 30, 2026

@jtobin jtobin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍 👍

(A couple of nits maybe worth addressing but I don't think they block anything.)

Comment thread backup/key_index.go Outdated
// maxKeyIndexAdvance bounds the number of DeriveNextKey RPCs an import
// can trigger for one family. This protects against resource exhaustion
// from a backup with a valid-looking but unreasonable marker.
maxKeyIndexAdvance = 10_000

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Fable argued this could probably stand to be bumped up ("each address consumes two keys, so ~5k addresses exceed the bound"). Seems reasonable? Arguably it could be dropped entirely as an "attacker" would already have the seed IIUC.

Comment thread backup/encoding.go

// maxKeyFamilyMarkers bounds the number of key-family markers decoded
// from an untrusted backup.
maxKeyFamilyMarkers = 100

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Fable points out that "every tapd key lives in only one family (212) and the export writes only one marker." The generality is probably unnecessary.

@github-project-automation github-project-automation Bot moved this from 🆕 New to 👀 In review in Taproot-Assets Project Board Jul 30, 2026
@jtobin

jtobin commented Jul 30, 2026

Copy link
Copy Markdown
Member

I thought the CI failure was a flake, but looks like it's related to the changes. Looks like the test just needs a quick fix (Fable):

The mechanism: the modified testRestoreLndFromSeed wipes Bob with bob.stop(true), assuming that destroys tapd's state. But stop(true) just does os.RemoveAll(BaseDir) (itest/tapd_harness.go:773) — it deletes the tapd data directory. On the sqlite backend the database file lives inside that directory, so the wipe is real and the test passes. On the postgres backend the database lives in a postgres fixture container provisioned at harness setup (tapd_harness.go:377) and torn down only at test cleanup — stop(true) doesn't touch it. Bob restarts, reconnects to the same postgres database with every asset still in it, the backup import skips them all as already present, and the PR's own assertion require.NotZero(importResp.NumImported) at send_test.go:2240 fails with exactly the observed "Should not be zero, but was 0".

The clean fix is to stand up a genuinely fresh tapd for the restored node — a new setupTapdHarness instance (which provisions its own postgres fixture) attached to the seed-restored lnd — rather than restarting the same harness and trusting stop(true) to have wiped state.

@GeorgeTsagk

Copy link
Copy Markdown
Member Author

Adding a complementary LND PR for this which should also close lightningnetwork/lnd#8698

@GeorgeTsagk
GeorgeTsagk force-pushed the codex/fix-seed-restore-key-index branch 3 times, most recently from d1d4ccf to c3e936a Compare August 18, 2026 11:43
@jtobin
jtobin self-requested a review August 19, 2026 07:39

@jtobin jtobin left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GeorgeTsagk does this one require lightningnetwork/lnd#11095 and lightninglabs/lndclient#288 before merge after the recent updates?

(EDIT: confirmed it does out-of-band.)

@jtobin
jtobin self-requested a review August 19, 2026 07:39
@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

1 similar comment
@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

4 similar comments
@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

The seed restore test wipes Bob before restoring LND, which it did by
stopping the tapd harness. Stopping a harness only removes its base
directory though. The sqlite database file lives in there, so the wipe is
real on that backend, but a postgres database lives in a fixture that is
only torn down at test cleanup, so nothing was wiped there at all.

On postgres, Bob therefore came back up with every asset still present,
the backup import skipped all of them as already known, and the test
failed on its own NumImported assertion. We now stand up a genuinely
fresh tapd, which provisions its own database, for the restored node.
Exporting a backup called DeriveNextKey purely to learn the key family's
current index. That permanently consumed one index per export, and it
recorded the *next* key rather than the last one actually used, so the
marker was always one index ahead of the true watermark.

The index is now read through ListAccounts, and the marker key itself is
derived read-only, which leaves the wallet untouched. The seed restore
itest asserts both properties directly: that an export does not move the
key family's index, and that the recorded marker is the last index the
family actually consumed.
Recording a key in LND implies recording every key that precedes it, so
restoring a key family's index is one DeriveAndStoreKey call per family
instead of one DeriveNextKey per index. LND keeps that operation monotonic
and bounds the number of keys it derives, so the local advance loop, the
rewind guard and the maxKeyIndexAdvance limit are no longer needed here.

Requires an LND with the DeriveAndStoreKey RPC
(lightningnetwork/lnd#11095) and the lndclient wrapper for it
(lightninglabs/lndclient#288).
@GeorgeTsagk
GeorgeTsagk force-pushed the codex/fix-seed-restore-key-index branch from c3e936a to 62c0895 Compare October 22, 2026 10:31
@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

2 similar comments
@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

@litbot-9000

Copy link
Copy Markdown
Collaborator

@jtobin: review reminder

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: 👀 In review

Development

Successfully merging this pull request may close these issues.

[bug]: full recovery (seed + backup import) resets address key derivation — new addresses reuse historical script keys, breaking receives

5 participants