Skip to content

init-wallet: let the watch-only wallet birthday be set, avoid the full rescan from 2017 - #98

Open
Roasbeef wants to merge 6 commits into
mainfrom
init-wallet-watch-only-birthday
Open

init-wallet: let the watch-only wallet birthday be set, avoid the full rescan from 2017#98
Roasbeef wants to merge 6 commits into
mainfrom
init-wallet-watch-only-birthday

Conversation

@Roasbeef

@Roasbeef Roasbeef commented Jul 24, 2026

Copy link
Copy Markdown
Member

In this PR, we fix #97: a watch-only wallet created by lndinit init-wallet --init-rpc.watch-only came out with no birthday, so lnd fell back to the
aezeed "Bitcoin Days Genesis" epoch of 2017-08-24 and rescanned the chain from
there. On mainnet today that's a walk of ~478k blocks and several hours parked at
Wallet is not synced to height <tip> yet, even for a brand new gateway that has
zero history to find (for 0 addrs, 0 outpoints in the logs, which is the tell).

The RPC could always express this, WatchOnly.master_key_birthday_timestamp has
been there since watch-only import landed. We just never populated it, and left
the field at 0, which lnd reads as "unknown, assume 2017". So we add an
--init-rpc.watch-only-birthday flag that fills it in.

Pulling on that thread turned up three more things in the same family, all fixed
here. See each commit message for the detailed reasoning w.r.t the individual
pieces.

Why the operator has to tell us

There's nowhere else to get the birthday from. It lives in the aezeed on the
signer side, and the accounts JSON we import here comes from lncli wallet accounts list, which only exports account xpubs, their derivation paths, and the
master key fingerprint. An xpub carries no notion of when it was created:

$ lncli wallet accounts list | jq -r '[.accounts[0] | keys[]] | join(", ")'
address_type, derivation_path, extended_public_key, external_key_count,
internal_key_count, master_key_fingerprint, name, watch_only

lightningnetwork/lnd#10993 fixes that end of it, teaching ListAccounts to
return the master key birthday so it can round-trip through the accounts file.
Once that lands, lndinit can read the birthday straight out of the file and this
flag becomes an override rather than a requirement. Until then the operator is the
only one who knows.

The flag

The value can be a Unix timestamp in seconds, an RFC3339 timestamp, or a plain
YYYY-MM-DD date read as midnight UTC. The bare timestamp is what InitWallet
and lncli createwatchonly's prompt take, but a calendar date is what an
operator actually knows and what reads sanely in a helm values file, so we take
both. It slots into
init-wallet.sh
next to the flags already there:

WALLET_INIT_FLAGS="--init-rpc.watch-only --init-rpc.accounts-file=... --init-rpc.watch-only-birthday=..."

Two values get rejected rather than passed through. A birthday before the
network's genesis block can only be a units mistake, and it would rescan from
block zero, which is worse than the default we're trying to avoid. A birthday
more than a day in the future is the same mistake in the other direction (usually
milliseconds handed to a field that wants seconds), and that one is worse than
slow: lnd would start scanning at the chain tip, where it can never see funds
the wallet already holds. A day of drift is allowed so clock skew between whatever
produced the timestamp and the pod running lndinit isn't fatal, and it stays
comfortably inside the 48 hours btcwallet rewinds the birthday by on its own
before it goes looking for the birthday block. The lower bound follows the
network, so a date that's fine on mainnet is still refused on regtest.

Passing the flag where nothing can act on it (any init type other than rpc, or
rpc without --init-rpc.watch-only) is an error. Quietly ignoring it would be
the same class of bug as the one this fixes: the operator does the right thing and
still ends up rescanning from 2017.

The recovery window is the same bug wearing a different hat

InitWalletRequest.RecoveryWindow was also left implicitly at zero, which lnd
reads as "recover nothing". For the fresh gateway in #97 that's correct, so zero
stays the default. For a wallet that already has history it isn't: the birthday
gets the rescan starting at the right height, and then the wallet never derives
far enough to notice its own addresses, so the funds stay invisible for a
different reason than before. lncli createwatchonly has always prompted for
this with a default of 2500; --init-rpc.recovery-window is the same knob.

Verbose mode was eating the warning

We warn when no birthday was given, since a multi-hour rescan is otherwise
invisible until you're already in it. That warning turned out to be unreachable:
-v set the log level to error, which made every logger.Info in the tool dead
code, including the "Reading seed from file" style progress lines that only exist
to be read. Since btclog's default handler is already at info, -v was in fact
reducing verbosity, which is the opposite of what the flag says it does. It now
means info, and it no longer clobbers an explicit --debuglevel. The charts all
run lndinit -v, so this is what makes the warning land where it needs to.

And one secret leak found on the way

Turning those info logs back on for -v runs made it worth looking at what they
actually print. store-secret was marshaling the k8s secret's entire ObjectMeta
into the log line, and a secret that has ever been touched by kubectl apply
carries the whole previously applied manifest, base64 data values included, in
its kubectl.kubernetes.io/last-applied-configuration annotation. That put a copy
of the secret in the pod log of the tool whose job is to keep it out of one. Those
two lines now log the name and namespace, matching the line directly above them.

Testing

Unit tests cover both parsers and every guard. For an end to end run, I stood up
a regtest stack shaped like the real thing: a signer lnd holding an aezeed
generated today, its accounts exported with wallet accounts list, and a
watch-only gateway with --remotesigner.enable pointed at it.

The one twist is the chain. A stock regtest chain has every block stamped with
roughly the same time, so every birthday resolves to the same block and the bug
hides. So I mined 200 blocks under setmocktime with timestamps spread from
2017-01 to 2026-06, ~17 days per block, which puts the aezeed epoch at height 13
and 2026-01-01 at height 190.

Without the flag, the bug reproduces exactly as reported:

[DBG] BTWL: Locating suitable block for birthday 2017-08-21 18:57:37 between blocks 0-200
[DBG] BTWL: Found birthday block: height=13, ...
[INF] BTWL: Started rescan from block ... (height 13) for 0 addrs, 0 outpoints
[DBG] BTWL: Rescanning from block height 14 to 200

With --init-rpc.watch-only-birthday=2026-01-01 (and identically with the
equivalent 1767225600):

[INF]: LNDINIT Using wallet birthday 2026-01-01T00:00:00Z
[DBG] BTWL: Locating suitable block for birthday 2025-12-29 16:00:00 between blocks 0-200
[DBG] BTWL: Found birthday block: height=190, ...
[INF] BTWL: Started rescan from block ... (height 190) for 0 addrs, 0 outpoints
[DBG] BTWL: Rescanning from block height 191 to 200

187 blocks of rescan down to 10. That first line is two days earlier than what I
passed in because btcwallet rewinds the birthday by 48 hours itself, which is
also why the drift window above is safe.

Adding --init-rpc.recovery-window=250 on top, the two compose the way they
should, with recovery starting from the birthday block rather than from 2017:

[INF]: LNDINIT Using wallet birthday 2026-01-01T00:00:00Z
[INF]: LNDINIT Using address look-ahead of 250 for recovery
[INF] LTND: Wallet recovery mode enabled with address lookahead of 250 addresses
[INF] BTWL: Seed birthday surpassed, starting recovery of wallet from height=191 ...
[INF] BTWL: Recovered addresses from blocks 191-301

The resulting wallet is a real one, not just one with a prettier rescan. Its
identity pubkey matches the signer's, every account comes back watch_only: true, newaddress succeeds (which only works if the derivation goes out to the
remote signer), and a coinbase paid to one of those addresses shows up as
confirmed balance:

$ lncli walletbalance
{ "total_balance": "3750000000", "confirmed_balance": "3750000000" }

Roasbeef added 5 commits July 24, 2026 16:42
In this commit, we make the -v/--verbose flag do what its own description
promises. Until now it set the log level to error, which meant that every
logger.Info call in the tool was dead code unless the operator also passed
an explicit --debuglevel. The progress lines we emit while reading secrets
and initializing a wallet exist precisely to be read, so a flag documented
as "turn on logging to stderr" should let them through.

This matters beyond cosmetics: warnings about a misconfiguration that only
manifests hours later were also being swallowed, and the helm charts that
drive lndinit in production all invoke it as "lndinit -v".
In this commit, we add an --init-rpc.watch-only-birthday flag that populates
WatchOnly.MasterKeyBirthdayTimestamp on the InitWallet request. Until now we
left that field at zero, and lnd reads an unset birthday as the aezeed
"Bitcoin Days Genesis" epoch of 2017-08-24. A freshly imported watch-only
gateway with no history at all therefore rescanned every block since then,
which on mainnet today is a walk of ~478k blocks and several hours of sitting
at "wallet is not synced to height <tip> yet" while matching against zero
addresses and zero outpoints.

The birthday has to come from the operator because there's nowhere else to
get it. It lives in the aezeed on the signer side, and "lncli wallet accounts
list", which produces the accounts JSON we import here, only exports the
account xpubs and their derivation paths. An xpub carries no notion of when
it was created.

The value can be given as a Unix timestamp in seconds, as an RFC3339
timestamp or as a plain YYYY-MM-DD date interpreted as midnight UTC. The bare
timestamp is what lnd's own InitWallet RPC and lncli's createwatchonly prompt
take, but a calendar date is what an operator actually knows and what reads
sanely in a helm values file, so we accept both.

Two values get rejected rather than passed through. A birthday before the
network's genesis block can only be a units mistake and would rescan from
block zero, which is worse than the default we're trying to avoid. A birthday
more than a day in the future is the same mistake in the other direction,
most often milliseconds handed to a field that wants seconds, and it would
make lnd start scanning at the chain tip where it can never see the funds the
wallet already holds. We tolerate a day of drift so clock skew between
whatever produced the timestamp and the pod running lndinit isn't fatal.

Finally, we log the birthday we settled on, and warn when there isn't one,
since the failure mode this fixes is otherwise invisible until the node has
already been rescanning for hours.
In this commit, we cover the birthday parser with a table test. Each accepted
notation maps to the same instant, the guard rails reject a timestamp before
genesis and one far enough in the future to be a units mistake, and the lower
bound follows the network rather than being hardcoded to mainnet, so a date
that's fine on mainnet is still refused on regtest.

We also assert that a birthday given without --init-rpc.watch-only fails
outright. Silently ignoring it there would be the same class of bug as the one
this fix is about: the operator does the right thing and still ends up with a
wallet that rescans from 2017.
In this commit, we document the new flag in the watch-only example, along with
a note on why it's needed at all: the accounts JSON only carries xpubs, so
without a birthday lnd has nothing to go on but the aezeed epoch. We also
point out which way to err. A birthday slightly too early costs a few extra
blocks of rescan, while one that's too late makes lnd skip the blocks the
wallet's funds are actually in.
In this commit, we cut the "Created secret"/"Updated secret" log lines down to
the secret's name and namespace, matching the line right above them that already
does exactly that.

Marshaling the whole ObjectMeta drags in the object's annotations, and a secret
that has ever been touched by kubectl apply carries the entire previously applied
manifest in kubectl.kubernetes.io/last-applied-configuration, base64 data values
and all. That put a copy of the secret in the pod log of the very tool whose job
is to keep it out of one. The name and namespace are the only parts of that JSON
anyone was reading anyway.
In this commit, we add an --init-rpc.recovery-window flag that populates
InitWalletRequest.RecoveryWindow, the address look-ahead lnd uses to scan for
keys that have already been seen on chain.

This is the same shape of gap as the birthday: the RPC field exists, lncli's
createwatchonly prompts for it with a default of 2500, and lndinit left it at
zero, which lnd reads as "recover nothing". For a brand new node that's the right
answer, which is why zero stays the default here. For a wallet that already has
history it isn't: the birthday makes the rescan start at the right height, and
then the wallet never derives far enough to notice its own addresses, so the
funds stay invisible for a different reason than before.

A negative look-ahead is rejected, as is a non-zero one on the file based init
path, which never looks at it. Silently dropping it there would land the operator
right back in the failure mode both of these flags exist to prevent.
@Roasbeef
Roasbeef force-pushed the init-wallet-watch-only-birthday branch from 694c85f to 7a61a1c Compare July 24, 2026 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

init-wallet watch-only import drops the wallet birthday, forcing a full rescan from the aezeed BDG epoch (2017-08-22)

1 participant