init-wallet: let the watch-only wallet birthday be set, avoid the full rescan from 2017 - #98
Open
Roasbeef wants to merge 6 commits into
Open
init-wallet: let the watch-only wallet birthday be set, avoid the full rescan from 2017#98Roasbeef wants to merge 6 commits into
Roasbeef wants to merge 6 commits into
Conversation
In this commit, we make the -v/--verbose flag do what its own description promises. Until now it set the log level to error, which meant that every logger.Info call in the tool was dead code unless the operator also passed an explicit --debuglevel. The progress lines we emit while reading secrets and initializing a wallet exist precisely to be read, so a flag documented as "turn on logging to stderr" should let them through. This matters beyond cosmetics: warnings about a misconfiguration that only manifests hours later were also being swallowed, and the helm charts that drive lndinit in production all invoke it as "lndinit -v".
In this commit, we add an --init-rpc.watch-only-birthday flag that populates WatchOnly.MasterKeyBirthdayTimestamp on the InitWallet request. Until now we left that field at zero, and lnd reads an unset birthday as the aezeed "Bitcoin Days Genesis" epoch of 2017-08-24. A freshly imported watch-only gateway with no history at all therefore rescanned every block since then, which on mainnet today is a walk of ~478k blocks and several hours of sitting at "wallet is not synced to height <tip> yet" while matching against zero addresses and zero outpoints. The birthday has to come from the operator because there's nowhere else to get it. It lives in the aezeed on the signer side, and "lncli wallet accounts list", which produces the accounts JSON we import here, only exports the account xpubs and their derivation paths. An xpub carries no notion of when it was created. The value can be given as a Unix timestamp in seconds, as an RFC3339 timestamp or as a plain YYYY-MM-DD date interpreted as midnight UTC. The bare timestamp is what lnd's own InitWallet RPC and lncli's createwatchonly prompt take, but a calendar date is what an operator actually knows and what reads sanely in a helm values file, so we accept both. Two values get rejected rather than passed through. A birthday before the network's genesis block can only be a units mistake and would rescan from block zero, which is worse than the default we're trying to avoid. A birthday more than a day in the future is the same mistake in the other direction, most often milliseconds handed to a field that wants seconds, and it would make lnd start scanning at the chain tip where it can never see the funds the wallet already holds. We tolerate a day of drift so clock skew between whatever produced the timestamp and the pod running lndinit isn't fatal. Finally, we log the birthday we settled on, and warn when there isn't one, since the failure mode this fixes is otherwise invisible until the node has already been rescanning for hours.
In this commit, we cover the birthday parser with a table test. Each accepted notation maps to the same instant, the guard rails reject a timestamp before genesis and one far enough in the future to be a units mistake, and the lower bound follows the network rather than being hardcoded to mainnet, so a date that's fine on mainnet is still refused on regtest. We also assert that a birthday given without --init-rpc.watch-only fails outright. Silently ignoring it there would be the same class of bug as the one this fix is about: the operator does the right thing and still ends up with a wallet that rescans from 2017.
In this commit, we document the new flag in the watch-only example, along with a note on why it's needed at all: the accounts JSON only carries xpubs, so without a birthday lnd has nothing to go on but the aezeed epoch. We also point out which way to err. A birthday slightly too early costs a few extra blocks of rescan, while one that's too late makes lnd skip the blocks the wallet's funds are actually in.
In this commit, we cut the "Created secret"/"Updated secret" log lines down to the secret's name and namespace, matching the line right above them that already does exactly that. Marshaling the whole ObjectMeta drags in the object's annotations, and a secret that has ever been touched by kubectl apply carries the entire previously applied manifest in kubectl.kubernetes.io/last-applied-configuration, base64 data values and all. That put a copy of the secret in the pod log of the very tool whose job is to keep it out of one. The name and namespace are the only parts of that JSON anyone was reading anyway.
Roasbeef
force-pushed
the
init-wallet-watch-only-birthday
branch
from
July 24, 2026 23:42
f46b46b to
694c85f
Compare
In this commit, we add an --init-rpc.recovery-window flag that populates InitWalletRequest.RecoveryWindow, the address look-ahead lnd uses to scan for keys that have already been seen on chain. This is the same shape of gap as the birthday: the RPC field exists, lncli's createwatchonly prompts for it with a default of 2500, and lndinit left it at zero, which lnd reads as "recover nothing". For a brand new node that's the right answer, which is why zero stays the default here. For a wallet that already has history it isn't: the birthday makes the rescan start at the right height, and then the wallet never derives far enough to notice its own addresses, so the funds stay invisible for a different reason than before. A negative look-ahead is rejected, as is a non-zero one on the file based init path, which never looks at it. Silently dropping it there would land the operator right back in the failure mode both of these flags exist to prevent.
Roasbeef
force-pushed
the
init-wallet-watch-only-birthday
branch
from
July 24, 2026 23:48
694c85f to
7a61a1c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, we fix #97: a watch-only wallet created by
lndinit init-wallet --init-rpc.watch-onlycame out with no birthday, solndfell back to theaezeed "Bitcoin Days Genesis" epoch of
2017-08-24and rescanned the chain fromthere. On mainnet today that's a walk of ~478k blocks and several hours parked at
Wallet is not synced to height <tip> yet, even for a brand new gateway that haszero history to find (
for 0 addrs, 0 outpointsin the logs, which is the tell).The RPC could always express this,
WatchOnly.master_key_birthday_timestamphasbeen there since watch-only import landed. We just never populated it, and left
the field at
0, whichlndreads as "unknown, assume 2017". So we add an--init-rpc.watch-only-birthdayflag that fills it in.Pulling on that thread turned up three more things in the same family, all fixed
here. See each commit message for the detailed reasoning w.r.t the individual
pieces.
Why the operator has to tell us
There's nowhere else to get the birthday from. It lives in the aezeed on the
signer side, and the accounts JSON we import here comes from
lncli wallet accounts list, which only exports account xpubs, their derivation paths, and themaster key fingerprint. An xpub carries no notion of when it was created:
lightningnetwork/lnd#10993 fixes that end of it, teaching
ListAccountstoreturn the master key birthday so it can round-trip through the accounts file.
Once that lands,
lndinitcan read the birthday straight out of the file and thisflag becomes an override rather than a requirement. Until then the operator is the
only one who knows.
The flag
The value can be a Unix timestamp in seconds, an RFC3339 timestamp, or a plain
YYYY-MM-DDdate read as midnight UTC. The bare timestamp is whatInitWalletand
lncli createwatchonly's prompt take, but a calendar date is what anoperator actually knows and what reads sanely in a helm values file, so we take
both. It slots into
init-wallet.shnext to the flags already there:
Two values get rejected rather than passed through. A birthday before the
network's genesis block can only be a units mistake, and it would rescan from
block zero, which is worse than the default we're trying to avoid. A birthday
more than a day in the future is the same mistake in the other direction (usually
milliseconds handed to a field that wants seconds), and that one is worse than
slow:
lndwould start scanning at the chain tip, where it can never see fundsthe wallet already holds. A day of drift is allowed so clock skew between whatever
produced the timestamp and the pod running
lndinitisn't fatal, and it stayscomfortably inside the 48 hours
btcwalletrewinds the birthday by on its ownbefore it goes looking for the birthday block. The lower bound follows the
network, so a date that's fine on mainnet is still refused on regtest.
Passing the flag where nothing can act on it (any init type other than
rpc, orrpcwithout--init-rpc.watch-only) is an error. Quietly ignoring it would bethe same class of bug as the one this fixes: the operator does the right thing and
still ends up rescanning from 2017.
The recovery window is the same bug wearing a different hat
InitWalletRequest.RecoveryWindowwas also left implicitly at zero, whichlndreads as "recover nothing". For the fresh gateway in #97 that's correct, so zero
stays the default. For a wallet that already has history it isn't: the birthday
gets the rescan starting at the right height, and then the wallet never derives
far enough to notice its own addresses, so the funds stay invisible for a
different reason than before.
lncli createwatchonlyhas always prompted forthis with a default of 2500;
--init-rpc.recovery-windowis the same knob.Verbose mode was eating the warning
We warn when no birthday was given, since a multi-hour rescan is otherwise
invisible until you're already in it. That warning turned out to be unreachable:
-vset the log level toerror, which made everylogger.Infoin the tool deadcode, including the "Reading seed from file" style progress lines that only exist
to be read. Since
btclog's default handler is already at info,-vwas in factreducing verbosity, which is the opposite of what the flag says it does. It now
means info, and it no longer clobbers an explicit
--debuglevel. The charts allrun
lndinit -v, so this is what makes the warning land where it needs to.And one secret leak found on the way
Turning those info logs back on for
-vruns made it worth looking at what theyactually print.
store-secretwas marshaling the k8s secret's entireObjectMetainto the log line, and a secret that has ever been touched by
kubectl applycarries the whole previously applied manifest, base64
datavalues included, inits
kubectl.kubernetes.io/last-applied-configurationannotation. That put a copyof the secret in the pod log of the tool whose job is to keep it out of one. Those
two lines now log the name and namespace, matching the line directly above them.
Testing
Unit tests cover both parsers and every guard. For an end to end run, I stood up
a regtest stack shaped like the real thing: a signer
lndholding an aezeedgenerated today, its accounts exported with
wallet accounts list, and awatch-only gateway with
--remotesigner.enablepointed at it.The one twist is the chain. A stock regtest chain has every block stamped with
roughly the same time, so every birthday resolves to the same block and the bug
hides. So I mined 200 blocks under
setmocktimewith timestamps spread from2017-01 to 2026-06, ~17 days per block, which puts the aezeed epoch at height 13
and
2026-01-01at height 190.Without the flag, the bug reproduces exactly as reported:
With
--init-rpc.watch-only-birthday=2026-01-01(and identically with theequivalent
1767225600):187 blocks of rescan down to 10. That first line is two days earlier than what I
passed in because
btcwalletrewinds the birthday by 48 hours itself, which isalso why the drift window above is safe.
Adding
--init-rpc.recovery-window=250on top, the two compose the way theyshould, with recovery starting from the birthday block rather than from 2017:
The resulting wallet is a real one, not just one with a prettier rescan. Its
identity pubkey matches the signer's, every account comes back
watch_only: true,newaddresssucceeds (which only works if the derivation goes out to theremote signer), and a coinbase paid to one of those addresses shows up as
confirmed balance: