Skip to content

bip138: encrypted backup - #109

Open
Sjors wants to merge 17 commits into
masterfrom
wip-encrypted-backup
Open

Sjors wants to merge 17 commits into
masterfrom
wip-encrypted-backup

Conversation

@Sjors

@Sjors Sjors commented Jan 14, 2026 •

Copy link
Copy Markdown
Owner

Implements BIP138 encrypted descriptor backups through bitcoin-wallet.

UNDERSTAND THE RISKS

  1. This might not go into Bitcoin Core, and I might not maintain this branch.
  2. It's partially vibe coded.

To back up an existing descriptor wallet (use the same data directory and network for both tools):

bitcoin-cli unloadwallet "existing"
bitcoin-wallet -wallet=existing encryptbackup > backup.txt
bitcoin-cli loadwallet "existing"

To import into a new watch-only wallet, set XPUB to a matching account xpub recovered from your seed or signing device:

bitcoin-cli -named createwallet wallet_name=recovered disable_private_keys=true blank=true
bitcoin-cli unloadwallet "recovered"
bitcoin-wallet -wallet=recovered -pubkey="$XPUB" decryptbackup < backup.txt
bitcoin-cli loadwallet "recovered"
bitcoin-cli -rpcwallet=recovered rescanblockchain 0

The backup contains descriptors, not private keys; keep your seed/private-key backup separately. Rescanning needs the relevant historical blocks. If the destination wallet already has a matching key, omit -pubkey to let it find the decryption key. inspectbackup reads unencrypted metadata from stdin; decryptbackup -pubkey=... without -wallet prints the decrypted descriptor content.

@pythcoiner

Copy link
Copy Markdown

in 7552892, H marker should also be allowed I think:

diff --git a/src/util/bip32.cpp b/src/util/bip32.cpp
index 94b1fcc027..14775dc79a 100644
--- a/src/util/bip32.cpp
+++ b/src/util/bip32.cpp
@@ -29,6 +29,9 @@ bool ParseHDKeypath(const std::string& keypath_str, std::vector<uint32_t>& keypa
         if (pos == std::string::npos) {
             pos = item.find('h');
         }
+        if (pos == std::string::npos) {
+            pos = item.find('H');
+        }
         if (pos != std::string::npos) {
             // The hardened tick can only be in the last index of the string
             if (pos != item.size() - 1) {
diff --git a/src/wallet/test/psbt_wallet_tests.cpp b/src/wallet/test/psbt_wallet_tests.cpp
index 5d8955e8c5..13c0f03753 100644
--- a/src/wallet/test/psbt_wallet_tests.cpp
+++ b/src/wallet/test/psbt_wallet_tests.cpp
@@ -135,6 +135,9 @@ BOOST_AUTO_TEST_CASE(parse_hd_keypath)
     BOOST_CHECK(ParseHDKeypath("m/0h/0h", keypath));
     BOOST_CHECK(!ParseHDKeypath("m/h0/0h", keypath));
 
+    BOOST_CHECK(ParseHDKeypath("m/0H/0H", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/H0/0H", keypath));
+
     BOOST_CHECK(ParseHDKeypath("m/0/0", keypath));
     BOOST_CHECK(!ParseHDKeypath("n/0/0", keypath));

Comment thread src/wallet/encryptedbackup.cpp Outdated
// CPubKey is either compressed (33 bytes) or uncompressed (65 bytes)
// In both cases, bytes 1-32 are the x-coordinate
uint256 result;
if (pubkey.size() >= 33) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not knowledgeable enough about the codebase, but is there some case where a CPubKey.size() could be 0 or 32?

it looks to me this if is confusing, because if it can resolve to false, we return a wrong (zeroized) key.

Comment thread src/wallet/encryptedbackup.h Outdated
* Handles:
* - Extended public keys (xpubs): extracts the pubkey and returns x-coordinate
* - Compressed public keys (33 bytes): strips the prefix byte
* - X-only public keys (32 bytes): returns as-is

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not what the function return, in that case it seems returning a zeroized key

Comment thread src/wallet/encryptedbackup.cpp Outdated
Comment on lines +123 to +126
// Require "m" prefix for BIP-xxxx paths
if (path_str.empty() || path_str[0] != 'm') {
return util::Error{Untranslated("Derivation path must start with 'm'")};
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m is optional in bip32

return util::Error{Untranslated("Truncated derivation path data")};
}

uint8_t child_count = data[pos++];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

        uint8_t child_count = data[pos++];
        if (child_count == 0) {
            return util::Error{Untranslated("Child count must be > 0")};
        }

@Sjors

Sjors commented Apr 15, 2026

Copy link
Copy Markdown
Owner Author

@pythcoiner thanks for the feedback, I plan to update this soon(tm).

@pythcoiner

Copy link
Copy Markdown

i'm working on a branch that is based on this, I'd like to to have it's test passing on the test vectors generated from the rust implem: pythcoiner#2

@pythcoiner

Copy link
Copy Markdown

@Sjors is there a reason to implement encryptbackup/decryptbackup/inspectbackup commands in wallet-tool rather than bitcoind?

@Sjors Sjors changed the title WIP: implemented encrypted backup BIP-xxxx WIP: implemented encrypted backup BIP138 May 22, 2026
@Sjors
Sjors force-pushed the wip-encrypted-backup branch from 90fe3c7 to e69813c Compare May 22, 2026 10:02
@Sjors

Sjors commented May 22, 2026 •

Copy link
Copy Markdown
Owner Author

in wallet-tool rather than bitcoind?

To keep the latter from becoming even more complex. Both binaries are shipped in the Bitcoin Core distribution, so it shouldn't be a problem. The core functionality lives should live in CWallet, so it can be exposed via the GUI later, if needed.

Rebased and updated test vectors, doing some additional refactoring now...

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from e69813c to d837001 Compare May 22, 2026 11:36
@Sjors

Sjors commented May 22, 2026

Copy link
Copy Markdown
Owner Author

I moved more functionality to CWallet so it can be used outside wallet-tool. Additionally, similar to bitcoin#34861, all calls go through the Wallet interface.

I added more test vectors from the BIP. Some vectors still to use BIPXXX, which I changed to BIP388 pending bitcoin/bips#1951 (comment).

While comparing this implementation with the BIP, my agent found a mismatch between prose and test vectors. Commit 9fa12f79b139f3cdb649c6c9a6f79cd1199f4259 changes the test vectors, but we could also change the prose - I haven't thought about it deeply.

@Sjors
Sjors force-pushed the wip-encrypted-backup branch 2 times, most recently from 8dc383f to d1016ab Compare May 22, 2026 14:41
@Sjors

Sjors commented May 22, 2026 •

Copy link
Copy Markdown
Owner Author

Got rid of 65c4a4e4777eef41858ca5838bba506f0be320e1 in favor of some trivial serialization code in encryptedbackup.cpp.

We're now actually using struct WalletDescriptorInfo introduced in 13096ee0ea2510ce886ba65f8d295a68da15eb8e.

I also cleaned up 17a25e655d7edf5a97042b6e4ea1c4431477a144 to make better use of existing crypto code (8330771899881c81c4eca8933103064df5af1d89 + e57aae0aa151bf2e3c6cc7398c6fdce174715815).

Misc. other cleanups in the first couple of commits.

I plan to refine the commits in this PR further another time, as that should give me a better sense which parts of the BIP are potentially problematic (i.e. actually tedious to implement, not just bad slop on my end).

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from d1016ab to 1e89b29 Compare May 22, 2026 15:34
@Sjors

Sjors commented May 22, 2026

Copy link
Copy Markdown
Owner Author

I changed 8330771899881c81c4eca8933103064df5af1d89 -> f3c5471068cd3b5a6d64015da1a6f95fba862f53 to instead have GetPubKeys filter by exclude_observable. That makes it more clear what the intention is, and why the extra complexity is probably justified. That said, I find the implementation confusing.

I added a test vector (see commit message) to clarify musig2(xpub1,xpub2) is allowed. It's impossible to derive the original xpub participants from an aggregated key on chain.

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from 1e89b29 to 80f46c9 Compare May 22, 2026 16:02
@Sjors

Sjors commented May 22, 2026

Copy link
Copy Markdown
Owner Author

f3c5471068cd3b5a6d64015da1a6f95fba862f53 -> f2482dc30edbd8d78d2a3a4560571e3ce02add37 now it makes more sense

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from 80f46c9 to ad79520 Compare May 22, 2026 16:56
@Sjors

Sjors commented May 22, 2026

Copy link
Copy Markdown
Owner Author

Bunch more cleanup, e.g.:

  • no more re-implementing EncryptChaCha20Poly1305
  • shrank wallet: BIP138 derivation path encoding a bit.
  • split the ChaCha20-Poly1305 stuff out of wallet: BIP138 full encrypted backup

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from ad79520 to b354c23 Compare May 26, 2026 16:04
@Sjors

Sjors commented May 26, 2026

Copy link
Copy Markdown
Owner Author

I implemented the descriptor format that I suggested in bitcoin/bips#1951 (comment). Also added a -compact argument to the export command to support the .txt format.

Also implemented the TLV change suggested here: https://github.com/bitcoin/bips/pull/1951/changes#r3304541662

decryptbackup can now import directly into a wallet. It can still optionally print an importdescriptors incantation. This involved pulling in bitcoin#34861 (first three commits).

Also pushed more cleanup:

  • use std::transform for the xor operation

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from b354c23 to 915eaed Compare May 27, 2026 07:57
@Sjors

Sjors commented May 27, 2026

Copy link
Copy Markdown
Owner Author

Repaired one of the intermediate commits that didn't pass CI.

@Sjors
Sjors force-pushed the wip-encrypted-backup branch from 915eaed to ed40eea Compare June 19, 2026 17:58
@Sjors
Sjors force-pushed the wip-encrypted-backup branch from 879f318 to cb6161a Compare September 23, 2026 18:26
@BenWestgate

Copy link
Copy Markdown

My daybreak blue agent produced an adverse security audit finding report for BIP 138 master when I was considering it for my project yesterday, would you like me to share it?

@Sjors

Sjors commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

@BenWestgate maybe share it in a gist to keep this thread focussed? Unless you have a specific patch that you just want me to take? Or is it in the spec, in which case an issue on the BIP repo makes more sense.

@BenWestgate

BenWestgate commented Sep 23, 2026 •

Copy link
Copy Markdown

Yes. It's a mix.

bip138-audit-2026-09-22.md findings
bip138-reviewed-revisions.md revisions
bip138-upstream-drafts-2026-09-22.md draft follow-up notes for proposed fixes/issues.

@BenWestgate

Copy link
Copy Markdown

Current wip-encrypted-backup accepts any s reconstructed from ci XOR BIP138_INDIVIDUAL_SECRET(p) if AEAD decryption succeeds, then imports the decrypted BIP380 descriptors.

I don't see a check that recomputes BIP138_DECRYPTION_SECRET from the eligible keys in the decrypted descriptor and requires it to equal that s.

Without that check, anyone who knows recipient key p can choose s, construct the matching ci, and encrypt an unrelated valid descriptor that Core will accept.

Could Core verify that binding before import? Successful decryption alone shouldn't imply descriptor activation/authenticity.

@pythcoiner

Copy link
Copy Markdown

Could Core verify that binding before import?

imho this is security circus, anyone knowing one of your xpub can just create a valid envelope just by encrypting only with this key, so this test will pass even with a "forged" encrypted payload

one check that could be enforced is verifying the key used for decryption is part of the descriptor, but I see some usecase where encrypting whith a 3rd party key not present in the descriptor can be usefull...

@Sjors

Sjors commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner Author

We could (further) emphasise in the BIP text that this scheme does not guarantee authenticity. The only way to do that is to involve a private key in the encryption or signing. We don't want that, because it would require custom firmware for hardware wallets. That said, maybe a future version could optionally sign it.

Also, I don't think Ben's proposal prevents this, but it's potentially useful to be able to add extra decryption keys that aren't in the descriptor.

@Sjors
Sjors marked this pull request as ready for review September 24, 2026 08:58
@Sjors
Sjors force-pushed the wip-encrypted-backup branch from cb6161a to a5b7e09 Compare September 25, 2026 08:37
@Sjors

Sjors commented Sep 25, 2026

Copy link
Copy Markdown
Owner Author

yuvicc and others added 15 commits November 7, 2026 17:03
The double spend pair reuses the transaction of the preceding coinbase
case, whose prevout was cleared with SetNull(). Only the prevout hash
was overwritten, so both transactions spent output index
COutPoint::NULL_INDEX, which does not exist. Template creation
therefore failed on a missing input even without a second spend.

Set the output index to 0 so the first transaction is valid and only
the double spend makes template creation fail.
…ng` double spend case

b2c7f25 test: spend an existing output in TestBasicMining double spend case (yuvicc)

Pull request description:

  The double spend case in `TestBasicMining` is meant to check that block template creation fails when two mempool transactions spend the same coin. It reuses the transaction from the preceding coinbase case, whose prevout was cleared with `SetNull()`, and only overwrites the prevout hash. The output index therefore stays `COutPoint::NULL_INDEX`, so both transactions spend an output of `txFirst[0]` that does not exist.

  Set the output index to 0, so the first transaction is a valid spend and only the second one makes template creation fail.

  Found while working on bitcoin#36692 and yuvicc#19.

ACKs for top commit:
  jpk68:
    ACK b2c7f25
  alexanderwiederin:
    ACK b2c7f25
  maflcko:
    lgtm ACK b2c7f25

Tree-SHA512: 3cf6f42f57a076cae340fb418296e815458251366a1b914015b779cafb063bb78db86efddafa704cc26e58efacdff37f424301214de662569500256da5918991
Add a shared RFC 3629 validator with tests for ASCII, multibyte boundaries, incomplete sequences, overlong encodings, surrogate code points, and values above U+10FFFF.
Preserve raw string bytes and validate them with the shared UTF-8 utility instead of decoding and normalizing malformed input. Keep escaped UTF-16 surrogate handling local, reject interrupted pairs, and correct supplementary-plane decoding.

Add regressions for malformed values and object keys, mixed raw bytes and escapes, and surrogate-pair boundaries.
Introduces WalletDescriptorInfo struct and DescriptorInfoToUniValue() helper
to avoid code duplication when serializing descriptor metadata to UniValue.

Refactors listdescriptors RPC to use the new helper.
Expose descriptor public-key collection that can omit keys observable from spent outputs. This lets encrypted-backup code derive recipient keys from non-observable descriptor xpubs without adding a separate BIP138-specific descriptor API.
Extract and normalize eligible descriptor BIP32 public key expressions for
BIP138 encrypted backups. Extended public keys that are not directly observable
from descriptor spends are converted to x-only keys, while literal pubkeys,
directly observable xpubs, and the BIP341 NUMS point are excluded.

For MuSig2 descriptors, participant xpubs remain eligible because the on-chain
aggregate key does not reveal them. Add a test vector for aggregate derivation
from bare participant xpubs using the BIP138 recipient vectors.
Add functions to compute the decryption secret and individual secrets
as specified in BIP138. The decryption secret is derived from sorted
public keys, and individual secrets allow any keyholder to decrypt.

Functions added:
- ComputeDecryptionSecret(): Hash sorted keys to derive decryption secret
- ComputeIndividualSecret(): Hash a single key to derive its individual secret
- ComputeAllIndividualSecrets(): Compute XOR'd secrets for all keys

Includes test vectors from the BIP specification.
Add functions for encoding and decoding BIP138 derivation paths as
count-prefixed arrays of big-endian child indices.

Encode paths in deterministic lexicographic order. Use the existing
ParseHDKeypath helper for textual path parsing in tests.

Includes test vectors from the BIP specification.
Add functions for encoding/decoding individual secrets as specified in BIP138:
- EncodeIndividualSecrets: encode sorted secrets to binary format
- DecodeIndividualSecrets: decode from binary format

Individual secrets allow any keyholder to derive the decryption secret using:
decryption_secret = ci XOR sha256("BIP_XXXX_INDIVIDUAL_SECRET" || pi)

Includes test vectors from the BIP specification.
Add functions for encoding and decoding BIP138 content type metadata.

Content types define what kind of data is in the encrypted payload:
- TYPE 0x01 encodes a 2-byte big-endian BIP number.
- TYPE 0x02 encodes vendor-specific content with CompactSize length.

Decode unknown TYPE values below 0x80 by skipping their length-prefixed
data. Reject TYPE 0x00 as reserved and TYPE values 0x80 or above as
unsupported upgrade-required content.
Add the BIP138 ChaCha20-Poly1305 algorithm identifier, nonce size, tag
size, and test coverage for the AEAD operation used by encrypted
backups.

The tests exercise a random roundtrip and the BIP test vectors against
the existing AEADChaCha20Poly1305 helper with an empty AAD.
Add binary and base64 encoding, encryption for all descriptor keys, and decryption helpers. Validate payloads and nonces, report excluded key expressions, pad secrets with decoys, and omit common derivation paths.

Include the full-backup vectors and roundtrip, malformed-content, and excluded-key tests.
Export every descriptor set as JSON or compact multipath content and encrypt for all included keys. Support optional derivation-path hints, metadata inspection, and decryption with an explicit xpub or wallet keys.

Keep decrypted content items separate internally so imports can process each document independently.
@Sjors
Sjors force-pushed the wip-encrypted-backup branch from a5b7e09 to a9dcbfc Compare November 9, 2026 13:24
@Sjors Sjors mentioned this pull request Nov 9, 2026
Sjors added 2 commits November 9, 2026 15:30
Import JSON documents and compact descriptors, including multiple content items and empty descriptor sets. Validate document fields and expand multipath descriptors while preserving birth times, ranges, and existing receive/change cursors.
Add encryptbackup, decryptbackup, and inspectbackup with compact output, optional derivation-path hints, explicit-xpub decryption, and import using wallet keys.

Exercise backup creation, recovery with every account xpub, metadata inspection, and JSON and compact imports in tool_wallet.py.
@Sjors
Sjors force-pushed the wip-encrypted-backup branch from a9dcbfc to 8f4c3d9 Compare November 10, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants