Skip to content

fix(crypto): make key rotation actually rotate, and prove reseal is interruption-safe - #384

Merged
Emmyt24 merged 2 commits into
Octo-Protocol-org:dev-branchfrom
Favourice01:fix/crypto/reseal-atomicity-audit
Sep 28, 2026
Merged

Emmyt24 merged 2 commits into
Octo-Protocol-org:dev-branchfrom
Favourice01:fix/crypto/reseal-atomicity-audit

Conversation

@Favourice01

Copy link
Copy Markdown

Closes #320

⚠️ Lead finding: migrate-keys could not rotate keys

The per-row atomicity this ticket asked about holds (see below). The audit also found a bigger gap in the rotation path itself:

  1. Nothing was ever selected for rotation. list_wallets_needing_reseal(SCHEME_V1) selected rows where sealed_scheme <> 1. seal only ever produces V1, whichever key did the sealing. So a MASTER_KEY → MASTER_KEY_NEXT run migrated zero rows and still logged "0 wallets remaining on old scheme". An operator following the runbook would then promote the new key, and every gas-tank seed would become unopenable.
  2. Legacy rows could never migrate. Scheme-0 rows (the only ones the query did select) failed in reseal, because open rejects scheme 0.
  3. The server picked the key by scheme tag. master_key_for_scheme returned MASTER_KEY_NEXT for every V1 row during a rotation window, so any gas tank not yet migrated failed to sign fee-bumps. Gas tanks created during the window were also sealed under the old key but opened with the new one.

The impact is on availability and key-compromise response. The threat model lists "zero-downtime rotation via sealed_scheme + bin/migrate-keys" as the master-key-compromise mitigation, and that mitigation did not work.

Fix

  • Tell keys apart with AES-GCM authentication, not the scheme tag. A record opened with the wrong key fails authentication cleanly.
    • migrate-keys pages over every sealed row. It skips rows that already open under the new key and reseals the rest from the old key. A row that opens under neither key aborts the run loudly instead of being skipped silently.
    • The server gets AppState::opening_keys() (next key first, then current), used by sponsor fee-bump signing. AppState::sealing_key() seals new gas tanks under the next key during a rotation. master_key_for_scheme is removed.
  • reseal_wallet's idempotency guard is now a compare-and-swap on the old ciphertext (WHERE sealed_ciphertext = $old) instead of the scheme, which doesn't change on a key rotation.
  • reseal accepts legacy scheme = 0 records. The crate docs say 0 is the same algorithm as V1. reseal opens them as V1 and writes an explicit V1 tag. open still rejects 0.
  • list_wallets_needing_reseal is replaced by list_sealed_wallets. The migration loop moves to bin/migrate-keys/src/lib.rs so tests run the exact production code. main.rs is now a thin wrapper.

Audit: per-row atomicity (confirmed)

Store::reseal_wallet (crates/store/src/lib.rs) sets sealed_ciphertext, sealed_nonce, sealed_salt and sealed_scheme in a single UPDATE statement. Postgres applies one statement atomically, so a crash can't leave a ciphertext from one sealing next to a nonce, salt or scheme from another. This PR keeps that property and documents it on the method.

Tests

bin/migrate-keys/tests/interruption_tests.rs gives each test its own throwaway Postgres database, seeds 12 wallets sealed under the old key, and runs migrate with batches of 4. A test-only hook panics in the task at the 6th write, which is the point between resealing in memory and persisting.

  • interrupting_migrate_keys_mid_batch_never_leaves_a_wallet_row_with_mismatched_scheme_and_ciphertext: after the crash, every row's four fields open under exactly one key with the original plaintext, and exactly the 5 rows written before the crash are rotated.
  • a_second_clean_run_after_interruption_completes_the_remaining_wallets_correctly: a clean run migrates the remaining 7 and skips the 5 already done, so every row is on the new key. A third run migrates 0.
  • reseal_upgrades_a_legacy_scheme_0_record_to_v1 (crypto unit test).

cargo test -p octo-crypto -p octo-migrate-keys passes, and so do the sponsor e2e and webhook tests (the fee-bump signing path).

Not covered by a dedicated test: the server trying the next key and then falling back to the current one mid-rotation. The sponsor tests exercise the single-key path through the same code.

…nterruption-safe

Audit: reseal_wallet writes all four sealed fields in one UPDATE, so a single row
is atomic. The audit also found that migrate-keys could not rotate anything. It
selected rows with sealed_scheme <> V1, but every record is V1 whichever key
sealed it, so a MASTER_KEY -> MASTER_KEY_NEXT run migrated nothing and still
reported "0 remaining". Legacy scheme-0 rows failed in open(). While
MASTER_KEY_NEXT was set, the server also opened every V1 row with the next key,
which broke gas tanks that had not been migrated.

- migrate-keys pages over all sealed rows and skips rows that already open under
  the new key; others are resealed from the old key. The loop moves into a
  library so tests can run it.
- reseal_wallet's guard is now a compare-and-swap on the old ciphertext, since
  the scheme does not change on a key rotation.
- reseal accepts legacy scheme-0 records (same algorithm) and upgrades them to V1.
- The server tries MASTER_KEY_NEXT first and falls back to MASTER_KEY when
  opening, and seals new gas tanks under the next key during a rotation.
- Adds interruption tests that crash a run mid-batch, then assert every row is
  wholly old or wholly new and that a second run finishes the rest.

Closes Octo-Protocol-org#320
@drips-wave

drips-wave Bot commented Sep 26, 2026

Copy link
Copy Markdown

@Favourice01 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

…seal-atomicity-audit

# Conflicts:
#	Cargo.lock
#	bin/migrate-keys/src/main.rs
@Emmyt24
Emmyt24 merged commit 9965dd1 into Octo-Protocol-org:dev-branch Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Confirm reseal never leaves a wallet row half-written if interrupted mid-operation

2 participants