From 1cdbf2ee982c73a47b5cf6fdec6b83302775e40a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 22:35:36 +0000 Subject: [PATCH 01/16] ci: re-pin standards callers to metadatastician/standards@a797d30 and relock MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit hyperpolymath/standards was transferred to metadatastician (same repo_id 1116521501, new owner_id 297579217); the API now answers 301. Since then the seven callers pinned at hyperpolymath/standards@a85aa8e end before any job exists (failure/startup_failure, zero jobs, labelled by file path). Re-point codeql, governance, hypatia-scan, mirror, rust-ci, scorecard and secret-scanner at metadatastician/standards@a797d308 (standards main #1394), the pin cicd-squabbler's main went green on (eeec5ed, 2026-11-04). actions.lock was regenerated by the estate tool, not by hand: scripts/relock-standards-pin.sh --to a797d308… (standards main 087f6bf) which rewrote the per-workflow edges, replaced the standards record with owner_id 297579217, and copied the two new leaf records it needs (actions/download-artifact v8.0.2, swatinem/rust-cache v2.9.3) from standards' own lock at the target. Validated locally: check-lock-sync.sh passes. check-lock-pin-contract.sh reports the same 11 tag-keyed leaf records as main (pre-existing, not gated here) and accepts the new caller-side standards edge. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- .github/workflows/actions.lock | 36 ++++++++++++++++------------ .github/workflows/codeql.yml | 4 ++-- .github/workflows/governance.yml | 2 +- .github/workflows/hypatia-scan.yml | 2 +- .github/workflows/mirror.yml | 2 +- .github/workflows/rust-ci.yml | 2 +- .github/workflows/scorecard.yml | 2 +- .github/workflows/secret-scanner.yml | 2 +- 8 files changed, 29 insertions(+), 23 deletions(-) diff --git a/.github/workflows/actions.lock b/.github/workflows/actions.lock index 3566e45..542aad8 100644 --- a/.github/workflows/actions.lock +++ b/.github/workflows/actions.lock @@ -15,7 +15,7 @@ workflows: - 'actions/upload-pages-artifact@v5.0.0' - 'haskell-actions/setup@v2.12.1' '.github/workflows/codeql.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/dashboard-check.yml': - 'actions/checkout@v7.0.1' - 'dtolnay/rust-toolchain@v1' @@ -30,21 +30,21 @@ workflows: - 'dtolnay/rust-toolchain@v1' - 'swatinem/rust-cache@v2.9.3' '.github/workflows/governance.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/hypatia-scan.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/idris-abi.yml': - 'actions/checkout@v7.0.1' '.github/workflows/label-triage.yml': [] '.github/workflows/labels.yml': [] '.github/workflows/mirror.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/rust-ci.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/scorecard.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/secret-scanner.yml': - - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7' + - 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' '.github/workflows/squabble.yml': - 'actions/checkout@v7.0.1' - 'hyperpolymath/cicd-squabbler@b625e05da04c0b72fe2ca19bd009841ff982f43d' @@ -79,6 +79,11 @@ dependencies: commit: 'sha1-368f82528645a54fb793d4d04e342629a3f51346' owner_id: 44036562 repo_id: 438112499 + 'actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333': + ref: 'v8.0.2' + commit: 'sha1-9000827ccba6bdab643e8b6fd33ac0654aef8333' + owner_id: 44036562 + repo_id: 192626254 'actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f': ref: 'v7.0.0' commit: 'sha1-bbbca2ddaa5d8feaa63e36b76fdaad77386f024f' @@ -156,14 +161,15 @@ dependencies: commit: 'sha1-c1a34884054fabf0e9de81dbf68f4ba7874e85f1' owner_id: 6759885 repo_id: 1275650185 - 'hyperpolymath/standards@a85aa8e70326cceedae169013a47daee554918b7': - ref: 'a85aa8e70326cceedae169013a47daee554918b7' - commit: 'sha1-a85aa8e70326cceedae169013a47daee554918b7' - owner_id: 6759885 + 'metadatastician/standards@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12': + ref: 'a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' + commit: 'sha1-a797d308c0517d4e2d21cc7f1cb08d41f82b1a12' + owner_id: 297579217 repo_id: 1116521501 uses: - 'actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9' - 'actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1' + - 'actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333' - 'actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9' - 'denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed' - 'dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8' @@ -173,7 +179,7 @@ dependencies: - 'goto-bus-stop/setup-zig@abea47f85e598557f500fa1fd2ab7464fcb39406' - 'ossf/scorecard-action@fa7117db83c52d787ae5a89f350999701505d78e' - 'oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6' - - 'swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6' + - 'swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0' - 'webfactory/ssh-agent@e83874834305fe9a4a2997156cb26c5de65a8555' 'ossf/scorecard-action@fa7117db83c52d787ae5a89f350999701505d78e': ref: 'v2.5.0' @@ -185,9 +191,9 @@ dependencies: commit: 'sha1-0c5077e51419868618aeaa5fe8019c62421857d6' owner_id: 108928776 repo_id: 512644635 - 'swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6': - ref: 'v2' - commit: 'sha1-6323deb102c322ba6fcbdcafc7e3dddab59af2b6' + 'swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0': + ref: 'v2.9.3' + commit: 'sha1-d1527c8f67ef9af357290475d4af4b0fab8f12e0' owner_id: 580492 repo_id: 298565987 'swatinem/rust-cache@v2.9.3': diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index ce0a91d..c55dccd 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -32,10 +32,10 @@ permissions: jobs: codeql-actions: - uses: hyperpolymath/standards/.github/workflows/codeql-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/codeql-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 with: language: actions codeql-rust: - uses: hyperpolymath/standards/.github/workflows/codeql-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/codeql-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 with: language: rust diff --git a/.github/workflows/governance.yml b/.github/workflows/governance.yml index fc91895..a114cbb 100644 --- a/.github/workflows/governance.yml +++ b/.github/workflows/governance.yml @@ -34,7 +34,7 @@ permissions: jobs: governance: - uses: hyperpolymath/standards/.github/workflows/governance-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/governance-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 permissions: actions: read contents: read diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index c2b4ebe..0fd655e 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -28,5 +28,5 @@ permissions: jobs: hypatia: - uses: hyperpolymath/standards/.github/workflows/hypatia-scan-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/hypatia-scan-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 secrets: inherit diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index 07ce99a..62dace9 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -14,7 +14,7 @@ permissions: jobs: mirror: - uses: hyperpolymath/standards/.github/workflows/mirror-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/mirror-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 secrets: inherit permissions: actions: read diff --git a/.github/workflows/rust-ci.yml b/.github/workflows/rust-ci.yml index 1817f74..96f6934 100644 --- a/.github/workflows/rust-ci.yml +++ b/.github/workflows/rust-ci.yml @@ -21,7 +21,7 @@ permissions: jobs: rust-ci: - uses: hyperpolymath/standards/.github/workflows/rust-ci-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/rust-ci-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 permissions: actions: read contents: read diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 53e16f6..954018e 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -17,5 +17,5 @@ jobs: contents: read security-events: write id-token: write - uses: hyperpolymath/standards/.github/workflows/scorecard-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/scorecard-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 secrets: inherit diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index 7461542..9be47cd 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -22,4 +22,4 @@ jobs: actions: read security-events: write contents: read - uses: hyperpolymath/standards/.github/workflows/secret-scanner-reusable.yml@a85aa8e70326cceedae169013a47daee554918b7 # standards main, 2026-10-19 (#1296) + uses: metadatastician/standards/.github/workflows/secret-scanner-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 From faefe321e2bd1e8eea69799d1ced7745a7086a86 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 22:39:15 +0000 Subject: [PATCH 02/16] docs(proofs): record the proof-statement audit and its findings in the ledger Adds docs/reports/audit/proof-statement-audit-2026-10-27.adoc: which "What Needs Proving" obligations are provable as written, which needed a code change first, and the probe evidence (P0 control, P1-P6) behind each verdict. A status preamble records what has since been fixed. Ledger rows, in both tables: - S-1, S-2 TESTED: content-store path confinement and atomic, repairing writes (fixed in #163; the four unit tests that pin them). - L-1, L-2 OPEN: the attestation log does not detect edits to uncovered fields or to the last entry (P1, P2), nor tail truncation (P3). - O-1 OPEN: ObliterationProof hard-codes storage_cleared and verifies for content never stored (P6). The attestation and content-store obligations now point at those rows. dashboard-check run locally: every TESTED row (the new S rows included) and the count checks pass; the PROVEN rows were not run here (no idris2 or agda in this container) and are left to the Dashboard Check workflow. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- PROOF-NEEDS.adoc | 16 +- .../proof-statement-audit-2026-10-27.adoc | 266 ++++++++++++++++++ 2 files changed, 280 insertions(+), 2 deletions(-) create mode 100644 docs/reports/audit/proof-statement-audit-2026-10-27.adoc diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index 32ae62f..80dba81 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -75,6 +75,11 @@ tables must agree. | T-6 | A failed execute leaves the filesystem unchanged, and each undo restores the state its execute started from (premises of T-1; single steps are TESTED by `+execute_then_undo_is_identity+`) | ASSUMED | not checked for the failure path | T-7 | Transaction ids (UUID v4) never collide within one log (the freshness premise of T-2) | ASSUMED | probabilistic, not checked | T-8 | The operation metadata log is saved atomically, and an undo whose `+mark_undone+` fails to persist can be retried | OPEN | by inspection: the log now takes the same write, sync, rename path as the transaction log (`+crate::transaction::write_atomic+`), and `+metadata::tests::save_replaces_the_log_atomically+` asserts it, but the crate has not been built since that edit, so the row stays OPEN until the test is green in CI. Retry is unchanged: `+undo+` still runs the undo before `+mark_undone+` persists, so a failed persist still leaves the file undone and the log not +| S-1 | A content hash that is not `+sha256:+` plus 64 lowercase hex digits never resolves to a path outside the store root | TESTED | `+content_store::tests::malformed_hash_never_leaves_the_store_root+`; fixed in #163 after audit probe P5 showed `+delete+` removing a file outside the store (`+docs/reports/audit/proof-statement-audit-2026-10-27.adoc+`) +| S-2 | `+store+` writes a blob by temporary file, sync and rename, and repairs a blob a crash left damaged instead of deduplicating against it | TESTED | `+content_store::tests::store_repairs_a_blob_truncated_by_a_crash+`, `+store_leaves_no_temp_files_and_count_ignores_strays+`; fixed in #163 (probe P4) +| L-1 | Editing any field of any audit-log entry makes `+verify_integrity+` fail | OPEN | false today, probes P1 and P2: algorithm, purpose, old_state, new_state, rotated_to and rotated_from are in neither the MAC nor the chain hash, and key_id, fingerprint and reason are chain-hashed only, so on the last entry nothing protects them. The fix is a MAC over every field (a scheme version bump) +| L-2 | Removing trailing audit-log entries makes `+verify_integrity+` fail | OPEN | false today, probe P3: every prefix of a valid log verifies. Needs an anchor held outside the log (a design decision: signed head, external counter, or witness) +| O-1 | An `+ObliterationProof+` attests that the content it names was erased | OPEN | false today, probe P6: `+ObliterationProof::generate+` sets `+storage_cleared: true+` unconditionally and `+verify_commitment+` holds for content that was never stored. Either compute the field from the erasure, or stop calling the object a proof | V-1 | valence-shell depends on `+reversible-core+` for reversible operation data | DESIGNED | ARCHITECTURE.adoc §Consumption status — no repository outside this workspace depends on the crate; Phase 2 is unstarted | V-2 | ochrance verifies januskey A2ML manifests | DESIGNED | `+crates/reversible-core/src/manifest.rs+` — `+ManifestEmitter+` has never been parsed by ochrance; the SHA-256/BLAKE3 boundary mismatch is unresolved | V-3 | aletheia consumes `+reversible-core+` | DESIGNED | ARCHITECTURE.adoc §Deferred — Phase 0 research, no operation types yet @@ -175,11 +180,11 @@ below are still open. |Weak keys break the key store's security model. **Off the `jk` path** — tracked here only while `+keys.rs+` remains in this tree (ADR-0001) -|Attestation chain |Chain of custody proofs are unforgeable |Tampered +|Attestation chain |Chain of custody proofs are unforgeable (false as written: L-1 and L-2 are OPEN, see the 2026-10-27 proof-statement audit) |Tampered attestations undermine trust |Content store integrity |Content-addressed storage never returns wrong -content |Hash collisions or bugs corrupt stored data +content (paths confined to the store and writes atomic: S-1 and S-2 are TESTED; reads verify the hash) |Hash collisions or bugs corrupt stored data |Transaction atomicity |Delta application is atomic and reversible (the model is proved, T-1 to T-3; the Rust is tested for partial-rollback retry, T-4, and failed saves, T-5; the premises T-6 and T-7 are assumed; the metadata log, T-8, now saves by rename but that is unverified) |Partial transactions corrupt state @@ -243,12 +248,19 @@ the status table above. |T-5: A save whose temporary file already exists (another writer's) fails without deleting that file or writing the log |TESTED |`crates/reversible-core/src/transaction.rs::a_save_never_removes_a_temporary_file_it_did_not_create` |`cargo test --locked -p reversible-core --lib a_save_never_removes_a_temporary_file_it_did_not_create` |T-5: The temporary file has the log's permissions from creation, before any content is written |TESTED |`crates/reversible-core/src/transaction.rs::the_temporary_file_is_restricted_before_any_content_is_written` |`cargo test --locked -p reversible-core --lib the_temporary_file_is_restricted_before_any_content_is_written` |T-5: The temporary file is opened with the log's mode, so it is never more readable than the log at any instant, not only after a later chmod (checked where the umask leaves owner read-write, the usual 022) |TESTED |`crates/reversible-core/src/transaction.rs::the_temporary_file_is_opened_with_the_logs_mode` |`cargo test --locked -p reversible-core --lib the_temporary_file_is_opened_with_the_logs_mode` +|S-1: A malformed content hash never resolves to a path outside the store root |TESTED |`crates/reversible-core/src/content_store.rs::malformed_hash_never_leaves_the_store_root` |`cargo test --locked -p reversible-core --lib malformed_hash_never_leaves_the_store_root` +|S-1: A well-formed content hash keeps the existing two-level blob layout |TESTED |`crates/reversible-core/src/content_store.rs::well_formed_hash_keeps_the_existing_layout` |`cargo test --locked -p reversible-core --lib well_formed_hash_keeps_the_existing_layout` +|S-2: Storing content again repairs a blob a crash left truncated |TESTED |`crates/reversible-core/src/content_store.rs::store_repairs_a_blob_truncated_by_a_crash` |`cargo test --locked -p reversible-core --lib store_repairs_a_blob_truncated_by_a_crash` +|S-2: A store leaves no temporary file behind, and count ignores stray ones |TESTED |`crates/reversible-core/src/content_store.rs::store_leaves_no_temp_files_and_count_ignores_strays` |`cargo test --locked -p reversible-core --lib store_leaves_no_temp_files_and_count_ignores_strays` |G-2: The Idris2 IsCNO / ReversibleOp pair corresponds to the Agda ReversiblePair (by transcription) |ASSUMED |`verification/proofs/agda/Reversibility.agda` |- |X-1: Hash one-wayness and physical erasure (cryptographic and physical assumptions) |ASSUMED |`src/abi/JanusKey/ABI/Proofs.idr` |- |G-3: The FFI bridge preserves grade 𝟙 (a homomorphism; resource-algebra lists homomorphisms as OPEN) |OPEN |`verification/proofs/agda/Reversibility.agda` |- |E-1: An attacker cannot recover obliterated content or use a revoked key (adversaryCannotKnow holds only by construction of the model) |OPEN |`src/abi/JanusKey/ABI/Types.idr` |- |E-1m: In a world model, an adversary whose observation excludes the store does not know what it holds (adversary-ignorant), obliteration leaves nothing to recover (obliterate-cuts-recovery, obliterate-ignorance), and rotation leaves no agent — the system included — able to use the old key (rotate-forgets-old); the fixtures AdversaryKnows and OldKeyStillUsable are refused |PROVEN |`verification/proofs/agda/Epistemic.agda::adversary-ignorant` |`bash scripts/check-agda.sh` |The six security obligations under "What Needs Proving" (obliteration, key derivation, attestation chain, content store, transaction atomicity, reversible-core) are proved |OPEN |`PROOF-NEEDS.adoc` |- +|L-1: Editing any field of any audit-log entry makes verify_integrity fail (false today: audit probes P1, P2) |OPEN |`crates/januskey-cli/src/attestation.rs` |- +|L-2: Removing trailing audit-log entries makes verify_integrity fail (false today: audit probe P3; needs an external anchor) |OPEN |`crates/januskey-cli/src/attestation.rs` |- +|O-1: An ObliterationProof attests that the content it names was erased (false today: storage_cleared is hard-coded, audit probe P6) |OPEN |`crates/januskey-cli/src/obliteration.rs` |- |T-8: The operation metadata log is saved atomically, and an undo whose `+mark_undone+` fails to persist can be retried (by inspection: `+MetadataStore::save+` calls `+crate::transaction::write_atomic+`, the path `+transaction.rs+` already tests in T-5; `+metadata::tests::save_replaces_the_log_atomically+` covers the log itself and has not been run) |OPEN |`crates/reversible-core/src/metadata.rs` |- |T-6: A failed execute leaves the filesystem unchanged, and each undo restores the state its execute started from (premises of T-1) |ASSUMED |`crates/januskey-cli/src/operations.rs` |- |T-7: Transaction ids (UUID v4) never collide within one log (the freshness premise of T-2) |ASSUMED |`crates/reversible-core/src/transaction.rs` |- diff --git a/docs/reports/audit/proof-statement-audit-2026-10-27.adoc b/docs/reports/audit/proof-statement-audit-2026-10-27.adoc new file mode 100644 index 0000000..04eb0c0 --- /dev/null +++ b/docs/reports/audit/proof-statement-audit-2026-10-27.adoc @@ -0,0 +1,266 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) Jonathan D.A. Jewell += JanusKey proof-statement audit (2026-10-27) +:toc: left + +[IMPORTANT] +.Status on 2026-11-04 — read this first +==== +This report records the audit as it was run on `main` at 10f0bc2. Since then: + +* *Fixed*: P4 and P5 (content store). Writes are now atomic and a damaged blob is repaired, and a malformed hash cannot leave the store root (#163). These are now ledger rows S-1 and S-2. +* *Fixed*: transaction-log persistence and resumable rollback (T-4 and T-5, #162 and #164). +* *Still open*: +** P1 to P3 (attestation log): uncovered fields, the last entry, and tail truncation. These are now ledger rows L-1 and L-2. +** P6 (`ObliterationProof` proves nothing about erasure). This is now ledger row O-1. + +The obligation statements in `PROOF-NEEDS.adoc` are the authority. Where this report and the ledger differ, the ledger wins. +==== + +Audited: januskey `main` at 10f0bc2, 2026-10-27. Purpose: before anyone proves anything, decide +which of the open obligations in `PROOF-NEEDS.adoc` are provable as written, which need a code +change first, and which are assumptions. No proofs were written. + +== How to read this + +* *Read*: `content_store.rs`, `obliteration.rs` (struct, `obliterate`, `obliterate_path`, +`secure_overwrite`), `transaction.rs`, `attestation.rs` (entry, hash, MAC, log, verify), +`keys.rs` (init/unlock/retrieve/rotate/revoke/KDF/wrap), `Types.idr` (`Knows`), +`PROOF-NEEDS.adoc`. +* *Not read*: the filesystem rollback executor (in `januskey-cli`), `metadata.rs`, +`operations.rs`, `keys.rs` `load_store`/`verify_kek`/`unwrap_key` bodies, `Proofs.idr` and +`Layout.idr` beyond the ledger, the Agda module beyond its header. +* *Run*: seven probe tests (a throwaway test file, not committed), on a +scratch copy of the repo, against the unmodified code. Each asserts what the code _does_, so a +passing probe is evidence the property is _false_. A control probe (an edit to a covered field is +detected) passes, so the harness itself is sound. P1 initially failed because my test edited a +protected field as well; I fixed the test and say so here. +* *Not run*: the Idris and Agda checkers. I rely on CI having passed on the #159 commits. +* Everything marked _unverified_ below is a hypothesis, not a finding. + +== Verdicts at a glance + +[cols=",,,",options="header",] +|=== +|# |Obligation (PROOF-NEEDS "What Needs Proving") |Verdict on the claim as written |Prove after +|1 |Content store integrity |Read-side is sound; *write side is not crash-safe* (P4) and a hash string can *steer paths outside the store* (P5) |code fix +|2 |Transaction atomicity |State machine already proven; *log persistence is not atomic*; rollback needs a failure model |code fix + model +|3 |Attestation chain "unforgeable" |*False as written*: three gaps confirmed by test (P1, P2, P3) |restate + code fix +|4 |Key derivation |Mostly not a proof job: parameters = compile-time assertion, nonces = bound, primitives = ASSUMED |restate +|5 |Obliteration |Logical (CAS) part provable as a model theorem; *"proof" object proves nothing* (P6); *key obliteration is not implemented*; physical erasure unprovable |restate + design +|6 |reversible-core undo/redo |Provable abstractly (nearly done: C-1 + per-op cancellation); the link to the Rust is the real work |model + conformance +|E-1 |Adversary cannot recover / revoked key unusable |*Vacuous as modelled*; revocation is an API guard, not a cryptographic one |delete or replace +|— |"Every operation is reversible (MPR)" |*Contradicts proven C-3* (obliteration is by design not reversible) |restate +|=== + +== Findings from the probes (evidence) + +[cols=",,",options="header",] +|=== +|Probe |Result |Meaning +|P0 control |actor edit detected |harness is valid +|P1 |editing `rotated_from` and `new_state` of a middle audit entry: `verify_integrity` still *valid* |those fields are in neither the MAC nor the chain hash +|P2 |editing `reason` of the *last* entry: still valid |the last entry's chain-hashed fields have no successor to protect them +|P3 |deleting trailing entries: still valid |verification accepts any prefix; no external anchor or count +|P4 |truncate a stored blob, call `store` again, `retrieve` -> integrity error |dedupe (`path.exists()`) never repairs a crash-truncated blob +|P5 |`ContentHash("sha256:ab" + absolute path)`: `content_path` is outside the root and `delete` removed my temp file |path is built by `Path::join`, and an absolute right-hand side replaces the root +|P6 |`ObliterationProof::generate` on content never stored: `verify_commitment` true, `storage_cleared` true |the proof is a public hash plus a hard-coded flag +|=== + +Reachability caveat for P5: it shows the _store API_ accepts such a hash. I have not shown that +attacker-controlled metadata can reach `delete`/`obliterate`. `ContentHash` derives `Deserialize` with a +public field and no validation, so that is where to look. Treat it as a security issue and fix it before +publishing details (the repository is public). + +[[1-content-store-integrity]] +== 1. Content store integrity + +*As written:* "Content-addressed storage never returns wrong content." + +* *Provable statement (R):* `retrieve(h) = Ok(x)` implies `H(x) = h`. This follows directly from the +guard in `retrieve`. It is true, cheap, and low value on its own. Combined with collision resistance +(X-1) it gives "x equals any content that was stored under h". +* *Round trip (W):* `store(c) = h` then `retrieve(h) = Ok(c)`. TESTED by the property test, *false under +a crash* (P4): a truncated file at the final path makes every later `store` of that content a no-op +and every `retrieve` fail. +* *Path confinement (C):* for every `h`, `content_path(h)` is inside `root`. *False* (P5). + +Do: fix C (validate `ContentHash` on construction/deserialization: `sha256:` plus 64 hex) and W (write to a +temp file, fsync, rename). Then prove R and C as lemmas about the validated type. Statements to avoid: +anything that restates the `verify` call. + +Difficulty: low. Kani or Verus can prove R and C +on the real function; unproven here (spike first). + +[[2-transaction-atomicity]] +== 2. Transaction atomicity + +*As written:* "Delta application is atomic and reversible." + +Split it: + +* *T1, lifecycle* (Pending -> Active -> Committed | RolledBack): PROVEN in `Proofs.idr`. Note the Rust +enum has three states (no `Pending`), so the Idris model and the Rust type already differ by one +state. The correspondence is assumed (same status as G-2). +* *T2, rollback restores state:* abstractly, undoing o1..on in reverse on state s gives s, given the +per-operation premise `undo o ∘ o = id`. That is C-1 (`seqCNO`) plus the TESTED premise +`execute_then_undo_is_identity`. Provable now; mostly done. What is missing is *partial failure*: what +if undo of o_k fails mid-way? The statement needs an error model (is undo idempotent and resumable?). +I did not read the executor, so I cannot say. +* *T3, crash atomicity of the log:* `TransactionManager::save` is `fs::write` (truncate, then write; no +temp file, no fsync, no rename). A crash mid-save leaves a truncated `transactions.json`, and +`TransactionManager::new` then fails with `MetadataCorrupted`. So "atomic" is *false for the +persistence layer*. After switching to temp + fsync + rename it is provable under a stated crash +model (POSIX rename atomicity is an assumption). + +Difficulty: T2 low-moderate; the crash model is the hard design choice (what does a crash do to a +directory and to file contents?). + +[[3-attestation-chain-unforgeable]] +== 3. Attestation chain "unforgeable" + +What the code actually does: each entry carries `attestation = HMAC-SHA256(k, scheme || 0 || event_id: timestamp:event_type:actor || 0 || previous_hash)`, and `previous_hash` is SHA-256 over a _different, +larger_ field set (adds `key_id`, `fingerprint`, `reason`). `k = SHA-256(KEK || "attestation")`, KEK from +Argon2id. + +Coverage, confirmed by probes: + +[cols=",,,",options="header",] +|=== +|Field |In MAC |In chain hash |Net +|event_id, timestamp, event_type, actor |yes |yes |protected +|key_id, fingerprint, reason |no |yes |protected except on the *last* entry (P2) +|algorithm, purpose, old_state, new_state, rotated_to, rotated_from |*no* |*no* |*unprotected* (P1) +|entries after the last kept one |n/a |n/a |*deletable* (P3) +|=== + +So "unforgeable" is false as written. Better statement, once fixed: + +____ +*A-1.* For an adversary without `k`: if `verify_integrity(L') = valid` then `L'` is a prefix of an +honestly produced log, or the adversary has forged an HMAC-SHA256 tag or found a SHA-256 collision. +____ + +Needed code changes first: MAC (or hash) over a canonical encoding of _every_ field; an anchor against +truncation (signed counter or head hash stored elsewhere, or an external witness). Also name the +adversary: the passphrase holder can always rewrite the log, since the MAC key is derived from the KEK +they hold. And `read_all` silently skips unparseable lines (a corrupted or injected line becomes a +deletion); decide whether that should fail. + +How to prove it, with ratings (first elegance/correctness, then dependability/security): + +[cols=",,,",options="header",] +|=== +|Option |Elegance |Dependability |Note +|Symbolic model in Agda (HMAC as an unforgeable function) |★★ |★★ |Cheap, but it assumes the conclusion for the crypto part. Same weakness as E-1. +|Game-based reduction on paper + conformance tests, mark HMAC security ASSUMED |★★★ |★★★ |Honest and sufficient for this repo. Recommended. +|Mechanised computational proof (EasyCrypt, CryptoVerif) |★★★ |★★★ |Correct but research-grade effort for a small gain. Different path, unexplored here. +|Replace the home-made chain with an existing transparency-log or signed-log design |★★ |★★★ |Removes the bespoke construction; real design change. +|=== + +Difficulty: statement and fix moderate; mechanised crypto proof very hard. + + +[[4-key-derivation]] +== 4. Key derivation + +"Sound keys" splits three ways, and only one part is a proof: + +* *Parameters* (Argon2id 64 MiB, t=3, p=4, 16-byte salt): a theorem about literals is exactly what the +ledger removed (`defaultMeetsOWASP`). Do it as a compile-time assertion +(`const _: () = assert!(...)`) in Rust. Status: TESTED (compile-time). +* *Nonce uniqueness* for AES-GCM key wrapping: nonces are random 96-bit per wrap. Statement: +after q wraps under one KEK, `P[collision] <= q(q-1)/2^97`. A birthday-bound lemma; paper-grade, and +NIST SP 800-38D caps random-nonce use near 2^32 invocations per key. Provable, practically irrelevant +at this scale; write it down as ASSUMED-with-bound unless you want a probability library in Agda. +* *Primitive security* (Argon2, AES-GCM, HMAC): ASSUMED (X-1 family). + +Smaller findings (unverified consequences): `attestation_key` is a plain `[u8;32]` in `AuditLog`, not +`Zeroize`d; the attestation key is `SHA-256(KEK || "attestation")` rather than HKDF (acceptable for a +uniform 32-byte KEK but non-standard). + +Not read: `load_store`, `verify_kek`, `unwrap_key`, and what the store-header `nonce` is used for. + +Difficulty: low. + +[[5-obliteration]] +== 5. Obliteration + +Three different claims are bundled: + +* *O-1, logical (CAS) completeness:* after `obliterate_path(p)`, no blob reachable only via `p` remains, +no log entry mentions `p`, shared blobs survive. TESTED today by three Rust tests. Provable as a model +theorem over (store map, log, reference sets): a partition argument, moderate. Ordering is sound +(shred, then purge) but not crash-atomic; a crash between them leaves entries that point to missing +blobs. Unverified hypothesis worth a test: if the store's compression flag differs from when a blob was +written, `content_path` misses the blob, `exists` is false, the shred is skipped silently, and the log +purge proceeds. +* *O-2, "proof of obliteration":* `ObliterationProof` is `SHA-256(content_hash || nonce || timestamp)` +plus a hard-coded `storage_cleared: true`. Anyone can produce one for any hash without erasing anything +(P6). The statement "a verifying proof implies the content was obliterated" is *false*. This is a +naming and documentation problem first (the module header says "formal obliteration proofs"). +* *O-3, physical erasure:* the code already says overwrite-in-place does not guarantee erasure on +CoW/journaling filesystems or flash. ASSUMED (X-1). Not provable. +* *Key obliteration:* `KeyState::Obliterated` is defined and read but *never assigned* anywhere in +`crates/`. The module comment itself recommends encrypt-then-shred. So the design that would make O-3 +tractable is not built. + +Options for the strong guarantee: + +[cols=",,,",options="header",] +|=== +|Option |Elegance |Dependability |Note +|Keep overwrite, relabel as best-effort, drop "proof" wording |★★ |★★ |Cheapest; honest. Does the least for users. +|Crypto-shredding: per-blob key, erasing the key is the erasure |★★★ |★★★ |Reduces the claim to AES-GCM secrecy + key erasure; key bytes still face the SSD problem, but 32 bytes can live in an OS keyring or memory only. Real design work. +|Two-tier: crypto-shred plus best-effort overwrite of blobs |★★ |★★★ |Defence in depth; most code. +|=== + +Difficulty: O-1 moderate; O-2 trivial once renamed (prove nothing, claim nothing); O-3 unprovable. + +[[6-reversible-core-undoredo-preserves-data-integrity]] +== 6. reversible-core: undo/redo preserves data integrity + +Same shape as T2. Abstract theorem: for a sequence of operations with the cancellation premise, undo +restores state. Provable now (nearly proven). I did not locate a "redo" in the code I read; the claim +should say what it covers. The expensive part is the *refinement link to the Rust* (G-2, G-3, capstone). + +Options for the link, with ratings: + +[cols=",,,",options="header",] +|=== +|Option |Elegance |Dependability |Note +|Keep the verified model; conformance tests against the Rust (current approach) |★★ |★★★ |Already working (`execute_then_undo_is_identity`); no proof of the code itself. +|Bounded model checking (Kani) on small kernels |★★★ |★★ |Proves the real code, bounded. Unproven on this repo; spike first. +|Verus/Creusot deductive proofs on kernels |★★★ |★★ |Heaviest; only for `content_store`-sized kernels. Not tried here. +|=== + +== E-1 and the capstone + +* *E-1 (`adversaryCannotKnow`):* `Knows` has no `Adversary` constructor, so `Knows Adversary a -> Void` +is true of _any_ model that omits the constructor. It states nothing. Delete it from the claims set (keep +as a note), or replace it with a symbolic derivability model plus an explicit assumption list. Likewise +`systemKnowsNewKey` is documented as not stating the negative. Revocation: `revoke`/`rotate` set a +`Revoked` flag and `retrieve` refuses it; the wrapped ciphertext stays in the store and anyone with the +KEK can unwrap it directly. So "a revoked key cannot be used" is an *API-level* property, not a +cryptographic one. Say that. +* *Capstone "every operation is reversible (MPR)":* C-3 proves a transaction containing an obliteration +is _not_ all-reversible. The capstone must be restated as the graded version G-1 already encodes: +every non-obliterating sequence is reversible; an obliterating step absorbs. + +== Recommended order + +[arabic] +. *Fix, then pin with tests* (no proof tooling): P5 path confinement, P4 atomic write, transaction log +atomic write, attestation field coverage + anchor, rename/qualify `ObliterationProof`. These are bugs, +not proof obligations; they change what is true. +. *Rewrite the six statements* in `PROOF-NEEDS.adoc` per this document, each with status +PROVEN / TESTED / ASSUMED / OPEN and the adversary named. This is the high-judgment step. +. *Then prove*, easiest first: R and C (content store), T2/item 6 (abstract), O-1 (model), A-1 (paper). +. Decide the crypto-shredding design before any proof of strong obliteration. + +== Beyond this repo (worth a list) + +* The five-status vocabulary and the dashboard-check ledger live in the estate canon (standards). +* `resource-algebra` must define homomorphisms before G-3 can move. +* The probe tests assert _current bad behaviour_. Do not merge them as-is. Turn each into a test that +fails today and passes after its fix. From 967edb8c496769009345919168b13f25c634060d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 22:51:47 +0000 Subject: [PATCH 03/16] docs(adr): propose obliteration by crypto-shredding, with the benchmark behind it ADR-0002 (status: proposed, nothing built) replaces the 3-pass overwrite, which cannot reach old bytes on SSDs, copy-on-write file systems, journals or snapshots, with per-blob random AES-256-GCM keys: erasing a blob is erasing its 32-byte key. It states where the key hierarchy must end (one root secret held where erasure is real), what becomes provable (O-1 restated as a finite-map lemma shaped like E-1m), the costs to reversibility (undo needs the store key), and four experiments to run before building, the decisive one a forensic scan of a btrfs loop image. benches/crypto_shred.rs measures the real store and obliterate_file against a prototype seal and key drop (one run, virtual disk, shape not absolute numbers): dropping a key costs ~0.74 ms whatever the blob size, overwriting a 16 MiB blob ~123 ms; sealing doubles store time at 16 MiB and is in the fsync noise below 64 KiB; a single key table is O(N) per shred (14 ms at 100k keys), so per-blob key files are proposed instead. cargo fmt --check and cargo clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 170 ++++++++++++++++++++++++++++ crates/januskey-cli/Cargo.toml | 5 + docs/adr/0002-crypto-shredding.adoc | 117 +++++++++++++++++++ 3 files changed, 292 insertions(+) create mode 100644 benches/crypto_shred.rs create mode 100644 docs/adr/0002-crypto-shredding.adoc diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs new file mode 100644 index 0000000..83f82b8 --- /dev/null +++ b/benches/crypto_shred.rs @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +// +// Crypto-shredding experiment for ADR-0002 (proposed). Nothing here is used by +// `jk`: it measures the real code where it exists and a prototype where it +// does not, so the design is argued from numbers. +// +// store: ContentStore::store (plaintext, temp/fsync/rename) against an +// AES-256-GCM seal under a fresh per-blob key followed by the same +// temp/fsync/rename write; `seal_only` is the cipher with no I/O. +// erase: obliterate_file on a blob (3-pass overwrite, unlink: the current +// path) against dropping the blob's key, either as one small key +// file (through the same obliterate_file) or as one entry removed +// from a key table that is rewritten atomically. +// +// Every stored payload is made unique per iteration, so dedupe never short- +// circuits a write. fsync cost depends on the device: numbers from a +// container's virtual disk say nothing about an SSD's flash translation layer. + +use aes_gcm::aead::{Aead, KeyInit}; +use aes_gcm::{Aes256Gcm, Nonce}; +use criterion::{ + black_box, criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion, Throughput, +}; +use januskey::obliteration::obliterate_file; +use januskey::{ContentHash, ContentStore}; +use rand::RngCore; +use std::fs; +use std::io::Write; +use std::path::Path; +use std::time::Duration; + +const SIZES: [usize; 4] = [4 << 10, 64 << 10, 1 << 20, 16 << 20]; +/// A table entry: 32-byte content address and 32-byte blob key. +const ENTRY: usize = 64; + +/// Make `data` differ from every earlier payload, so the store cannot dedupe. +fn make_unique(data: &mut [u8], n: &mut u64) { + *n += 1; + data[..8].copy_from_slice(&n.to_le_bytes()); +} + +/// Write `bytes` to `dir/name` the way the store does: temporary file, +/// fsync, rename, then fsync the directory. +fn write_synced(dir: &Path, name: &str, bytes: &[u8]) { + let tmp = dir.join(format!(".tmp-{name}")); + let mut f = fs::File::create(&tmp).unwrap(); + f.write_all(bytes).unwrap(); + f.sync_all().unwrap(); + fs::rename(&tmp, dir.join(name)).unwrap(); + fs::File::open(dir).unwrap().sync_all().unwrap(); +} + +/// Seal `data` under a fresh random key; returns (key, nonce || ciphertext). +fn seal(data: &[u8]) -> ([u8; 32], Vec) { + let mut key = [0u8; 32]; + let mut nonce = [0u8; 12]; + rand::rng().fill_bytes(&mut key); + rand::rng().fill_bytes(&mut nonce); + let cipher = Aes256Gcm::new_from_slice(&key).unwrap(); + let ct = cipher.encrypt(Nonce::from_slice(&nonce), data).unwrap(); + let mut out = Vec::with_capacity(12 + ct.len()); + out.extend_from_slice(&nonce); + out.extend_from_slice(&ct); + (key, out) +} + +fn bench_store(c: &mut Criterion) { + let mut g = c.benchmark_group("crypto_shred/store"); + g.sample_size(10).measurement_time(Duration::from_secs(3)); + for size in SIZES { + g.throughput(Throughput::Bytes(size as u64)); + g.bench_with_input(BenchmarkId::new("plaintext", size), &size, |b, &size| { + let dir = tempfile::tempdir().unwrap(); + let store = ContentStore::new(dir.path().to_path_buf(), false).unwrap(); + let (mut data, mut n) = (vec![0xCDu8; size], 0u64); + b.iter(|| { + make_unique(&mut data, &mut n); + black_box(store.store(&data).unwrap()); + }); + }); + g.bench_with_input(BenchmarkId::new("sealed", size), &size, |b, &size| { + let dir = tempfile::tempdir().unwrap(); + let (mut data, mut n) = (vec![0xCDu8; size], 0u64); + b.iter(|| { + make_unique(&mut data, &mut n); + let hash = ContentHash::from_bytes(&data); + let (key, sealed) = seal(&data); + write_synced(dir.path(), hash.raw_hash(), &sealed); + black_box(key); + }); + }); + g.bench_with_input(BenchmarkId::new("seal_only", size), &size, |b, &size| { + let (mut data, mut n) = (vec![0xCDu8; size], 0u64); + b.iter(|| { + make_unique(&mut data, &mut n); + black_box(seal(&data)); + }); + }); + } + g.finish(); +} + +fn bench_erase(c: &mut Criterion) { + let mut g = c.benchmark_group("crypto_shred/erase"); + g.sample_size(10).measurement_time(Duration::from_secs(3)); + + // The current path: overwrite the blob itself, whatever its size. + for size in SIZES { + g.bench_with_input( + BenchmarkId::new("overwrite_blob", size), + &size, + |b, &size| { + let dir = tempfile::tempdir().unwrap(); + let data = vec![0xEFu8; size]; + b.iter_batched( + || { + let p = dir.path().join("blob"); + fs::write(&p, &data).unwrap(); + p + }, + |p| black_box(obliterate_file(&p).unwrap()), + BatchSize::PerIteration, + ); + }, + ); + } + + // Crypto-shred, one key file per blob: the same overwrite, on 32 bytes. + g.bench_function("drop_key_file", |b| { + let dir = tempfile::tempdir().unwrap(); + b.iter_batched( + || { + let p = dir.path().join("key"); + let mut key = [0u8; 32]; + rand::rng().fill_bytes(&mut key); + fs::write(&p, key).unwrap(); + p + }, + |p| black_box(obliterate_file(&p).unwrap()), + BatchSize::PerIteration, + ); + }); + + // Crypto-shred, one key table: remove an entry, rewrite the table. + for entries in [1_000usize, 10_000, 100_000] { + g.bench_with_input( + BenchmarkId::new("drop_key_table", entries), + &entries, + |b, &entries| { + let dir = tempfile::tempdir().unwrap(); + let mut table = vec![0u8; entries * ENTRY]; + rand::rng().fill_bytes(&mut table); + let mut victim = 0usize; + b.iter(|| { + victim = (victim + 1) % entries; + let mut next = Vec::with_capacity(table.len() - ENTRY); + next.extend_from_slice(&table[..victim * ENTRY]); + next.extend_from_slice(&table[(victim + 1) * ENTRY..]); + write_synced(dir.path(), "keys", &next); + black_box(next.len()); + }); + }, + ); + } + g.finish(); +} + +criterion_group!(benches, bench_store, bench_erase); +criterion_main!(benches); diff --git a/crates/januskey-cli/Cargo.toml b/crates/januskey-cli/Cargo.toml index 3c98b8f..573416b 100644 --- a/crates/januskey-cli/Cargo.toml +++ b/crates/januskey-cli/Cargo.toml @@ -62,6 +62,11 @@ name = "januskey_benchmarks" harness = false path = "../../benches/januskey_benchmarks.rs" +[[bench]] +name = "crypto_shred" +harness = false +path = "../../benches/crypto_shred.rs" + [[bin]] name = "jk" path = "src/main.rs" diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc new file mode 100644 index 0000000..366fff9 --- /dev/null +++ b/docs/adr/0002-crypto-shredding.adoc @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) Jonathan D.A. Jewell += ADR-0002 — Obliteration by crypto-shredding +:toc: left + +* *Status:* proposed. Not accepted, and nothing is built. The benchmark in `benches/crypto_shred.rs` is the only code. +* *Date:* 2026-11-04 +* *Related:* +** ADR-0001: `jk` is reversible file operations, not a key system. +** `PROOF-NEEDS.adoc` rows O-1 (OPEN), X-1 (ASSUMED) and E-1m (PROVEN). +** `docs/reports/audit/proof-statement-audit-2026-10-27.adoc` §5. + +== 1. Context + +`jk obliterate` today calls `secure_overwrite`: three passes over the file, an fsync after each pass, then an unlink. It records an `ObliterationProof` whose `storage_cleared` is hard-coded to `true` (row O-1). + +Two things are wrong with treating this as erasure. + +* *The overwrite does not reach the old bytes on the media that matter.* +** On an SSD, the flash translation layer writes each pass to fresh cells. +** On a copy-on-write file system (btrfs, ZFS, APFS) each pass is a new extent. +** A journal, a snapshot or a backup keeps its own copy. +** NIST SP 800-88 Rev. 1 warns that overwriting may not reach all of the storage on a flash device. DoD 5220.22-M, which the code cites, dropped its overwrite procedure in the 2006 revision. Row X-1 is therefore an assumption that is false on most current disks. +* *The content address survives the erasure.* A blob is addressed by SHA-256 of its plaintext. That address stays in `obliterations.json` and in the proof (`content_hash`). Anyone who can guess low-entropy content, such as a short password or a yes/no field, can confirm the guess after obliteration. + +== 2. Proposal + +*Crypto-shredding:* each blob is stored encrypted under its own random key, and erasing a blob means erasing its key. + +[arabic] +. *Sealing.* `store` seals the content with AES-256-GCM under a fresh 256-bit key from the OS CSPRNG and a 96-bit random nonce. The blob file is `version byte ‖ nonce ‖ ciphertext ‖ tag`. +** *Never* use a convergent key (key = H(content)). Such a key can be recomputed from the content, so it cannot be shredded. +. *Deduplication is unchanged.* The key belongs to the blob, not to a path. A blob shared by two paths keeps its key until the last path that needs it is obliterated, which is the rule `obliterate_path` already applies to blobs. +. *Key storage.* Keep one small key file per blob under `.januskey/keys/` rather than a single table. A table rewrite is O(N) per shred (§3). Key files are wrapped under a store key; see §4 for where the store key comes from. +. *Shred.* `obliterate` and `gc` remove the blob's key file, then the blob. Once the key file is gone, every copy of the ciphertext anywhere (flash cells, snapshots, backups) is ciphertext under a key that no longer exists. +. *Addresses.* Use HMAC-SHA-256 under a per-store index key instead of a bare SHA-256, so that an address alone does not confirm a guessed plaintext. The index key is in turn an erasure target when the whole store is destroyed. +. *Proof object.* `ObliterationProof` loses `storage_cleared` and `overwrite_passes`. It records the key id that was destroyed, and is honestly named a *record*, not a proof. + +== 3. Measured cost (E1, done) + +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04. Conditions: 4 vCPU, a container's virtual disk, one run, and medians shown with Criterion's 95% intervals in the raw output. + +These figures are evidence about *shape*, not about real disks. fsync on a virtual disk is not fsync on an SSD. + +[cols="2,1,1,1,1",options="header"] +|=== +|Store (per blob) |4 KiB |64 KiB |1 MiB |16 MiB +|`ContentStore::store`, plaintext (today) |1.39 ms |1.95 ms |4.40 ms |40.2 ms +|seal + same synced write (prototype) |1.02 ms |1.57 ms |6.43 ms |81.1 ms +|seal only, no I/O |3.7 µs |54 µs |1.72 ms |40.0 ms +|=== + +[cols="2,1",options="header"] +|=== +|Erase |Time +|`obliterate_file` on the blob, 4 KiB (today) |0.72 ms +|`obliterate_file` on the blob, 1 MiB (today) |6.04 ms +|`obliterate_file` on the blob, 16 MiB (today) |122.9 ms +|drop one 32-byte key file (same `obliterate_file`) |0.74 ms, *independent of blob size* +|drop one entry from a key table, 1 000 / 10 000 / 100 000 keys |0.80 / 2.22 / 14.2 ms +|=== + +What the numbers say: + +* *Small blobs.* Below about 64 KiB, fsync dominates and the cipher is in the noise. The prototype is even faster than today's store, because it skips the store's two-level directory creation. That gap is an artefact of the prototype, not a gain from sealing. +* *Large blobs.* At 16 MiB, sealing doubles store time. The cipher ran at about 400 MiB/s at that size, against about 1.1 GiB/s on small inputs, which points at allocation and copying rather than AES itself. +* *Erasure.* Erasure becomes constant-time in blob size. On a 16 MiB blob it is about 166× cheaper, and on any medium it is a claim that can be made, whereas overwriting is not. +* *Key layout.* A single key table makes every shred rewrite the whole table, so it was rejected in favour of per-blob key files. + +== 4. Where the recursion ends (the honest limit) + +Crypto-shredding does not remove the physical-erasure assumption. It *shrinks* it: from "every copy of every blob, of any size, on any medium" to "one small secret held where erasure is real". + +A deleted key file can itself survive on flash. That is why key files are wrapped, and why the root must live somewhere else. + +[cols="3,1,1,4",options="header"] +|=== +|Root of the key hierarchy |Elegance |Dependability |Aspects +|Passphrase via Argon2id (already a dependency), plus a per-epoch random salt kept only in memory and in an OS keyring |★★ |★★ |Usable everywhere. Privacy is only as good as the passphrase. A shred becomes final when the epoch is rotated and the old epoch secret is gone. +|OS keyring or TPM / Secure Enclave sealed key |★★★ |★★★ |Hardware erase semantics. Platform-specific, so a Rust/SPARK seam and an Idris2 ABI per platform. This is the estate's FFI pattern. +|No root at rest: the key is supplied per session, for example by a hardware token |★★★ |★★ |The strongest privacy. Undo needs the token present, which changes how `jk` is used, and ADR-0001 says that is not the product. +|=== + +Recommended for a first build: the passphrase plus epoch rotation, with the TPM/keyring root as a later, separate seam. + +== 5. Statements this makes provable + +* *O-1, restated:* "after `shred(h)`, the live key store holds no key for `h`, and every stored copy of `h` is a sealed blob under a key with that id". +** This is a finite-map model in the Agda kernel. +** It has the same shape as `obliterate-cuts-recovery` (E-1m): recovery needs the key, and the key is not in any reachable state. +** It is provable, and then TESTED against the Rust through a conformance test. +* *X-1 becomes two assumptions:* +** AES-256-GCM is IND-CPA and INT-CTXT for one message per key. One message per key also removes nonce reuse as a failure mode. +** Erasing the root or epoch secret is real erasure. +* *C-3 is unchanged:* obliteration stays by design irreversible, and `undo` after a shred has nothing to restore, as today. + +== 6. Consequences + +* *Identity (ADR-0001).* Encryption is a *storage* detail here, as SHA-256 addressing already is. `jk` gains no key lifecycle, rotation UI or key export. The `jk-keys` lifecycle manager stays out of the product. The only shared code is the AEAD and KDF crates already in the dependency graph, so no new dependency is needed. +* *Format and migration.* Blobs gain a version byte. A store with plaintext blobs is read as today, and new writes are sealed. Old plaintext blobs need an explicit `jk migrate` that seals and shreds them. Until that runs, the old plaintext is exactly as exposed as it is now, and the docs must say so. +* *Reversibility.* Every `undo` now needs the store key. A lost passphrase loses history. That is a new failure mode, and it needs a recovery story (an escrow file, or an accepted loss) before acceptance. +* *Wording.* The DoD 5220.22-M and "cryptographic proof of erasure" claims are removed from the code, docs and ledger. + +== 7. Experiments before building (proposed) + +* *E2: real media.* Repeat E1 on ext4 over an NVMe SSD and on btrfs. Report the fsync share of store time. +* *E3: cipher choice.* Compare ChaCha20-Poly1305 with AES-256-GCM on a CPU without AES-NI. Check that the 16 MiB seal is limited by allocation, not by the cipher, by sealing in place (`encrypt_in_place`). +* *E4: forensic acceptance test.* This is the decisive one. Build a btrfs file system on a loop device, store a marker payload, obliterate it, then scan the raw image for the marker. +** *Expected for today's overwrite:* the marker is found (the CoW extents survive). +** *Expected with sealing:* the marker is never found, because the plaintext is never written. +** It becomes a TESTED row if it holds. +* *E5: shred cost with the store key wrapped.* Add one key unwrap and rewrap to the key-file path, and confirm the cost stays constant in blob size. + +== 8. Beyond this repository + +* The same erasure-on-flash problem applies to any estate repository that promises deletion: consent and retention tooling, and obliteration in other stores. If this ADR is accepted, the root-key seam (§4) is a candidate for a shared crate rather than a januskey-only module. +* `docs/security/THREAT_MODEL.adoc` covers only the key store. It needs a section on the obliteration path whichever way this ADR is decided. From 56e64ea45b3c49f83937cf98bd268a901003b8a1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 22:54:04 +0000 Subject: [PATCH 04/16] ci: name secrets instead of inheriting them across the owner boundary; narrow S-2 GitHub permits `secrets: inherit` only within one organisation or enterprise. The re-pin moved the callee from the hyperpolymath user account to the metadatastician organisation, so hypatia-scan, mirror and scorecard now pass exactly the secrets their reusables declare: HYPATIA_SCAN_PAT; the seven optional mirror keys; none for scorecard. cicd-squabbler already dropped every `inherit` for the same reason. This is also what SonarCloud flagged (hypatia-scan.yml:32, mirror.yml:18, scorecard.yml:21: the C security rating on new code). S-2 claimed the store's write is atomic, but no test injects a mid-write failure; it is atomic by construction only. The row and the obligation text now say so, and claim only what the two tests show: a damaged blob is repaired and no temporary file is left. Checked: actionlint 1.7.7 (checksum-verified) on all seven callers, check-lock-sync.sh, YAML parse of the three edited callers, and dashboard-check (TESTED rows and counts pass; PROVEN rows need idris2/agda, absent here). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- .github/workflows/hypatia-scan.yml | 6 +++++- .github/workflows/mirror.yml | 12 +++++++++++- .github/workflows/scorecard.yml | 3 ++- PROOF-NEEDS.adoc | 4 ++-- .../audit/proof-statement-audit-2026-10-27.adoc | 2 +- 5 files changed, 21 insertions(+), 6 deletions(-) diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index 0fd655e..ffa84bc 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -29,4 +29,8 @@ permissions: jobs: hypatia: uses: metadatastician/standards/.github/workflows/hypatia-scan-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 - secrets: inherit + # Named, not `inherit`: GitHub refuses inheritance from a user account + # (hyperpolymath) into an organisation (metadatastician). The reusable + # declares this one optional secret. + secrets: + HYPATIA_SCAN_PAT: ${{ secrets.HYPATIA_SCAN_PAT }} diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index 62dace9..1940333 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -15,7 +15,17 @@ permissions: jobs: mirror: uses: metadatastician/standards/.github/workflows/mirror-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 - secrets: inherit + # Named, not `inherit`: metadatastician is an organisation and hyperpolymath a + # user account, and GitHub refuses `secrets: inherit` across that boundary. + # These are the secrets mirror-reusable.yml declares (all optional). + secrets: + GITLAB_SSH_KEY: ${{ secrets.GITLAB_SSH_KEY }} + BITBUCKET_SSH_KEY: ${{ secrets.BITBUCKET_SSH_KEY }} + CODEBERG_SSH_KEY: ${{ secrets.CODEBERG_SSH_KEY }} + SOURCEHUT_SSH_KEY: ${{ secrets.SOURCEHUT_SSH_KEY }} + DISROOT_SSH_KEY: ${{ secrets.DISROOT_SSH_KEY }} + GITEA_SSH_KEY: ${{ secrets.GITEA_SSH_KEY }} + RADICLE_KEY: ${{ secrets.RADICLE_KEY }} permissions: actions: read contents: read diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 954018e..93560d1 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -18,4 +18,5 @@ jobs: security-events: write id-token: write uses: metadatastician/standards/.github/workflows/scorecard-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 - secrets: inherit + # No `secrets:` block: scorecard-reusable.yml declares none, and GitHub + # refuses `secrets: inherit` from a user account into an organisation. diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index 80dba81..1ec8377 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -76,7 +76,7 @@ tables must agree. | T-7 | Transaction ids (UUID v4) never collide within one log (the freshness premise of T-2) | ASSUMED | probabilistic, not checked | T-8 | The operation metadata log is saved atomically, and an undo whose `+mark_undone+` fails to persist can be retried | OPEN | by inspection: the log now takes the same write, sync, rename path as the transaction log (`+crate::transaction::write_atomic+`), and `+metadata::tests::save_replaces_the_log_atomically+` asserts it, but the crate has not been built since that edit, so the row stays OPEN until the test is green in CI. Retry is unchanged: `+undo+` still runs the undo before `+mark_undone+` persists, so a failed persist still leaves the file undone and the log not | S-1 | A content hash that is not `+sha256:+` plus 64 lowercase hex digits never resolves to a path outside the store root | TESTED | `+content_store::tests::malformed_hash_never_leaves_the_store_root+`; fixed in #163 after audit probe P5 showed `+delete+` removing a file outside the store (`+docs/reports/audit/proof-statement-audit-2026-10-27.adoc+`) -| S-2 | `+store+` writes a blob by temporary file, sync and rename, and repairs a blob a crash left damaged instead of deduplicating against it | TESTED | `+content_store::tests::store_repairs_a_blob_truncated_by_a_crash+`, `+store_leaves_no_temp_files_and_count_ignores_strays+`; fixed in #163 (probe P4) +| S-2 | `+store+` repairs a blob a crash left damaged instead of deduplicating against it, and leaves no temporary file behind | TESTED | not claimed: that the write itself is atomic. `+store+` writes by temporary file, sync and rename, but by construction only; no test injects a failure mid-write. `+content_store::tests::store_repairs_a_blob_truncated_by_a_crash+`, `+store_leaves_no_temp_files_and_count_ignores_strays+`; fixed in #163 (probe P4) | L-1 | Editing any field of any audit-log entry makes `+verify_integrity+` fail | OPEN | false today, probes P1 and P2: algorithm, purpose, old_state, new_state, rotated_to and rotated_from are in neither the MAC nor the chain hash, and key_id, fingerprint and reason are chain-hashed only, so on the last entry nothing protects them. The fix is a MAC over every field (a scheme version bump) | L-2 | Removing trailing audit-log entries makes `+verify_integrity+` fail | OPEN | false today, probe P3: every prefix of a valid log verifies. Needs an anchor held outside the log (a design decision: signed head, external counter, or witness) | O-1 | An `+ObliterationProof+` attests that the content it names was erased | OPEN | false today, probe P6: `+ObliterationProof::generate+` sets `+storage_cleared: true+` unconditionally and `+verify_commitment+` holds for content that was never stored. Either compute the field from the erasure, or stop calling the object a proof @@ -184,7 +184,7 @@ here only while `+keys.rs+` remains in this tree (ADR-0001) attestations undermine trust |Content store integrity |Content-addressed storage never returns wrong -content (paths confined to the store and writes atomic: S-1 and S-2 are TESTED; reads verify the hash) |Hash collisions or bugs corrupt stored data +content (paths confined to the store and damaged blobs repaired: S-1 and S-2 are TESTED; the write is atomic by construction, not by test; reads verify the hash) |Hash collisions or bugs corrupt stored data |Transaction atomicity |Delta application is atomic and reversible (the model is proved, T-1 to T-3; the Rust is tested for partial-rollback retry, T-4, and failed saves, T-5; the premises T-6 and T-7 are assumed; the metadata log, T-8, now saves by rename but that is unverified) |Partial transactions corrupt state diff --git a/docs/reports/audit/proof-statement-audit-2026-10-27.adoc b/docs/reports/audit/proof-statement-audit-2026-10-27.adoc index 04eb0c0..e3e72dc 100644 --- a/docs/reports/audit/proof-statement-audit-2026-10-27.adoc +++ b/docs/reports/audit/proof-statement-audit-2026-10-27.adoc @@ -8,7 +8,7 @@ ==== This report records the audit as it was run on `main` at 10f0bc2. Since then: -* *Fixed*: P4 and P5 (content store). Writes are now atomic and a damaged blob is repaired, and a malformed hash cannot leave the store root (#163). These are now ledger rows S-1 and S-2. +* *Fixed*: P4 and P5 (content store). A damaged blob is now repaired, writes go by temporary file and rename (by construction; no test injects a mid-write failure), and a malformed hash cannot leave the store root (#163). These are now ledger rows S-1 and S-2. * *Fixed*: transaction-log persistence and resumable rollback (T-4 and T-5, #162 and #164). * *Still open*: ** P1 to P3 (attestation log): uncovered fields, the last entry, and tail truncation. These are now ledger rows L-1 and L-2. From 85482350340f72471099eea3869d8172a938f7dd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:02:02 +0000 Subject: [PATCH 05/16] docs(adr-0002): correct the shred claim, its scope, the migration and the store cost Four review findings on the proposal, all correct: - Removing a key file is not erasure: a snapshot or CoW extent can keep it, still wrapped under the live epoch key. A shred is now "pending" until the epoch it was wrapped under is rotated away, and O-1 is restated over that rotation; per-shred versus batched rotation is left as an acceptance decision. - The guarantee covers content-store copies only. The working file was plaintext on the medium before jk saw it; E4 gains a product-flow case that is expected to find the marker, so the limit stays tested. - HMAC addresses would orphan every ContentHash in the operation log. Migration now versions ContentHash, rewrites the log in one atomic save, and shreds the SHA-addressed copies only after that commits. - The sealed-store benchmark omitted the key file's own synced write. Re-measured with it: a sealed 4 KiB store is ~1.7x the plaintext one (2.69 vs 1.61 ms), not "in the noise"; the cost is fsync count. cargo fmt --check and clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 7 ++++-- docs/adr/0002-crypto-shredding.adoc | 37 ++++++++++++++++++++++------- 2 files changed, 33 insertions(+), 11 deletions(-) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index 83f82b8..7480136 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -7,7 +7,8 @@ // // store: ContentStore::store (plaintext, temp/fsync/rename) against an // AES-256-GCM seal under a fresh per-blob key followed by the same -// temp/fsync/rename write; `seal_only` is the cipher with no I/O. +// temp/fsync/rename write for the blob and again for its key file; +// `seal_only` is the cipher with no I/O. // erase: obliterate_file on a blob (3-pass overwrite, unlink: the current // path) against dropping the blob's key, either as one small key // file (through the same obliterate_file) or as one entry removed @@ -87,7 +88,9 @@ fn bench_store(c: &mut Criterion) { let hash = ContentHash::from_bytes(&data); let (key, sealed) = seal(&data); write_synced(dir.path(), hash.raw_hash(), &sealed); - black_box(key); + // The key must be durable too: one more synced write. It is + // stored unwrapped here; wrapping adds microseconds (E5). + write_synced(dir.path(), &format!("{}.key", hash.raw_hash()), &key); }); }); g.bench_with_input(BenchmarkId::new("seal_only", size), &size, |b, &size| { diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 366fff9..f965a9f 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -32,7 +32,16 @@ Two things are wrong with treating this as erasure. ** *Never* use a convergent key (key = H(content)). Such a key can be recomputed from the content, so it cannot be shredded. . *Deduplication is unchanged.* The key belongs to the blob, not to a path. A blob shared by two paths keeps its key until the last path that needs it is obliterated, which is the rule `obliterate_path` already applies to blobs. . *Key storage.* Keep one small key file per blob under `.januskey/keys/` rather than a single table. A table rewrite is O(N) per shred (§3). Key files are wrapped under a store key; see §4 for where the store key comes from. -. *Shred.* `obliterate` and `gc` remove the blob's key file, then the blob. Once the key file is gone, every copy of the ciphertext anywhere (flash cells, snapshots, backups) is ciphertext under a key that no longer exists. +. *Shred.* `obliterate` and `gc` remove the blob's key file, then the blob. Removing the key file is *not* yet erasure: +** A snapshot, a backup or a copy-on-write extent can keep the deleted key file. +** That retained copy is still wrapped under the current store or epoch key, so it can be unwrapped. +** A shred is therefore *pending* until the wrapping key it was made under is destroyed. That happens through epoch rotation: rewrap every live key under a fresh epoch key, then destroy the old one. +** Only then is every retained copy of the blob's ciphertext under a key that cannot be recovered. +** Rotation costs O(live keys). It can run on every shred, which makes shreds final at once but costly, or in batches, which leaves a bounded window in which shreds are pending. That trade-off is a decision for acceptance (§4). +. *Scope: content-store copies only.* The guarantee covers what JanusKey writes: blobs and keys. It does not cover the working file itself, which `jk obliterate` targets too. +** That file existed in plaintext on the same medium before JanusKey ever saw it, so CoW extents, flash remapping, snapshots or backups may still hold its plaintext. +** Covering it needs encryption from the file's creation, meaning an encrypted volume underneath, which is outside JanusKey. +** The record says which copies the shred covers, and the docs say the working file is overwritten best-effort only. . *Addresses.* Use HMAC-SHA-256 under a per-store index key instead of a bare SHA-256, so that an address alone does not confirm a guessed plaintext. The index key is in turn an erasure target when the whole store is destroyed. . *Proof object.* `ObliterationProof` loses `storage_cleared` and `overwrite_passes`. It records the key id that was destroyed, and is honestly named a *record*, not a proof. @@ -46,10 +55,12 @@ These figures are evidence about *shape*, not about real disks. fsync on a virtu |=== |Store (per blob) |4 KiB |64 KiB |1 MiB |16 MiB |`ContentStore::store`, plaintext (today) |1.39 ms |1.95 ms |4.40 ms |40.2 ms -|seal + same synced write (prototype) |1.02 ms |1.57 ms |6.43 ms |81.1 ms +|seal + synced blob write + synced key-file write (prototype; key unwrapped) |2.69 ms |3.33 ms |8.11 ms |85.5 ms |seal only, no I/O |3.7 µs |54 µs |1.72 ms |40.0 ms |=== +The sealed row was re-measured after review. The first run wrote only the blob, so it left out the second durable write that every sealed blob needs, and it understated small-blob cost. The plaintext row in that re-run (1.61 / 1.82 / 4.75 / 40.3 ms) agreed with the first run within its intervals. Wrapping the key and any crash coordination between the two writes are still not included (E5). + [cols="2,1",options="header"] |=== |Erase |Time @@ -62,8 +73,8 @@ These figures are evidence about *shape*, not about real disks. fsync on a virtu What the numbers say: -* *Small blobs.* Below about 64 KiB, fsync dominates and the cipher is in the noise. The prototype is even faster than today's store, because it skips the store's two-level directory creation. That gap is an artefact of the prototype, not a gain from sealing. -* *Large blobs.* At 16 MiB, sealing doubles store time. The cipher ran at about 400 MiB/s at that size, against about 1.1 GiB/s on small inputs, which points at allocation and copying rather than AES itself. +* *Small blobs.* Below about 64 KiB, the cipher costs microseconds, but the second synced write, for the key file, makes a sealed store about 1.7× slower (2.69 against 1.61 ms at 4 KiB). The cost is fsync count, not cryptography. Grouping key files into one synced write per transaction is the obvious lever, at the price of the O(N) rewrite rejected below. E2 must measure this on real media. +* *Large blobs.* At 16 MiB, sealing roughly doubles store time. The cipher ran at about 400 MiB/s at that size, against about 1.1 GiB/s on small inputs, which points at allocation and copying rather than AES itself. * *Erasure.* Erasure becomes constant-time in blob size. On a 16 MiB blob it is about 166× cheaper, and on any medium it is a claim that can be made, whereas overwriting is not. * *Key layout.* A single key table makes every shred rewrite the whole table, so it was rejected in favour of per-blob key files. @@ -85,7 +96,7 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ == 5. Statements this makes provable -* *O-1, restated:* "after `shred(h)`, the live key store holds no key for `h`, and every stored copy of `h` is a sealed blob under a key with that id". +* *O-1, restated:* "after `shred(h)` and the epoch rotation that follows it, no reachable state, retained copies of the key directory included, holds an unwrappable key for `h`; and every stored copy of `h` is a sealed blob under a key with that id". The working file is outside this statement (§2). ** This is a finite-map model in the Agda kernel. ** It has the same shape as `obliterate-cuts-recovery` (E-1m): recovery needs the key, and the key is not in any reachable state. ** It is provable, and then TESTED against the Rust through a conformance test. @@ -98,6 +109,10 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ * *Identity (ADR-0001).* Encryption is a *storage* detail here, as SHA-256 addressing already is. `jk` gains no key lifecycle, rotation UI or key export. The `jk-keys` lifecycle manager stays out of the product. The only shared code is the AEAD and KDF crates already in the dependency graph, so no new dependency is needed. * *Format and migration.* Blobs gain a version byte. A store with plaintext blobs is read as today, and new writes are sealed. Old plaintext blobs need an explicit `jk migrate` that seals and shreds them. Until that runs, the old plaintext is exactly as exposed as it is now, and the docs must say so. +** *Addresses are part of the history.* The operation log serialises each blob's `ContentHash`, and `undo` passes that exact value to `ContentStore::retrieve`. Moving blobs to HMAC addresses would leave every historical operation pointing at a blob that no longer exists. +** `ContentHash` therefore becomes versioned (`sha256:` or `hmac-sha256:`), and `retrieve` resolves either form. +** Migration proceeds in three steps: seal each blob at its new address; rewrite every reference in the operation log in one atomic save (the log already saves by write, sync and rename, row T-8); only then shred the SHA-addressed copies. +** A crash between the steps leaves both copies, never neither. The SHA address keeps its dictionary exposure until the rewrite commits. * *Reversibility.* Every `undo` now needs the store key. A lost passphrase loses history. That is a new failure mode, and it needs a recovery story (an escrow file, or an accepted loss) before acceptance. * *Wording.* The DoD 5220.22-M and "cryptographic proof of erasure" claims are removed from the code, docs and ledger. @@ -105,10 +120,14 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ * *E2: real media.* Repeat E1 on ext4 over an NVMe SSD and on btrfs. Report the fsync share of store time. * *E3: cipher choice.* Compare ChaCha20-Poly1305 with AES-256-GCM on a CPU without AES-NI. Check that the 16 MiB seal is limited by allocation, not by the cipher, by sealing in place (`encrypt_in_place`). -* *E4: forensic acceptance test.* This is the decisive one. Build a btrfs file system on a loop device, store a marker payload, obliterate it, then scan the raw image for the marker. -** *Expected for today's overwrite:* the marker is found (the CoW extents survive). -** *Expected with sealing:* the marker is never found, because the plaintext is never written. -** It becomes a TESTED row if it holds. +* *E4: forensic acceptance test.* This is the decisive one. Build a btrfs file system on a loop device, run the steps below, then scan the raw image for a marker payload. There are two cases. +** *Store-only.* Store the marker straight into the content store, then shred it and rotate the epoch. +*** Expected for today's overwrite: the marker is found, because the CoW extents survive. +*** Expected with sealing: the marker is never found, because the store never writes plaintext. +*** It becomes a TESTED row if it holds. +** *Product flow.* Create a working file containing the marker, record it with `jk`, then `jk obliterate` it. +*** Expected in both designs: the marker *is* found, from the working file's own extents. +*** This case is kept as a test of the scope limit in §2, so that no one later reads the store-only result as covering the working file. * *E5: shred cost with the store key wrapped.* Add one key unwrap and rewrap to the key-file path, and confirm the cost stays constant in blob size. == 8. Beyond this repository From ecc5bca545e6db085ada23a1b2bec6c90461437f Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:15:34 +0000 Subject: [PATCH 06/16] docs(adr-0002): measure the final shred and the default store; redact old records Three more review findings, all correct, and two change the design: - A final shred must re-seal every live key under a fresh epoch key. Measured: one sealed key table costs 1.2 / 3.9 / 34 ms at 1k / 10k / 100k keys (one synced write); one file per key costs 1.2 s at 1k keys. Key storage therefore flips to the single sealed table, and the "166x" comparison of a pending key drop against today's overwrite is withdrawn. - jk compresses by default, and the benchmark had compression off with compressible payloads. It now uses random (incompressible) payloads and measures the default store and compress-then-seal: sealing adds ~1.7x at 4 KiB (the key's fsync) and ~7% at 16 MiB, where gzip on incompressible input is the dominant cost today (545 vs 40 ms). - obliterations.json keeps the SHA-256 address of content already erased, so migration cannot rewrite it. The ADR now proposes redacting hash, nonce and commitment from legacy records, as a decision for acceptance. E5 now covers the full final shred; E6 measures compressible payloads and skipping gzip when it does not pay. fmt and clippy --benches pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 147 +++++++++++++++++++--------- docs/adr/0002-crypto-shredding.adoc | 69 ++++++++----- 2 files changed, 145 insertions(+), 71 deletions(-) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index 7480136..2482f98 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -5,24 +5,32 @@ // `jk`: it measures the real code where it exists and a prototype where it // does not, so the design is argued from numbers. // -// store: ContentStore::store (plaintext, temp/fsync/rename) against an -// AES-256-GCM seal under a fresh per-blob key followed by the same -// temp/fsync/rename write for the blob and again for its key file; +// store: ContentStore::store with compression off and with it on (the +// `jk` default), against the proposed path: gzip, then an +// AES-256-GCM seal under a fresh per-blob key, then the same +// temp/fsync/rename write for the blob and again for its key file. // `seal_only` is the cipher with no I/O. // erase: obliterate_file on a blob (3-pass overwrite, unlink: the current -// path) against dropping the blob's key, either as one small key -// file (through the same obliterate_file) or as one entry removed -// from a key table that is rewritten atomically. +// path) against crypto-shredding, in its two stages: +// pending: drop the blob's key file (through the same +// obliterate_file); copies retained elsewhere are still +// wrapped under the live epoch key; +// final: drop the entry and re-seal every live key under a +// fresh epoch key, either as one table (one synced +// write) or as one file per key (one synced write each). // -// Every stored payload is made unique per iteration, so dedupe never short- -// circuits a write. fsync cost depends on the device: numbers from a -// container's virtual disk say nothing about an SSD's flash translation layer. +// Payloads are random, so incompressible: the worst case for gzip, which then +// costs time and saves nothing. Each is made unique per iteration, so dedupe +// never short-circuits a write. fsync cost depends on the device: numbers +// from a container's virtual disk say nothing about an SSD's flash +// translation layer. use aes_gcm::aead::{Aead, KeyInit}; use aes_gcm::{Aes256Gcm, Nonce}; use criterion::{ black_box, criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion, Throughput, }; +use flate2::{write::GzEncoder, Compression}; use januskey::obliteration::obliterate_file; use januskey::{ContentHash, ContentStore}; use rand::RngCore; @@ -32,8 +40,16 @@ use std::path::Path; use std::time::Duration; const SIZES: [usize; 4] = [4 << 10, 64 << 10, 1 << 20, 16 << 20]; -/// A table entry: 32-byte content address and 32-byte blob key. -const ENTRY: usize = 64; +/// A key-table entry: 32-byte content address, then the blob key wrapped +/// under the epoch key (12-byte nonce, 32-byte key, 16-byte tag). +const ENTRY: usize = 32 + 12 + 32 + 16; + +/// `size` random bytes: incompressible, the worst case for gzip. +fn random_payload(size: usize) -> Vec { + let mut data = vec![0u8; size]; + rand::rng().fill_bytes(&mut data); + data +} /// Make `data` differ from every earlier payload, so the store cannot dedupe. fn make_unique(data: &mut [u8], n: &mut u64) { @@ -66,35 +82,49 @@ fn seal(data: &[u8]) -> ([u8; 32], Vec) { (key, out) } +/// gzip at the store's level, as `ContentStore::write_blob` does. +fn gzip(data: &[u8]) -> Vec { + let mut enc = GzEncoder::new(Vec::new(), Compression::default()); + enc.write_all(data).unwrap(); + enc.finish().unwrap() +} + fn bench_store(c: &mut Criterion) { let mut g = c.benchmark_group("crypto_shred/store"); g.sample_size(10).measurement_time(Duration::from_secs(3)); for size in SIZES { g.throughput(Throughput::Bytes(size as u64)); - g.bench_with_input(BenchmarkId::new("plaintext", size), &size, |b, &size| { - let dir = tempfile::tempdir().unwrap(); - let store = ContentStore::new(dir.path().to_path_buf(), false).unwrap(); - let (mut data, mut n) = (vec![0xCDu8; size], 0u64); - b.iter(|| { - make_unique(&mut data, &mut n); - black_box(store.store(&data).unwrap()); - }); - }); - g.bench_with_input(BenchmarkId::new("sealed", size), &size, |b, &size| { - let dir = tempfile::tempdir().unwrap(); - let (mut data, mut n) = (vec![0xCDu8; size], 0u64); - b.iter(|| { - make_unique(&mut data, &mut n); - let hash = ContentHash::from_bytes(&data); - let (key, sealed) = seal(&data); - write_synced(dir.path(), hash.raw_hash(), &sealed); - // The key must be durable too: one more synced write. It is - // stored unwrapped here; wrapping adds microseconds (E5). - write_synced(dir.path(), &format!("{}.key", hash.raw_hash()), &key); + for (name, compression) in [("plaintext_raw", false), ("plaintext_default", true)] { + g.bench_with_input(BenchmarkId::new(name, size), &size, |b, &size| { + let dir = tempfile::tempdir().unwrap(); + let store = ContentStore::new(dir.path().to_path_buf(), compression).unwrap(); + let (mut data, mut n) = (random_payload(size), 0u64); + b.iter(|| { + make_unique(&mut data, &mut n); + black_box(store.store(&data).unwrap()); + }); }); - }); + } + g.bench_with_input( + BenchmarkId::new("sealed_default", size), + &size, + |b, &size| { + let dir = tempfile::tempdir().unwrap(); + let (mut data, mut n) = (random_payload(size), 0u64); + b.iter(|| { + make_unique(&mut data, &mut n); + let hash = ContentHash::from_bytes(&data); + // Compress before sealing: ciphertext does not compress. + let (key, sealed) = seal(&gzip(&data)); + write_synced(dir.path(), hash.raw_hash(), &sealed); + // The key must be durable too: one more synced write. It is + // stored unwrapped here; wrapping adds microseconds (E5). + write_synced(dir.path(), &format!("{}.key", hash.raw_hash()), &key); + }); + }, + ); g.bench_with_input(BenchmarkId::new("seal_only", size), &size, |b, &size| { - let (mut data, mut n) = (vec![0xCDu8; size], 0u64); + let (mut data, mut n) = (random_payload(size), 0u64); b.iter(|| { make_unique(&mut data, &mut n); black_box(seal(&data)); @@ -115,7 +145,7 @@ fn bench_erase(c: &mut Criterion) { &size, |b, &size| { let dir = tempfile::tempdir().unwrap(); - let data = vec![0xEFu8; size]; + let data = random_payload(size); b.iter_batched( || { let p = dir.path().join("blob"); @@ -129,15 +159,13 @@ fn bench_erase(c: &mut Criterion) { ); } - // Crypto-shred, one key file per blob: the same overwrite, on 32 bytes. - g.bench_function("drop_key_file", |b| { + // Pending shred: drop one key file, the same overwrite on 32 bytes. + g.bench_function("pending_drop_key_file", |b| { let dir = tempfile::tempdir().unwrap(); b.iter_batched( || { let p = dir.path().join("key"); - let mut key = [0u8; 32]; - rand::rng().fill_bytes(&mut key); - fs::write(&p, key).unwrap(); + fs::write(&p, random_payload(32)).unwrap(); p }, |p| black_box(obliterate_file(&p).unwrap()), @@ -145,27 +173,56 @@ fn bench_erase(c: &mut Criterion) { ); }); - // Crypto-shred, one key table: remove an entry, rewrite the table. + // Final shred, one table: drop the entry, re-seal the whole table under a + // fresh epoch key, one synced write. (Destroying the old epoch secret is + // not measured: where it lives is the open question of ADR §4.) for entries in [1_000usize, 10_000, 100_000] { g.bench_with_input( - BenchmarkId::new("drop_key_table", entries), + BenchmarkId::new("final_reseal_table", entries), &entries, |b, &entries| { let dir = tempfile::tempdir().unwrap(); - let mut table = vec![0u8; entries * ENTRY]; - rand::rng().fill_bytes(&mut table); + let table = random_payload(entries * ENTRY); let mut victim = 0usize; b.iter(|| { victim = (victim + 1) % entries; let mut next = Vec::with_capacity(table.len() - ENTRY); next.extend_from_slice(&table[..victim * ENTRY]); next.extend_from_slice(&table[(victim + 1) * ENTRY..]); - write_synced(dir.path(), "keys", &next); - black_box(next.len()); + let (epoch_key, sealed) = seal(&next); + write_synced(dir.path(), "keys", &sealed); + black_box(epoch_key); }); }, ); } + + // Final shred, one file per key: rewrap every live key file under a fresh + // epoch key, one synced write each. + let entries = 1_000usize; + g.bench_with_input( + BenchmarkId::new("final_rewrap_files", entries), + &entries, + |b, &entries| { + let dir = tempfile::tempdir().unwrap(); + let names: Vec = (0..entries).map(|i| format!("{i:08}.key")).collect(); + b.iter(|| { + let mut epoch_key = [0u8; 32]; + rand::rng().fill_bytes(&mut epoch_key); + let cipher = Aes256Gcm::new_from_slice(&epoch_key).unwrap(); + for name in &names { + let mut nonce = [0u8; 12]; + rand::rng().fill_bytes(&mut nonce); + let blob_key = random_payload(32); + let wrapped = cipher + .encrypt(Nonce::from_slice(&nonce), blob_key.as_slice()) + .unwrap(); + write_synced(dir.path(), name, &wrapped); + } + black_box(epoch_key); + }); + }, + ); g.finish(); } diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index f965a9f..f26197c 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -31,13 +31,15 @@ Two things are wrong with treating this as erasure. . *Sealing.* `store` seals the content with AES-256-GCM under a fresh 256-bit key from the OS CSPRNG and a 96-bit random nonce. The blob file is `version byte ‖ nonce ‖ ciphertext ‖ tag`. ** *Never* use a convergent key (key = H(content)). Such a key can be recomputed from the content, so it cannot be shredded. . *Deduplication is unchanged.* The key belongs to the blob, not to a path. A blob shared by two paths keeps its key until the last path that needs it is obliterated, which is the rule `obliterate_path` already applies to blobs. -. *Key storage.* Keep one small key file per blob under `.januskey/keys/` rather than a single table. A table rewrite is O(N) per shred (§3). Key files are wrapped under a store key; see §4 for where the store key comes from. -. *Shred.* `obliterate` and `gc` remove the blob's key file, then the blob. Removing the key file is *not* yet erasure: -** A snapshot, a backup or a copy-on-write extent can keep the deleted key file. -** That retained copy is still wrapped under the current store or epoch key, so it can be unwrapped. -** A shred is therefore *pending* until the wrapping key it was made under is destroyed. That happens through epoch rotation: rewrap every live key under a fresh epoch key, then destroy the old one. -** Only then is every retained copy of the blob's ciphertext under a key that cannot be recovered. -** Rotation costs O(live keys). It can run on every shred, which makes shreds final at once but costly, or in batches, which leaves a bounded window in which shreds are pending. That trade-off is a decision for acceptance (§4). +. *Key storage.* Keep all blob keys in *one key table* (`.januskey/keys`), sealed as a whole under the current *epoch key*. See §4 for where the epoch key comes from. +** An earlier draft proposed one key file per blob, because dropping a single file is cheap. +** But making a shred final requires every live key to be re-sealed under a fresh epoch key. With per-blob files that is one synced write per live key: 1.2 s for only 1 000 keys (§3). +** The single table makes a final shred one synced write, whatever the number of keys. +. *Shred.* `obliterate` and `gc` drop the blob's entry and write the table again under a *fresh* epoch key. Then they destroy the old epoch key, then delete the blob. +** Destroying the old epoch key is what makes the shred final. A snapshot, a backup or a copy-on-write extent can keep an old copy of the table, but that copy is sealed under an epoch key that no longer exists. +** If the old epoch key is not destroyed, for example because the new table is written but the old key survives, the shred is only *pending*. Every retained copy of the old table, and so of the blob's key, can still be opened. +** The cost of a final shred is O(live keys): 1.2 ms at 1 000 keys, 34 ms at 100 000 (§3). +** Shreds could instead be batched, so that one rotation finalises many. That leaves a window in which a shred is pending. The ADR's default is a final shred every time; batching is an option to decide at acceptance. . *Scope: content-store copies only.* The guarantee covers what JanusKey writes: blobs and keys. It does not cover the working file itself, which `jk obliterate` targets too. ** That file existed in plaintext on the same medium before JanusKey ever saw it, so CoW extents, flash remapping, snapshots or backups may still hold its plaintext. ** Covering it needs encryption from the file's creation, meaning an encrypted volume underneath, which is outside JanusKey. @@ -47,42 +49,50 @@ Two things are wrong with treating this as erasure. == 3. Measured cost (E1, done) -`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04. Conditions: 4 vCPU, a container's virtual disk, one run, and medians shown with Criterion's 95% intervals in the raw output. +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after two rounds of review corrections. -These figures are evidence about *shape*, not about real disks. fsync on a virtual disk is not fsync on an SSD. +* *Conditions:* 4 vCPU, a container's virtual disk, one run, and medians, with Criterion's 95% intervals in the raw output. +* *Payloads:* random, so incompressible. That is the worst case for gzip: it costs time and saves nothing. Compressible content would make every gzip row faster, and the sealed row with it, since that row compresses before sealing. +* *Limits:* these figures are evidence about *shape*, not about real disks. fsync on a virtual disk is not fsync on an SSD. [cols="2,1,1,1,1",options="header"] |=== |Store (per blob) |4 KiB |64 KiB |1 MiB |16 MiB -|`ContentStore::store`, plaintext (today) |1.39 ms |1.95 ms |4.40 ms |40.2 ms -|seal + synced blob write + synced key-file write (prototype; key unwrapped) |2.69 ms |3.33 ms |8.11 ms |85.5 ms -|seal only, no I/O |3.7 µs |54 µs |1.72 ms |40.0 ms +|`ContentStore::store`, compression off |0.92 ms |2.10 ms |6.05 ms |40.1 ms +|`ContentStore::store`, compression on (the `jk` default, today) |1.70 ms |4.40 ms |35.7 ms |545 ms +|proposed: gzip, then seal, then a synced blob write and a synced key write (key unwrapped) |2.88 ms |5.22 ms |39.0 ms |582 ms +|seal only, no I/O |3.7 µs |55 µs |1.65 ms |38.6 ms |=== -The sealed row was re-measured after review. The first run wrote only the blob, so it left out the second durable write that every sealed blob needs, and it understated small-blob cost. The plaintext row in that re-run (1.61 / 1.82 / 4.75 / 40.3 ms) agreed with the first run within its intervals. Wrapping the key and any crash coordination between the two writes are still not included (E5). - [cols="2,1",options="header"] |=== |Erase |Time -|`obliterate_file` on the blob, 4 KiB (today) |0.72 ms -|`obliterate_file` on the blob, 1 MiB (today) |6.04 ms -|`obliterate_file` on the blob, 16 MiB (today) |122.9 ms -|drop one 32-byte key file (same `obliterate_file`) |0.74 ms, *independent of blob size* -|drop one entry from a key table, 1 000 / 10 000 / 100 000 keys |0.80 / 2.22 / 14.2 ms +|today: `obliterate_file` on the blob, 4 KiB / 64 KiB / 1 MiB / 16 MiB |0.71 / 0.98 / 6.76 / 124 ms +|*pending* shred: drop one 32-byte key file (the same `obliterate_file`) |0.72 ms +|*final* shred, one table: drop the entry and re-seal the table under a fresh epoch key, 1 000 / 10 000 / 100 000 live keys |1.17 / 3.91 / 33.8 ms +|*final* shred, one file per key: rewrap every key file under a fresh epoch key, 1 000 live keys |1 210 ms |=== +Destroying the old epoch secret is not in any row. Its cost depends on where it lives, which is §4's open question. + What the numbers say: -* *Small blobs.* Below about 64 KiB, the cipher costs microseconds, but the second synced write, for the key file, makes a sealed store about 1.7× slower (2.69 against 1.61 ms at 4 KiB). The cost is fsync count, not cryptography. Grouping key files into one synced write per transaction is the obvious lever, at the price of the O(N) rewrite rejected below. E2 must measure this on real media. -* *Large blobs.* At 16 MiB, sealing roughly doubles store time. The cipher ran at about 400 MiB/s at that size, against about 1.1 GiB/s on small inputs, which points at allocation and copying rather than AES itself. -* *Erasure.* Erasure becomes constant-time in blob size. On a 16 MiB blob it is about 166× cheaper, and on any medium it is a claim that can be made, whereas overwriting is not. -* *Key layout.* A single key table makes every shred rewrite the whole table, so it was rejected in favour of per-blob key files. +* *Sealing is cheap next to today's default.* Against the store `jk` actually runs (gzip on), sealing adds 1.2 ms at 4 KiB, about 1.7× because of the key's own synced write, and about 7% at 16 MiB. The cost is in fsync count and compression, not in the cipher. +** An earlier version of this section said sealing doubles large-blob store time. That compared against the store with compression off, which is not the default. +* *Compression is the large-blob cost today.* gzip on incompressible input is the slowest step by an order of magnitude: 545 against 40 ms at 16 MiB. +** That observation is independent of this ADR. +** Skipping compression when it does not pay, or a faster codec, would help either design. +* *A final shred costs O(live keys), not O(blob size).* With one sealed table it is one synced write. That is cheaper than today's overwrite of a 16 MiB blob even at 100 000 keys (34 against 124 ms), but dearer than overwriting a 1 MiB blob at that key count. +* *Unlike overwriting, the claim holds on any medium*, subject to §4. +* *Key layout.* One file per key makes final shreds unworkable (1.2 s at 1 000 keys). It is why §2 chose the single table. +** The pending-shred row (0.72 ms) is only the first step of a shred, not an erasure. +** An earlier version compared that row with today's overwrite (a "166×" saving). That comparison was wrong, and it is withdrawn. == 4. Where the recursion ends (the honest limit) Crypto-shredding does not remove the physical-erasure assumption. It *shrinks* it: from "every copy of every blob, of any size, on any medium" to "one small secret held where erasure is real". -A deleted key file can itself survive on flash. That is why key files are wrapped, and why the root must live somewhere else. +An old copy of the key table can itself survive on flash. That is why the table is sealed under an epoch key, why each shred rotates that key, and why the root of the epoch keys must live somewhere else. [cols="3,1,1,4",options="header"] |=== @@ -96,7 +106,7 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ == 5. Statements this makes provable -* *O-1, restated:* "after `shred(h)` and the epoch rotation that follows it, no reachable state, retained copies of the key directory included, holds an unwrappable key for `h`; and every stored copy of `h` is a sealed blob under a key with that id". The working file is outside this statement (§2). +* *O-1, restated:* "after `shred(h)` and the epoch rotation that follows it, no reachable state, retained copies of the key table included, holds an openable key for `h`; and every stored copy of `h` is a sealed blob under a key with that id". The working file is outside this statement (§2). ** This is a finite-map model in the Agda kernel. ** It has the same shape as `obliterate-cuts-recovery` (E-1m): recovery needs the key, and the key is not in any reachable state. ** It is provable, and then TESTED against the Rust through a conformance test. @@ -113,6 +123,12 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ ** `ContentHash` therefore becomes versioned (`sha256:` or `hmac-sha256:`), and `retrieve` resolves either form. ** Migration proceeds in three steps: seal each blob at its new address; rewrite every reference in the operation log in one atomic save (the log already saves by write, sync and rename, row T-8); only then shred the SHA-addressed copies. ** A crash between the steps leaves both copies, never neither. The SHA address keeps its dictionary exposure until the rewrite commits. +** *Obliteration records hold addresses too.* Each `ObliterationRecord` and its nested proof in `obliterations.json` carry the old SHA-256 `ContentHash`, together with the nonce and commitment derived from it. For a blob already obliterated there is no plaintext left from which to compute an HMAC address. +*** Rewriting those records is impossible, so migration must either *redact* or *retain*. +*** *Redact:* keep the record's id, time, reason and legal basis, and remove the hash, nonce and commitment. The record can then no longer answer "was this content obliterated?". +*** *Retain:* keep the hash, with the dictionary exposure stated in the docs. +*** Proposed default: redact, because the record exists to say *that* something was erased, not *what*. This is a decision for acceptance, because some legal-basis uses may need the "what". +*** New records written after migration carry only the key id and the HMAC address. * *Reversibility.* Every `undo` now needs the store key. A lost passphrase loses history. That is a new failure mode, and it needs a recovery story (an escrow file, or an accepted loss) before acceptance. * *Wording.* The DoD 5220.22-M and "cryptographic proof of erasure" claims are removed from the code, docs and ledger. @@ -128,7 +144,8 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ ** *Product flow.* Create a working file containing the marker, record it with `jk`, then `jk obliterate` it. *** Expected in both designs: the marker *is* found, from the working file's own extents. *** This case is kept as a test of the scope limit in §2, so that no one later reads the store-only result as covering the working file. -* *E5: shred cost with the store key wrapped.* Add one key unwrap and rewrap to the key-file path, and confirm the cost stays constant in blob size. +* *E5: the full final shred.* Add the epoch-secret destruction (§4's chosen root) and crash coordination between the table write and the blob deletion to the `final_reseal_table` case. Confirm the cost stays independent of blob size and linear in live keys. +* *E6: compression.* Repeat the store rows with compressible payloads (text, source code). Then measure skipping gzip when it does not shrink the input. This is a candidate fix to today's store whatever happens to this ADR. == 8. Beyond this repository From 08f10fa84ce534a07fd01d52dbcd6a1540219615 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:24:20 +0000 Subject: [PATCH 07/16] docs(adr-0002): journal the keys on store, protect the index key, keep migration open MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fourth review round, three findings, all correct: - The benchmark still wrote a key file per blob though §2 had moved to one sealed table, and rewriting that table on every store would cost O(live keys). Stores now append one sealed entry to a key journal and sync it; shreds fold the journal into the table they rewrite anyway. Re-measured against the default (gzip) store: within run-to-run noise at 4 and 64 KiB, +8% at 1 MiB, +12% at 16 MiB. The earlier "1.7x at 4 KiB" measured the rejected per-file layout and is withdrawn. - The HMAC index key had no protection scheme. It is now derived from the §4 root with HKDF under its own label, never written down, stable for the store's lifetime, and only as strong as that root. - Migration exposure does not end when the log rewrite commits: the old SHA-addressed blobs are plaintext under predictable names until shredded. Migration stays marked incomplete and resumes cleanup on every invocation until the last old copy is gone. fmt and clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 19 +++++++++++---- docs/adr/0002-crypto-shredding.adoc | 37 +++++++++++++++++++---------- 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index 2482f98..f4826e3 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -8,7 +8,8 @@ // store: ContentStore::store with compression off and with it on (the // `jk` default), against the proposed path: gzip, then an // AES-256-GCM seal under a fresh per-blob key, then the same -// temp/fsync/rename write for the blob and again for its key file. +// temp/fsync/rename write for the blob, plus one synced append of +// its key to the key journal (the layout ADR-0002 §2 chooses). // `seal_only` is the cipher with no I/O. // erase: obliterate_file on a blob (3-pass overwrite, unlink: the current // path) against crypto-shredding, in its two stages: @@ -110,6 +111,11 @@ fn bench_store(c: &mut Criterion) { &size, |b, &size| { let dir = tempfile::tempdir().unwrap(); + let mut journal = fs::OpenOptions::new() + .create(true) + .append(true) + .open(dir.path().join("keys.journal")) + .unwrap(); let (mut data, mut n) = (random_payload(size), 0u64); b.iter(|| { make_unique(&mut data, &mut n); @@ -117,9 +123,14 @@ fn bench_store(c: &mut Criterion) { // Compress before sealing: ciphertext does not compress. let (key, sealed) = seal(&gzip(&data)); write_synced(dir.path(), hash.raw_hash(), &sealed); - // The key must be durable too: one more synced write. It is - // stored unwrapped here; wrapping adds microseconds (E5). - write_synced(dir.path(), &format!("{}.key", hash.raw_hash()), &key); + // The key must be durable too. A store appends one entry + // to the key journal and syncs it; the sealed table is + // rewritten only by a shred, which rewrites it anyway. + // Wrapping the entry adds microseconds (E5). + let mut entry = [0u8; ENTRY]; + entry[..32].copy_from_slice(&key); + journal.write_all(&entry).unwrap(); + journal.sync_all().unwrap(); }); }, ); diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index f26197c..4a39142 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -31,11 +31,12 @@ Two things are wrong with treating this as erasure. . *Sealing.* `store` seals the content with AES-256-GCM under a fresh 256-bit key from the OS CSPRNG and a 96-bit random nonce. The blob file is `version byte ‖ nonce ‖ ciphertext ‖ tag`. ** *Never* use a convergent key (key = H(content)). Such a key can be recomputed from the content, so it cannot be shredded. . *Deduplication is unchanged.* The key belongs to the blob, not to a path. A blob shared by two paths keeps its key until the last path that needs it is obliterated, which is the rule `obliterate_path` already applies to blobs. -. *Key storage.* Keep all blob keys in *one key table* (`.januskey/keys`), sealed as a whole under the current *epoch key*. See §4 for where the epoch key comes from. -** An earlier draft proposed one key file per blob, because dropping a single file is cheap. -** But making a shred final requires every live key to be re-sealed under a fresh epoch key. With per-blob files that is one synced write per live key: 1.2 s for only 1 000 keys (§3). -** The single table makes a final shred one synced write, whatever the number of keys. -. *Shred.* `obliterate` and `gc` drop the blob's entry and write the table again under a *fresh* epoch key. Then they destroy the old epoch key, then delete the blob. +. *Key storage.* Keep all blob keys in *one key table* (`.januskey/keys`), sealed as a whole under the current *epoch key*, plus an append-only *key journal*. See §4 for where the epoch key comes from. +** A `store` appends one entry, sealed under the current epoch key, to the journal and syncs it. That costs the same at any number of keys; rewriting the whole table on every store would not. +** A shred folds the journal into the table as part of the rewrite it does anyway. +** Journal entries are sealed under the same epoch key as the table, so the rotation below covers retained copies of the journal too. +** An earlier draft proposed one key file per blob, because dropping a single file is cheap. But making a shred final requires every live key to be re-sealed under a fresh epoch key. With per-blob files that is one synced write per live key: 1.2 s for only 1 000 keys (§3). The table makes a final shred one synced write. +. *Shred.* `obliterate` and `gc` drop the blob's entry, fold in the journal, and write the table again under a *fresh* epoch key, starting an empty journal. Then they destroy the old epoch key, then delete the blob. ** Destroying the old epoch key is what makes the shred final. A snapshot, a backup or a copy-on-write extent can keep an old copy of the table, but that copy is sealed under an epoch key that no longer exists. ** If the old epoch key is not destroyed, for example because the new table is written but the old key survives, the shred is only *pending*. Every retained copy of the old table, and so of the blob's key, can still be opened. ** The cost of a final shred is O(live keys): 1.2 ms at 1 000 keys, 34 ms at 100 000 (§3). @@ -44,12 +45,17 @@ Two things are wrong with treating this as erasure. ** That file existed in plaintext on the same medium before JanusKey ever saw it, so CoW extents, flash remapping, snapshots or backups may still hold its plaintext. ** Covering it needs encryption from the file's creation, meaning an encrypted volume underneath, which is outside JanusKey. ** The record says which copies the shred covers, and the docs say the working file is overwritten best-effort only. -. *Addresses.* Use HMAC-SHA-256 under a per-store index key instead of a bare SHA-256, so that an address alone does not confirm a guessed plaintext. The index key is in turn an erasure target when the whole store is destroyed. +. *Addresses.* Use HMAC-SHA-256 under a per-store *index key* instead of a bare SHA-256, so that an address alone does not confirm a guessed plaintext. +** That only holds if the attacker who reads the store cannot also read the index key. The index key is therefore part of the protected hierarchy and never sits beside the store in plaintext. +** It is derived from the root of §4 with HKDF-SHA-256 under its own label. It is never written down, and is recomputed when the store is opened. +** It is stable for the store's lifetime, because rotating it would re-address every blob. So it is not rotated per shred. +** Destroying the root destroys it too, which is a whole-store erasure. +** The protection is exactly as strong as the root. An attacker who holds the store *and* the passphrase (or the hardware root) can still confirm guesses. . *Proof object.* `ObliterationProof` loses `storage_cleared` and `overwrite_passes`. It records the key id that was destroyed, and is honestly named a *record*, not a proof. == 3. Measured cost (E1, done) -`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after two rounds of review corrections. +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after three rounds of review corrections. * *Conditions:* 4 vCPU, a container's virtual disk, one run, and medians, with Criterion's 95% intervals in the raw output. * *Payloads:* random, so incompressible. That is the worst case for gzip: it costs time and saves nothing. Compressible content would make every gzip row faster, and the sealed row with it, since that row compresses before sealing. @@ -59,11 +65,13 @@ Two things are wrong with treating this as erasure. |=== |Store (per blob) |4 KiB |64 KiB |1 MiB |16 MiB |`ContentStore::store`, compression off |0.92 ms |2.10 ms |6.05 ms |40.1 ms -|`ContentStore::store`, compression on (the `jk` default, today) |1.70 ms |4.40 ms |35.7 ms |545 ms -|proposed: gzip, then seal, then a synced blob write and a synced key write (key unwrapped) |2.88 ms |5.22 ms |39.0 ms |582 ms +|`ContentStore::store`, compression on (the `jk` default, today) |1.34 ms |3.98 ms |36.3 ms |525 ms +|proposed: gzip, then seal, then a synced blob write and one synced append to the key journal (entry unwrapped) |1.63 ms |3.66 ms |39.1 ms |588 ms |seal only, no I/O |3.7 µs |55 µs |1.65 ms |38.6 ms |=== +The default-store and proposed rows come from one paired re-run, made after the layout moved to the key journal. An earlier run of the default store measured 1.70 / 4.40 / 35.7 / 545 ms. The spread at 4 and 64 KiB between those runs is as large as the difference between the two rows, so read small-blob differences as noise. + [cols="2,1",options="header"] |=== |Erase |Time @@ -77,8 +85,10 @@ Destroying the old epoch secret is not in any row. Its cost depends on where it What the numbers say: -* *Sealing is cheap next to today's default.* Against the store `jk` actually runs (gzip on), sealing adds 1.2 ms at 4 KiB, about 1.7× because of the key's own synced write, and about 7% at 16 MiB. The cost is in fsync count and compression, not in the cipher. -** An earlier version of this section said sealing doubles large-blob store time. That compared against the store with compression off, which is not the default. +* *Sealing is cheap next to today's default.* Against the store `jk` actually runs (gzip on), the proposed path is within run-to-run noise at 4 and 64 KiB, and adds about 8% at 1 MiB and 12% at 16 MiB. The cost is in compression and fsync count, not in the cipher. +** Two earlier versions of this section were wrong. +** One said sealing doubles large-blob store time. That compared against the store with compression off, which is not the default. +** The other gave 1.7× at 4 KiB. That measured a separate key *file* per blob, with its own temp file, rename and directory fsync, which the chosen layout does not do. * *Compression is the large-blob cost today.* gzip on incompressible input is the slowest step by an order of magnitude: 545 against 40 ms at 16 MiB. ** That observation is independent of this ADR. ** Skipping compression when it does not pay, or a faster codec, would help either design. @@ -122,7 +132,10 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ ** *Addresses are part of the history.* The operation log serialises each blob's `ContentHash`, and `undo` passes that exact value to `ContentStore::retrieve`. Moving blobs to HMAC addresses would leave every historical operation pointing at a blob that no longer exists. ** `ContentHash` therefore becomes versioned (`sha256:` or `hmac-sha256:`), and `retrieve` resolves either form. ** Migration proceeds in three steps: seal each blob at its new address; rewrite every reference in the operation log in one atomic save (the log already saves by write, sync and rename, row T-8); only then shred the SHA-addressed copies. -** A crash between the steps leaves both copies, never neither. The SHA address keeps its dictionary exposure until the rewrite commits. +** A crash between the steps leaves both copies, never neither. That cuts both ways. +*** The old SHA-addressed blobs are plaintext with predictable names. Exposure ends only when *every* old copy is shredded, not when the log rewrite commits. +*** Migration therefore records itself as incomplete, in a marker it writes first and removes last. Every `jk` invocation resumes the cleanup until the last old copy is gone. +*** `jk status` reports a store with a migration still in progress as still exposed. ** *Obliteration records hold addresses too.* Each `ObliterationRecord` and its nested proof in `obliterations.json` carry the old SHA-256 `ContentHash`, together with the nonce and commitment derived from it. For a blob already obliterated there is no plaintext left from which to compute an HMAC address. *** Rewriting those records is impossible, so migration must either *redact* or *retain*. *** *Redact:* keep the record's id, time, reason and legal basis, and remove the hash, nonce and commitment. The record can then no longer answer "was this content obliterated?". From bc701e428cdca5308e4c98ea060be811b13e305f Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:35:02 +0000 Subject: [PATCH 08/16] docs(adr-0002): two-root hierarchy, per-message subkeys, honest shred costs Fifth review round, six findings, all correct: - Two roots: a stable Argon2id store root (index key via HKDF, never rotated) and a random keyring-held epoch secret replaced on every final shred (epoch key = HKDF over the store root, salted with it). Rotation no longer re-addresses blobs, and the passphrase alone does not reopen a retained table. - One message per AEAD key now holds for epoch-level messages too: the table and each journal entry get their own HKDF subkey by sequence number. X-1 is split into AEAD, erasure and KDF assumptions. - Migration cleanup removes only reachable files; plaintext written before migration may persist in CoW extents, flash, snapshots and backups. jk status reports that permanently; a fresh sealed store is recommended where exposure matters. - Benchmark: the pending shred is now a journal tombstone (0.20 ms), not the rejected per-file delete; the final shred times the whole open, drop and re-seal on 64-byte plaintext rows (0.96 / 2.91 / 26.2 ms at 1k / 10k / 100k keys), replacing reseal-only figures on 92-byte rows. fmt and clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 73 +++++++++++++++++------------ docs/adr/0002-crypto-shredding.adoc | 60 +++++++++++++++++------- 2 files changed, 88 insertions(+), 45 deletions(-) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index f4826e3..de18f55 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -13,12 +13,13 @@ // `seal_only` is the cipher with no I/O. // erase: obliterate_file on a blob (3-pass overwrite, unlink: the current // path) against crypto-shredding, in its two stages: -// pending: drop the blob's key file (through the same -// obliterate_file); copies retained elsewhere are still -// wrapped under the live epoch key; -// final: drop the entry and re-seal every live key under a -// fresh epoch key, either as one table (one synced -// write) or as one file per key (one synced write each). +// pending: append a tombstone for the blob to the key journal +// and sync it; the old table, and so the key, can still +// be opened under the live epoch key; +// final: open the sealed table, drop the entry and re-seal +// every live key under a fresh epoch key, as one table +// (one synced write). For contrast, the rejected layout: +// one file per key (one synced write each). // // Payloads are random, so incompressible: the worst case for gzip, which then // costs time and saves nothing. Each is made unique per iteration, so dedupe @@ -41,9 +42,12 @@ use std::path::Path; use std::time::Duration; const SIZES: [usize; 4] = [4 << 10, 64 << 10, 1 << 20, 16 << 20]; -/// A key-table entry: 32-byte content address, then the blob key wrapped -/// under the epoch key (12-byte nonce, 32-byte key, 16-byte tag). -const ENTRY: usize = 32 + 12 + 32 + 16; +/// A plaintext key-table row: 32-byte content address and 32-byte blob key. +/// The table is sealed as one message, so its rows are not wrapped one by one. +const TABLE_ROW: usize = 32 + 32; +/// A key-journal entry: one table row sealed on its own (12-byte nonce, +/// the row, 16-byte tag), since entries are appended one at a time. +const JOURNAL_ENTRY: usize = 12 + TABLE_ROW + 16; /// `size` random bytes: incompressible, the worst case for gzip. fn random_payload(size: usize) -> Vec { @@ -83,6 +87,13 @@ fn seal(data: &[u8]) -> ([u8; 32], Vec) { (key, out) } +/// Open `nonce || ciphertext` sealed by [`seal`] under `key`. +fn open(key: &[u8; 32], sealed: &[u8]) -> Vec { + let cipher = Aes256Gcm::new_from_slice(key).unwrap(); + let (nonce, ct) = sealed.split_at(12); + cipher.decrypt(Nonce::from_slice(nonce), ct).unwrap() +} + /// gzip at the store's level, as `ContentStore::write_blob` does. fn gzip(data: &[u8]) -> Vec { let mut enc = GzEncoder::new(Vec::new(), Compression::default()); @@ -127,7 +138,7 @@ fn bench_store(c: &mut Criterion) { // to the key journal and syncs it; the sealed table is // rewritten only by a shred, which rewrites it anyway. // Wrapping the entry adds microseconds (E5). - let mut entry = [0u8; ENTRY]; + let mut entry = [0u8; JOURNAL_ENTRY]; entry[..32].copy_from_slice(&key); journal.write_all(&entry).unwrap(); journal.sync_all().unwrap(); @@ -170,36 +181,40 @@ fn bench_erase(c: &mut Criterion) { ); } - // Pending shred: drop one key file, the same overwrite on 32 bytes. - g.bench_function("pending_drop_key_file", |b| { + // Pending shred: append a tombstone for the blob to the key journal and + // sync it. The live epoch key still opens the old table. + g.bench_function("pending_journal_tombstone", |b| { let dir = tempfile::tempdir().unwrap(); - b.iter_batched( - || { - let p = dir.path().join("key"); - fs::write(&p, random_payload(32)).unwrap(); - p - }, - |p| black_box(obliterate_file(&p).unwrap()), - BatchSize::PerIteration, - ); + let mut journal = fs::OpenOptions::new() + .create(true) + .append(true) + .open(dir.path().join("keys.journal")) + .unwrap(); + b.iter(|| { + journal.write_all(&random_payload(JOURNAL_ENTRY)).unwrap(); + journal.sync_all().unwrap(); + }); }); - // Final shred, one table: drop the entry, re-seal the whole table under a - // fresh epoch key, one synced write. (Destroying the old epoch secret is - // not measured: where it lives is the open question of ADR §4.) + // Final shred, one table: open (authenticate and decrypt) the sealed + // table, drop the entry, re-seal the rest under a fresh epoch key, one + // synced write. Not measured: folding in a non-empty journal (one more + // open per entry since the last shred) and destroying the old epoch + // secret, whose cost depends on where ADR §4 puts it. for entries in [1_000usize, 10_000, 100_000] { g.bench_with_input( - BenchmarkId::new("final_reseal_table", entries), + BenchmarkId::new("final_open_drop_reseal_table", entries), &entries, |b, &entries| { let dir = tempfile::tempdir().unwrap(); - let table = random_payload(entries * ENTRY); + let (old_key, sealed_table) = seal(&random_payload(entries * TABLE_ROW)); let mut victim = 0usize; b.iter(|| { victim = (victim + 1) % entries; - let mut next = Vec::with_capacity(table.len() - ENTRY); - next.extend_from_slice(&table[..victim * ENTRY]); - next.extend_from_slice(&table[(victim + 1) * ENTRY..]); + let table = open(&old_key, &sealed_table); + let mut next = Vec::with_capacity(table.len() - TABLE_ROW); + next.extend_from_slice(&table[..victim * TABLE_ROW]); + next.extend_from_slice(&table[(victim + 1) * TABLE_ROW..]); let (epoch_key, sealed) = seal(&next); write_synced(dir.path(), "keys", &sealed); black_box(epoch_key); diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 4a39142..fffed74 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -39,8 +39,8 @@ Two things are wrong with treating this as erasure. . *Shred.* `obliterate` and `gc` drop the blob's entry, fold in the journal, and write the table again under a *fresh* epoch key, starting an empty journal. Then they destroy the old epoch key, then delete the blob. ** Destroying the old epoch key is what makes the shred final. A snapshot, a backup or a copy-on-write extent can keep an old copy of the table, but that copy is sealed under an epoch key that no longer exists. ** If the old epoch key is not destroyed, for example because the new table is written but the old key survives, the shred is only *pending*. Every retained copy of the old table, and so of the blob's key, can still be opened. -** The cost of a final shred is O(live keys): 1.2 ms at 1 000 keys, 34 ms at 100 000 (§3). -** Shreds could instead be batched, so that one rotation finalises many. That leaves a window in which a shred is pending. The ADR's default is a final shred every time; batching is an option to decide at acceptance. +** The cost of a final shred is O(live keys): opening, dropping and re-sealing took about 1 ms at 1 000 keys and 26 ms at 100 000 (§3). +** Shreds could instead be batched, so that one rotation finalises many. A batched shred appends a *tombstone* for the blob to the key journal and syncs it (0.2 ms, §3); the blob is unusable through `jk` at once, but the old table and journal, and so the key, stay openable under the live epoch key until the next rotation. The ADR's default is a final shred every time; batching, and how long a shred may stay pending, are decisions for acceptance. . *Scope: content-store copies only.* The guarantee covers what JanusKey writes: blobs and keys. It does not cover the working file itself, which `jk obliterate` targets too. ** That file existed in plaintext on the same medium before JanusKey ever saw it, so CoW extents, flash remapping, snapshots or backups may still hold its plaintext. ** Covering it needs encryption from the file's creation, meaning an encrypted volume underneath, which is outside JanusKey. @@ -55,7 +55,7 @@ Two things are wrong with treating this as erasure. == 3. Measured cost (E1, done) -`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after three rounds of review corrections. +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after five rounds of review corrections. * *Conditions:* 4 vCPU, a container's virtual disk, one run, and medians, with Criterion's 95% intervals in the raw output. * *Payloads:* random, so incompressible. That is the worst case for gzip: it costs time and saves nothing. Compressible content would make every gzip row faster, and the sealed row with it, since that row compresses before sealing. @@ -76,12 +76,15 @@ The default-store and proposed rows come from one paired re-run, made after the |=== |Erase |Time |today: `obliterate_file` on the blob, 4 KiB / 64 KiB / 1 MiB / 16 MiB |0.71 / 0.98 / 6.76 / 124 ms -|*pending* shred: drop one 32-byte key file (the same `obliterate_file`) |0.72 ms -|*final* shred, one table: drop the entry and re-seal the table under a fresh epoch key, 1 000 / 10 000 / 100 000 live keys |1.17 / 3.91 / 33.8 ms +|*pending* shred (batched): append a tombstone to the key journal and sync it |0.20 ms +|*final* shred, one table: open (authenticate and decrypt) the sealed table, drop the entry, re-seal it under a fresh epoch key, one synced write; 64-byte rows; 1 000 / 10 000 / 100 000 live keys |0.96 / 2.91 / 26.2 ms |*final* shred, one file per key: rewrap every key file under a fresh epoch key, 1 000 live keys |1 210 ms |=== -Destroying the old epoch secret is not in any row. Its cost depends on where it lives, which is §4's open question. +Two costs are not in any row: + +* *Destroying the old epoch secret.* Its cost depends on where it lives (§4). +* *Folding a non-empty journal into the table.* That is one more open per journal entry since the last shred, which is zero when every shred is final. What the numbers say: @@ -92,11 +95,13 @@ What the numbers say: * *Compression is the large-blob cost today.* gzip on incompressible input is the slowest step by an order of magnitude: 545 against 40 ms at 16 MiB. ** That observation is independent of this ADR. ** Skipping compression when it does not pay, or a faster codec, would help either design. -* *A final shred costs O(live keys), not O(blob size).* With one sealed table it is one synced write. That is cheaper than today's overwrite of a 16 MiB blob even at 100 000 keys (34 against 124 ms), but dearer than overwriting a 1 MiB blob at that key count. +* *A final shred costs O(live keys), not O(blob size).* With one sealed table it is one synced write. That is cheaper than today's overwrite of a 16 MiB blob even at 100 000 keys (26 against 124 ms), but dearer than overwriting a 1 MiB blob at that key count. * *Unlike overwriting, the claim holds on any medium*, subject to §4. * *Key layout.* One file per key makes final shreds unworkable (1.2 s at 1 000 keys). It is why §2 chose the single table. -** The pending-shred row (0.72 ms) is only the first step of a shred, not an erasure. -** An earlier version compared that row with today's overwrite (a "166×" saving). That comparison was wrong, and it is withdrawn. +** The pending-shred row is only the first step of a batched shred, not an erasure. +** An earlier version measured a pending shred as deleting a per-blob key file (0.72 ms). That operation belongs to the rejected layout. +** An earlier version also compared that figure with today's overwrite (a "166×" saving). That comparison was wrong, and it is withdrawn. +** Earlier final-shred figures (1.17 / 3.91 / 33.8 ms) re-sealed rows sized as if each key were wrapped on its own (92 bytes). They also skipped opening the old table. The current row times the whole open, drop and re-seal on 64-byte rows. == 4. Where the recursion ends (the honest limit) @@ -107,22 +112,43 @@ An old copy of the key table can itself survive on flash. That is why the table [cols="3,1,1,4",options="header"] |=== |Root of the key hierarchy |Elegance |Dependability |Aspects -|Passphrase via Argon2id (already a dependency), plus a per-epoch random salt kept only in memory and in an OS keyring |★★ |★★ |Usable everywhere. Privacy is only as good as the passphrase. A shred becomes final when the epoch is rotated and the old epoch secret is gone. +|Passphrase plus a keyring-held epoch secret (see the hierarchy below) |★★ |★★ |Usable everywhere. Privacy is only as good as the passphrase. A shred becomes final when the epoch secret is replaced and the old one is gone from the keyring. |OS keyring or TPM / Secure Enclave sealed key |★★★ |★★★ |Hardware erase semantics. Platform-specific, so a Rust/SPARK seam and an Idris2 ABI per platform. This is the estate's FFI pattern. |No root at rest: the key is supplied per session, for example by a hardware token |★★★ |★★ |The strongest privacy. Undo needs the token present, which changes how `jk` is used, and ADR-0001 says that is not the product. |=== Recommended for a first build: the passphrase plus epoch rotation, with the TPM/keyring root as a later, separate seam. +*The hierarchy has two roots, because the index key must stay stable while epoch keys must change.* + +* *Store root.* Argon2id over the passphrase, with a salt fixed when the store is created and stored beside it. +** It never changes. +** It is never written down. +* *Index key.* HKDF-SHA-256 over the store root, under the label `"index"` (§2). +** It is stable for the store's lifetime, so rotation never re-addresses blobs. +* *Epoch secret.* 256 random bits, held in the OS keyring (or TPM), never on the store's medium. +** Every final shred replaces it. +** It is not derivable from the passphrase, so a retained copy of the store together with the passphrase does not recover it. +* *Epoch key.* HKDF-SHA-256 over the store root, salted with the epoch secret, under the label `"epoch"`. +** Opening the table needs both the passphrase and the current epoch secret. +** Removing the old epoch secret from the keyring finalises every shred made under it. +* *Erasure scope.* "Erasing the epoch secret is real erasure" is now an assumption about the keyring's storage, not about the store's medium. On platforms whose keyring persists to the same flash, the honest claim is no stronger than that keyring's own deletion. E2 must check this per platform. + == 5. Statements this makes provable * *O-1, restated:* "after `shred(h)` and the epoch rotation that follows it, no reachable state, retained copies of the key table included, holds an openable key for `h`; and every stored copy of `h` is a sealed blob under a key with that id". The working file is outside this statement (§2). ** This is a finite-map model in the Agda kernel. ** It has the same shape as `obliterate-cuts-recovery` (E-1m): recovery needs the key, and the key is not in any reachable state. ** It is provable, and then TESTED against the Rust through a conformance test. -* *X-1 becomes two assumptions:* -** AES-256-GCM is IND-CPA and INT-CTXT for one message per key. One message per key also removes nonce reuse as a failure mode. -** Erasing the root or epoch secret is real erasure. +* *X-1 becomes three assumptions:* +** *AEAD security.* AES-256-GCM is IND-CPA and INT-CTXT for *one message per key*. +*** Blob keys meet that by construction. +*** The epoch key would not, because the table and every journal entry under one epoch are separate messages. +*** So each of them is sealed under its own subkey: HKDF-SHA-256 over the epoch key with the message's sequence number (0 for the table, then 1, 2, … for journal entries). +*** Every AEAD key therefore still encrypts exactly one message, and random-nonce collisions and per-key GCM usage limits stop being part of the assumption. +*** The design must never reuse a sequence number within an epoch, which is a property for the model to check. +** *Erasure.* Erasing the epoch secret where §4 keeps it is real erasure. +** *Key derivation.* HKDF-SHA-256 is a PRF and Argon2id resists offline guessing at the chosen cost. * *C-3 is unchanged:* obliteration stays by design irreversible, and `undo` after a shred has nothing to restore, as today. == 6. Consequences @@ -133,9 +159,11 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ ** `ContentHash` therefore becomes versioned (`sha256:` or `hmac-sha256:`), and `retrieve` resolves either form. ** Migration proceeds in three steps: seal each blob at its new address; rewrite every reference in the operation log in one atomic save (the log already saves by write, sync and rename, row T-8); only then shred the SHA-addressed copies. ** A crash between the steps leaves both copies, never neither. That cuts both ways. -*** The old SHA-addressed blobs are plaintext with predictable names. Exposure ends only when *every* old copy is shredded, not when the log rewrite commits. -*** Migration therefore records itself as incomplete, in a marker it writes first and removes last. Every `jk` invocation resumes the cleanup until the last old copy is gone. -*** `jk status` reports a store with a migration still in progress as still exposed. +*** The old SHA-addressed blobs are plaintext with predictable names. Migration records itself as incomplete, in a marker it writes first and removes last. Every `jk` invocation resumes cleanup until no reachable old file remains. +*** Even then, cleanup removes only the *reachable* files. Plaintext the store wrote before migration may still exist in copy-on-write extents, on remapped flash, in snapshots and in backups. That is §1's own premise, and no epoch key covers those copies. +*** So a migrated store is never reported as unexposed. `jk status` reports "plaintext from before migration may persist on the medium, its snapshots and backups", permanently, for any store that ever held plaintext blobs. +*** Removing that exposure needs the medium, its snapshots and its backups to be sanitised outside JanusKey. +*** For data whose exposure matters, the ADR recommends starting a fresh sealed store over migrating an old one. ** *Obliteration records hold addresses too.* Each `ObliterationRecord` and its nested proof in `obliterations.json` carry the old SHA-256 `ContentHash`, together with the nonce and commitment derived from it. For a blob already obliterated there is no plaintext left from which to compute an HMAC address. *** Rewriting those records is impossible, so migration must either *redact* or *retain*. *** *Redact:* keep the record's id, time, reason and legal basis, and remove the hash, nonce and commitment. The record can then no longer answer "was this content obliterated?". From 90328b79f7302101e7b0209ebbaffcd082ab2b80 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:43:37 +0000 Subject: [PATCH 09/16] docs(adr-0002): crash-safe rotation by generation, measured journal fold, valid E4 Sixth review round, three findings, all correct: - Rotation could strand the live table: the keyring and the file system cannot be updated atomically. Tables and epoch secrets now carry a generation; a shred adds secret n+1, writes keys., renames a `current` pointer, then removes secret n and the old files. Open recovers a mid-rotation store to exactly one openable generation, and E5 now injects a crash after each step. - Folding the journal is never zero in practice (the victim's own store leaves an entry). Measured: 0.42 / 0.45 / 0.77 / 3.44 ms for 10 / 100 / 1k / 10k entries; the final-shred cost is now stated as O(live keys + journal entries), and the earlier "zero" claim is withdrawn. - E4 runs with compression off in both arms and requires a positive pre-shred control, so a gzip-hidden marker cannot pass as erasure. fmt and clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 27 ++++++++++++++++++++++- docs/adr/0002-crypto-shredding.adoc | 33 +++++++++++++++++++++++------ 2 files changed, 53 insertions(+), 7 deletions(-) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index de18f55..4c31684 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -18,7 +18,8 @@ // be opened under the live epoch key; // final: open the sealed table, drop the entry and re-seal // every live key under a fresh epoch key, as one table -// (one synced write). For contrast, the rejected layout: +// (one synced write), plus folding in the journal. For +// contrast, the rejected layout: // one file per key (one synced write each). // // Payloads are random, so incompressible: the worst case for gzip, which then @@ -223,6 +224,30 @@ fn bench_erase(c: &mut Criterion) { ); } + // Folding the journal into the table during a final shred: open each entry + // stored since the last rotation (each under its own subkey) and start an + // empty journal, one synced write. Added to the table cost above. + for journal_len in [10usize, 100, 1_000, 10_000] { + g.bench_with_input( + BenchmarkId::new("fold_journal", journal_len), + &journal_len, + |b, &journal_len| { + let dir = tempfile::tempdir().unwrap(); + let entries: Vec<([u8; 32], Vec)> = (0..journal_len) + .map(|_| seal(&random_payload(TABLE_ROW))) + .collect(); + b.iter(|| { + let mut rows = Vec::with_capacity(journal_len * TABLE_ROW); + for (key, sealed) in &entries { + rows.extend_from_slice(&open(key, sealed)); + } + write_synced(dir.path(), "keys.journal", &[]); + black_box(rows.len()); + }); + }, + ); + } + // Final shred, one file per key: rewrap every live key file under a fresh // epoch key, one synced write each. let entries = 1_000usize; diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index fffed74..80cd8f4 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -39,7 +39,21 @@ Two things are wrong with treating this as erasure. . *Shred.* `obliterate` and `gc` drop the blob's entry, fold in the journal, and write the table again under a *fresh* epoch key, starting an empty journal. Then they destroy the old epoch key, then delete the blob. ** Destroying the old epoch key is what makes the shred final. A snapshot, a backup or a copy-on-write extent can keep an old copy of the table, but that copy is sealed under an epoch key that no longer exists. ** If the old epoch key is not destroyed, for example because the new table is written but the old key survives, the shred is only *pending*. Every retained copy of the old table, and so of the blob's key, can still be opened. -** The cost of a final shred is O(live keys): opening, dropping and re-sealing took about 1 ms at 1 000 keys and 26 ms at 100 000 (§3). +** The cost of a final shred is O(live keys + journal entries since the last rotation). +*** Opening, dropping and re-sealing the table took about 1 ms at 1 000 keys and 26 ms at 100 000. +*** Folding the journal adds 0.4 ms at 100 entries and 3.4 ms at 10 000 (§3). +** *Rotation is crash-safe only with generations.* The keyring and the file system cannot be updated in one atomic step, so tables and epoch secrets are numbered by generation. +*** A shred moving from generation _n_ to _n+1_: +**** stores the new epoch secret in the keyring under _n+1_, keeping _n_; +**** writes `keys.`, with an empty journal for _n+1_, and syncs it; +**** atomically renames a small `current` pointer file to name _n+1_; +**** removes secret _n_ from the keyring; +**** deletes `keys.`, its journal, and the blob. +*** On open, a store whose keyring holds both _n_ and _n+1_ is mid-rotation: +**** if `current` names _n+1_, the open finishes the last two steps; +**** if `current` still names _n_, the open discards secret _n+1_ and `keys.`. +*** A crash at any point leaves exactly one openable generation. +*** The shred is final only once secret _n_ has left the keyring. ** Shreds could instead be batched, so that one rotation finalises many. A batched shred appends a *tombstone* for the blob to the key journal and syncs it (0.2 ms, §3); the blob is unusable through `jk` at once, but the old table and journal, and so the key, stay openable under the live epoch key until the next rotation. The ADR's default is a final shred every time; batching, and how long a shred may stay pending, are decisions for acceptance. . *Scope: content-store copies only.* The guarantee covers what JanusKey writes: blobs and keys. It does not cover the working file itself, which `jk obliterate` targets too. ** That file existed in plaintext on the same medium before JanusKey ever saw it, so CoW extents, flash remapping, snapshots or backups may still hold its plaintext. @@ -55,7 +69,7 @@ Two things are wrong with treating this as erasure. == 3. Measured cost (E1, done) -`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after five rounds of review corrections. +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after six rounds of review corrections. * *Conditions:* 4 vCPU, a container's virtual disk, one run, and medians, with Criterion's 95% intervals in the raw output. * *Payloads:* random, so incompressible. That is the worst case for gzip: it costs time and saves nothing. Compressible content would make every gzip row faster, and the sealed row with it, since that row compresses before sealing. @@ -78,13 +92,16 @@ The default-store and proposed rows come from one paired re-run, made after the |today: `obliterate_file` on the blob, 4 KiB / 64 KiB / 1 MiB / 16 MiB |0.71 / 0.98 / 6.76 / 124 ms |*pending* shred (batched): append a tombstone to the key journal and sync it |0.20 ms |*final* shred, one table: open (authenticate and decrypt) the sealed table, drop the entry, re-seal it under a fresh epoch key, one synced write; 64-byte rows; 1 000 / 10 000 / 100 000 live keys |0.96 / 2.91 / 26.2 ms +|*final* shred, add for the journal: open each entry stored since the last rotation and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.42 / 0.45 / 0.77 / 3.44 ms |*final* shred, one file per key: rewrap every key file under a fresh epoch key, 1 000 live keys |1 210 ms |=== -Two costs are not in any row: +Not in any row: * *Destroying the old epoch secret.* Its cost depends on where it lives (§4). -* *Folding a non-empty journal into the table.* That is one more open per journal entry since the last shred, which is zero when every shred is final. +* *The generation protocol's extra writes:* one keyring write and one pointer rename (§2). + +The journal row is never zero in practice, even when every shred is final: any store since the last rotation leaves an entry, including the victim's own. An earlier version of this section claimed it was zero. What the numbers say: @@ -177,7 +194,10 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ * *E2: real media.* Repeat E1 on ext4 over an NVMe SSD and on btrfs. Report the fsync share of store time. * *E3: cipher choice.* Compare ChaCha20-Poly1305 with AES-256-GCM on a CPU without AES-NI. Check that the 16 MiB seal is limited by allocation, not by the cipher, by sealing in place (`encrypt_in_place`). -* *E4: forensic acceptance test.* This is the decisive one. Build a btrfs file system on a loop device, run the steps below, then scan the raw image for a marker payload. There are two cases. +* *E4: forensic acceptance test.* This is the decisive one. Build a btrfs file system on a loop device, run the steps below, then scan the raw image for a marker payload. +** *Compression is off in both arms.* `jk` compresses by default, and a gzip stream would hide a literal marker from the scan even with no erasure at all. That false negative would make the sealed path look as if it passed. +** *A positive control is required.* Before any shred or obliterate, the scan must find the marker in today's store. If it does not, the run is invalid, not a pass. +** There are two cases. ** *Store-only.* Store the marker straight into the content store, then shred it and rotate the epoch. *** Expected for today's overwrite: the marker is found, because the CoW extents survive. *** Expected with sealing: the marker is never found, because the store never writes plaintext. @@ -185,7 +205,8 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ ** *Product flow.* Create a working file containing the marker, record it with `jk`, then `jk obliterate` it. *** Expected in both designs: the marker *is* found, from the working file's own extents. *** This case is kept as a test of the scope limit in §2, so that no one later reads the store-only result as covering the working file. -* *E5: the full final shred.* Add the epoch-secret destruction (§4's chosen root) and crash coordination between the table write and the blob deletion to the `final_reseal_table` case. Confirm the cost stays independent of blob size and linear in live keys. +* *E5: the full final shred.* Time the generation protocol of §2 end to end, including the keyring writes for §4's chosen root and the pointer rename. Confirm the cost stays independent of blob size and linear in live keys plus journal entries. +** Inject a crash after each of the protocol's steps. Check that the next open recovers to exactly one openable generation, and that no step leaves the shred reported final while secret _n_ is still in the keyring. * *E6: compression.* Repeat the store rows with compressible payloads (text, source code). Then measure skipping gzip when it does not shrink the input. This is a candidate fix to today's store whatever happens to this ADR. == 8. Beyond this repository From b6fb58682c4e4f1819a65c8c2e4b247595a6c231 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:44:11 +0000 Subject: [PATCH 10/16] docs(adr-0002): serialise journal appends with rotation A store appending under generation n after a concurrent shred folded that journal, but before secret n was removed, would leave its blob key in no table under a destroyed secret: permanent loss of undo content. Every journal append and every whole rotation now holds one exclusive store lock, and journal entries carry their generation so a store that waited across a rotation re-seals under the new one. Stricter than the transaction log's out-of-scope stance on concurrent writers (T-5), because the failure here is key loss, not a lost update. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- docs/adr/0002-crypto-shredding.adoc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 80cd8f4..be94043 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -42,6 +42,10 @@ Two things are wrong with treating this as erasure. ** The cost of a final shred is O(live keys + journal entries since the last rotation). *** Opening, dropping and re-sealing the table took about 1 ms at 1 000 keys and 26 ms at 100 000. *** Folding the journal adds 0.4 ms at 100 entries and 3.4 ms at 10 000 (§3). +** *Stores and rotations are serialised.* Suppose a `jk` process appended to the journal under generation _n_ after a concurrent shred had folded that journal but before it removed secret _n_. That blob's key would be in no table and sealed under a destroyed secret, and its undo content would be lost permanently. +*** So every journal append and every whole rotation runs under one exclusive store lock (`.januskey/keys.lock`, an OS file lock). +*** As a second guard, each journal entry carries the generation it was sealed under. A store that finds `current` changed while it waited for the lock re-seals under the new generation. +*** This is stricter than the rest of `jk` today: concurrent writers are out of scope for the transaction log (T-5). It is required here because the failure is permanent key loss, not a lost update. ** *Rotation is crash-safe only with generations.* The keyring and the file system cannot be updated in one atomic step, so tables and epoch secrets are numbered by generation. *** A shred moving from generation _n_ to _n+1_: **** stores the new epoch secret in the keyring under _n+1_, keeping _n_; From 6890fc02ebfc3dbf88704482e30554f00421a2b1 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:54:28 +0000 Subject: [PATCH 11/16] docs(adr-0002): durable pointer, shared reader lock, key-before-blob; fix TESTED count Seventh review round, five findings, all correct: - PROOF-NEEDS: the proof-grade section still said 23 TESTED rows; the ledger now has 27 (the four S-1/S-2 rows). Corrected. - The generation pointer is now written by temp, fsync, rename and a directory fsync before secret n is removed, and recovery picks the generation that actually opens rather than trusting `current`. - retrieve and undo hold the store lock shared for the whole use of a key, so a reader cannot finish an undo after a final shred reported the content gone. - A store syncs its journal entry before renaming the sealed blob into place, and dedupe accepts an existing blob only if its key exists: a crash can leave a key without a blob, never a blob without a key. - The journal-fold benchmark now derives every subkey (HKDF expand, one HMAC-SHA-256 block) inside the timed loop: 0.45 / 0.70 / 2.28 / 18.0 ms for 10 / 100 / 1k / 10k entries, replacing figures that used pre-derived keys. fmt and clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- PROOF-NEEDS.adoc | 2 +- benches/crypto_shred.rs | 38 ++++++++++++++++++++++++++--- docs/adr/0002-crypto-shredding.adoc | 19 +++++++++------ 3 files changed, 47 insertions(+), 12 deletions(-) diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index 1ec8377..ef054ef 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -134,7 +134,7 @@ one, so the receipt covers these rows as they stand. Mathesis status for them: The other seven PROVEN rows — A-1, A-2, C-1 to C-5 — are Idris2 theorems. Under PROOF-POLICY §1 those are interface and ABI contracts, so the theory ladder does not grade them: they are PROVEN in the vocabulary above and nothing more. The -same holds for the 23 TESTED rows, which record observed behaviour and never +same holds for the 27 TESTED rows, which record observed behaviour and never grading. First-party Rust behaviour belongs to **Creusot** (not SPARK, and not Idris2); no Creusot contract exists in this tree, which is why no Rust row is graded. diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index 4c31684..b1a4053 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -34,9 +34,11 @@ use criterion::{ black_box, criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion, Throughput, }; use flate2::{write::GzEncoder, Compression}; +use hmac::{Hmac, Mac}; use januskey::obliteration::obliterate_file; use januskey::{ContentHash, ContentStore}; use rand::RngCore; +use sha2::Sha256; use std::fs; use std::io::Write; use std::path::Path; @@ -88,6 +90,28 @@ fn seal(data: &[u8]) -> ([u8; 32], Vec) { (key, out) } +/// Seal `data` under a given key; returns nonce || ciphertext. +fn seal_under(key: &[u8; 32], data: &[u8]) -> Vec { + let mut nonce = [0u8; 12]; + rand::rng().fill_bytes(&mut nonce); + let cipher = Aes256Gcm::new_from_slice(key).unwrap(); + let ct = cipher.encrypt(Nonce::from_slice(&nonce), data).unwrap(); + let mut out = Vec::with_capacity(12 + ct.len()); + out.extend_from_slice(&nonce); + out.extend_from_slice(&ct); + out +} + +/// The per-message subkey of ADR §5: HKDF-SHA-256 expand of the epoch key +/// with the message's sequence number. One output block, so this is exactly +/// HMAC-SHA-256(epoch key, info || 0x01) with info = the sequence number. +fn subkey(epoch_key: &[u8; 32], seq: u64) -> [u8; 32] { + let mut mac = as Mac>::new_from_slice(epoch_key).unwrap(); + mac.update(&seq.to_le_bytes()); + mac.update(&[1]); + mac.finalize().into_bytes().into() +} + /// Open `nonce || ciphertext` sealed by [`seal`] under `key`. fn open(key: &[u8; 32], sealed: &[u8]) -> Vec { let cipher = Aes256Gcm::new_from_slice(key).unwrap(); @@ -233,13 +257,19 @@ fn bench_erase(c: &mut Criterion) { &journal_len, |b, &journal_len| { let dir = tempfile::tempdir().unwrap(); - let entries: Vec<([u8; 32], Vec)> = (0..journal_len) - .map(|_| seal(&random_payload(TABLE_ROW))) + let mut epoch_key = [0u8; 32]; + rand::rng().fill_bytes(&mut epoch_key); + // Entry i is sealed under the subkey for sequence number i + 1 + // (0 is the table), as ADR §5 specifies. + let entries: Vec> = (0..journal_len as u64) + .map(|i| seal_under(&subkey(&epoch_key, i + 1), &random_payload(TABLE_ROW))) .collect(); b.iter(|| { let mut rows = Vec::with_capacity(journal_len * TABLE_ROW); - for (key, sealed) in &entries { - rows.extend_from_slice(&open(key, sealed)); + for (i, sealed) in entries.iter().enumerate() { + // Only the epoch key is held: derive each subkey here. + let key = subkey(&epoch_key, i as u64 + 1); + rows.extend_from_slice(&open(&key, sealed)); } write_synced(dir.path(), "keys.journal", &[]); black_box(rows.len()); diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index be94043..9c227f2 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -33,6 +33,10 @@ Two things are wrong with treating this as erasure. . *Deduplication is unchanged.* The key belongs to the blob, not to a path. A blob shared by two paths keeps its key until the last path that needs it is obliterated, which is the rule `obliterate_path` already applies to blobs. . *Key storage.* Keep all blob keys in *one key table* (`.januskey/keys`), sealed as a whole under the current *epoch key*, plus an append-only *key journal*. See §4 for where the epoch key comes from. ** A `store` appends one entry, sealed under the current epoch key, to the journal and syncs it. That costs the same at any number of keys; rewriting the whole table on every store would not. +** *Key before blob.* The journal entry is synced *before* the sealed blob is renamed into place. +*** A crash in between leaves a key with no blob. That is harmless, and the next fold drops it. +*** It never leaves a blob with no key. That case would be permanent loss, because deduplication would accept the existing blob on a retry. +*** Deduplication accepts an existing blob only if the table or journal holds its key; otherwise the blob is rewritten. ** A shred folds the journal into the table as part of the rewrite it does anyway. ** Journal entries are sealed under the same epoch key as the table, so the rotation below covers retained copies of the journal too. ** An earlier draft proposed one key file per blob, because dropping a single file is cheap. But making a shred final requires every live key to be re-sealed under a fresh epoch key. With per-blob files that is one synced write per live key: 1.2 s for only 1 000 keys (§3). The table makes a final shred one synced write. @@ -41,21 +45,22 @@ Two things are wrong with treating this as erasure. ** If the old epoch key is not destroyed, for example because the new table is written but the old key survives, the shred is only *pending*. Every retained copy of the old table, and so of the blob's key, can still be opened. ** The cost of a final shred is O(live keys + journal entries since the last rotation). *** Opening, dropping and re-sealing the table took about 1 ms at 1 000 keys and 26 ms at 100 000. -*** Folding the journal adds 0.4 ms at 100 entries and 3.4 ms at 10 000 (§3). +*** Folding the journal adds 0.7 ms at 100 entries and 18 ms at 10 000 (§3). Each entry needs its own subkey derivation and AES key schedule. ** *Stores and rotations are serialised.* Suppose a `jk` process appended to the journal under generation _n_ after a concurrent shred had folded that journal but before it removed secret _n_. That blob's key would be in no table and sealed under a destroyed secret, and its undo content would be lost permanently. *** So every journal append and every whole rotation runs under one exclusive store lock (`.januskey/keys.lock`, an OS file lock). +*** Readers take the same lock *shared*, for the whole of a `retrieve` or `undo` that uses a key. Without it, a process could open generation _n_, keep the victim's key or plaintext, and finish an undo after the shred reported the content gone. That would recreate obliterated data. A final shred waits for readers to finish. *** As a second guard, each journal entry carries the generation it was sealed under. A store that finds `current` changed while it waited for the lock re-seals under the new generation. *** This is stricter than the rest of `jk` today: concurrent writers are out of scope for the transaction log (T-5). It is required here because the failure is permanent key loss, not a lost update. ** *Rotation is crash-safe only with generations.* The keyring and the file system cannot be updated in one atomic step, so tables and epoch secrets are numbered by generation. *** A shred moving from generation _n_ to _n+1_: **** stores the new epoch secret in the keyring under _n+1_, keeping _n_; **** writes `keys.`, with an empty journal for _n+1_, and syncs it; -**** atomically renames a small `current` pointer file to name _n+1_; +**** writes a small `current` pointer naming _n+1_ by temporary file, fsync and rename, then fsyncs the directory, so the switch is durable *before* the next step; **** removes secret _n_ from the keyring; **** deletes `keys.`, its journal, and the blob. *** On open, a store whose keyring holds both _n_ and _n+1_ is mid-rotation: -**** if `current` names _n+1_, the open finishes the last two steps; -**** if `current` still names _n_, the open discards secret _n+1_ and `keys.`. +**** recovery chooses the generation that actually opens, not merely the one `current` names: it tries `current`'s generation first, then the other; +**** it finishes or undoes the rotation from there, and never discards the only generation whose secret is still present. *** A crash at any point leaves exactly one openable generation. *** The shred is final only once secret _n_ has left the keyring. ** Shreds could instead be batched, so that one rotation finalises many. A batched shred appends a *tombstone* for the blob to the key journal and syncs it (0.2 ms, §3); the blob is unusable through `jk` at once, but the old table and journal, and so the key, stay openable under the live epoch key until the next rotation. The ADR's default is a final shred every time; batching, and how long a shred may stay pending, are decisions for acceptance. @@ -73,7 +78,7 @@ Two things are wrong with treating this as erasure. == 3. Measured cost (E1, done) -`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after six rounds of review corrections. +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after seven rounds of review corrections. * *Conditions:* 4 vCPU, a container's virtual disk, one run, and medians, with Criterion's 95% intervals in the raw output. * *Payloads:* random, so incompressible. That is the worst case for gzip: it costs time and saves nothing. Compressible content would make every gzip row faster, and the sealed row with it, since that row compresses before sealing. @@ -96,7 +101,7 @@ The default-store and proposed rows come from one paired re-run, made after the |today: `obliterate_file` on the blob, 4 KiB / 64 KiB / 1 MiB / 16 MiB |0.71 / 0.98 / 6.76 / 124 ms |*pending* shred (batched): append a tombstone to the key journal and sync it |0.20 ms |*final* shred, one table: open (authenticate and decrypt) the sealed table, drop the entry, re-seal it under a fresh epoch key, one synced write; 64-byte rows; 1 000 / 10 000 / 100 000 live keys |0.96 / 2.91 / 26.2 ms -|*final* shred, add for the journal: open each entry stored since the last rotation and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.42 / 0.45 / 0.77 / 3.44 ms +|*final* shred, add for the journal: derive each entry's subkey from the epoch key, open it, and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.45 / 0.70 / 2.28 / 18.0 ms |*final* shred, one file per key: rewrap every key file under a fresh epoch key, 1 000 live keys |1 210 ms |=== @@ -105,7 +110,7 @@ Not in any row: * *Destroying the old epoch secret.* Its cost depends on where it lives (§4). * *The generation protocol's extra writes:* one keyring write and one pointer rename (§2). -The journal row is never zero in practice, even when every shred is final: any store since the last rotation leaves an entry, including the victim's own. An earlier version of this section claimed it was zero. +The journal row is never zero in practice, even when every shred is final: any store since the last rotation leaves an entry, including the victim's own. An earlier version of this section claimed it was zero. An earlier journal row (0.42 / 0.45 / 0.77 / 3.44 ms) used keys derived before timing began; the current row derives every subkey inside the measurement. What the numbers say: From 63b7c884b6db3447c0b01d591ab3c976dde18936 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 23:56:32 +0000 Subject: [PATCH 12/16] docs(proofs): cite the Agda receipt that covers E-1m; scope S-2's temp-file claim CodeRabbit review findings, verified: - The Grade 3 receipt (run 40867211509 at 4afb710) predates Epistemic.agda and --cubical-compatible: its check-agda.sh checked only Reversibility and TransactionAtomicity, so it never covered E-1m. The receipt is now run 41687715055 at fc188c4, which ran the current gate (all three modules, --safe --without-K --cubical-compatible, Agda 2.6.4.3, resource-algebra cf9235c); gate, modules, sibling fetch and workflow are byte-identical from fc188c4 to this head. - S-2: store ignores the remove_file error on its write and rename failure paths, so only a successful store is shown to leave no temporary file. Both rows now say so. - ADR-0002: the new generation's files and their directory are synced before the pointer switch; the journal-fold row is labelled as the read-and-clear part, the merge being inside the table re-seal. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- PROOF-NEEDS.adoc | 18 +++++++++++------- docs/adr/0002-crypto-shredding.adoc | 4 ++-- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc index ef054ef..6513abe 100644 --- a/PROOF-NEEDS.adoc +++ b/PROOF-NEEDS.adoc @@ -76,7 +76,7 @@ tables must agree. | T-7 | Transaction ids (UUID v4) never collide within one log (the freshness premise of T-2) | ASSUMED | probabilistic, not checked | T-8 | The operation metadata log is saved atomically, and an undo whose `+mark_undone+` fails to persist can be retried | OPEN | by inspection: the log now takes the same write, sync, rename path as the transaction log (`+crate::transaction::write_atomic+`), and `+metadata::tests::save_replaces_the_log_atomically+` asserts it, but the crate has not been built since that edit, so the row stays OPEN until the test is green in CI. Retry is unchanged: `+undo+` still runs the undo before `+mark_undone+` persists, so a failed persist still leaves the file undone and the log not | S-1 | A content hash that is not `+sha256:+` plus 64 lowercase hex digits never resolves to a path outside the store root | TESTED | `+content_store::tests::malformed_hash_never_leaves_the_store_root+`; fixed in #163 after audit probe P5 showed `+delete+` removing a file outside the store (`+docs/reports/audit/proof-statement-audit-2026-10-27.adoc+`) -| S-2 | `+store+` repairs a blob a crash left damaged instead of deduplicating against it, and leaves no temporary file behind | TESTED | not claimed: that the write itself is atomic. `+store+` writes by temporary file, sync and rename, but by construction only; no test injects a failure mid-write. `+content_store::tests::store_repairs_a_blob_truncated_by_a_crash+`, `+store_leaves_no_temp_files_and_count_ignores_strays+`; fixed in #163 (probe P4) +| S-2 | `+store+` repairs a blob a crash left damaged instead of deduplicating against it, and a successful store leaves no temporary file behind | TESTED | not claimed: cleanup after a failed write or rename, whose removal error `+store+` ignores, so a temporary file can remain; not claimed: that the write itself is atomic. `+store+` writes by temporary file, sync and rename, but by construction only; no test injects a failure mid-write. `+content_store::tests::store_repairs_a_blob_truncated_by_a_crash+`, `+store_leaves_no_temp_files_and_count_ignores_strays+`; fixed in #163 (probe P4) | L-1 | Editing any field of any audit-log entry makes `+verify_integrity+` fail | OPEN | false today, probes P1 and P2: algorithm, purpose, old_state, new_state, rotated_to and rotated_from are in neither the MAC nor the chain hash, and key_id, fingerprint and reason are chain-hashed only, so on the last entry nothing protects them. The fix is a MAC over every field (a scheme version bump) | L-2 | Removing trailing audit-log entries makes `+verify_integrity+` fail | OPEN | false today, probe P3: every prefix of a valid log verifies. Needs an anchor held outside the log (a design decision: signed head, external counter, or witness) | O-1 | An `+ObliterationProof+` attests that the content it names was erased | OPEN | false today, probe P6: `+ObliterationProof::generate+` sets `+storage_cleared: true+` unconditionally and `+verify_commitment+` holds for content that was never stored. Either compute the field from the erasure, or stop calling the object a proof @@ -124,11 +124,15 @@ their artefact is Agda (`+verification/proofs/agda/+`). They are **Grade 3**: each type-checks in Agda and none has been translated to an independent foundational system, which is mathesis §5's definition as written. The receipt is a green `+agda-proofs.yml+` run -(https://github.com/hyperpolymath/januskey/actions/runs/40867211509[40867211509]) -at commit `+4afb710e86646658f667f10bd6274ee47416bfc7+`, Agda 2.6.4.3, with -`+resource-algebra+` pinned at `+cf9235c+`. The modules, the gate script, the -sibling fetch and the workflow are byte-identical at that commit and at this -one, so the receipt covers these rows as they stand. Mathesis status for them: +(https://github.com/hyperpolymath/januskey/actions/runs/41687715055[41687715055]) +at commit `+fc188c4+`, Agda 2.6.4.3, with `+resource-algebra+` pinned at +`+cf9235c+`. That run used the current gate (`+Reversibility+`, +`+TransactionAtomicity+` and `+Epistemic+` under `+--safe --without-K +--cubical-compatible+`), and the modules, the gate script, the sibling fetch and +the workflow are byte-identical at that commit and at this one, so the receipt +covers these rows, E-1m included, as they stand. (An earlier receipt cited here, +run 40867211509 at `+4afb710+`, predates `+Epistemic.agda+` and the +`+--cubical-compatible+` flag, so it never covered E-1m.) Mathesis status for them: `+AGDA-VERIFIED+`. The other seven PROVEN rows — A-1, A-2, C-1 to C-5 — are Idris2 theorems. Under @@ -251,7 +255,7 @@ the status table above. |S-1: A malformed content hash never resolves to a path outside the store root |TESTED |`crates/reversible-core/src/content_store.rs::malformed_hash_never_leaves_the_store_root` |`cargo test --locked -p reversible-core --lib malformed_hash_never_leaves_the_store_root` |S-1: A well-formed content hash keeps the existing two-level blob layout |TESTED |`crates/reversible-core/src/content_store.rs::well_formed_hash_keeps_the_existing_layout` |`cargo test --locked -p reversible-core --lib well_formed_hash_keeps_the_existing_layout` |S-2: Storing content again repairs a blob a crash left truncated |TESTED |`crates/reversible-core/src/content_store.rs::store_repairs_a_blob_truncated_by_a_crash` |`cargo test --locked -p reversible-core --lib store_repairs_a_blob_truncated_by_a_crash` -|S-2: A store leaves no temporary file behind, and count ignores stray ones |TESTED |`crates/reversible-core/src/content_store.rs::store_leaves_no_temp_files_and_count_ignores_strays` |`cargo test --locked -p reversible-core --lib store_leaves_no_temp_files_and_count_ignores_strays` +|S-2: A successful store leaves no temporary file behind, and count ignores stray ones |TESTED |`crates/reversible-core/src/content_store.rs::store_leaves_no_temp_files_and_count_ignores_strays` |`cargo test --locked -p reversible-core --lib store_leaves_no_temp_files_and_count_ignores_strays` |G-2: The Idris2 IsCNO / ReversibleOp pair corresponds to the Agda ReversiblePair (by transcription) |ASSUMED |`verification/proofs/agda/Reversibility.agda` |- |X-1: Hash one-wayness and physical erasure (cryptographic and physical assumptions) |ASSUMED |`src/abi/JanusKey/ABI/Proofs.idr` |- |G-3: The FFI bridge preserves grade 𝟙 (a homomorphism; resource-algebra lists homomorphisms as OPEN) |OPEN |`verification/proofs/agda/Reversibility.agda` |- diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 9c227f2..81ec79a 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -54,7 +54,7 @@ Two things are wrong with treating this as erasure. ** *Rotation is crash-safe only with generations.* The keyring and the file system cannot be updated in one atomic step, so tables and epoch secrets are numbered by generation. *** A shred moving from generation _n_ to _n+1_: **** stores the new epoch secret in the keyring under _n+1_, keeping _n_; -**** writes `keys.`, with an empty journal for _n+1_, and syncs it; +**** writes `keys.`, with an empty journal for _n+1_, syncs both files, then syncs the directory so their entries are durable; **** writes a small `current` pointer naming _n+1_ by temporary file, fsync and rename, then fsyncs the directory, so the switch is durable *before* the next step; **** removes secret _n_ from the keyring; **** deletes `keys.`, its journal, and the blob. @@ -101,7 +101,7 @@ The default-store and proposed rows come from one paired re-run, made after the |today: `obliterate_file` on the blob, 4 KiB / 64 KiB / 1 MiB / 16 MiB |0.71 / 0.98 / 6.76 / 124 ms |*pending* shred (batched): append a tombstone to the key journal and sync it |0.20 ms |*final* shred, one table: open (authenticate and decrypt) the sealed table, drop the entry, re-seal it under a fresh epoch key, one synced write; 64-byte rows; 1 000 / 10 000 / 100 000 live keys |0.96 / 2.91 / 26.2 ms -|*final* shred, add for the journal: derive each entry's subkey from the epoch key, open it, and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.45 / 0.70 / 2.28 / 18.0 ms +|*final* shred, add for the journal (reading and clearing only; merging the rows into the table is part of the table row above, which re-seals the whole table anyway): derive each entry's subkey from the epoch key, open it, and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.45 / 0.70 / 2.28 / 18.0 ms |*final* shred, one file per key: rewrap every key file under a fresh epoch key, 1 000 live keys |1 210 ms |=== From 4a8950edd9186740c14bb0a5dd49413065de5079 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 00:07:33 +0000 Subject: [PATCH 13/16] ci(mirror): forward the three R2 backup secrets; ADR-0002 eighth review round mirror-reusable.yml at a797d30 declares ten optional secrets; the named mapping that replaced `secrets: inherit` forwarded only the seven forge keys, so an enabled R2 bundle backup would silently receive empty credentials. R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY and R2_AGE_RECIPIENT are now passed too. Every other caller's mapping was re-checked against its reusable's declared secrets with a YAML parser: all match. ADR-0002, four findings, all correct: - Dedupe accepts an existing blob only if it authenticates under the mapped key; a key's mere presence is not enough after a repair crash. - Rotation recovery guarantees at least one openable generation (both are, by design, between the table write and the pointer switch), selects the committed one, and ends with exactly one; E5 checks that. - Per-message subkeys use a random 256-bit salt carried in the message, not a sequence number, which a torn-append recovery could reuse. - The fold benchmark reads the salted journal from disk inside the timed loop: 0.48 / 0.74 / 2.43 / 19.1 ms for 10 / 100 / 1k / 10k. actionlint, fmt and clippy --benches -D warnings pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- .github/workflows/mirror.yml | 5 +++- benches/crypto_shred.rs | 43 ++++++++++++++++++----------- docs/adr/0002-crypto-shredding.adoc | 18 ++++++------ 3 files changed, 40 insertions(+), 26 deletions(-) diff --git a/.github/workflows/mirror.yml b/.github/workflows/mirror.yml index 1940333..f2b820c 100644 --- a/.github/workflows/mirror.yml +++ b/.github/workflows/mirror.yml @@ -17,7 +17,7 @@ jobs: uses: metadatastician/standards/.github/workflows/mirror-reusable.yml@a797d308c0517d4e2d21cc7f1cb08d41f82b1a12 # Named, not `inherit`: metadatastician is an organisation and hyperpolymath a # user account, and GitHub refuses `secrets: inherit` across that boundary. - # These are the secrets mirror-reusable.yml declares (all optional). + # These are the ten secrets mirror-reusable.yml declares (all optional). secrets: GITLAB_SSH_KEY: ${{ secrets.GITLAB_SSH_KEY }} BITBUCKET_SSH_KEY: ${{ secrets.BITBUCKET_SSH_KEY }} @@ -26,6 +26,9 @@ jobs: DISROOT_SSH_KEY: ${{ secrets.DISROOT_SSH_KEY }} GITEA_SSH_KEY: ${{ secrets.GITEA_SSH_KEY }} RADICLE_KEY: ${{ secrets.RADICLE_KEY }} + R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }} + R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }} + R2_AGE_RECIPIENT: ${{ secrets.R2_AGE_RECIPIENT }} permissions: actions: read contents: read diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index b1a4053..fbd8c39 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -48,9 +48,11 @@ const SIZES: [usize; 4] = [4 << 10, 64 << 10, 1 << 20, 16 << 20]; /// A plaintext key-table row: 32-byte content address and 32-byte blob key. /// The table is sealed as one message, so its rows are not wrapped one by one. const TABLE_ROW: usize = 32 + 32; -/// A key-journal entry: one table row sealed on its own (12-byte nonce, -/// the row, 16-byte tag), since entries are appended one at a time. -const JOURNAL_ENTRY: usize = 12 + TABLE_ROW + 16; +/// The random salt a journal entry carries to select its subkey (ADR §5). +const SALT: usize = 32; +/// A key-journal entry: one table row sealed on its own, since entries are +/// appended one at a time: salt, 12-byte nonce, the row, 16-byte tag. +const JOURNAL_ENTRY: usize = SALT + 12 + TABLE_ROW + 16; /// `size` random bytes: incompressible, the worst case for gzip. fn random_payload(size: usize) -> Vec { @@ -103,11 +105,11 @@ fn seal_under(key: &[u8; 32], data: &[u8]) -> Vec { } /// The per-message subkey of ADR §5: HKDF-SHA-256 expand of the epoch key -/// with the message's sequence number. One output block, so this is exactly -/// HMAC-SHA-256(epoch key, info || 0x01) with info = the sequence number. -fn subkey(epoch_key: &[u8; 32], seq: u64) -> [u8; 32] { +/// with the message's random salt. One output block, so this is exactly +/// HMAC-SHA-256(epoch key, info || 0x01) with info = the salt. +fn subkey(epoch_key: &[u8; 32], salt: &[u8]) -> [u8; 32] { let mut mac = as Mac>::new_from_slice(epoch_key).unwrap(); - mac.update(&seq.to_le_bytes()); + mac.update(salt); mac.update(&[1]); mac.finalize().into_bytes().into() } @@ -259,17 +261,26 @@ fn bench_erase(c: &mut Criterion) { let dir = tempfile::tempdir().unwrap(); let mut epoch_key = [0u8; 32]; rand::rng().fill_bytes(&mut epoch_key); - // Entry i is sealed under the subkey for sequence number i + 1 - // (0 is the table), as ADR §5 specifies. - let entries: Vec> = (0..journal_len as u64) - .map(|i| seal_under(&subkey(&epoch_key, i + 1), &random_payload(TABLE_ROW))) - .collect(); + // Each entry is salt || nonce || ciphertext, sealed under the + // subkey its own salt selects (ADR §5), written to disk once. + let mut journal = Vec::with_capacity(journal_len * JOURNAL_ENTRY); + for _ in 0..journal_len { + let salt = random_payload(SALT); + journal.extend_from_slice(&salt); + let row = random_payload(TABLE_ROW); + journal.extend_from_slice(&seal_under(&subkey(&epoch_key, &salt), &row)); + } + let path = dir.path().join("keys.journal.full"); + fs::write(&path, &journal).unwrap(); b.iter(|| { + // Timed: read the journal from disk, derive each subkey + // from the epoch key alone, open every entry, then start + // an empty journal. Merging the rows is in the table row. + let bytes = fs::read(&path).unwrap(); let mut rows = Vec::with_capacity(journal_len * TABLE_ROW); - for (i, sealed) in entries.iter().enumerate() { - // Only the epoch key is held: derive each subkey here. - let key = subkey(&epoch_key, i as u64 + 1); - rows.extend_from_slice(&open(&key, sealed)); + for entry in bytes.chunks_exact(JOURNAL_ENTRY) { + let (salt, sealed) = entry.split_at(SALT); + rows.extend_from_slice(&open(&subkey(&epoch_key, salt), sealed)); } write_synced(dir.path(), "keys.journal", &[]); black_box(rows.len()); diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 81ec79a..65d18c8 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -36,7 +36,7 @@ Two things are wrong with treating this as erasure. ** *Key before blob.* The journal entry is synced *before* the sealed blob is renamed into place. *** A crash in between leaves a key with no blob. That is harmless, and the next fold drops it. *** It never leaves a blob with no key. That case would be permanent loss, because deduplication would accept the existing blob on a retry. -*** Deduplication accepts an existing blob only if the table or journal holds its key; otherwise the blob is rewritten. +*** Deduplication accepts an existing blob only if it *authenticates* (the AEAD open succeeds) under the key the table or journal maps to its address. Otherwise the blob is rewritten. A key merely being present is not enough: a repair can crash after syncing a new key but before replacing the old ciphertext, and that key would not open the old blob. ** A shred folds the journal into the table as part of the rewrite it does anyway. ** Journal entries are sealed under the same epoch key as the table, so the rotation below covers retained copies of the journal too. ** An earlier draft proposed one key file per blob, because dropping a single file is cheap. But making a shred final requires every live key to be re-sealed under a fresh epoch key. With per-blob files that is one synced write per live key: 1.2 s for only 1 000 keys (§3). The table makes a final shred one synced write. @@ -45,7 +45,7 @@ Two things are wrong with treating this as erasure. ** If the old epoch key is not destroyed, for example because the new table is written but the old key survives, the shred is only *pending*. Every retained copy of the old table, and so of the blob's key, can still be opened. ** The cost of a final shred is O(live keys + journal entries since the last rotation). *** Opening, dropping and re-sealing the table took about 1 ms at 1 000 keys and 26 ms at 100 000. -*** Folding the journal adds 0.7 ms at 100 entries and 18 ms at 10 000 (§3). Each entry needs its own subkey derivation and AES key schedule. +*** Folding the journal adds 0.7 ms at 100 entries and 19 ms at 10 000 (§3). Each entry needs its own subkey derivation and AES key schedule. ** *Stores and rotations are serialised.* Suppose a `jk` process appended to the journal under generation _n_ after a concurrent shred had folded that journal but before it removed secret _n_. That blob's key would be in no table and sealed under a destroyed secret, and its undo content would be lost permanently. *** So every journal append and every whole rotation runs under one exclusive store lock (`.januskey/keys.lock`, an OS file lock). *** Readers take the same lock *shared*, for the whole of a `retrieve` or `undo` that uses a key. Without it, a process could open generation _n_, keep the victim's key or plaintext, and finish an undo after the shred reported the content gone. That would recreate obliterated data. A final shred waits for readers to finish. @@ -61,7 +61,7 @@ Two things are wrong with treating this as erasure. *** On open, a store whose keyring holds both _n_ and _n+1_ is mid-rotation: **** recovery chooses the generation that actually opens, not merely the one `current` names: it tries `current`'s generation first, then the other; **** it finishes or undoes the rotation from there, and never discards the only generation whose secret is still present. -*** A crash at any point leaves exactly one openable generation. +*** A crash at any point leaves *at least one* openable generation. Between steps 2 and 4 both are openable by design. Recovery then selects deterministically: the generation `current` names if it opens, else the other. It then finishes or rolls back the rotation, so after recovery exactly one remains. *** The shred is final only once secret _n_ has left the keyring. ** Shreds could instead be batched, so that one rotation finalises many. A batched shred appends a *tombstone* for the blob to the key journal and syncs it (0.2 ms, §3); the blob is unusable through `jk` at once, but the old table and journal, and so the key, stay openable under the live epoch key until the next rotation. The ADR's default is a final shred every time; batching, and how long a shred may stay pending, are decisions for acceptance. . *Scope: content-store copies only.* The guarantee covers what JanusKey writes: blobs and keys. It does not cover the working file itself, which `jk obliterate` targets too. @@ -78,7 +78,7 @@ Two things are wrong with treating this as erasure. == 3. Measured cost (E1, done) -`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after seven rounds of review corrections. +`cargo bench -p januskey --bench crypto_shred` measured these figures on 2026-11-04, after eight rounds of review corrections. * *Conditions:* 4 vCPU, a container's virtual disk, one run, and medians, with Criterion's 95% intervals in the raw output. * *Payloads:* random, so incompressible. That is the worst case for gzip: it costs time and saves nothing. Compressible content would make every gzip row faster, and the sealed row with it, since that row compresses before sealing. @@ -101,7 +101,7 @@ The default-store and proposed rows come from one paired re-run, made after the |today: `obliterate_file` on the blob, 4 KiB / 64 KiB / 1 MiB / 16 MiB |0.71 / 0.98 / 6.76 / 124 ms |*pending* shred (batched): append a tombstone to the key journal and sync it |0.20 ms |*final* shred, one table: open (authenticate and decrypt) the sealed table, drop the entry, re-seal it under a fresh epoch key, one synced write; 64-byte rows; 1 000 / 10 000 / 100 000 live keys |0.96 / 2.91 / 26.2 ms -|*final* shred, add for the journal (reading and clearing only; merging the rows into the table is part of the table row above, which re-seals the whole table anyway): derive each entry's subkey from the epoch key, open it, and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.45 / 0.70 / 2.28 / 18.0 ms +|*final* shred, add for the journal (reading and clearing only; merging the rows into the table is part of the table row above, which re-seals the whole table anyway): read the journal from disk, derive each entry's subkey from the epoch key and its salt, open it, and start an empty journal, one synced write; 10 / 100 / 1 000 / 10 000 entries |0.48 / 0.74 / 2.43 / 19.1 ms |*final* shred, one file per key: rewrap every key file under a fresh epoch key, 1 000 live keys |1 210 ms |=== @@ -110,7 +110,7 @@ Not in any row: * *Destroying the old epoch secret.* Its cost depends on where it lives (§4). * *The generation protocol's extra writes:* one keyring write and one pointer rename (§2). -The journal row is never zero in practice, even when every shred is final: any store since the last rotation leaves an entry, including the victim's own. An earlier version of this section claimed it was zero. An earlier journal row (0.42 / 0.45 / 0.77 / 3.44 ms) used keys derived before timing began; the current row derives every subkey inside the measurement. +The journal row is never zero in practice, even when every shred is final: any store since the last rotation leaves an entry, including the victim's own. An earlier version of this section claimed it was zero. An earlier journal row (0.42 / 0.45 / 0.77 / 3.44 ms) used keys derived before timing began; the current row reads the journal from disk and derives every subkey inside the measurement. What the numbers say: @@ -170,9 +170,9 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ ** *AEAD security.* AES-256-GCM is IND-CPA and INT-CTXT for *one message per key*. *** Blob keys meet that by construction. *** The epoch key would not, because the table and every journal entry under one epoch are separate messages. -*** So each of them is sealed under its own subkey: HKDF-SHA-256 over the epoch key with the message's sequence number (0 for the table, then 1, 2, … for journal entries). +*** So each of them is sealed under its own subkey: HKDF-SHA-256 over the epoch key with a random 256-bit salt that the message carries in its header. *** Every AEAD key therefore still encrypts exactly one message, and random-nonce collisions and per-key GCM usage limits stop being part of the assumption. -*** The design must never reuse a sequence number within an epoch, which is a property for the model to check. +*** A salt rather than a sequence number, because a sequence can be reused. Recovery that drops a torn final journal entry would hand its number to the next append, while a snapshot or CoW extent may still hold the torn entry's ciphertext. Two messages would then share one subkey. A fresh random salt is reused only with negligible probability (2^-128 by the birthday bound at 2^64 messages), and needs no durable counter. ** *Erasure.* Erasing the epoch secret where §4 keeps it is real erasure. ** *Key derivation.* HKDF-SHA-256 is a PRF and Argon2id resists offline guessing at the chosen cost. * *C-3 is unchanged:* obliteration stays by design irreversible, and `undo` after a shred has nothing to restore, as today. @@ -215,7 +215,7 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ *** Expected in both designs: the marker *is* found, from the working file's own extents. *** This case is kept as a test of the scope limit in §2, so that no one later reads the store-only result as covering the working file. * *E5: the full final shred.* Time the generation protocol of §2 end to end, including the keyring writes for §4's chosen root and the pointer rename. Confirm the cost stays independent of blob size and linear in live keys plus journal entries. -** Inject a crash after each of the protocol's steps. Check that the next open recovers to exactly one openable generation, and that no step leaves the shred reported final while secret _n_ is still in the keyring. +** Inject a crash after each of the protocol's steps. Check that every crash point leaves at least one openable generation, that recovery selects the committed one and ends with exactly one, and that no step leaves the shred reported final while secret _n_ is still in the keyring. * *E6: compression.* Repeat the store rows with compressible payloads (text, source code). Then measure skipping gzip when it does not shrink the input. This is a candidate fix to today's store whatever happens to this ADR. == 8. Beyond this repository From c0c606aaa2bf042b66709a3c2e5ab66c4592c363 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 00:14:32 +0000 Subject: [PATCH 14/16] docs(adr-0002): require a checked protocol model before acceptance; record open questions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ninth review round, five findings, all correct, and the pattern is now clear: each prose fix to the key-table, journal and rotation protocol exposes the next edge. Prose review cannot close that class; a model can. The ADR now states, as a precondition for acceptance, that the protocol be written as a state machine and checked for crash consistency (TLA+ or the Agda kernel beside T-1..T-3), with the four properties it must hold. New §9 records what remains open rather than patching it in prose: duplicate journal keys after a key-first crash; metadata-only new_content_hash references that migration cannot re-address; locked, zeroized in-process keys and tables before a shred is final (AV-2); and the benchmark exclusions. §3 now labels the table row as excluding the table read, and the table+journal sum as overcounting one directory fsync. The two benchmark entry points gain doc comments. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 5 +++++ docs/adr/0002-crypto-shredding.adoc | 13 +++++++++++++ 2 files changed, 18 insertions(+) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index fbd8c39..f758d33 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -128,6 +128,9 @@ fn gzip(data: &[u8]) -> Vec { enc.finish().unwrap() } +/// Store cost per blob: today's `ContentStore::store`, with compression off and +/// on (the `jk` default), against the proposed gzip-then-seal path and the +/// cipher alone. fn bench_store(c: &mut Criterion) { let mut g = c.benchmark_group("crypto_shred/store"); g.sample_size(10).measurement_time(Duration::from_secs(3)); @@ -183,6 +186,8 @@ fn bench_store(c: &mut Criterion) { g.finish(); } +/// Erase cost: today's overwrite of a blob against the pending and final stages +/// of a crypto-shred, and the rejected one-file-per-key layout for contrast. fn bench_erase(c: &mut Criterion) { let mut g = c.benchmark_group("crypto_shred/erase"); g.sample_size(10).measurement_time(Duration::from_secs(3)); diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 65d18c8..829d288 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -4,6 +4,7 @@ :toc: left * *Status:* proposed. Not accepted, and nothing is built. The benchmark in `benches/crypto_shred.rs` is the only code. +* *Precondition for acceptance:* §2's key-table, journal and rotation protocol is a sketch refined through nine rounds of review, and §9 lists what is still open. Before acceptance it must be written as a state machine and checked mechanically for crash consistency at every step, either in TLA+ or in the Agda kernel beside T-1 to T-3. The checked properties are: no blob without an openable key, at least one openable generation, no reused subkey, and a final shred leaving no openable copy of the victim's key. Prose review cannot close this class of question; a model can. * *Date:* 2026-11-04 * *Related:* ** ADR-0001: `jk` is reversible file operations, not a key system. @@ -109,6 +110,9 @@ Not in any row: * *Destroying the old epoch secret.* Its cost depends on where it lives (§4). * *The generation protocol's extra writes:* one keyring write and one pointer rename (§2). +* *Reading the sealed table from disk.* The table row opens a table already in memory, so the file read (up to about 6.4 MiB at 100 000 keys) is excluded. + +Adding the table row and the journal row also overcounts by one directory fsync: each row does its own, while §2's protocol syncs the directory once for both files. The sum is therefore an upper bound on that part, and a lower bound overall because of the exclusions above. The journal row is never zero in practice, even when every shred is final: any store since the last rotation leaves an entry, including the victim's own. An earlier version of this section claimed it was zero. An earlier journal row (0.42 / 0.45 / 0.77 / 3.44 ms) used keys derived before timing began; the current row reads the journal from disk and derives every subkey inside the measurement. @@ -222,3 +226,12 @@ Recommended for a first build: the passphrase plus epoch rotation, with the TPM/ * The same erasure-on-flash problem applies to any estate repository that promises deletion: consent and retention tooling, and obliteration in other stores. If this ADR is accepted, the root-key seam (§4) is a candidate for a shared crate rather than a januskey-only module. * `docs/security/THREAT_MODEL.adoc` covers only the key store. It needs a section on the obliteration path whichever way this ADR is decided. + +== 9. Open questions before acceptance + +These were raised in review and are *not* resolved by this proposal. Each belongs to the formal model required above, or to E5, rather than to further prose. + +* *Duplicate keys after a key-first crash.* A store can sync its journal entry and then crash before publishing the blob. A retry can then append a second key for the same address and publish under it. A fold must not keep the older key just because it exists. Either a retry reuses the pending key, or the fold keeps only the mapping that authenticates the published blob. +* *Hashes with no stored plaintext.* `Modify` and `Create` record a `new_content_hash` whose content is never stored, and the working file may since have changed or gone. Migration cannot compute an HMAC address for such references. Like legacy obliteration records (§6), they must be redacted or retained with the exposure stated. +* *Keys in process memory.* A rotation loads the old epoch secret, derives the epoch key and decrypts a table holding the victim's key. Under the threat model's memory-disclosure vector (AV-2: core dumps, swap, cold boot), dropping the old secret from the keyring is not enough. The secret, the derived keys, the decrypted table and the removed row must be held in locked memory and zeroized before the shred is reported final. +* *Benchmark completeness.* The exclusions listed in §3 (table I/O, the shared directory fsync, the keyring and pointer writes) are measured together in E5, as one end-to-end final shred. From dfb2efc851a4db63334faddfd923e6e6745f5423 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 00:17:22 +0000 Subject: [PATCH 15/16] fix(bench): use as_chunks for the fixed-size journal entries CI's clippy (Rust 1.99) denies chunks_exact with a constant size (clippy::chunks_exact_to_as_chunks), failing "Rust Build + Unit Tests" on 4a8950e. Reproduced with cargo +1.99.0 clippy --all --all-targets -D warnings; passes after this change on 1.99 and on 1.97, and the fold_journal benchmark still runs. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- benches/crypto_shred.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/benches/crypto_shred.rs b/benches/crypto_shred.rs index f758d33..40e45f1 100644 --- a/benches/crypto_shred.rs +++ b/benches/crypto_shred.rs @@ -283,7 +283,7 @@ fn bench_erase(c: &mut Criterion) { // an empty journal. Merging the rows is in the table row. let bytes = fs::read(&path).unwrap(); let mut rows = Vec::with_capacity(journal_len * TABLE_ROW); - for entry in bytes.chunks_exact(JOURNAL_ENTRY) { + for entry in bytes.as_chunks::().0 { let (salt, sealed) = entry.split_at(SALT); rows.extend_from_slice(&open(&subkey(&epoch_key, salt), sealed)); } From 195d1a32ad38a9d4ed66e1f9b95cb02903501b8f Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 5 Nov 2026 00:25:41 +0000 Subject: [PATCH 16/16] docs(adr-0002): correct the model invariant and an overclaim; record round-ten questions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two corrections to text that was wrong as written: - The acceptance precondition's invariant "no blob without an openable key" contradicts shredding itself (a shredded blob has none). It now reads: every live blob has an openable key and every shredded blob has none. - §3 compared the measured table-reseal portion (26 ms) with the full 16 MiB overwrite (124 ms) as if it were the complete shred. It now says the figures do not establish that a complete shred beats overwriting at any size, and leaves that to E5. Recorded in §9 rather than patched into prose: never re-seal under a pending key on retry; the migration marker must be durably written before any migration mutation; and two benchmark baselines that are not like for like (flat vs two-level blob layout; obliterate_file's extra read-and-hash versus the manager path). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_013CuBToxr5h9catptGiTsFX --- docs/adr/0002-crypto-shredding.adoc | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/adr/0002-crypto-shredding.adoc b/docs/adr/0002-crypto-shredding.adoc index 829d288..97285b6 100644 --- a/docs/adr/0002-crypto-shredding.adoc +++ b/docs/adr/0002-crypto-shredding.adoc @@ -4,7 +4,7 @@ :toc: left * *Status:* proposed. Not accepted, and nothing is built. The benchmark in `benches/crypto_shred.rs` is the only code. -* *Precondition for acceptance:* §2's key-table, journal and rotation protocol is a sketch refined through nine rounds of review, and §9 lists what is still open. Before acceptance it must be written as a state machine and checked mechanically for crash consistency at every step, either in TLA+ or in the Agda kernel beside T-1 to T-3. The checked properties are: no blob without an openable key, at least one openable generation, no reused subkey, and a final shred leaving no openable copy of the victim's key. Prose review cannot close this class of question; a model can. +* *Precondition for acceptance:* §2's key-table, journal and rotation protocol is a sketch refined through nine rounds of review, and §9 lists what is still open. Before acceptance it must be written as a state machine and checked mechanically for crash consistency at every step, either in TLA+ or in the Agda kernel beside T-1 to T-3. The checked properties are: every live (not tombstoned or shredded) blob has an openable key and every shredded blob has none, at least one openable generation, no reused subkey, and a final shred leaving no openable copy of the victim's key. Prose review cannot close this class of question; a model can. * *Date:* 2026-11-04 * *Related:* ** ADR-0001: `jk` is reversible file operations, not a key system. @@ -125,7 +125,7 @@ What the numbers say: * *Compression is the large-blob cost today.* gzip on incompressible input is the slowest step by an order of magnitude: 545 against 40 ms at 16 MiB. ** That observation is independent of this ADR. ** Skipping compression when it does not pay, or a faster codec, would help either design. -* *A final shred costs O(live keys), not O(blob size).* With one sealed table it is one synced write. That is cheaper than today's overwrite of a 16 MiB blob even at 100 000 keys (26 against 124 ms), but dearer than overwriting a 1 MiB blob at that key count. +* *A final shred costs O(live keys), not O(blob size).* With one sealed table it is one synced write. The *measured portion* (opening, dropping and re-sealing the table: 26 ms at 100 000 keys) is below today's 124 ms overwrite of a 16 MiB blob and above the 6.8 ms overwrite of a 1 MiB blob. The complete shred adds the table read, a non-empty journal, the keyring writes and the pointer rename, and the keyring cost depends on §4's backend. So these figures do not establish that a complete shred beats overwriting at any size; E5 measures that. * *Unlike overwriting, the claim holds on any medium*, subject to §4. * *Key layout.* One file per key makes final shreds unworkable (1.2 s at 1 000 keys). It is why §2 chose the single table. ** The pending-shred row is only the first step of a batched shred, not an erasure. @@ -235,3 +235,6 @@ These were raised in review and are *not* resolved by this proposal. Each belong * *Hashes with no stored plaintext.* `Modify` and `Create` record a `new_content_hash` whose content is never stored, and the working file may since have changed or gone. Migration cannot compute an HMAC address for such references. Like legacy obliteration records (§6), they must be redacted or retained with the exposure stated. * *Keys in process memory.* A rotation loads the old epoch secret, derives the epoch key and decrypts a table holding the victim's key. Under the threat model's memory-disclosure vector (AV-2: core dumps, swap, cold boot), dropping the old secret from the keyring is not enough. The secret, the derived keys, the decrypted table and the removed row must be held in locked memory and zeroized before the shred is reported final. * *Benchmark completeness.* The exclusions listed in §3 (table I/O, the shared directory fsync, the keyring and pointer writes) are measured together in E5, as one end-to-end final shred. +* *Retrying a store after a key-first crash.* If the first attempt staged ciphertext before crashing, re-sealing under the same pending key would make that key protect two messages (a retained temporary file or CoW extent may keep the first). So a retry either publishes the exact authenticated staged ciphertext, or creates a new key and lets the fold discard the stale mapping. It never re-seals under a pending key. +* *Durable migration marker.* The marker must be written by temporary file, fsync, rename and directory fsync *before* any migration mutation, as the generation pointer is. Otherwise a power loss can lose it after the log has been rewritten, and nothing then resumes the cleanup. +* *Benchmark baselines.* Two comparisons in §3 are not like for like. The sealed-store arm writes every blob into one directory, while `ContentStore::store` uses its two-level hash-prefix layout. Today's erase baseline times `obliterate_file`, which also reads and hashes the whole file before overwriting it, whereas the content-store path (`ObliterationManager::obliterate`) already holds the hash. The first can distort small-blob overhead and its scaling; the second inflates today's overwrite figures. E2 and E5 use the product layout and the manager path.