BeamMCP.Signer.COSE.HashEnvelope: a file of any size signed by its digest, the hash algorithm inside the signed bytes

.github/workflows/ci.yml

@@ -63,9 +63,15 @@ jobs:
       - run: mix credo --strict
       - run: mix hex.audit
       - run: mix test
-      # The box leg's ML-DSA-87 signature and N6 carrier, for the cross-verify job to check with
-      # implementations outside Elixir (tools/mldsa_sample.exs; the key pair is a published ACVP
-      # test vector).
+      # The hash envelope's done-when: a 10 GiB file (sparse, so no disk is written) signed in
+      # constant memory and verified holding only the digest and the signature, its digest held
+      # to sha384sum's (test/beam_mcp/signer/cose_hash_envelope_ten_gb_test.exs).
+      - name: a 10 GiB file signed in constant memory (hash envelope)
+        if: matrix.leg == 'box'
+        run: mix test --only ten_gb
+      # The box leg's ML-DSA-87 signature, N6 carrier and hash envelope, for the cross-verify job
+      # to check with implementations outside Elixir (tools/mldsa_sample.exs; the key pair is a
+      # published ACVP test vector).
       - name: an ML-DSA-87 signature for other implementations to verify
         if: matrix.leg == 'box'
         run: mix run tools/mldsa_sample.exs mldsa87-sample
@@ -85,7 +91,8 @@ jobs:
   # BeamMCP.Signer.MLDSA87's output, verified by two ML-DSA implementations other than the
   # OpenSSL that made it: Go's crypto/mldsa (which also derives the public key from the seed,
   # as OTP cannot) and AWS-LC through aws-lc-rs. Each must accept the signature and refuse it
-  # over the message with one bit flipped (tools/cross_verify/).
+  # over the message with one bit flipped; then read the N6 carrier and the hash envelope with
+  # its own CBOR and COSE code, build ToBeSigned itself and verify (tools/cross_verify/).
   cross-verify:
     name: ML-DSA-87 cross-verify (Go crypto/mldsa, aws-lc-rs)
     needs: test
@@ -101,10 +108,10 @@ jobs:
         with:
           go-version: "1.27.3"
           cache: false
-      - name: Go crypto/mldsa, and the N6 carrier read with fxamacker/cbor
+      - name: Go crypto/mldsa, and the N6 carrier and hash envelope read with fxamacker/cbor
         working-directory: tools/cross_verify/go
         run: go run . "$GITHUB_WORKSPACE/mldsa87-sample"
-      - name: aws-lc-rs, and the N6 carrier read with coset and ciborium (the toolchain and every crate pinned)
+      - name: aws-lc-rs, and the N6 carrier and hash envelope read with coset and ciborium (the toolchain and every crate pinned)
         working-directory: tools/cross_verify/rust
         run: |
           rustup toolchain install 1.92.0 --profile minimal

CHANGELOG.md

@@ -7,6 +7,35 @@ SPDX-License-Identifier: Apache-2.0
 
 ## [Unreleased]
 
+### Added: `BeamMCP.Signer.COSE.HashEnvelope`, large artifacts signed by their digest (non-claimed)
+
+- A new public module, outside core's `BeamMCP.Signer` behaviour, which is unchanged; `COSE`
+  still exports only `sign/2`. `sign_stream/2` hashes any enumerable of binaries with
+  `:crypto.hash_init/update/final`, so memory is one chunk whatever the size.
+  `sign_digest/2` takes a digest the host already has.
+- **Output:** the N6 single-signer carrier, with the digest as its payload, signed as an RFC 9995
+  COSE Hash Envelope under ML-DSA-87. The protected header is `{1: -50, 2: [258], 4: kid,
+  16: typ, 258: -43 | -44, 259: preimage type, ? 260: location}`; label 3 is never written.
+- **Why:** a raw signature over a digest leaves the hash algorithm unsigned and the digest
+  indistinguishable from content. Here both are in the signed bytes, and `crit` makes a verifier
+  that does not know hash envelopes refuse.
+- **Options:** the two keys, read as `MLDSA87` reads them (the seed function is called after
+  the last chunk); `:typ` and `:preimage_type` (required); `:location` (optional);
+  `:payload_hash` (`:sha384`, the default, or `:sha512`). Every option is judged before the
+  first chunk is read. New errors: `:no_typ`, `{:typ, :invalid | :full_payload}`,
+  `:no_preimage_type`, `{:preimage_type, :invalid}`, `{:location, :invalid}`,
+  `{:payload_hash, :invalid}`, `{:digest, :wrong_length}`, `{:stream, :not_binary |
+  :unreadable}`.
+- **Non-claimed:** the signature is pure ML-DSA-87, not HashML-DSA, but no NSA, NIST or IETF text
+  yet names the construction, so it is not a claimed path.
+- **Tests:** a 10 GiB file (sparse), signed on CI's box leg (`mix test --only ten_gb`), with the
+  VM's memory growth held under 64 MiB and the digest held to `sha384sum`'s; a test-side
+  verifier that holds only the envelope and the public key. CI's Go and Rust cross-verify jobs
+  read the box leg's envelope. coset 0.4.2 refuses it over `crit` [258] (it predates RFC 9995),
+  and the Rust check asserts that refusal, then verifies with coset's ToBeSigned.
+- `BeamMCP.Signer.COSE`: the option walk, the key read and the kid moved to a private
+  `BeamMCP.Signer.COSE.Keys`, shared with the hash envelope; its answers do not change.
+
 ### Added: `BeamMCP.Signer.COSE`, ML-DSA-87 in COSE with the algorithm and key signed
 
 - A third module implementing core's `BeamMCP.Signer` behaviour, unchanged. It returns

README.md

@@ -9,7 +9,7 @@ SPDX-License-Identifier: Apache-2.0
 
 Signers for [`beam_mcp`](https://github.com/ScriptKittyOS/beam_mcp)'s canonical connectome
 bytes, through OTP's `:crypto`. Three modules, each implementing core's `BeamMCP.Signer`
-behaviour (`sign/2`) and nothing else:
+behaviour (`sign/2`) and nothing else, and a fourth for artifacts too large to hold whole:
 
 - **`BeamMCP.Signer.Ed25519`**: Ed25519 (RFC 8032), a 64-byte signature.
 - **`BeamMCP.Signer.MLDSA87`**: ML-DSA-87 (FIPS 204), the post-quantum signature CNSA 2.0
@@ -18,8 +18,11 @@ behaviour (`sign/2`) and nothing else:
 - **`BeamMCP.Signer.COSE`**: the same ML-DSA-87 signature inside COSE (RFC 9964, algorithm
   -50), with the algorithm and the key's thumbprint in the signed bytes; see [COSE](#cose)
   below.
+- **`BeamMCP.Signer.COSE.HashEnvelope`** (non-claimed): a file of any size, streamed and
+  signed by its SHA-384 as an RFC 9995 COSE Hash Envelope under the same ML-DSA-87. It is not
+  a `BeamMCP.Signer`; see [Hash envelope](#hash-envelope-non-claimed) below.
 
-**What they sign.** Exactly the bytes `BeamMCP.Connectome.Canonical.encode/2` produces, as
+**What the three sign.** Exactly the bytes `BeamMCP.Connectome.Canonical.encode/2` produces, as
 `Canonical.signature/3` hands them over; the signature comes back beside them. Nothing else:
 no envelope byte moves, no verdict or receipt is added.
 
@@ -144,11 +147,55 @@ The schema number and `typ` are provisional and change when the shared registry
 The encoder is this package's own: no Elixir library writes deterministic CBOR or COSE with
 ML-DSA. It is held to RFC 8949 Appendix A and to RFC 9964 Appendix A.2, byte for byte.
 
+## Hash envelope (non-claimed)
+
+`BeamMCP.Signer.COSE.HashEnvelope` signs an artifact by its digest, so a 10 GB clip, tile
+bundle or model file never has to be in memory whole. The host streams it in, and memory stays
+one chunk and a hash state:
+
+```elixir
+{:ok, envelope} =
+  BeamMCP.Signer.COSE.HashEnvelope.sign_stream(chunks,   # say, a file read 1 MiB at a time
+    private_key: fn -> seed end, public_key: fn -> public_key end,
+    typ: "application/vnd.example.b80.test_hash.v0+cose", preimage_type: "video/mp4")
+```
+
+`sign_digest/2` takes a digest the host already has. What comes back is the same carrier as
+COSE's, with the digest as its payload:
+
+```
+{0: 0xFFFF0003, 1: h'<SHA-384 of the artifact>', 2: [h'<protected>', {}, null, h'<signature>']}
+
+protected = {1: -50, 2: [258], 4: h'<kid>', 16: "<typ>",
+             258: -43,                                 ; payload-hash-alg: SHA-384
+             259: "video/mp4",                         ; preimage-content-type
+             260: "<location>"}                        ; optional hint, never trusted
+```
+
+- **What it fixes.** A raw signature over a digest leaves the hash algorithm unsigned and the
+  digest indistinguishable from content. Here both the hash algorithm and the preimage's type
+  are in the protected header.
+- **How a verifier reads it.** `crit` lists 258, so a COSE verifier that does not know hash
+  envelopes refuses the signature. One that does needs only the envelope and the public key.
+  It rebuilds `["Signature1", protected, h'', digest]` and verifies. When it holds the
+  artifact, it also checks the digest.
+- **The `typ` is the host's** and required. It is a lowercase media type ending `+cose` that
+  names the kind of statement; the connectome's `typ` is refused. The preimage type is per
+  artifact. `payload_hash: :sha512` gives SHA-512 (-44).
+- **Non-claimed.** The signature is pure ML-DSA-87, not HashML-DSA, but no NSA, NIST or IETF
+  text yet names this construction. A verified envelope is judged non-claimed, and
+  `BeamMCP.Signer.COSE` stays the claimed path.
+- **Tested.** CI's box leg signs a 10 GiB file, holds the VM's memory growth under 64 MiB and
+  compares the digest with `sha384sum`'s. The Go and Rust verifiers read each run's envelope.
+  `coset` refuses it over `crit` (it predates RFC 9995), and the Rust check confirms that
+  refusal before verifying with coset's ToBeSigned.
+
 ## Any module
 
-This package reads the key options, and for COSE the issuer, and nothing else
-(`opts[:private_key]`, for ML-DSA-87 `opts[:public_key]`, for COSE `opts[:issuer]`) -- no
-environment variable, no file, no application config, no default.
+This package reads the key options, for COSE the issuer, and for the hash envelope its own
+options listed above, and nothing else (`opts[:private_key]`, for ML-DSA-87 `opts[:public_key]`,
+for COSE `opts[:issuer]`). There is no environment variable, no file and no application config.
+No key, issuer or `typ` has a default.
 Where the keys live between calls is the host's decision.
 
 **Core is a separate package.** `beam_mcp` holds no key and calls no signing primitive; it is

docs/architecture.md

@@ -7,7 +7,8 @@ SPDX-License-Identifier: Apache-2.0
 
 Three modules, one function each, one dependency direction. Ed25519 is drawn below; ML-DSA-87
 takes the same path with two keys and a check before it returns; COSE builds a statement around
-the bytes and has ML-DSA-87 sign that.
+the bytes and has ML-DSA-87 sign that. A fourth, the hash envelope, sits outside the behaviour
+and has ML-DSA-87 sign a streamed artifact's digest.
 
 ```text
  host application
@@ -55,6 +56,14 @@ the bytes and has ML-DSA-87 sign that.
   them, and returns the N6 single-signer carrier. Its one `:crypto` call is SHA-384; it calls
   no signing primitive. Its encoder, `BeamMCP.Signer.COSE.CBOR` (private), writes
   deterministic CBOR under N6's rules and refuses anything else by name.
+- **`BeamMCP.Signer.COSE.HashEnvelope`** (`lib/beam_mcp/signer/cose/hash_envelope.ex`), outside
+  the behaviour and non-claimed. It takes `sign_digest/2` (a digest the host has) or
+  `sign_stream/2` (any enumerable of binaries, hashed with `:crypto.hash_init/update/final`).
+  It judges every option, the public key and the hash before reading a chunk. It builds the
+  RFC 9995 protected header (`{1: -50, 2: [258], 4: kid, 16: typ, 258, 259, ?260}`), has
+  `BeamMCP.Signer.MLDSA87` sign ToBeSigned over the digest, and returns the same carrier, the
+  digest as its payload. It and COSE share `BeamMCP.Signer.COSE.Keys` (private) for the option
+  walk, the key read and the kid.
 - **`beam_mcp`** (a dependency, `~> 0.7`): defines the behaviour and produces the bytes. It
   holds no key, calls no signing primitive and does not depend on this package.
 - **Erlang/OTP `:crypto`**: performs Ed25519, ML-DSA-87 and SHA-384, through OpenSSL. This

docs/assurance-case.md

@@ -26,6 +26,13 @@ given bytes by SHA-384 and length together with a protected header naming the al
 and the key (its SHA-384 thumbprint, computed from the public key); changing any of them makes
 the signature invalid. It reads the keys and the issuer from nowhere else and keeps nothing.**
 
+**Given the same two keys, a `typ` and a preimage type, `BeamMCP.Signer.COSE.HashEnvelope`
+returns the same carrier, its payload the SHA-384 (or SHA-512) of the stream or digest given.
+Its ML-DSA-87 signature covers that digest together with a protected header naming the
+algorithm, the key, the `typ`, the hash algorithm (258, listed in `crit`) and the preimage type
+(259). Changing any of them makes the signature invalid. Memory is independent of the stream's
+length. Every option is judged before the stream is read. It is a non-claimed path.**
+
 ## Threat model
 
 The asset is the host's private key; the goal is that this package adds no way for it to be
@@ -40,6 +47,8 @@ read, kept or misused beyond the host's own call.
 | A signature over bytes other than those given, or a non-standard signature | yes | the bytes go to `:crypto.sign/4` unaltered, with the algorithm and `:none` as literals and no options argument (census); tests verify with `:crypto.verify/5` and with core's `encode!/2` bytes |
 | ML-DSA-87: a signature that does not verify under the key the host publishes (a seed and public key that are not a pair, a faulted signature) | yes | every signature is verified under the given public key, and must not verify over bytes one bit away, before it is returned; otherwise a named error and no signature |
 | The algorithm or the key named beside a signature swapped (scheme and key id are host-asserted, outside the signed bytes) | yes, for `BeamMCP.Signer.COSE` | alg and kid are in the COSE protected header, inside ToBeSigned; tests, and Go and Rust verifiers in CI, show the signature fails once either is changed. The two raw-signature modules do not address it, by their shape |
+| A digest taken for the content, or signed under an unnamed hash (a raw signature over a digest) | yes, for the hash envelope | 258 and 259 are in the protected header, and 258 is in `crit`, so a COSE verifier that does not know RFC 9995 refuses rather than misreads (coset does, in CI); label 3 is never written |
+| A large artifact exhausting memory, or hashed and then refused | yes, for the hash envelope | `:crypto.hash_init/update/final` over the host's stream, one chunk at a time (a 10 GiB test holds VM growth under 64 MiB); every option is judged before the first chunk; the seed function is called only after the last |
 | COSE output a verifier reads differently (non-deterministic CBOR, an extra header label, a tagged or attached Sign1) | yes | the encoder writes RFC 8949 §4.2.1 under N6's rules and refuses everything else by name, held to RFC 8949 Appendix A and RFC 9964 A.2 byte for byte; the header is exactly `{1, 4, 16}`; two non-Elixir libraries (fxamacker/cbor, coset) re-read it in CI |
 | Weak randomness | yes | Ed25519 uses none (deterministic, RFC 8032). ML-DSA-87 is hedged: OpenSSL's DRBG supplies the per-signature `rnd`; were it constant, signing would be FIPS 204's deterministic variant, which is also approved. No key is generated here |
 | The host's key storage, the node the host runs, OpenSSL defects | no | the host's and upstream's; stated in `SECURITY.md` |
@@ -60,7 +69,7 @@ read, kept or misused beyond the host's own call.
 
 | principle | here |
 |---|---|
-| Economy of mechanism | three modules, one public function each, and a private encoder; remote calls `:crypto.sign/4`, `:crypto.verify/5` for ML-DSA-87's check, and `:crypto.hash/2` (SHA-384) in COSE, pinned by census |
+| Economy of mechanism | three modules, one public function each; the hash envelope, two; a private encoder and key helpers; remote calls `:crypto.sign/4`, `:crypto.verify/5` for ML-DSA-87's check, `:crypto.hash/2` (SHA-384) in COSE, and the streaming hash calls in the hash envelope, pinned by census |
 | Fail-safe defaults | no default key; a missing or malformed key is an error, never a signature |
 | Complete mediation | the key is checked on every call; nothing is cached |
 | Open design | standard Ed25519 and ML-DSA-87, verifiable by any implementation; the source is Apache-2.0 |

docs/roadmap.md

@@ -39,8 +39,9 @@ is ever needed it follows the same rule, decided in the open.
 A third module wraps the second rather than adding an algorithm: `BeamMCP.Signer.COSE` signs
 the same ML-DSA-87 inside COSE, so the algorithm and the key are named in the signed bytes and
 not only beside them, in the deterministic-CBOR form (N6) Script Kitty's systems share. The
-RFC 9995 hash envelope, for artifacts too large to hand over whole, is next, as its own mode
-and not a claimed one until its standing under CNSA 2.0 is settled.
+RFC 9995 hash envelope, for artifacts too large to hand over whole, is its own module beside
+it (`BeamMCP.Signer.COSE.HashEnvelope`), outside the behaviour. It is not a claimed path until
+its standing under CNSA 2.0 is settled.
 
 ## How it changes
 

docs/security-review.md

@@ -22,7 +22,8 @@ boundaries in `docs/assurance-case.md`.
 Each item is answered with what was read, what was tried, and what was found.
 
 1. **One source for the key.** Read `lib/beam_mcp/signer/ed25519.ex`,
-   `lib/beam_mcp/signer/mldsa87.ex` and `lib/beam_mcp/signer/cose.ex` in full. Is each key read
+   `lib/beam_mcp/signer/mldsa87.ex`, `lib/beam_mcp/signer/cose.ex`, `lib/beam_mcp/signer/cose/keys.ex`
+   and `lib/beam_mcp/signer/cose/hash_envelope.ex` in full. Is each key read
    from its option only (`:private_key`, and `:public_key` for ML-DSA-87 and COSE)? Does the census's regex and call list still describe the
    compiled module (`test/beam_mcp/signer/no_key_source_test.exs`)?
 2. **Nothing kept.** No process state, table, persistent term, application environment or
@@ -39,6 +40,9 @@ Each item is answered with what was read, what was tried, and what was found.
    from the public key, and does ToBeSigned cover the body the carrier carries? Does the
    encoder (`lib/beam_mcp/signer/cose/cbor.ex`) refuse every float, tag, non-NFC text and
    non-integer key, and does a carrier re-read by an outside library re-encode identically?
+   For the hash envelope: is the header exactly `{1: -50, 2: [258], 4, 16, 258, 259}` plus an
+   optional 260, never label 3? Is every option judged before a chunk is read, and the seed
+   function called only after the last? Does the 10 GiB test's memory bound still hold?
 5. **Randomness.** Ed25519 uses none; ML-DSA-87's comes from OpenSSL's DRBG. Confirm that no
    key or nonce is generated in `lib/`.
 6. **Supply chain and release.** `mix.lock` against `mix hex.audit`; the `beam_mcp` requirement;