diff --git a/.claude/board/entries/2026-10-26-d-tflp-65-landing.md b/.claude/board/entries/2026-10-26-d-tflp-65-landing.md new file mode 100644 index 000000000..fd489d4f7 --- /dev/null +++ b/.claude/board/entries/2026-10-26-d-tflp-65-landing.md @@ -0,0 +1,20 @@ +# 2026-10-26 — D-TFLP 6.5 landing: softmax evidence + 8-bit admission + +**Status:** MEASURED (lab). Docs and probes only; no crate, ABI or kernel change. Does not move any +`D-TFLP-n` status row: D-TFLP-4 (softmax in ndarray) stays Queued. + +The Amendment 6.5 lab work moves from research branch `claude/bpe-tokenization-architecture-3xd4eh` +(not rebased) to `main` as two draft PRs: + +- **#1800**: softmax fixture replay (runs under `cargo test`, no TFLM) and the certified preparation + gate. While landing, the gate gained a table↔calibration check: it recomputes the table from the + claimed (scale, β) and refuses `table_identity_mismatch`. It also gained separate outputs for the + numerical cache key `(mult, shift)` and the semantic receipt. Re-run results: 601/529/72 known and + 514/446/68 held-out, identical to the research run; fixtures regenerate byte-identical. +- **this PR**: `pal256-admission-v1/admit` (17 tests). The identity claim is narrowed to + TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET: 21 instantiations on ndarray `af4910a1`, with pairwise-distinct + signatures. `HANDOFF.md` maps it onto D-NAM §2 / PZL-4, the D-NFR S5 binding store (MISSING) and + DT64-QB (#1799). + +OPEN: generation identity owner (EXO-152); binding store (D-NFR S5); contract placement of `Receipt`; +accumulator-wrapping semantics (proposal kept on the research branch, in no landing PR). diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index f8029be9c..155282a88 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -482 entries, 2026-08-06 .. 2026-10-26. +483 entries, 2026-08-06 .. 2026-10-26. | date | entry id | finding | file | |---|---|---|---| @@ -34,6 +34,7 @@ exactly one of them, never both. | 2026-10-26 | `D-DR256-OD` | | [2026-10-26-dr256-on-demand-l1.md](2026-10-26-dr256-on-demand-l1.md) | | 2026-10-26 | `D-DR256` | | [2026-10-26-dr256-global-union-lab.md](2026-10-26-dr256-global-union-lab.md) | | 2026-10-26 | `D-DQM` | FAIL | [2026-10-26-dqm-p1-chunk-touched.md](2026-10-26-dqm-p1-chunk-touched.md) | +| 2026-10-26 | `D-TFLP` | | [2026-10-26-d-tflp-65-landing.md](2026-10-26-d-tflp-65-landing.md) | | 2026-10-26 | `coresearch-lance13-masked-read-dr64` | | [2026-10-26-coresearch-lance13-masked-read-dr64.md](2026-10-26-coresearch-lance13-masked-read-dr64.md) | | 2026-10-25 | `D-DRF-W8` | | [2026-10-25-drf-w8-budget.md](2026-10-25-drf-w8-budget.md) | | 2026-10-25 | `D-DRF-W4` | | [2026-10-25-drf-w4-handoff.md](2026-10-25-drf-w4-handoff.md) | diff --git a/.claude/probes/pal256-admission-v1/HANDOFF.md b/.claude/probes/pal256-admission-v1/HANDOFF.md new file mode 100644 index 000000000..cca8f202c --- /dev/null +++ b/.claude/probes/pal256-admission-v1/HANDOFF.md @@ -0,0 +1,51 @@ +# Handoff: 8-bit admission ↔ DT64 / Quack / D-NAM + +**Status:** lab evidence plus integration notes. This document proposes no runtime, store, ABI or +registry, and it does not duplicate the DT64 or D-NAM documents. It says where the admission contract +in `admit/` would attach, and what must exist before that can happen. + +## What exists + +- **ndarray `Codec8`** (`src/hpc/codec8.rs`, on master). Codec identity exists only in the + type parameters; a payload is a bare `&[i8]`. There is no production consumer on main, only lab + probes and `examples/codec8_asm_probe.rs`. +- **`admit/` (this directory).** It provides the receipt, the expectation and `bind`, with 17 tests and a + 6×6 matrix in which only the diagonal binds. Identity is derived from the sealed traits, and the + claim is scoped to the 21 pinned instantiations (TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET). +- **Softmax certification, #1800.** It is the same distinction one level up: a numerical cache key + `(mult, shift)` against a semantic receipt `(scale, zp, β, operator, source)`. Two calibrations can share a + table and still need separate receipts. + +## Where it would attach (planned, not built) + +| owner | what it needs from this | status there | +|---|---|---| +| D-NAM capstone §2 "Recipe receipt" (`.claude/plans/d-nam-palette-native-lut-capstone-v1.md`) | the field set of `Receipt` (codec signature, scale/zp, operator, generation, source, payload digest) as the 8-bit half of a recipe receipt | PROPOSAL; [EXO-152](https://linear.app/exo-red/issue/EXO-152) | +| PZL-4 (same plan): "two same-shape layers with different weights/calibration must refuse wrong table" | `bind` refusing N4 (generation), N6 (scale/zp) and N7 (same bytes, different receipt), and #1800's `table_identity_mismatch` | waits for owner-issued generation identity | +| D-NFR S5, **binding store, MISSING** (`.claude/plans/d-native-repr-contract-v1.md`) | the record a binding would carry. `admit` is not that store and must not become a second one | MISSING; D-NFR-1 is the probe | +| DT64-QB (#1799, with stages #1794–#1798) | DT64 supplies source-attested typed semantic bindings and borrowed lanes. For an 8-bit lane, the receipt is what makes "which code space" part of that binding instead of part of a Rust type that disappears at storage | docs only | +| Quack / RelAlg / R2IL / mask-risc | none directly. They execute relational algebra over already-bound lanes. A matching receipt is not permission to reuse an escaped Keep mask (#1792 merged; escaped authority still open) | unchanged | + +ndarray owns numerics and D-NAM owns numerical methods; ONNX/TFLite stay leaf sources, and model +import is not a prerequisite for any of this. + +## Open, with owners + +1. **Generation identity:** who issues `codebook_generation` (EXO-152 / PZL-4). Until then the field is + compared but not minted by anyone. +2. **Binding store:** where a bound receipt lives and when it is invalidated (D-NFR S5 × S6). This is not + decided here. +3. **Contract placement:** whether the `Receipt` fields enter a contract type at all. That is an owner + decision; this lab crate makes no claim. +4. **Accumulator wrapping:** biased VPDPBUSD/TDPBUSD wrap by design. A five-line `wrapping_*` proposal is + on the research branch (`int8-semantics-v1/accum_wrapping65.diff`). It makes debug match release but + removes the out-of-range signal. It is not in any landing PR; it needs an owner decision on + overflow semantics first. +5. **Outside the pinned codec set:** a new `Rounding`, `NanPolicy`, `CodeSpace` or `Stores` impl needs the + scope test extended before the identity claim covers it. + +## Not claimed + +- No source-system compatibility beyond the tested codec set and the softmax calibrations in #1800. +- No source permissions, no writes and no semantic identity from a digest. +- No change to Quack/DR architecture, Codec8 semantics or the decoder's handling of −128. diff --git a/.claude/probes/pal256-admission-v1/README.md b/.claude/probes/pal256-admission-v1/README.md new file mode 100644 index 000000000..9b9bf2400 --- /dev/null +++ b/.claude/probes/pal256-admission-v1/README.md @@ -0,0 +1,68 @@ +# pal256-admission-v1 (D-TFLP 6.5, lab) + +These lab crates exist for the 8-bit code-space question: which consumer may interpret which bytes. +No production file in ndarray or lance-graph is changed, and neither crate is a workspace member. + +| path | role | +|---|---| +| `admit/` | P0 semantic admission. `bind(artifact, expect)` admits 8-bit bytes for one consumer codec, or refuses with a named `Refusal`, before any value is interpreted | +| `replay/` | TFLM-free replay of the softmax reciprocal fixtures. It lands separately with the softmax certification work; see its own README | +| `matrix65.json` | 6 producers × 6 consumers; only the diagonal binds | +| `disable65_p0.txt` | disable runs: each admission check replaced by `if false && …`, and the test that must fail | +| `HANDOFF.md` | how this meets DT64, Quack and D-NAM, and what is deliberately not built | + +## The defect it addresses + +Amendment 6.4 finding F4 / N8.1: `Codec8` decodes −128 as NaN. That is +correct, because it is that codec's declared policy. The defect is that a payload is a bare `&[i8]`. The +producer's code space lives only in the producer's Rust type, so any consumer type can be applied to +the stored bytes. An asymmetric weight of −128 then silently becomes NaN. + +The decoder is not changed. Admission happens once, at preparation time. + +## Contract (`admit/src/lib.rs`) + +- **`Receipt`**, carried by every artifact. It holds: + - the codec signature; + - the scale and zero-point bits; + - the operator; + - the codebook generation; + - an optional source identity (revision, backend); + - the length and the payload SHA-256. +- **`Expect`**: what the consumer requires. +- **`bind`**: checks length, integrity, storage, code space, NaN policy, rounding, scale, zero point, + operator, generation and source. It then range-scans the payload. Each mismatch has its own `Refusal`. +- **`Bound::decode`**: decodes with no further checks. + +`CodecSignature` is derived from `Codec8`'s own sealed traits: +- the raw stored bytes for −128 and 0; +- `MIN` and `MAX`; +- `NAN_CODE`; +- the rounding of four half-ties. + +There is no name table and no registry. + +**Identity claim: TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET.** The pinned set is every instantiation the +sealed traits admit on ndarray master `af4910a1`, 21 types in all. A test checks that their signatures +are pairwise distinct. A codec impl added to ndarray later is outside the claim until it is added to that test. + +## Tests (`admit/tests/admission.rs`, 17) + +| group | cases | +|---|---| +| positive | asymmetric −128 is an ordinary value (P1/P2); SymNaN −128 is the declared sentinel (P3); Sym −127..127 (P4); a TFLM symmetric weight still rejects −128 (P5); signed and biased indexing agree on every code (P6) | +| negative | F4 after the fix (N1); SymNaN→Asym (N2); biased U8→raw signed (N3); wrong generation (N4); rounding (N5); scale / zero point (N6); same bytes with an incompatible receipt (N7); one corrupted byte (N8); missing or mismatched source (N9); truncated payload (N10) | +| baseline | `f4_before_wrong_interpretation_accepted`: without admission, the wrong interpretation goes through | +| scope | `signatures_are_pairwise_distinct_on_the_pinned_codec_set` | + +In every negative control, the payload bytes are identical in the accepted arm and the refused arm. +Only the consumer or a single receipt field changes. + +```sh +# ndarray must sit five levels up (…/ndarray next to the lance-graph checkout) +cd admit && cargo test --release && cargo run --release --example matrix ../matrix65.json +``` + +Measured on ndarray master `af4910a1`: 17 passed, and `matrix65.json` regenerates byte-identical. +Collapsing the rounding signature to one half-tie makes the scope test fail +(`TiesEven` and `Floor` collide), so the test is not vacuous. diff --git a/.claude/probes/pal256-admission-v1/admit/Cargo.toml b/.claude/probes/pal256-admission-v1/admit/Cargo.toml new file mode 100644 index 000000000..c8bab174f --- /dev/null +++ b/.claude/probes/pal256-admission-v1/admit/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "admit" +version = "0.0.0" +edition = "2021" +publish = false + +# D-TFLP Amendment 6.5 P0 lab adapter: semantic admission of 8-bit artifacts against a consumer +# codec. Lab only; not a workspace member; does not change ndarray. +# ndarray is five levels up: admit -> pal256-admission-v1 -> probes -> .claude -> lance-graph -> /home/user. +[dependencies] +ndarray = { path = "../../../../../ndarray", default-features = false, features = ["std"] } +sha2 = "0.10" + +[profile.release] +debug = 0 + +[workspace] diff --git a/.claude/probes/pal256-admission-v1/admit/examples/matrix.rs b/.claude/probes/pal256-admission-v1/admit/examples/matrix.rs new file mode 100644 index 000000000..f57a2dacc --- /dev/null +++ b/.claude/probes/pal256-admission-v1/admit/examples/matrix.rs @@ -0,0 +1,68 @@ +//! Amendment 6.5 P0: code-space compatibility matrix. Each producer encodes its own full code range +//! (every valid code, plus its NaN sentinel if it has one); the artifact is offered to every consumer. +//! A cell is "bind" or the refusal reason. Usage: matrix OUT.json +use admit::*; +use ndarray::hpc::codec8::{Asym, Calibration, CodeSpace, NaNAtMin, NanPolicy, NoNaN, Stores, Sym, TiesAway, TiesEven, I8, U8}; + +struct One; +impl Calibration for One { + const SCALE: f32 = 1.0; + const ZERO_POINT: i32 = 0; +} + +fn payload, S: CodeSpace, N: NanPolicy>() -> Vec { + let mut v: Vec = (S::MIN..=S::MAX).map(T::store).collect(); + if let Some(n) = N::NAN_CODE { + v.push(T::store(n)); + } + v +} + +macro_rules! codecs { + ($m:ident) => { + $m!( + ("I8/Asym/TiesAway/NoNaN (TfliteAct)", I8, Asym, TiesAway, NoNaN), + ("I8/Asym/TiesEven/NoNaN (OnnxQ)", I8, Asym, TiesEven, NoNaN), + ("I8/Sym/TiesAway/NoNaN (TfliteWeight)", I8, Sym, TiesAway, NoNaN), + ("I8/Sym/TiesAway/NaNAtMin (SymNaN)", I8, Sym, TiesAway, NaNAtMin), + ("U8/Asym/TiesAway/NoNaN (biased)", U8, Asym, TiesAway, NoNaN), + ("U8/Sym/TiesAway/NaNAtMin (biased SymNaN)", U8, Sym, TiesAway, NaNAtMin) + ) + }; +} + +macro_rules! run { + ($(($pn:expr, $pt:ty, $ps:ty, $pr:ty, $pnn:ty)),*) => {{ + let mut arts: Vec<(&str, Artifact)> = Vec::new(); + $( arts.push(($pn, produce::<$pt, $ps, $pr, $pnn, One>(&payload::<$pt, $ps, $pnn>(), "lab.op", 1, None))); )* + arts + }}; +} + +macro_rules! consume { + ($arts:expr; $(($cn:expr, $ct:ty, $cs:ty, $cr:ty, $cnn:ty)),*) => {{ + let mut rows: Vec = Vec::new(); + for (pn, a) in $arts.iter() { + let mut cells: Vec = Vec::new(); + $( + let cell = match bind(a, &Expect::<$ct, $cs, $cr, $cnn, One>::new("lab.op", 1, None)) { + Ok(_) => "bind".to_string(), + Err(r) => format!("{:?}", r).split([' ', '{']).next().unwrap().to_string(), + }; + cells.push(format!("\"{}\": \"{}\"", $cn, cell)); + )* + rows.push(format!(" \"{}\": {{{}}}", pn, cells.join(", "))); + } + rows + }}; +} + +fn main() { + let out = std::env::args().nth(1).expect("OUT.json"); + let arts = codecs!(run); + macro_rules! go { ($($t:tt)*) => { consume!(arts; $($t)*) }; } + let rows = codecs!(go); + let s = format!("{{\n \"rows_are_producers_columns_are_consumers\": true,\n \"matrix\": {{\n{}\n }}\n}}\n", rows.join(",\n")); + std::fs::write(&out, &s).unwrap(); + print!("{s}"); +} diff --git a/.claude/probes/pal256-admission-v1/admit/src/lib.rs b/.claude/probes/pal256-admission-v1/admit/src/lib.rs new file mode 100644 index 000000000..62efe8a7e --- /dev/null +++ b/.claude/probes/pal256-admission-v1/admit/src/lib.rs @@ -0,0 +1,283 @@ +//! D-TFLP Amendment 6.5 P0: the smallest safe binding contract between an 8-bit artifact and the +//! consumer that interprets it. Lab adapter only. +//! +//! The defect (Amendment 6.4 F4 / N8.1) is not in the decoder. `Codec8` +//! correctly decodes -128 as NaN, because that is its declared policy. The defect is that a payload +//! is a bare `&[i8]`: the producer's code space is carried only by the producer's Rust type and is +//! gone once the bytes are stored, so any consumer type can be applied to them. +//! +//! The contract here: an artifact carries a [`Receipt`]; a consumer states what it requires as an +//! [`Expect`]; [`bind`] compares the two field by field and checks integrity and the payload's +//! code range BEFORE any value is interpreted, and refuses with a named [`Refusal`]. Binding happens +//! once, at preparation time; a [`Bound`] view then decodes without further checks. +//! +//! No registry is introduced. A codec's identity is a [`CodecSignature`] DERIVED from `Codec8`'s +//! own sealed traits: the code range (`CodeSpace::MIN/MAX`), the NaN code (`NanPolicy::NAN_CODE`), +//! the raw bytes `Stores::store` writes for -128 and 0 (which separates I8 from biased U8), and the +//! rounding observed on four half-ties (which separates TiesAway, TiesEven and Floor). +//! +//! Scope of the identity claim: TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET. The pinned set is every +//! `Codec8` instantiation the sealed traits admit in ndarray `src/hpc/codec8.rs` as of +//! master `af4910a1`: 5 `Stores` pairs x 3 roundings with `NoNaN`, plus the 2 `Sym` pairs x 3 +//! roundings with `NaNAtMin`, i.e. 21 types. `tests/admission.rs` checks that their 21 signatures +//! are pairwise distinct. The signature is therefore injective on that set, nothing more: a new +//! `Rounding`, `NanPolicy`, `CodeSpace` or `Stores` impl is not covered until it is added to that +//! test (four half-ties cannot separate every conceivable rounding rule). +//! +//! Byte equality is not identity: [`bind`] compares receipts, so two artifacts with identical +//! payload bytes and different receipts bind to different consumers. + +use core::marker::PhantomData; +use ndarray::hpc::codec8::{Calibration, Codec8, Codec8Error, CodeSpace, Linear, NanPolicy, Rounding, Stores}; +use sha2::{Digest, Sha256}; + +/// Behavioural identity of a `Codec8` type, read from the codec's own traits. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct CodecSignature { + /// Raw stored byte for code -128 (I8: 0x80, biased U8: 0x00). + pub raw_of_min: u8, + /// Raw stored byte for code 0 (I8: 0x00, biased U8: 0x80). + pub raw_of_zero: u8, + /// Smallest value code. + pub min: i32, + /// Largest value code. + pub max: i32, + /// The NaN sentinel code, if the codec reserves one. + pub nan_code: Option, + /// `R::round` of 0.5, 1.5, 2.5, -0.5 (as i32): TiesAway 1,2,3,-1; TiesEven 0,2,2,0; Floor 0,1,2,-1. + pub rounding: [i32; 4], +} + +fn raw_byte(e: E) -> u8 { + assert_eq!(core::mem::size_of::(), 1, "an 8-bit codec stores one byte per code"); + // SAFETY: E is a one-byte Copy type (i8 or u8), checked above; reading it as u8 is a bit copy. + unsafe { core::mem::transmute_copy::(&e) } +} + +/// The signature of `Codec8`. Only the types' own associated items are used. +pub fn signature() -> CodecSignature +where + T: Stores, + S: CodeSpace, + R: Rounding, + N: NanPolicy, +{ + CodecSignature { + raw_of_min: raw_byte(T::store(-128)), + raw_of_zero: raw_byte(T::store(0)), + min: S::MIN, + max: S::MAX, + nan_code: N::NAN_CODE, + rounding: [R::round(0.5) as i32, R::round(1.5) as i32, R::round(2.5) as i32, R::round(-0.5) as i32], + } +} + +/// Where the bytes came from, when SourceParity needs it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct SourceIdentity { + /// Source revision, e.g. the TFLM checkout. + pub revision: String, + /// Kernel set / backend, e.g. "reference". + pub backend: String, +} + +/// What an artifact says about its payload. Everything a consumer needs to interpret the bytes. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Receipt { + /// Logical code space, storage, NaN policy, rounding and saturation (saturation is the code + /// range: encoders clamp to `min..=max`). + pub codec: CodecSignature, + /// `f32::to_bits` of the calibration scale. + pub scale_bits: u32, + /// Calibration zero point. + pub zero_point: i32, + /// The operator contract the bytes were prepared for (e.g. "fc.weight.per_tensor"). + pub operator: String, + /// Calibration / codebook generation the bytes belong to. + pub codebook_generation: u64, + /// Source identity, when the artifact claims SourceParity. + pub source: Option, + /// Number of codes in the payload. + pub len: usize, + /// SHA-256 of the payload bytes. + pub payload_sha256: [u8; 32], +} + +/// A stored artifact: a receipt and its raw payload bytes. +#[derive(Debug, Clone)] +pub struct Artifact { + /// The receipt. + pub receipt: Receipt, + /// The payload exactly as stored. + pub payload: Vec, +} + +fn sha256(b: &[u8]) -> [u8; 32] { + Sha256::digest(b).into() +} + +/// Produce an artifact from codes encoded by `Codec8` with calibration `C`. +pub fn produce(codes: &[T::Elem], operator: &str, codebook_generation: u64, source: Option) -> Artifact +where + T: Stores, + S: CodeSpace, + R: Rounding, + N: NanPolicy, + C: Calibration, +{ + let payload: Vec = codes.iter().map(|&e| raw_byte(e)).collect(); + Artifact { + receipt: Receipt { + codec: signature::(), + scale_bits: C::SCALE.to_bits(), + zero_point: C::ZERO_POINT, + operator: operator.to_string(), + codebook_generation, + source, + len: payload.len(), + payload_sha256: sha256(&payload), + }, + payload, + } +} + +/// Why a binding was refused. Every variant is decided before any payload value is interpreted. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Refusal { + /// Payload length differs from the receipt. + LengthMismatch { receipt: usize, payload: usize }, + /// Payload bytes do not hash to the receipt's digest. + IntegrityMismatch, + /// Same code range but a different stored byte layout (e.g. biased U8 vs raw I8). + StorageMismatch, + /// Different code range. + CodeSpaceMismatch { artifact: (i32, i32), consumer: (i32, i32) }, + /// Different NaN sentinel. + NanPolicyMismatch { artifact: Option, consumer: Option }, + /// Different rounding mode. + RoundingMismatch, + /// Different calibration scale. + ScaleMismatch, + /// Different calibration zero point. + ZeroPointMismatch { artifact: i32, consumer: i32 }, + /// Prepared for a different operator contract. + OperatorMismatch, + /// Belongs to a different calibration / codebook generation. + GenerationMismatch { artifact: u64, consumer: u64 }, + /// Source identity required and missing or different. + SourceMismatch, + /// A stored code outside the code space that is not the NaN code. + InvalidCode { index: usize, code: i32 }, +} + +/// What a consumer requires, stated from its own codec type `Codec8` and calibration `C`. +pub struct Expect { + /// Operator contract the consumer implements. + pub operator: String, + /// The generation it was prepared against. + pub codebook_generation: u64, + /// Required source identity (SourceParity consumers); `None` accepts any source. + pub source: Option, + _m: PhantomData (T, S, R, N, C)>, +} + +impl Expect { + /// A consumer requirement. + pub fn new(operator: &str, codebook_generation: u64, source: Option) -> Self { + Expect { operator: operator.to_string(), codebook_generation, source, _m: PhantomData } + } +} + +/// An artifact admitted for one consumer type. Holds the payload; decodes through the consumer's codec. +pub struct Bound<'a, T, S, R, N, C> { + payload: &'a [u8], + _m: PhantomData (T, S, R, N, C)>, +} + +/// Admit `a` for the consumer described by `e`. Checks, in order: length and integrity, codec +/// signature (storage, code space, NaN policy, rounding), calibration, operator, generation, +/// source, then the payload's code range. Returns the first refusal. +pub fn bind<'a, T, S, R, N, C>(a: &'a Artifact, e: &Expect) -> Result, Refusal> +where + T: Stores, + S: Linear, + R: Rounding, + N: NanPolicy, + C: Calibration, +{ + let r = &a.receipt; + if r.len != a.payload.len() { + return Err(Refusal::LengthMismatch { receipt: r.len, payload: a.payload.len() }); + } + if sha256(&a.payload) != r.payload_sha256 { + return Err(Refusal::IntegrityMismatch); + } + let want = signature::(); + if (r.codec.min, r.codec.max) != (want.min, want.max) { + return Err(Refusal::CodeSpaceMismatch { artifact: (r.codec.min, r.codec.max), consumer: (want.min, want.max) }); + } + if (r.codec.raw_of_min, r.codec.raw_of_zero) != (want.raw_of_min, want.raw_of_zero) { + return Err(Refusal::StorageMismatch); + } + if r.codec.nan_code != want.nan_code { + return Err(Refusal::NanPolicyMismatch { artifact: r.codec.nan_code, consumer: want.nan_code }); + } + if r.codec.rounding != want.rounding { + return Err(Refusal::RoundingMismatch); + } + if r.scale_bits != C::SCALE.to_bits() { + return Err(Refusal::ScaleMismatch); + } + if r.zero_point != C::ZERO_POINT { + return Err(Refusal::ZeroPointMismatch { artifact: r.zero_point, consumer: C::ZERO_POINT }); + } + if r.operator != e.operator { + return Err(Refusal::OperatorMismatch); + } + if r.codebook_generation != e.codebook_generation { + return Err(Refusal::GenerationMismatch { artifact: r.codebook_generation, consumer: e.codebook_generation }); + } + if let Some(src) = &e.source { + if r.source.as_ref() != Some(src) { + return Err(Refusal::SourceMismatch); + } + } + // Code-range scan: the signature matched, so the layout is the consumer's own. + for (index, &b) in a.payload.iter().enumerate() { + let code = load_raw::(b); + if (code < S::MIN || code > S::MAX) && N::NAN_CODE != Some(code) { + return Err(Refusal::InvalidCode { index, code }); + } + } + Ok(Bound { payload: &a.payload, _m: PhantomData }) +} + +fn load_raw, S: CodeSpace>(b: u8) -> i32 { + // T::Elem is one byte (checked when the artifact was produced and by `raw_byte`). + assert_eq!(core::mem::size_of::(), 1); + // SAFETY: T::Elem is a one-byte Copy type (i8 or u8); this is a bit copy. + let e: T::Elem = unsafe { core::mem::transmute_copy::(&b) }; + T::load(e) +} + +impl Bound<'_, T, S, R, N, C> +where + T: Stores, + S: Linear, + R: Rounding, + N: NanPolicy, + C: Calibration, +{ + /// Decode the admitted payload with the consumer's own codec. + pub fn decode(&self, dst: &mut [f32]) -> Result<(), Codec8Error> { + let elems: Vec = self + .payload + .iter() + .map(|&b| { + // SAFETY: as in `load_raw`. + unsafe { core::mem::transmute_copy::(&b) } + }) + .collect(); + Codec8::::decode_checked::(&elems, dst) + } +} diff --git a/.claude/probes/pal256-admission-v1/admit/tests/admission.rs b/.claude/probes/pal256-admission-v1/admit/tests/admission.rs new file mode 100644 index 000000000..963ccc836 --- /dev/null +++ b/.claude/probes/pal256-admission-v1/admit/tests/admission.rs @@ -0,0 +1,230 @@ +//! Amendment 6.5 P0 positive tests and negative controls. +//! +//! Rule for every negative control: the payload bytes are IDENTICAL in the accepted and the refused +//! arm. Only the consumer (or one receipt field) changes. A refusal caused by different input +//! values would not show that admission works. +use admit::*; +use ndarray::hpc::codec8::{Asym, Calibration, Codec8, Codec8Error, Floor, NaNAtMin, NoNaN, Sym, TiesAway, TiesEven, UnitFull, I8, U8}; + +struct One; +impl Calibration for One { + const SCALE: f32 = 1.0; + const ZERO_POINT: i32 = 0; +} +struct Half; +impl Calibration for Half { + const SCALE: f32 = 0.5; + const ZERO_POINT: i32 = 0; +} +struct OneZp5; +impl Calibration for OneZp5 { + const SCALE: f32 = 1.0; + const ZERO_POINT: i32 = 5; +} + +type SymNaN = Codec8; + +const OP: &str = "lab.op"; +fn src() -> Option { + Some(SourceIdentity { revision: "5f64e4ad".into(), backend: "reference".into() }) +} +fn asym_payload() -> Vec { + vec![-128, -127, -1, 0, 127] +} +fn asym_artifact() -> Artifact { + produce::(&asym_payload(), OP, 7, src()) +} + +// ---------------------------------------------------------------- positive + +#[test] +fn p1_p2_asym_minus128_and_neighbours_are_ordinary_values() { + let a = asym_artifact(); + let b = bind(&a, &Expect::::new(OP, 7, src())).unwrap(); + let mut out = [0f32; 5]; + b.decode(&mut out).unwrap(); + assert_eq!(out, [-128.0, -127.0, -1.0, 0.0, 127.0]); +} + +#[test] +fn p3_symnan_minus128_is_the_declared_sentinel() { + let a = produce::(&[-128, -127, 0, 127], OP, 7, None); + let b = bind(&a, &Expect::::new(OP, 7, None)).unwrap(); + let mut out = [0f32; 4]; + b.decode(&mut out).unwrap(); + assert!(out[0].is_nan()); + assert_eq!(&out[1..], &[-127.0, 0.0, 127.0]); +} + +#[test] +fn p4_sym_minus127_to_127_all_valid() { + let codes: Vec = (-127..=127).collect(); + let a = produce::(&codes, OP, 7, None); + let b = bind(&a, &Expect::::new(OP, 7, None)).unwrap(); + let mut out = vec![0f32; codes.len()]; + b.decode(&mut out).unwrap(); + assert!(out.iter().zip(&codes).all(|(&v, &c)| v == c as f32)); +} + +#[test] +fn p5_tflm_symmetric_weight_still_rejects_minus128() { + // A producer that CLAIMS Sym/NoNaN (TfliteWeight) but stores -128: refused at bind, by the + // code-range scan, with the position. + let a = produce::(&[3, -128, 4], OP, 7, None); + let r = bind(&a, &Expect::::new(OP, 7, None)); + assert_eq!(r.err(), Some(Refusal::InvalidCode { index: 1, code: -128 })); + // and the decoder's own check agrees, unchanged + let mut out = [0f32; 3]; + assert_eq!(Codec8::::decode_checked::(&[3, -128, 4], &mut out), Err(Codec8Error::InvalidCode { index: 1 })); +} + +#[test] +fn p6_signed_and_biased_indexing_agree_on_every_code() { + use ndarray::hpc::codec8::Stores; + // a 256-entry table indexed by the biased byte; the signed path indexes it with code + 128 + let table: Vec = (0..256u16).map(|i| i.wrapping_mul(40503)).collect(); + for c in -128..=127i32 { + let biased = >::store(c); + let signed = >::store(c); + assert_eq!(biased, (signed as u8) ^ 0x80, "storage relation at {c}"); + assert_eq!(table[biased as usize], table[(i32::from(signed) + 128) as usize], "index at {c}"); + assert_eq!(>::load(biased), c); + assert_eq!(>::load(signed), c); + if c > -128 { + assert_eq!(>::load(>::store(c)), c); + } + } +} + +// ---------------------------------------------------------------- negative controls + +/// The original F4: the bare decoder applied to an Asym payload ACCEPTS it and reads -128 as NaN. +#[test] +fn f4_before_wrong_interpretation_accepted() { + let p = asym_payload(); + let mut out = [0f32; 5]; + SymNaN::decode_checked::(&p, &mut out).unwrap(); + assert!(out[0].is_nan(), "baseline: the bare decoder turns an Asym value into NaN"); +} + +/// F4 after: the same bytes through `bind` are refused before interpretation, and the identical +/// payload binds to its own consumer (two-sided). +#[test] +fn n1_f4_after_asym_payload_to_symnan_refused() { + let a = asym_artifact(); + let r = bind(&a, &Expect::::new(OP, 7, src())).err(); + assert_eq!(r, Some(Refusal::CodeSpaceMismatch { artifact: (-128, 127), consumer: (-127, 127) })); + assert!(bind(&a, &Expect::::new(OP, 7, src())).is_ok()); +} + +#[test] +fn n2_symnan_payload_to_asym_consumer_refused() { + let a = produce::(&[-128, 1, 2], OP, 7, None); + assert!(matches!(bind(&a, &Expect::::new(OP, 7, None)).err(), Some(Refusal::CodeSpaceMismatch { .. }))); + // Sym/NoNaN consumer: same range, different sentinel + assert_eq!( + bind(&a, &Expect::::new(OP, 7, None)).err(), + Some(Refusal::NanPolicyMismatch { artifact: Some(-128), consumer: None }) + ); + assert!(bind(&a, &Expect::::new(OP, 7, None)).is_ok()); +} + +#[test] +fn n3_biased_u8_payload_to_raw_signed_consumer_refused() { + let codes: Vec = asym_payload().iter().map(|&c| >::store(i32::from(c))).collect(); + let a = produce::(&codes, OP, 7, None); + assert_eq!(bind(&a, &Expect::::new(OP, 7, None)).err(), Some(Refusal::StorageMismatch)); + let b = bind(&a, &Expect::::new(OP, 7, None)).unwrap(); + let mut out = [0f32; 5]; + b.decode(&mut out).unwrap(); + assert_eq!(out, [-128.0, -127.0, -1.0, 0.0, 127.0]); +} + +#[test] +fn n4_wrong_codebook_generation_refused() { + let a = asym_artifact(); + assert_eq!( + bind(&a, &Expect::::new(OP, 8, src())).err(), + Some(Refusal::GenerationMismatch { artifact: 7, consumer: 8 }) + ); +} + +#[test] +fn n5_wrong_rounding_refused() { + let a = asym_artifact(); + assert_eq!(bind(&a, &Expect::::new(OP, 7, src())).err(), Some(Refusal::RoundingMismatch)); +} + +#[test] +fn n6_wrong_scale_or_zero_point_refused() { + let a = asym_artifact(); + assert_eq!(bind(&a, &Expect::::new(OP, 7, src())).err(), Some(Refusal::ScaleMismatch)); + assert_eq!( + bind(&a, &Expect::::new(OP, 7, src())).err(), + Some(Refusal::ZeroPointMismatch { artifact: 0, consumer: 5 }) + ); +} + +#[test] +fn n7_same_bytes_incompatible_receipt_refused() { + // Two artifacts with identical payload bytes and digests, prepared for different operators. + let a = asym_artifact(); + let b = produce::(&asym_payload(), "other.op", 7, src()); + assert_eq!(a.payload, b.payload); + assert_eq!(a.receipt.payload_sha256, b.receipt.payload_sha256); + let e = Expect::::new(OP, 7, src()); + assert!(bind(&a, &e).is_ok()); + assert_eq!(bind(&b, &e).err(), Some(Refusal::OperatorMismatch)); +} + +#[test] +fn n8_one_corrupted_byte_refused() { + let mut a = asym_artifact(); + a.payload[2] ^= 0x01; + assert_eq!(bind(&a, &Expect::::new(OP, 7, src())).err(), Some(Refusal::IntegrityMismatch)); +} + +#[test] +fn n9_source_identity_required_and_mismatched_refused() { + let a = produce::(&asym_payload(), OP, 7, None); + assert_eq!(bind(&a, &Expect::::new(OP, 7, src())).err(), Some(Refusal::SourceMismatch)); + let wrong = Some(SourceIdentity { revision: "8f6596d".into(), backend: "reference".into() }); + let b = asym_artifact(); + assert_eq!(bind(&b, &Expect::::new(OP, 7, wrong)).err(), Some(Refusal::SourceMismatch)); +} + +#[test] +fn n10_truncated_payload_refused() { + let mut a = asym_artifact(); + a.payload.pop(); + assert_eq!(bind(&a, &Expect::::new(OP, 7, src())).err(), Some(Refusal::LengthMismatch { receipt: 5, payload: 4 })); +} + +// ---------------------------------------------------------------- pinned codec set + +/// The identity claim is scoped to this list: every instantiation ndarray's sealed traits admit. +/// Adding a codec impl to ndarray without adding it here leaves it outside the claim. +#[test] +fn signatures_are_pairwise_distinct_on_the_pinned_codec_set() { + macro_rules! sigs { + ($( ($t:ty, $s:ty, $n:ty) ),*) => {{ + let mut v = Vec::new(); + $( + v.push((concat!(stringify!($t), "/", stringify!($s), "/TiesAway/", stringify!($n)), signature::<$t, $s, TiesAway, $n>())); + v.push((concat!(stringify!($t), "/", stringify!($s), "/TiesEven/", stringify!($n)), signature::<$t, $s, TiesEven, $n>())); + v.push((concat!(stringify!($t), "/", stringify!($s), "/Floor/", stringify!($n)), signature::<$t, $s, Floor, $n>())); + )* + v + }}; + } + let all = sigs!( + (I8, Asym, NoNaN), (I8, Sym, NoNaN), (U8, Asym, NoNaN), (U8, Sym, NoNaN), (U8, UnitFull, NoNaN), + (I8, Sym, NaNAtMin), (U8, Sym, NaNAtMin) + ); + assert_eq!(all.len(), 21); + for (i, (a, sa)) in all.iter().enumerate() { + for (b, sb) in &all[i + 1..] { + assert_ne!(sa, sb, "{a} and {b} share a signature"); + } + } +} diff --git a/.claude/probes/pal256-admission-v1/disable65_p0.txt b/.claude/probes/pal256-admission-v1/disable65_p0.txt new file mode 100644 index 000000000..e1f284f62 --- /dev/null +++ b/.claude/probes/pal256-admission-v1/disable65_p0.txt @@ -0,0 +1,8 @@ +P0 disable runs (each check replaced by `if false && ...`, anchor asserted to occur exactly once, file restored after): +code-space -> FAILED n1_f4_after_asym_payload_to_symnan_refused, n2_symnan_payload_to_asym_consumer_refused + (F4 is then still refused, as NanPolicyMismatch: the NaN-policy check is a second line of defence) +storage -> FAILED n3_biased_u8_payload_to_raw_signed_consumer_refused +integrity -> FAILED n8_one_corrupted_byte_refused +range-scan -> FAILED p5_tflm_symmetric_weight_still_rejects_minus128 +operator -> FAILED n7_same_bytes_incompatible_receipt_refused +All other tests stayed green in each run (14 or 15 of 16 passed). diff --git a/.claude/probes/pal256-admission-v1/matrix65.json b/.claude/probes/pal256-admission-v1/matrix65.json new file mode 100644 index 000000000..51f4301ef --- /dev/null +++ b/.claude/probes/pal256-admission-v1/matrix65.json @@ -0,0 +1,11 @@ +{ + "rows_are_producers_columns_are_consumers": true, + "matrix": { + "I8/Asym/TiesAway/NoNaN (TfliteAct)": {"I8/Asym/TiesAway/NoNaN (TfliteAct)": "bind", "I8/Asym/TiesEven/NoNaN (OnnxQ)": "RoundingMismatch", "I8/Sym/TiesAway/NoNaN (TfliteWeight)": "CodeSpaceMismatch", "I8/Sym/TiesAway/NaNAtMin (SymNaN)": "CodeSpaceMismatch", "U8/Asym/TiesAway/NoNaN (biased)": "StorageMismatch", "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": "CodeSpaceMismatch"}, + "I8/Asym/TiesEven/NoNaN (OnnxQ)": {"I8/Asym/TiesAway/NoNaN (TfliteAct)": "RoundingMismatch", "I8/Asym/TiesEven/NoNaN (OnnxQ)": "bind", "I8/Sym/TiesAway/NoNaN (TfliteWeight)": "CodeSpaceMismatch", "I8/Sym/TiesAway/NaNAtMin (SymNaN)": "CodeSpaceMismatch", "U8/Asym/TiesAway/NoNaN (biased)": "StorageMismatch", "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": "CodeSpaceMismatch"}, + "I8/Sym/TiesAway/NoNaN (TfliteWeight)": {"I8/Asym/TiesAway/NoNaN (TfliteAct)": "CodeSpaceMismatch", "I8/Asym/TiesEven/NoNaN (OnnxQ)": "CodeSpaceMismatch", "I8/Sym/TiesAway/NoNaN (TfliteWeight)": "bind", "I8/Sym/TiesAway/NaNAtMin (SymNaN)": "NanPolicyMismatch", "U8/Asym/TiesAway/NoNaN (biased)": "CodeSpaceMismatch", "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": "StorageMismatch"}, + "I8/Sym/TiesAway/NaNAtMin (SymNaN)": {"I8/Asym/TiesAway/NoNaN (TfliteAct)": "CodeSpaceMismatch", "I8/Asym/TiesEven/NoNaN (OnnxQ)": "CodeSpaceMismatch", "I8/Sym/TiesAway/NoNaN (TfliteWeight)": "NanPolicyMismatch", "I8/Sym/TiesAway/NaNAtMin (SymNaN)": "bind", "U8/Asym/TiesAway/NoNaN (biased)": "CodeSpaceMismatch", "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": "StorageMismatch"}, + "U8/Asym/TiesAway/NoNaN (biased)": {"I8/Asym/TiesAway/NoNaN (TfliteAct)": "StorageMismatch", "I8/Asym/TiesEven/NoNaN (OnnxQ)": "StorageMismatch", "I8/Sym/TiesAway/NoNaN (TfliteWeight)": "CodeSpaceMismatch", "I8/Sym/TiesAway/NaNAtMin (SymNaN)": "CodeSpaceMismatch", "U8/Asym/TiesAway/NoNaN (biased)": "bind", "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": "CodeSpaceMismatch"}, + "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": {"I8/Asym/TiesAway/NoNaN (TfliteAct)": "CodeSpaceMismatch", "I8/Asym/TiesEven/NoNaN (OnnxQ)": "CodeSpaceMismatch", "I8/Sym/TiesAway/NoNaN (TfliteWeight)": "StorageMismatch", "I8/Sym/TiesAway/NaNAtMin (SymNaN)": "StorageMismatch", "U8/Asym/TiesAway/NoNaN (biased)": "CodeSpaceMismatch", "U8/Sym/TiesAway/NaNAtMin (biased SymNaN)": "bind"} + } +}