Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .claude/board/entries/2026-10-26-d-tflp-65-landing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# 2026-10-26 — D-TFLP 6.5 landing: softmax evidence + 8-bit admission

**Status:** MEASURED (lab). Docs and probes only; no crate, ABI or kernel change. Does not move any
`D-TFLP-n` status row: D-TFLP-4 (softmax in ndarray) stays Queued.

The Amendment 6.5 lab work moves from research branch `claude/bpe-tokenization-architecture-3xd4eh`
(not rebased) to `main` as two draft PRs:

- **#1800**: softmax fixture replay (runs under `cargo test`, no TFLM) and the certified preparation
gate. While landing, the gate gained a table↔calibration check: it recomputes the table from the
claimed (scale, β) and refuses `table_identity_mismatch`. It also gained separate outputs for the
numerical cache key `(mult, shift)` and the semantic receipt. Re-run results: 601/529/72 known and
514/446/68 held-out, identical to the research run; fixtures regenerate byte-identical.
- **this PR**: `pal256-admission-v1/admit` (17 tests). The identity claim is narrowed to
TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET: 21 instantiations on ndarray `af4910a1`, with pairwise-distinct
signatures. `HANDOFF.md` maps it onto D-NAM §2 / PZL-4, the D-NFR S5 binding store (MISSING) and
DT64-QB (#1799).

OPEN: generation identity owner (EXO-152); binding store (D-NFR S5); contract placement of `Receipt`;
accumulator-wrapping semantics (proposal kept on the research branch, in no landing PR).
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

482 entries, 2026-08-06 .. 2026-10-26.
483 entries, 2026-08-06 .. 2026-10-26.

| date | entry id | finding | file |
|---|---|---|---|
Expand All @@ -34,6 +34,7 @@ exactly one of them, never both.
| 2026-10-26 | `D-DR256-OD` | | [2026-10-26-dr256-on-demand-l1.md](2026-10-26-dr256-on-demand-l1.md) |
| 2026-10-26 | `D-DR256` | | [2026-10-26-dr256-global-union-lab.md](2026-10-26-dr256-global-union-lab.md) |
| 2026-10-26 | `D-DQM` | FAIL | [2026-10-26-dqm-p1-chunk-touched.md](2026-10-26-dqm-p1-chunk-touched.md) |
| 2026-10-26 | `D-TFLP` | | [2026-10-26-d-tflp-65-landing.md](2026-10-26-d-tflp-65-landing.md) |
| 2026-10-26 | `coresearch-lance13-masked-read-dr64` | | [2026-10-26-coresearch-lance13-masked-read-dr64.md](2026-10-26-coresearch-lance13-masked-read-dr64.md) |
| 2026-10-25 | `D-DRF-W8` | | [2026-10-25-drf-w8-budget.md](2026-10-25-drf-w8-budget.md) |
| 2026-10-25 | `D-DRF-W4` | | [2026-10-25-drf-w4-handoff.md](2026-10-25-drf-w4-handoff.md) |
Expand Down
51 changes: 51 additions & 0 deletions .claude/probes/pal256-admission-v1/HANDOFF.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Handoff: 8-bit admission ↔ DT64 / Quack / D-NAM

**Status:** lab evidence plus integration notes. This document proposes no runtime, store, ABI or
registry, and it does not duplicate the DT64 or D-NAM documents. It says where the admission contract
in `admit/` would attach, and what must exist before that can happen.

## What exists

- **ndarray `Codec8<T, S, R, N>`** (`src/hpc/codec8.rs`, on master). Codec identity exists only in the
type parameters; a payload is a bare `&[i8]`. There is no production consumer on main, only lab
probes and `examples/codec8_asm_probe.rs`.
- **`admit/` (this directory).** It provides the receipt, the expectation and `bind`, with 17 tests and a
6×6 matrix in which only the diagonal binds. Identity is derived from the sealed traits, and the
claim is scoped to the 21 pinned instantiations (TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET).
- **Softmax certification, #1800.** It is the same distinction one level up: a numerical cache key
`(mult, shift)` against a semantic receipt `(scale, zp, β, operator, source)`. Two calibrations can share a
table and still need separate receipts.

## Where it would attach (planned, not built)

| owner | what it needs from this | status there |
|---|---|---|
| D-NAM capstone §2 "Recipe receipt" (`.claude/plans/d-nam-palette-native-lut-capstone-v1.md`) | the field set of `Receipt` (codec signature, scale/zp, operator, generation, source, payload digest) as the 8-bit half of a recipe receipt | PROPOSAL; [EXO-152](https://linear.app/exo-red/issue/EXO-152) |
| PZL-4 (same plan): "two same-shape layers with different weights/calibration must refuse wrong table" | `bind` refusing N4 (generation), N6 (scale/zp) and N7 (same bytes, different receipt), and #1800's `table_identity_mismatch` | waits for owner-issued generation identity |
| D-NFR S5, **binding store, MISSING** (`.claude/plans/d-native-repr-contract-v1.md`) | the record a binding would carry. `admit` is not that store and must not become a second one | MISSING; D-NFR-1 is the probe |
| DT64-QB (#1799, with stages #1794–#1798) | DT64 supplies source-attested typed semantic bindings and borrowed lanes. For an 8-bit lane, the receipt is what makes "which code space" part of that binding instead of part of a Rust type that disappears at storage | docs only |
| Quack / RelAlg / R2IL / mask-risc | none directly. They execute relational algebra over already-bound lanes. A matching receipt is not permission to reuse an escaped Keep mask (#1792 merged; escaped authority still open) | unchanged |

ndarray owns numerics and D-NAM owns numerical methods; ONNX/TFLite stay leaf sources, and model
import is not a prerequisite for any of this.

## Open, with owners

1. **Generation identity:** who issues `codebook_generation` (EXO-152 / PZL-4). Until then the field is
compared but not minted by anyone.
2. **Binding store:** where a bound receipt lives and when it is invalidated (D-NFR S5 × S6). This is not
decided here.
3. **Contract placement:** whether the `Receipt` fields enter a contract type at all. That is an owner
decision; this lab crate makes no claim.
4. **Accumulator wrapping:** biased VPDPBUSD/TDPBUSD wrap by design. A five-line `wrapping_*` proposal is
on the research branch (`int8-semantics-v1/accum_wrapping65.diff`). It makes debug match release but
removes the out-of-range signal. It is not in any landing PR; it needs an owner decision on
overflow semantics first.
5. **Outside the pinned codec set:** a new `Rounding`, `NanPolicy`, `CodeSpace` or `Stores` impl needs the
scope test extended before the identity claim covers it.

## Not claimed

- No source-system compatibility beyond the tested codec set and the softmax calibrations in #1800.
- No source permissions, no writes and no semantic identity from a digest.
- No change to Quack/DR architecture, Codec8 semantics or the decoder's handling of −128.
68 changes: 68 additions & 0 deletions .claude/probes/pal256-admission-v1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# pal256-admission-v1 (D-TFLP 6.5, lab)

These lab crates exist for the 8-bit code-space question: which consumer may interpret which bytes.
No production file in ndarray or lance-graph is changed, and neither crate is a workspace member.

| path | role |
|---|---|
| `admit/` | P0 semantic admission. `bind(artifact, expect)` admits 8-bit bytes for one consumer codec, or refuses with a named `Refusal`, before any value is interpreted |
| `replay/` | TFLM-free replay of the softmax reciprocal fixtures. It lands separately with the softmax certification work; see its own README |
| `matrix65.json` | 6 producers × 6 consumers; only the diagonal binds |
| `disable65_p0.txt` | disable runs: each admission check replaced by `if false && …`, and the test that must fail |
| `HANDOFF.md` | how this meets DT64, Quack and D-NAM, and what is deliberately not built |

## The defect it addresses

Amendment 6.4 finding F4 / N8.1: `Codec8<I8, Sym, TiesAway, NaNAtMin>` decodes −128 as NaN. That is
correct, because it is that codec's declared policy. The defect is that a payload is a bare `&[i8]`. The
producer's code space lives only in the producer's Rust type, so any consumer type can be applied to
the stored bytes. An asymmetric weight of −128 then silently becomes NaN.

The decoder is not changed. Admission happens once, at preparation time.

## Contract (`admit/src/lib.rs`)

- **`Receipt`**, carried by every artifact. It holds:
- the codec signature;
- the scale and zero-point bits;
- the operator;
- the codebook generation;
- an optional source identity (revision, backend);
- the length and the payload SHA-256.
- **`Expect<T, S, R, N, C>`**: what the consumer requires.
- **`bind`**: checks length, integrity, storage, code space, NaN policy, rounding, scale, zero point,
operator, generation and source. It then range-scans the payload. Each mismatch has its own `Refusal`.
- **`Bound::decode`**: decodes with no further checks.

`CodecSignature` is derived from `Codec8`'s own sealed traits:
- the raw stored bytes for −128 and 0;
- `MIN` and `MAX`;
- `NAN_CODE`;
- the rounding of four half-ties.

There is no name table and no registry.

**Identity claim: TESTED_COMPATIBLE_FOR_PINNED_CODEC_SET.** The pinned set is every instantiation the
sealed traits admit on ndarray master `af4910a1`, 21 types in all. A test checks that their signatures
are pairwise distinct. A codec impl added to ndarray later is outside the claim until it is added to that test.

## Tests (`admit/tests/admission.rs`, 17)

| group | cases |
|---|---|
| positive | asymmetric −128 is an ordinary value (P1/P2); SymNaN −128 is the declared sentinel (P3); Sym −127..127 (P4); a TFLM symmetric weight still rejects −128 (P5); signed and biased indexing agree on every code (P6) |
| negative | F4 after the fix (N1); SymNaN→Asym (N2); biased U8→raw signed (N3); wrong generation (N4); rounding (N5); scale / zero point (N6); same bytes with an incompatible receipt (N7); one corrupted byte (N8); missing or mismatched source (N9); truncated payload (N10) |
| baseline | `f4_before_wrong_interpretation_accepted`: without admission, the wrong interpretation goes through |
| scope | `signatures_are_pairwise_distinct_on_the_pinned_codec_set` |

In every negative control, the payload bytes are identical in the accepted arm and the refused arm.
Only the consumer or a single receipt field changes.

```sh
# ndarray must sit five levels up (…/ndarray next to the lance-graph checkout)
cd admit && cargo test --release && cargo run --release --example matrix ../matrix65.json
```

Measured on ndarray master `af4910a1`: 17 passed, and `matrix65.json` regenerates byte-identical.
Collapsing the rounding signature to one half-tie makes the scope test fail
(`TiesEven` and `Floor` collide), so the test is not vacuous.
17 changes: 17 additions & 0 deletions .claude/probes/pal256-admission-v1/admit/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[package]
name = "admit"
version = "0.0.0"
edition = "2021"
publish = false

# D-TFLP Amendment 6.5 P0 lab adapter: semantic admission of 8-bit artifacts against a consumer
# codec. Lab only; not a workspace member; does not change ndarray.
# ndarray is five levels up: admit -> pal256-admission-v1 -> probes -> .claude -> lance-graph -> /home/user.
[dependencies]
ndarray = { path = "../../../../../ndarray", default-features = false, features = ["std"] }
sha2 = "0.10"

[profile.release]
debug = 0

[workspace]
68 changes: 68 additions & 0 deletions .claude/probes/pal256-admission-v1/admit/examples/matrix.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
//! Amendment 6.5 P0: code-space compatibility matrix. Each producer encodes its own full code range
//! (every valid code, plus its NaN sentinel if it has one); the artifact is offered to every consumer.
//! A cell is "bind" or the refusal reason. Usage: matrix OUT.json
use admit::*;
use ndarray::hpc::codec8::{Asym, Calibration, CodeSpace, NaNAtMin, NanPolicy, NoNaN, Stores, Sym, TiesAway, TiesEven, I8, U8};

struct One;
impl Calibration for One {
const SCALE: f32 = 1.0;
const ZERO_POINT: i32 = 0;
}

fn payload<T: Stores<S>, S: CodeSpace, N: NanPolicy<S>>() -> Vec<T::Elem> {
let mut v: Vec<T::Elem> = (S::MIN..=S::MAX).map(T::store).collect();
if let Some(n) = N::NAN_CODE {
v.push(T::store(n));
}
v
}

macro_rules! codecs {
($m:ident) => {
$m!(
("I8/Asym/TiesAway/NoNaN (TfliteAct)", I8, Asym, TiesAway, NoNaN),
("I8/Asym/TiesEven/NoNaN (OnnxQ)", I8, Asym, TiesEven, NoNaN),
("I8/Sym/TiesAway/NoNaN (TfliteWeight)", I8, Sym, TiesAway, NoNaN),
("I8/Sym/TiesAway/NaNAtMin (SymNaN)", I8, Sym, TiesAway, NaNAtMin),
("U8/Asym/TiesAway/NoNaN (biased)", U8, Asym, TiesAway, NoNaN),
("U8/Sym/TiesAway/NaNAtMin (biased SymNaN)", U8, Sym, TiesAway, NaNAtMin)
)
};
}

macro_rules! run {
($(($pn:expr, $pt:ty, $ps:ty, $pr:ty, $pnn:ty)),*) => {{
let mut arts: Vec<(&str, Artifact)> = Vec::new();
$( arts.push(($pn, produce::<$pt, $ps, $pr, $pnn, One>(&payload::<$pt, $ps, $pnn>(), "lab.op", 1, None))); )*
arts
}};
}

macro_rules! consume {
($arts:expr; $(($cn:expr, $ct:ty, $cs:ty, $cr:ty, $cnn:ty)),*) => {{
let mut rows: Vec<String> = Vec::new();
for (pn, a) in $arts.iter() {
let mut cells: Vec<String> = Vec::new();
$(
let cell = match bind(a, &Expect::<$ct, $cs, $cr, $cnn, One>::new("lab.op", 1, None)) {
Ok(_) => "bind".to_string(),
Err(r) => format!("{:?}", r).split([' ', '{']).next().unwrap().to_string(),
};
cells.push(format!("\"{}\": \"{}\"", $cn, cell));
)*
rows.push(format!(" \"{}\": {{{}}}", pn, cells.join(", ")));
}
rows
}};
}

fn main() {
let out = std::env::args().nth(1).expect("OUT.json");
let arts = codecs!(run);
macro_rules! go { ($($t:tt)*) => { consume!(arts; $($t)*) }; }
let rows = codecs!(go);
let s = format!("{{\n \"rows_are_producers_columns_are_consumers\": true,\n \"matrix\": {{\n{}\n }}\n}}\n", rows.join(",\n"));
std::fs::write(&out, &s).unwrap();
print!("{s}");
}
Loading
Loading