Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .claude/board/INTEGRATION_PLANS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 2026-11-04 — D-DRISA decision memos after the local register proof → `.claude/plans/d-drisa-decision-memos-v1.md`

**Status:** PROPOSED, owned by this session (EXO-153 stages 1 and 3). **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DRISA-1-LAB, D-DRISA-3-LAB, D-DRISA-M1..M7; [entry](entries/2026-11-04-d-drisa-1-register-proof.md). Architecture: #1953 (D-DRISA-0, a GPT-session plan adopted after review). Seven owner questions the local proof leaves open: a layout witness on the SoA (the first missing connection), a global id → tile index or co-addressed selections, result-register lifetime, source-location receipt, DR → Lance authority, the carry words as ALU, async loopback.
**Confidence:** each memo cites a lab measurement or a source line; no ruling made.
## 2026-11-04 — D-DRISA-0: virtual DR registers × Abakus ALU × async data-local cognitive shader

**Status: PROPOSED / DOCS ONLY.** [Architecture](../plans/d-drisa-remote-register-shader-v1.md) · [staged PR charter](../plans/d-drisa-staged-pr-charter-v1.md) · [57 independent falsifiers](../plans/d-drisa-falsifier-matrix-v1.md) · [entry](entries/2026-11-04-d-drisa-0-remote-register-contract.md) · [STATUS_BOARD](STATUS_BOARD.md) D-DRISA · [EXO-153](https://linear.app/exo-red/issue/EXO-153) · [EXO-50](https://linear.app/exo-red/issue/EXO-50) bus. **New hypothesis:** use existing DR frame as data-local virtual register namespace and preexisting Quack/R2IL/mask-risc/ndarray words as the typed ALU, including shipped Abakus half/full adders where arithmetic helps, with a bounded asynchronous remote descriptor and compact result. Do not send raw pointers, transport payloads by default, copy canonical values into DR, or create new global row mappings. Network zero-copy is *not* claimed; Arrow C same-process ABI and optional Flight RPC are distinguished. Nine separately gated PR stages, no production implementation. This does not replace D-DRAM-0/P3 or claim #1950's copied-lane prepared binding as address-only memoization. **D4/S-INGRESS, EXO-46/89/151, security and THOUGHT/WORLD admission remain open.**
Expand Down
14 changes: 14 additions & 0 deletions .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,20 @@
| D-DRAM-P3-7 | route once per frame, lab measurement | Queued | this session; production GATED EXO-46 |
| D-DRAM-P3-8 | = D-DRAM-4 end-to-end, decision memo | GATED | D4 + S-INGRESS (#1788) |

**D-DRISA lab (2026-11-04): this session executes EXO-153 stages 1 and 3** ([memos](../plans/d-drisa-decision-memos-v1.md); architecture #1953, a GPT-session plan adopted after review).

| D-id | Deliverable | Status | Owner / gate |
| --- | --- | --- | --- |
| D-DRISA-1-LAB | sealed DR frame as a local register file over borrowed SoA | Measured: 13/13 tests, 6 disables red, 0 allocations/query, +0.04–0.6 µs revalidation; [entry](entries/2026-11-04-d-drisa-1-register-proof.md) | this session; lab crate `drisa-register-probe` |
| D-DRISA-3-LAB | k-of-n / Kleene / Bag on the shipped carry words | In stacked PR | this session; ndarray / Quack owners |
| D-DRISA-M1 | layout witness on the canonical SoA | Proposed | SoA / DR owner |
| D-DRISA-M2 | global id → tile index, or co-addressed selections | Proposed | EXO-46 |
| D-DRISA-M3 | result-register lifetime across calls | Proposed | result owner |
| D-DRISA-M4 | source-location receipt | Deferred | EXO-46/89/151 |
| D-DRISA-M5 | DR → Lance row authority | Unchanged, GATED | D4 / S-INGRESS |
| D-DRISA-M6 | carry words as the default ALU | Proposed: refuse on current evidence | ndarray / Quack owners |
| D-DRISA-M7 | async loopback (stage 2) as a lab | Proposed: after review of stage 1 | runtime / security review |

Verified: ndarray #402 half/full adder words and #390 byte sum shipped, `Abacus<BITS>` lab-only; DR `LocalMap` is identity-only; H14c/H14d and #1633 selective Lance reads are lab. Unverified: exact operator-mentioned `ndarray::simd::5x24::*` signature. **Borrow/prefetch default; materialized foreign facet only under D-DRF-1 §5 admission.**

## D-DT64-7 — behaviour addressing and registration: Frozen, Learned, Explore over one classid (2026-10-31)
Expand Down
68 changes: 68 additions & 0 deletions .claude/board/entries/2026-11-04-d-drisa-1-register-proof.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# D-DRISA-1: a sealed DR frame works as a local register file over borrowed SoA, once the SoA states its layout

**Status:** MEASURED-LAB (local, one process). Owner: this session (EXO-153 stage 1). Probe: `crates/drisa-register-probe/` (`src/lib.rs`, `tests/integration/register.rs`, `examples/drisa_bench.rs`), excluded from the workspace. Architecture: #1953 (D-DRISA-0, a plan written by a GPT session, adopted after review). No production change, no transport, no new public type.

## Census: what the register contract can be built from

Read at source on `main` `05f03abb4`.

| Piece | Where | Status |
|---|---|---|
| Sealed frame, compared by issue (not by value) | `dt64::dr::Frame`, `FrameStore::seal/check` (`dr.rs:404`, `:445`) | EXISTING |
| Versioned tile maps, old frames replay their layout | `MapTable::advance/map/map_at` (`dr.rs:871`, `:913`, `:921`) | EXISTING |
| Counted global → local routing; per-tile digest | `LocalMap::local` counts lookups (`:605`), `TileMap::digest` (`:780`) | EXISTING |
| Locate one id across tiles | `MapTable::locate` reads EVERY listed tile (`:955`) | EXISTING; O(tiles) per id by design (ambiguity check) |
| Focus / Scope / Guards | `dr.rs:496`, `:503`, `:509` | EXISTING; `Focus` is a tile revision, not a lease |
| Map vs value lifetime | `dt64::ValueDemand::{MapOnly, Values}` (`lib.rs:189`) | EXISTING, no executor reads it |
| Borrowed operands | `mask-risc` `Planes { masks: &[&[u64]], lanes: &[LaneRef] }`, `Operand::{Plane, Scratch}` (`ir.rs:7`, `:73`) | EXISTING |
| Execution | `mask-risc::execute` / `execute_into` (`exec.rs:763`, `:1343`) over `ndarray::simd` | WIRED |
| Stale scratch refused | `ExecError::ScratchReadBeforeWrite` (`value.rs:128`) | EXISTING; the plan's "StaleSlot" is the R2IL probe's `FoldError::StaleSlot`, not mask-risc |
| A statement of which frame/tile digests a SoA's rows are laid out under | none found in the closed space above | **MISSING** (the lab `Soa` carries it) |
| A global id → tile index | none; `locate` scans tiles | **MISSING**, owner decision (EXO-46) |
| Source-location receipt / lease host | only excluded lab crates (#1951 census) | **BLOCKED** EXO-46/89/151 |
| Cross-call result register lifetime | no owner | **BLOCKED**, owner decision |
| DR → Lance `_rowid` authority | none | **BLOCKED** D4 / S-INGRESS |

## The proof

`RegisterFile` is a descriptor: the frame, the tile digests it was bound under, a scope, the demanded field positions, a value witness when values were demanded, and one map-only register (the identity selection, routed from global ids at bind). It holds no reference into the SoA and no value. At execution it re-checks every operand and hands `mask-risc` planes that borrow the SoA's own slices.

The eight properties of the mission, each with the test that checks it:

| # | Property | Result |
|---|---|---|
| 1 | Registers name coordinates, not values | `execution_borrows_the_soa_and_copies_no_value`: executed lanes and the presence plane have the SoA's own addresses; the file owns only the selection bits (8 KiB for 65,536 rows) |
| 2 | Methods name typed operands, not rows | `MethodOp` names field positions and constants; compiled once to an unchanged `mask-risc` `Program` |
| 3 | No second ALU | every answer comes from `mask-risc::execute` |
| 4 | One binding serves many methods without rerouting | `many_methods_reuse_one_binding_without_rerouting`: 256 rounds × 4 methods, 0 lookups after bind |
| 5 | Lifetimes tied to frame, layout, scope, source, values | each check has its own refusal and its own red disable (below) |
| 6 | A value write invalidates value-scoped registers, not the map | `a_value_write_invalidates_values_but_not_the_map`: COUNT stays valid and correct; value methods and the derived predicate register refuse; re-taking the value witness routes nothing; the stale derived register still refuses until recomputed |
| 7 | An invalid register refuses, never answers wrong or empty | `Refusal` has a variant per cause; the oracle tests show the lax versions answer wrong |
| 8 | No mandatory payload gather | no lane is copied; the only owned buffers are the selection and the optional derived register (one bit per row each) |

All methods equal an independent oracle (global id → value, no map or position) on DR64 (1 tile), DR256 (1 tile) and DR256 × 256 tiles (65,536 identities, local order permuted per tile, 10% selection that includes absent identities). 13/13 tests.

**Disable runs** (commit first, patch with an asserted anchor, `git checkout`): frame check off → 1 red; layout digest compare off → 2 red; value witness off (lanes) → 1 red; value witness off (derived register) → 1 red; scope check off → 1 red; duplicate-identity check off → 1 red. Restored: 13/13.

**Review fixes (Codex, #1955).** The value witness is now `(incarnation, revision)`: a rebuilt SoA restarted its revision at zero and was accepted against a witness from the old one. `Compiled` now records the lane schema it was resolved against, and `run` refuses `SchemaMismatch` on a file whose lanes name other fields. Two falsifiers with controls; disables: incarnation pinned → 1 red, schema check off → 1 red. 15/15.

## Cost (µs per query, round robin over 4 methods, 3 processes)

| fixture | A direct planes | C one binding, revalidated per query | allocations per query (C) |
|---|---|---|---|
| DR64, 1 tile | 0.09–0.20 | 0.14–0.25 | 0 |
| DR256, 1 tile | 0.09–0.27 | 0.13–0.32 | 0 |
| DR256 × 256 tiles, Q ≥ 64 | 6.5–6.7 | 6.8–7.1 | 0 |

Revalidating the register file costs about 0.04–0.6 µs per query over direct planes. Arm B (bind per query) costs 1.2–1.6 µs (DR64) and 3.7–4.3 µs (DR256) per query, 5 allocations and 7 / 26 lookups. At 256 tiles arm B **fails the gate** (above 1,000 µs per query): routing 6,554 ids scans the tile maps one after another, 842,138 lookups per bind, because no global id → tile index exists. That is the cost a binding pays once, and the measured reason the index (or a co-addressed producer) is an owner question.

## Findings

1. **The first missing source-level connection is a layout witness on the canonical SoA**: a statement that its rows are laid out under frame version V with tile digests D. Without it a binding cannot prove that the planes it borrows correspond to the frame; with it (the lab `Soa::layout`), a permuted or foreign layout is refused, and turning that check off gives a wrong answer.
2. **Frame currency is policy; correctness rests on the layout digest.** `MapTable::map` answers old frames by replaying their layout, so an old binding over an unchanged SoA still answers correctly (the lax-frame control). After an extension, the old SoA is refused under the new frame by its digest, not by the frame check.
3. **Map and value lifetimes separate cleanly**, exactly as `ValueDemand` declares; no executor reads `ValueDemand` today.
4. **Routing is the only per-query cost a binding removes.** Execution itself is identical (A vs C differ by the checks).

## Not shown

Remote or asynchronous execution, network or RDMA zero-copy, Lance I/O, source leases, cross-call result registers (the derived register here lives inside one in-process object; giving it a lifetime across calls is an owner decision). Decision memos: `.claude/plans/d-drisa-decision-memos-v1.md`.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,11 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

573 entries, 2026-08-06 .. 2026-11-04.
574 entries, 2026-08-06 .. 2026-11-04.

| date | entry id | finding | file |
|---|---|---|---|
| 2026-11-04 | `D-DRISA-1` | | [2026-11-04-d-drisa-1-register-proof.md](2026-11-04-d-drisa-1-register-proof.md) |
| 2026-11-04 | `D-DRISA-0` | | [2026-11-04-d-drisa-0-remote-register-contract.md](2026-11-04-d-drisa-0-remote-register-contract.md) |
| 2026-11-03 | `D-XLOG-META-WIRE-P0` | | [2026-11-03-d-xlog-meta-wire-p0.md](2026-11-03-d-xlog-meta-wire-p0.md) |
| 2026-11-03 | `D-GRAMMAR-WHEN-1` | | [2026-11-03-d-grammar-when-1.md](2026-11-03-d-grammar-when-1.md) |
Expand Down
83 changes: 83 additions & 0 deletions .claude/plans/d-drisa-decision-memos-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# D-DRISA decision memos: the owner questions the local register proof leaves open

> **Status:** PROPOSED. Written by this session after D-DRISA-1 (local register proof) and D-DRISA-3 (ALU semantics). Each memo states the question, the source it rests on, a proposed ruling, the falsifier that would decide it, and what follows from approval or refusal. Nothing here is a decision; each needs the named owner. Architecture: #1953 (D-DRISA-0, a plan by a GPT session, adopted after review). Tracking: EXO-153; handoff: EXO-50.

## D-DRISA-M1: a layout witness on the canonical SoA

**Question.** May the canonical SoA state which DR frame version and tile digests its rows are laid out under, so a register binding can verify the planes it borrows?

**Source.** `MapTable` owns layout (`dr.rs:805`); `TileMap::digest` (`:780`); nothing on the SoA side states the layout (D-DRISA-1 census). The lab `Soa::layout` carries `(TileId, digest)` per tile.

**Proposed ruling.** The SoA owner records `(MapTable identity, RouteVersion, per-tile digest)` when it lays rows out, read-only to consumers. No new map, no copy of ids.

**Falsifier.** D-DRISA-1 `a_permuted_coordinate_assignment_is_refused_and_would_be_wrong`: same ids, another local order. Refused with the witness; wrong answer without it.

**Approve:** register binding becomes verifiable without trusting the caller. **Refuse:** every binding must trust that planes and frame agree; a permuted layout answers wrong silently.

## D-DRISA-M2: a global id → tile index (EXO-46)

**Question.** Who owns routing a set of global ids to tiles? `MapTable::locate` reads every tile per id.

**Source.** `dr.rs:955`. D-DRISA-1 arm B at 256 tiles: 842,138 lookups per bind, above 1,000 µs per query (failed gate).

**Proposed ruling.** Either (a) an index owned by `MapTable`, versioned with it, or (b) no index: callers obtain selections co-addressed (the #1948 C′ form) so routing never happens per query. Not a new hash table outside DR.

**Falsifier.** The same arm B with the chosen mechanism: lookups per id ≤ 1 and bind under 1,000 µs at 256 tiles, with the D-DRISA-1 oracle unchanged.

**Approve (a):** binding cost falls to O(ids). **Approve (b):** no new structure; selections must arrive co-addressed. **Refuse both:** register files must be bound once and reused, never per query.

## D-DRISA-M3: result-register lifetime across calls

**Question.** May a derived mask (the D-DRISA-1 derived predicate register) outlive one call?

**Source.** In-program scratch is epoch-safe (`ExecError::ScratchReadBeforeWrite`). The derived register in the lab lives in one in-process object, stamped with the value revision; nothing owns it across calls.

**Proposed ruling.** A bounded result register keyed by `(frame issue, layout digests, scope, value revision)`, evicted without changing truth, refusing on any mismatch. No global mask store.

**Falsifier.** D-DRISA-1 `a_value_write_invalidates_values_but_not_the_map` and `disabling_the_value_check_returns_a_wrong_derived_answer`, run across two calls; plus #1953 N9 (reuse on another frame/scope/source must refuse).

**Approve:** repeated value predicates need not be recomputed. **Refuse:** results are scalars or egress masks only, recomputed per call.

## D-DRISA-M4: source-location receipt (EXO-46 / EXO-89 / EXO-151)

**Question.** Who hosts a receipt of resolved canonical locations?

**Source.** #1951 census: no production lease host; `drf-focus-probe::FocusKey` and `lance-graph-nsl::FocusLease` are excluded lab crates. D-DRISA-1 needed no receipt: within one process the SoA is borrowed directly.

**Proposed ruling.** Defer. D-DRISA-1 shows a local binding needs a layout witness (M1), not a location receipt. A receipt matters only once the source is not directly borrowable (Lance, remote).

**Falsifier.** #1953 R8–R16 once a host exists.

## D-DRISA-M5: DR → Lance row authority (D4 / S-INGRESS)

**Question.** Unchanged from #1950 and #1951: exactly one attested DR ordinal → `_rowid` seam, with ingress counted.

**Source.** #1950 `where_lance_arm` (`BLOCKED_D4`); ingress measured as a copy (0.4–5.4 µs for one range, 123,162 µs for 3.26M ranges).

**Proposed ruling.** No change by this work. D-DRISA-1 adds one input: the layout witness (M1) is the same kind of statement on the Lance side (a frame → fragment/row binding), so M1 and D4 should be designed together.

## D-DRISA-M6: the Abakus ALU as an instruction family

**Question.** Should the shipped carry words become the ALU for threshold, Kleene or Bag instructions?

**Source.** D-DRISA-3: correct on every case (19/19 tests, 7 disables red), but 1.5–4.8× slower than native words on the same facade in all eight measured shapes, including 3-of-5 against its Boolean expansion and Kleene against two byte counters. P3-5 (#1954): 1.7–4.0× slower for plain k-of-n.

**Proposed ruling.** Refuse as the default ALU on this evidence. Keep the semantics (the tests and the oracle) as the reference; lower k-of-n, Kleene and Bag thresholds to `U8x64` byte counters. Revisit only with a wide carry word (ndarray owner) or a shape byte counters cannot hold (counts above 255).

**Falsifier.** A carry implementation that beats `U8x64` on any of the eight shapes, with the D-DRISA-3 oracle unchanged.

## D-DRISA-M7: asynchronous loopback (stage 2)

**Question.** May stage 2 (in-process queue, deadline, cancel) proceed as a lab?

**Source.** #1953 charter PR2. D-DRISA-1 gives the descriptor it would carry: frame issue, layout digests, scope, value revision, compiled method.

**Proposed ruling.** Yes, as a lab, after review of D-DRISA-1. A queued descriptor must re-run the same revalidation at execution, so a source change between admit and run is refused by the existing checks (N1, N4, N12).

**Falsifier.** #1953 N1–N9 against the D-DRISA-1 refusals.

## Checklist

- [x] D-DRISA-1 local register proof (`entries/2026-11-04-d-drisa-1-register-proof.md`)
- [ ] D-DRISA-3 ALU semantics (stacked PR)
- [ ] Owner rulings M1–M7
5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,11 @@ exclude = [
# lance-graph-dt64 and the contract only. Verify via
# `cargo test --manifest-path crates/dt64-admission-probe/Cargo.toml`.
"crates/dt64-admission-probe",
# D-DRISA-1 (EXO-153) LAB: a sealed DR frame read as a register file over
# borrowed SoA planes. Own workspace root; path-deps dt64, mask-risc and
# the contract only. Verify via
# `cargo test --manifest-path crates/drisa-register-probe/Cargo.toml`.
"crates/drisa-register-probe",
]
resolver = "2"

Expand Down
21 changes: 21 additions & 0 deletions crates/drisa-register-probe/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# D-DRISA-1 (EXO-153) LAB: a witnessed DR frame read as a logical register
# file over borrowed canonical SoA planes. NOT a workspace member, nothing
# depends on it, no public contract. Verify with
# cargo test --manifest-path crates/drisa-register-probe/Cargo.toml
# cargo run --release --manifest-path crates/drisa-register-probe/Cargo.toml --example drisa_bench
[package]
name = "drisa-register-probe"
version = "0.0.0"
edition = "2021"
publish = false
license = "MIT OR Apache-2.0"
description = "D-DRISA-1 lab: does a sealed DR frame bind typed register operands to borrowed SoA planes, reused across operations without rerouting, copying payload, or answering stale?"

# Own workspace root, like the other lab probes.
[workspace]

[dependencies]
# Consumed read-only through their public APIs; none is changed.
lance-graph-dt64 = { path = "../lance-graph-dt64" }
lance-graph-mask-risc = { path = "../lance-graph-mask-risc" }
lance-graph-contract = { path = "../lance-graph-contract" }
Loading
Loading