Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .claude/board/AGENT_LOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
## 2026-11-03 — D-DRAM-1 census (Haiku harvest, Opus verify)

- Two `harvest-scout` runs: ndarray `src/benches/examples/tests` for the "5×24" spellings, `half_add_mask` / `full_add_mask` / `sum8_u8_add_masked` and `Abacus`; lance-graph `crates/` and `.claude/plans/` for focus/lease/seal/map/locator symbols.
- Checks on the main thread, all held: `half_add_mask` at `simd_masking_ops.rs:6523`, `full_add_mask` at `:6545`, the re-export at `simd.rs:1062`, `sum8_u8_add_masked` at `simd_scalar.rs:550`, `U24x5` at `examples/microtile/tile.rs:60` (plus its "lab fixture only" line); `FocusKey`/`SourceId`/`check_seal` in `drf-focus-probe`, `FocusLease` in `lance-graph-nsl`, `Focus`/`Guards` in `dt64/dr.rs`; workspace membership read in `Cargo.toml`.
- Misses: the ndarray run ran `carry` over all of `src/` instead of the matched files only and hit its cap (self-reported). The lance-graph run hit its cap on three patterns and listed substring false positives (`allocator`, `release(`, `RelationshipMapping`), all self-reported. Neither card concluded or edited. Retired: none.
- Result: `entries/2026-11-03-d-dram-1-census.md`. Written by the orchestrating main thread only.

## 2026-10-22 — Stage B E1 probe (Opus spec, Sonnet draft)

- One Sonnet drafter from a written spec (no cargo). The main thread replaced its hand-written variable expansion (a second rewriter outside egglog) with an egglog constructor cost, fixed a test that pinned egglog's cost tie-break, withheld the out-of-fold egglog timing pending an operator ruling, compiled, and ran 6 disable runs (all red).
Expand Down
5 changes: 5 additions & 0 deletions .claude/board/INTEGRATION_PLANS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
## 2026-11-03 — D-DRAM-P3: P2 open ends integrated with the D-DRAM-0 falsifier map → `.claude/plans/d-dram-p3-integrated-v1.md`

**Status:** PLAN, owned by this session (operator). **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DRAM-P3-1..8. Credits P2 (#1950) against the pre-registered #1949 gates (L9, L10, C2, C4 measured in the lab; L1/L2/L6/A3/A4 partial). Orders eleven owned items by signal per effort: three need no outside decision (ReadPolicy range count, non-vacuous projection cases, the D-DRAM-1 census). Production and gated items keep their owners; nothing merges automatically.
**Confidence:** plan; every motive is a measured P2 number, no new measurement yet.

## 2026-11-03 — D-DRAM-0: DR address memoization + adaptive Abakus mask lens → `.claude/plans/d-dram-abacus-address-memoization-v1.md`

**Status:** PROPOSED / DOCS ONLY. **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DRAM; [entry](entries/2026-11-03-d-dram-abacus-address-memoization.md); [EXO-50](https://linear.app/exo-red/issue/EXO-50/agent-message-bus-gpt-xgrok-ccsession-id) coordination. Separate focus-scoped, witnessed receipt for already resolved canonical SoA locations, *no copy of values and no mutation of `LocalMap::globals`*. Quack dynamic ClassView/WideFieldMask scope can test shipped ndarray half/full-adder words for k-of-n masks; operator's exact 5×24 symbols require source verification. Existing Lance masked reads stay in their owner, D4/S-INGRESS gate the DR↔Lance binding, and generic payload gather remains prohibited outside the narrow D-DRF-1 §5 exception. Gates D-DRAM-1..5 are proposals, not authorized production tasks.
Expand Down
13 changes: 13 additions & 0 deletions .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,19 @@
| D-DRAM-4 | combined ClassView selective borrowing + Lance read probe | GATED for end-to-end DR→Lance | D4 and S-INGRESS from #1788 |
| D-DRAM-5 | engineering-parity and cost verdict, handoff | Queued | independent falsifiers and owner decision |

**D-DRAM-P3 (2026-11-03): this session owns the integrated list** ([plan](../plans/d-dram-p3-integrated-v1.md)). P2 (#1950) already measured, in the lab, L9, L10, C2 and C4 (with ids given under D4), and partly measured L1/L2/L6/A3/A4.

| D-id | Deliverable | Status | Owner / gate |
| --- | --- | --- | --- |
| D-DRAM-P3-1 | `ReadPolicy` weighs range count (clustered 10% misrouted 3.8–4.6×) | Done in lab: `page_rows` rule; clustered 10–50% now 1.2–7.8× faster, scattered unchanged | this session; lab crate |
| D-DRAM-P3-2 | non-vacuous `project` k4/k16 sparse/mixed cases | Queued | this session; research branch |
| D-DRAM-P3-3 | = D-DRAM-1 read-only census | Done: [entry](entries/2026-11-03-d-dram-1-census.md); only `half_add_mask`/`full_add_mask` shipped, `U24x5`/`Abacus` lab-only, no production lease | this session; report to EXO-46 / ndarray owner |
| D-DRAM-P3-4 | EXISTS early exit per tile (0.16× DuckDB at 10M) | Queued | this session drafts; mask-risc owner reviews |
| D-DRAM-P3-5 | = D-DRAM-3 k-of-n Abakus bench, A1/A2/A5–A7 | Queued, after P3-3 | this session; no new opcode/type |
| D-DRAM-P3-6 | ordered u8-in-u32 compare proposal | Queued | this session proposes; ndarray owner |
| D-DRAM-P3-7 | route once per frame, lab measurement | Queued | this session; production GATED EXO-46 |
| D-DRAM-P3-8 | = D-DRAM-4 end-to-end, decision memo | GATED | D4 + S-INGRESS (#1788) |

Verified: ndarray #402 half/full adder words and #390 byte sum shipped, `Abacus<BITS>` lab-only; DR `LocalMap` is identity-only; H14c/H14d and #1633 selective Lance reads are lab. Unverified: exact operator-mentioned `ndarray::simd::5x24::*` signature. **Borrow/prefetch default; materialized foreign facet only under D-DRF-1 §5 admission.**

## D-DT64-7 — behaviour addressing and registration: Frozen, Learned, Explore over one classid (2026-10-31)
Expand Down
36 changes: 36 additions & 0 deletions .claude/board/entries/2026-11-03-d-dram-1-census.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# D-DRAM-1 (= D-DRAM-P3-3): census of the shipped carry words, the "5×24" kernels, and the DR lease owner

**Status:** VERIFIED-IN-CODE, read-only. ndarray `master` at `b3d7919`, lance-graph `main` at `58e01cfab`. Owner: this session (D-DRAM-P3). Each listed item was read at the cited line. The search space was closed: ndarray `src/`, `benches/`, `examples/`, `tests/`; lance-graph `crates/` and `.claude/plans/`; each searched with named spellings. An item not found is reported as "not found under these spellings", never as absent.

## ndarray: what a k-of-n consumer may use

| Word | Where | Status |
|---|---|---|
| `half_add_mask(a, carry) -> (u64, u64)` | `src/simd_masking_ops.rs:6523`, re-exported `src/simd.rs:1062` | **Shipped, public** `const fn`: 64 independent lanes, carry to the next significance column in the same lane |
| `full_add_mask(a, b, c) -> (u64, u64)` | `src/simd_masking_ops.rs:6545`, re-exported `src/simd.rs:1062` | **Shipped, public** `const fn`: 3:2 carry-save compressor on 64 lanes |
| `U64x8::sum8_u8_add_masked(self, a: U8x64, keep: u64)` | per backend, e.g. `src/simd_scalar.rs:550` (read), plus avx512, avx2, avx, neon, wasm, nightly (harvested) | **Shipped**: masked vertical byte sums (#390), a different operation from bit-sliced counting |
| `U24x5` | `examples/microtile/tile.rs:60` | **Lab fixture only**: five 24-bit fields plus 8 metadata bits per 128-bit microtile. `tile.rs:23` says: "Lab fixture only: no public type, no production caller." |
| `Abacus`, `Abacus<BITS>`, `AccAbacus` | `examples/u9_abacus/abacus.rs:60`, `examples/u9_domino/domino.rs:316` | **Lab fixture only**. `src/` holds one comment (`simd_masking_ops.rs:6504`) and no type. |

The "5×24" spellings searched over `src/` (`5x24`, `5×24`, `5_24`, `24x5`, `x24`) matched only hex literals. **The only 5×24-like name is the `U24x5` microtile packing in `examples/`**, so #1949's UNVERIFIED mark stands, now with a reason: no shipped 5×24 kernel family was found under these spellings.

**Consequence for D-DRAM-3 / P3-5:** the k-of-n bench uses only `half_add_mask` and `full_add_mask`. Importing `U24x5` or `Abacus` would hit #1949's stop condition "unverified 5×24 API claimed as shipped". Promoting either to a public type is an ndarray-owner decision and needs a real consumer.

## lance-graph: who owns the DR map, and where a lease exists

| Item | Where | Crate status |
|---|---|---|
| `MapTable` (versioned coordinate table), `LocalMap` (local → global identity), `Frame`, `FrameStore` | `crates/lance-graph-dt64/src/dr.rs:805`, `:556`, `:469`, `:379` | workspace **member** (`Cargo.toml:23`): the DR map owner |
| `Guards { frame, focus, scope }`, `Guards::check` | `dr.rs:509`, `:526` | member. Here `Focus` is a tile *revision*: "Which tile of the frame is read" (`dr.rs:493-498`). It is not a lease. |
| `FocusKey { source: SourceId { dataset, branch, version, index_seal }, map, scope, facet }`, `check_seal` | `crates/drf-focus-probe/src/lib.rs:62`, `:24-30`, `:132` | **excluded lab** (`Cargo.toml` exclude). Its `Focus` gathers one `i32` facet column, the D-DRF-1 §5 derived cache, not an address receipt (`lib.rs:1-4`, `:143-160`). |
| `FocusLease` (one feature table per source; `StaleSeal` refusal) | `crates/lance-graph-nsl/src/lib.rs:380-398` | **excluded lab** (`Cargo.toml` exclude) |
| `SealedFacetLane` (Lance-side lane witness used by #1633) | `crates/lance-graph-contract/src/ordered_lane.rs:180` | member (contract) |

**Consequence for D-DRAM-2 / P3-9:** under these spellings, every focus or lease candidate outside dt64 is in an excluded lab crate. dt64's `Focus` is a revision, not a lease. So #1949's "existing focus lease to host disposable locator metadata" has no production host today.

- The nearest key shape is `drf-focus-probe`'s `FocusKey`: full source witness, map, scope and facet. That matches the receipt key #1949 §2.1 sketches.
- Its value, however, is a gathered column, not a locator.

A receipt therefore needs a production owner. D-DRAM-2 stays **GATED on EXO-46 / EXO-151**, and this census is the input to that decision.

**Not done:** the harvest capped on `carry`, on the full `MapTable`/`Frame` hit list and on the plan-reference list. The items above were read; the uncapped tails were not enumerated.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

569 entries, 2026-08-06 .. 2026-11-03.
570 entries, 2026-08-06 .. 2026-11-03.

| date | entry id | finding | file |
|---|---|---|---|
Expand All @@ -35,6 +35,7 @@ exactly one of them, never both.
| 2026-11-03 | `D-GLOBAL-WHERE-P2` | | [2026-11-03-d-global-where-p2.md](2026-11-03-d-global-where-p2.md) |
| 2026-11-03 | `D-GLOBAL-WHERE` | | [2026-11-03-d-global-where-p0.md](2026-11-03-d-global-where-p0.md) |
| 2026-11-03 | `D-DRAM-0` | | [2026-11-03-d-dram-abacus-address-memoization.md](2026-11-03-d-dram-abacus-address-memoization.md) |
| 2026-11-03 | `D-DRAM-1` | | [2026-11-03-d-dram-1-census.md](2026-11-03-d-dram-1-census.md) |
| 2026-11-03 | `D-DR-FOLD-MAP` | | [2026-11-03-d-dr-fold-map.md](2026-11-03-d-dr-fold-map.md) |
| 2026-11-03 | `D-COGNITIVE-POLYFILL` | | [2026-11-03-d-cognitive-polyfill-p0.md](2026-11-03-d-cognitive-polyfill-p0.md) |
| 2026-11-03 | `D-BAG-FOLD` | | [2026-11-03-d-bag-fold-witnessed-existential.md](2026-11-03-d-bag-fold-witnessed-existential.md) |
Expand Down
87 changes: 87 additions & 0 deletions .claude/plans/d-dram-p3-integrated-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# D-DRAM-P3: D-GLOBAL-WHERE P2 open ends integrated with the D-DRAM-0 falsifier map

> **Status: PLAN, owned by this session (operator, 2026-11-03: "integrate 1949
> ideas … and take ownership").** It merges the open ends of D-GLOBAL-WHERE P2
> (#1950) into the D-DRAM-0 plan (#1949,
> `.claude/plans/d-dram-abacus-address-memoization-v1.md`). Ownership means:
> - this session executes the lab and read-only items;
> - it prepares production changes as draft PRs, for owner review;
> - it drives each gated item to the named decision with measured inputs.
>
> Ownership does not lift any gate: EXO-46, EXO-89, EXO-151, D4 and S-INGRESS
> stay with their owners. Nothing here is authorized to merge automatically.

## 1. What P2 already measured, against #1949's gates

P2 measured some #1949 gates before #1949 was written. Each gate is credited
at the strength it was measured, no higher.

| #1949 gate | What P2 measured | Status |
|---|---|---|
| L9 (full / ALLOW / BLOCK / attested range; 0.1, 1, 10, 50%; scattered and clustered) | Every cell, 1M and 10M, plus k = 1/4/16. Mask ingress and bytes accounted separately. `BLOCKED_D4` retained. | **MEASURED-LAB**, with ids given (D4) |
| L10 (1M and 10M; several DR256 relations; identity and Bag/Boolean oracles; no dense matrix, no new paging) | `compose`: DR composition beyond 64K at 1M and 10M against an independent oracle; no dense adjacency. | **MEASURED-LAB**; Bag multiplicity not separately varied |
| L1 / L2 (locator reuse, miss → hit) | Only the *route binding* is reused, not a locator receipt (C1 correction): break-even at Q ≥ 2; 158,198 → 1,012 µs per query at Q = 256. | **PARTIAL**: binding reuse measured; receipts not built (D-DRAM-2 GATED) |
| L6 (stale / foreign refusal) | The stale-binding guard refuses after `replace_tile`; with the guard disabled, the stale binding is accepted. The Lance lane is refused for a stale version, another dataset, and a forged witness. | **PARTIAL**: no scope or palette cases |
| A3 (FieldMask 64 vs WideFieldMask 256; 32-lane mapping; noncontiguous fields) | `project`: 64 logical properties over 32 lanes, U/P/S layouts. The Lance arm resolves ClassView fields through `predicate_iri`; the ordinal and label traps fire. | **PARTIAL**: k4/k16 sparse and mixed AND cases have oracle count 0 (vacuous) |
| A4 (packed 4×8, only after codec attestation) | C2: there is **no** ordered u8-in-u32 compare; composing existing words costs one pass per 2-bit prefix (up to 5× slower than a hand loop). | **NO-EXISTING-KERNEL**, recorded; codec not attested |
| C2 (beyond 64K, independent oracle) | `compose`, as for L10. | **MEASURED-LAB** |
| C3 (DuckDB engineering lessons) | `ENGINEERING-PARITY.md`: literature-first matrix; WHEN vs WHETHER. | **DONE** (reference) |
| C4 (`BLOCKED_D4`, `S_INGRESS_COPY`) | Arm C reports `BLOCKED_D4`; ingress is measured as a copy (0.4–5.4 µs for one range, 123,162 µs for 3.26M ranges). | **DONE**; adopt the exact label `S_INGRESS_COPY` in the arm's output |

Not touched by P2: L3–L5, L7, L8, A1, A2, A5–A7, C1.

## 2. Owned work items

Ordered by signal per unit of effort. **Owner: this session** for every row.
The gate column names who must decide before anything outside the lab
changes.

| ID | Item | Gates it serves | Measured motive | Effort | Gate / where it lands |
|---|---|---|---|---|---|
| P3-1 | `ReadPolicy` weighs range count, not only coverage | L9 | Clustered 10% range sent to a full scan: 3.8× (1M), 4.6× (10M) slower | low | lab crate `crepe-plan-probe`; none |
| P3-2 | Make the `project` k4/k16 sparse and mixed AND cases non-vacuous | A3 | Oracle count 0 cannot catch a wrong answer | low | research branch; none |
| P3-3 | **D-DRAM-1 census** (read-only): the exact `ndarray::simd` 5×24 symbols and signatures, the existing DR focus/locator lease, and the one DR map owner | prerequisite for A1–A7 and L1–L8 | #1949 marks the 5×24 API UNVERIFIED; building on it unverified is a STOP condition | low | read-only; report to EXO-46 / ndarray owner |
| P3-4 | EXISTS stops at the first match per tile, plus a can-stay-silent test on an empty mask | C3 | The only DuckDB loss: 0.16× / 0.43× at 10M | low | **production mask-risc**: draft PR, mask-risc owner reviews |
| P3-5 | **D-DRAM-3**: k-of-n threshold on the shipped `half_add_mask` / `full_add_mask` vs native AND/OR/COUNT, gates A1, A2, A5, A6, A7 | A-family | No k-of-n measurement exists; P2's WHERE shapes are AND/OR/NOT only | medium | lab; after P3-3; no new opcode or public type |
| P3-6 | Ordered u8-in-u32 compare word | A4, A7 | P2 C2 measured the gap | medium | ndarray owner; proposal only, after P3-3 and a codec attestation |
| P3-7 | Route once per frame in the composition route stage | L1, L2, C1 | Routing dominates `compose` (35,914 of 50,178 µs) | medium | lab measurement now; a production binding is GATED on EXO-46 |
| P3-8 | **D-DRAM-4** end-to-end DR → Lance: ids from DR instead of given | L9, C1, C4 | The integrated path is still unmeasured | — | GATED on D4 + S-INGRESS (#1788); this session prepares the decision memo with the measured costs |
| P3-9 | **D-DRAM-2** locator receipts (L1–L8 proper) | L-family | — | — | GATED on EXO-46, EXO-89, EXO-151 |
| P3-10 | Evaluation direction from presence density | C3 | P0 W4 wrong-direction arms; #1945 | medium | Quack owner |
| P3-11 | Bind-time folding of tautologies and contradictions | C3 | `adv_contradictory_17` at PARITY | low–medium | Quack owner |

## 3. Execution order

1. **P3-1, P3-2, P3-3** first: none needs anyone else's decision. P3-3 runs
as read-only harvest (Haiku cards, orchestrator verifies every hit).
2. **P3-4** as a draft PR on its own branch, once P3-1 is in.
3. **P3-5** after P3-3 has pinned the exact symbols. If the census finds no
shipped 5×24 family, P3-5 uses only `half_add_mask` / `full_add_mask` and
says so.
4. **P3-7** lab measurement; **P3-6** proposal.
5. **P3-8** decision memo to the D4 / S-INGRESS owner.

## 4. Stop conditions (from #1949 §7, kept verbatim in force)

Stop on any of:
- a new global → row-id store;
- a fresh SIMD backend in Quack;
- changed field-presence meaning;
- payload duplication by default;
- a locator outliving its lease;
- an unverified 5×24 API claimed as shipped;
- an unauthorized Gather;
- a new ClassId or F###;
- a production patch merged without its owner.

## 5. Checklist

- [x] P3-1 `ReadPolicy` range count, with a can-fire and a can-stay-silent test (disable runs: 2 and 6 tests red)
- [ ] P3-2 non-vacuous `project` cases
- [x] P3-3 D-DRAM-1 census written (`entries/2026-11-03-d-dram-1-census.md`)
- [ ] P3-4 EXISTS early exit, draft PR
- [ ] P3-5 D-DRAM-3 k-of-n bench (A1, A2, A5–A7)
- [ ] P3-6 ordered compare proposal
- [ ] P3-7 route-once lab measurement
- [ ] P3-8 D4 / S-INGRESS decision memo
- [ ] `S_INGRESS_COPY` label in the Lance arm output (C4)
1 change: 1 addition & 0 deletions crates/crepe-plan-probe/examples/addressed_read.rs
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ async fn main() {
let p = ReadPolicy {
coalesce_gap: g,
full_permille: u32::MAX,
page_rows: 0,
};
let (t, ..) = run(&rules, &d, &st, &lanes, p, k64).await;
print!(" {:>8.3}", ms(t));
Expand Down
Loading
Loading