From daf15b291e09e884dcda5f46e574013035ed93f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 23:07:28 +0000 Subject: [PATCH 1/3] plan: D-DRAM-P3, P2 open ends integrated with the D-DRAM-0 falsifier map Credits what D-GLOBAL-WHERE P2 (#1950) already measured against the gates pre-registered in #1949: - measured in the lab: L9, L10, C2, C4; - partly measured: L1/L2, L6, A3, A4. Orders eleven owned items by signal per effort. Three of them need no outside decision: ReadPolicy weighing the range count, non-vacuous projection cases, and the D-DRAM-1 census. Production and gated items keep their owners (mask-risc, ndarray, Quack, EXO-46/89/151, D4/S-INGRESS). #1949's stop conditions stay in force. Adds the D-DRAM-P3 rows to STATUS_BOARD, an INTEGRATION_PLANS entry, and the regenerated supersession index. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01XAqKMNZqKXSaeb9g2ypywF --- .claude/board/INTEGRATION_PLANS.md | 5 ++ .claude/board/STATUS_BOARD.md | 13 ++++ .claude/plans/d-dram-p3-integrated-v1.md | 87 ++++++++++++++++++++++++ 3 files changed, 105 insertions(+) create mode 100644 .claude/plans/d-dram-p3-integrated-v1.md diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 4d669202e..72efec009 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,8 @@ +## 2026-11-03 — D-DRAM-P3: P2 open ends integrated with the D-DRAM-0 falsifier map → `.claude/plans/d-dram-p3-integrated-v1.md` + +**Status:** PLAN, owned by this session (operator). **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DRAM-P3-1..8. Credits P2 (#1950) against the pre-registered #1949 gates (L9, L10, C2, C4 measured in the lab; L1/L2/L6/A3/A4 partial). Orders eleven owned items by signal per effort: three need no outside decision (ReadPolicy range count, non-vacuous projection cases, the D-DRAM-1 census). Production and gated items keep their owners; nothing merges automatically. +**Confidence:** plan; every motive is a measured P2 number, no new measurement yet. + ## 2026-11-03 — D-DRAM-0: DR address memoization + adaptive Abakus mask lens → `.claude/plans/d-dram-abacus-address-memoization-v1.md` **Status:** PROPOSED / DOCS ONLY. **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DRAM; [entry](entries/2026-11-03-d-dram-abacus-address-memoization.md); [EXO-50](https://linear.app/exo-red/issue/EXO-50/agent-message-bus-gpt-xgrok-ccsession-id) coordination. Separate focus-scoped, witnessed receipt for already resolved canonical SoA locations, *no copy of values and no mutation of `LocalMap::globals`*. Quack dynamic ClassView/WideFieldMask scope can test shipped ndarray half/full-adder words for k-of-n masks; operator's exact 5×24 symbols require source verification. Existing Lance masked reads stay in their owner, D4/S-INGRESS gate the DR↔Lance binding, and generic payload gather remains prohibited outside the narrow D-DRF-1 §5 exception. Gates D-DRAM-1..5 are proposals, not authorized production tasks. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 45ccfa884..95b09c021 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -11,6 +11,19 @@ | D-DRAM-4 | combined ClassView selective borrowing + Lance read probe | GATED for end-to-end DR→Lance | D4 and S-INGRESS from #1788 | | D-DRAM-5 | engineering-parity and cost verdict, handoff | Queued | independent falsifiers and owner decision | +**D-DRAM-P3 (2026-11-03): this session owns the integrated list** ([plan](../plans/d-dram-p3-integrated-v1.md)). P2 (#1950) already measured, in the lab, L9, L10, C2 and C4 (with ids given under D4), and partly measured L1/L2/L6/A3/A4. + +| D-id | Deliverable | Status | Owner / gate | +| --- | --- | --- | --- | +| D-DRAM-P3-1 | `ReadPolicy` weighs range count (clustered 10% misrouted 3.8–4.6×) | In progress | this session; lab crate | +| D-DRAM-P3-2 | non-vacuous `project` k4/k16 sparse/mixed cases | Queued | this session; research branch | +| D-DRAM-P3-3 | = D-DRAM-1 read-only census | In progress | this session; report to EXO-46 / ndarray owner | +| D-DRAM-P3-4 | EXISTS early exit per tile (0.16× DuckDB at 10M) | Queued | this session drafts; mask-risc owner reviews | +| D-DRAM-P3-5 | = D-DRAM-3 k-of-n Abakus bench, A1/A2/A5–A7 | Queued, after P3-3 | this session; no new opcode/type | +| D-DRAM-P3-6 | ordered u8-in-u32 compare proposal | Queued | this session proposes; ndarray owner | +| D-DRAM-P3-7 | route once per frame, lab measurement | Queued | this session; production GATED EXO-46 | +| D-DRAM-P3-8 | = D-DRAM-4 end-to-end, decision memo | GATED | D4 + S-INGRESS (#1788) | + Verified: ndarray #402 half/full adder words and #390 byte sum shipped, `Abacus` lab-only; DR `LocalMap` is identity-only; H14c/H14d and #1633 selective Lance reads are lab. Unverified: exact operator-mentioned `ndarray::simd::5x24::*` signature. **Borrow/prefetch default; materialized foreign facet only under D-DRF-1 §5 admission.** ## D-DT64-7 — behaviour addressing and registration: Frozen, Learned, Explore over one classid (2026-10-31) diff --git a/.claude/plans/d-dram-p3-integrated-v1.md b/.claude/plans/d-dram-p3-integrated-v1.md new file mode 100644 index 000000000..da4b82e86 --- /dev/null +++ b/.claude/plans/d-dram-p3-integrated-v1.md @@ -0,0 +1,87 @@ +# D-DRAM-P3: D-GLOBAL-WHERE P2 open ends integrated with the D-DRAM-0 falsifier map + +> **Status: PLAN, owned by this session (operator, 2026-11-03: "integrate 1949 +> ideas … and take ownership").** It merges the open ends of D-GLOBAL-WHERE P2 +> (#1950) into the D-DRAM-0 plan (#1949, +> `.claude/plans/d-dram-abacus-address-memoization-v1.md`). Ownership means: +> - this session executes the lab and read-only items; +> - it prepares production changes as draft PRs, for owner review; +> - it drives each gated item to the named decision with measured inputs. +> +> Ownership does not lift any gate: EXO-46, EXO-89, EXO-151, D4 and S-INGRESS +> stay with their owners. Nothing here is authorized to merge automatically. + +## 1. What P2 already measured, against #1949's gates + +P2 measured some #1949 gates before #1949 was written. Each gate is credited +at the strength it was measured, no higher. + +| #1949 gate | What P2 measured | Status | +|---|---|---| +| L9 (full / ALLOW / BLOCK / attested range; 0.1, 1, 10, 50%; scattered and clustered) | Every cell, 1M and 10M, plus k = 1/4/16. Mask ingress and bytes accounted separately. `BLOCKED_D4` retained. | **MEASURED-LAB**, with ids given (D4) | +| L10 (1M and 10M; several DR256 relations; identity and Bag/Boolean oracles; no dense matrix, no new paging) | `compose`: DR composition beyond 64K at 1M and 10M against an independent oracle; no dense adjacency. | **MEASURED-LAB**; Bag multiplicity not separately varied | +| L1 / L2 (locator reuse, miss → hit) | Only the *route binding* is reused, not a locator receipt (C1 correction): break-even at Q ≥ 2; 158,198 → 1,012 µs per query at Q = 256. | **PARTIAL**: binding reuse measured; receipts not built (D-DRAM-2 GATED) | +| L6 (stale / foreign refusal) | The stale-binding guard refuses after `replace_tile`; with the guard disabled, the stale binding is accepted. The Lance lane is refused for a stale version, another dataset, and a forged witness. | **PARTIAL**: no scope or palette cases | +| A3 (FieldMask 64 vs WideFieldMask 256; 32-lane mapping; noncontiguous fields) | `project`: 64 logical properties over 32 lanes, U/P/S layouts. The Lance arm resolves ClassView fields through `predicate_iri`; the ordinal and label traps fire. | **PARTIAL**: k4/k16 sparse and mixed AND cases have oracle count 0 (vacuous) | +| A4 (packed 4×8, only after codec attestation) | C2: there is **no** ordered u8-in-u32 compare; composing existing words costs one pass per 2-bit prefix (up to 5× slower than a hand loop). | **NO-EXISTING-KERNEL**, recorded; codec not attested | +| C2 (beyond 64K, independent oracle) | `compose`, as for L10. | **MEASURED-LAB** | +| C3 (DuckDB engineering lessons) | `ENGINEERING-PARITY.md`: literature-first matrix; WHEN vs WHETHER. | **DONE** (reference) | +| C4 (`BLOCKED_D4`, `S_INGRESS_COPY`) | Arm C reports `BLOCKED_D4`; ingress is measured as a copy (0.4–5.4 µs for one range, 123,162 µs for 3.26M ranges). | **DONE**; adopt the exact label `S_INGRESS_COPY` in the arm's output | + +Not touched by P2: L3–L5, L7, L8, A1, A2, A5–A7, C1. + +## 2. Owned work items + +Ordered by signal per unit of effort. **Owner: this session** for every row. +The gate column names who must decide before anything outside the lab +changes. + +| ID | Item | Gates it serves | Measured motive | Effort | Gate / where it lands | +|---|---|---|---|---|---| +| P3-1 | `ReadPolicy` weighs range count, not only coverage | L9 | Clustered 10% range sent to a full scan: 3.8× (1M), 4.6× (10M) slower | low | lab crate `crepe-plan-probe`; none | +| P3-2 | Make the `project` k4/k16 sparse and mixed AND cases non-vacuous | A3 | Oracle count 0 cannot catch a wrong answer | low | research branch; none | +| P3-3 | **D-DRAM-1 census** (read-only): the exact `ndarray::simd` 5×24 symbols and signatures, the existing DR focus/locator lease, and the one DR map owner | prerequisite for A1–A7 and L1–L8 | #1949 marks the 5×24 API UNVERIFIED; building on it unverified is a STOP condition | low | read-only; report to EXO-46 / ndarray owner | +| P3-4 | EXISTS stops at the first match per tile, plus a can-stay-silent test on an empty mask | C3 | The only DuckDB loss: 0.16× / 0.43× at 10M | low | **production mask-risc**: draft PR, mask-risc owner reviews | +| P3-5 | **D-DRAM-3**: k-of-n threshold on the shipped `half_add_mask` / `full_add_mask` vs native AND/OR/COUNT, gates A1, A2, A5, A6, A7 | A-family | No k-of-n measurement exists; P2's WHERE shapes are AND/OR/NOT only | medium | lab; after P3-3; no new opcode or public type | +| P3-6 | Ordered u8-in-u32 compare word | A4, A7 | P2 C2 measured the gap | medium | ndarray owner; proposal only, after P3-3 and a codec attestation | +| P3-7 | Route once per frame in the composition route stage | L1, L2, C1 | Routing dominates `compose` (35,914 of 50,178 µs) | medium | lab measurement now; a production binding is GATED on EXO-46 | +| P3-8 | **D-DRAM-4** end-to-end DR → Lance: ids from DR instead of given | L9, C1, C4 | The integrated path is still unmeasured | — | GATED on D4 + S-INGRESS (#1788); this session prepares the decision memo with the measured costs | +| P3-9 | **D-DRAM-2** locator receipts (L1–L8 proper) | L-family | — | — | GATED on EXO-46, EXO-89, EXO-151 | +| P3-10 | Evaluation direction from presence density | C3 | P0 W4 wrong-direction arms; #1945 | medium | Quack owner | +| P3-11 | Bind-time folding of tautologies and contradictions | C3 | `adv_contradictory_17` at PARITY | low–medium | Quack owner | + +## 3. Execution order + +1. **P3-1, P3-2, P3-3** first: none needs anyone else's decision. P3-3 runs + as read-only harvest (Haiku cards, orchestrator verifies every hit). +2. **P3-4** as a draft PR on its own branch, once P3-1 is in. +3. **P3-5** after P3-3 has pinned the exact symbols. If the census finds no + shipped 5×24 family, P3-5 uses only `half_add_mask` / `full_add_mask` and + says so. +4. **P3-7** lab measurement; **P3-6** proposal. +5. **P3-8** decision memo to the D4 / S-INGRESS owner. + +## 4. Stop conditions (from #1949 §7, kept verbatim in force) + +Stop on any of: +- a new global → row-id store; +- a fresh SIMD backend in Quack; +- changed field-presence meaning; +- payload duplication by default; +- a locator outliving its lease; +- an unverified 5×24 API claimed as shipped; +- an unauthorized Gather; +- a new ClassId or F###; +- a production patch merged without its owner. + +## 5. Checklist + +- [ ] P3-1 `ReadPolicy` range count, with a can-fire and a can-stay-silent test +- [ ] P3-2 non-vacuous `project` cases +- [ ] P3-3 D-DRAM-1 census written +- [ ] P3-4 EXISTS early exit, draft PR +- [ ] P3-5 D-DRAM-3 k-of-n bench (A1, A2, A5–A7) +- [ ] P3-6 ordered compare proposal +- [ ] P3-7 route-once lab measurement +- [ ] P3-8 D4 / S-INGRESS decision memo +- [ ] `S_INGRESS_COPY` label in the Lance arm output (C4) From 6e17d4b0973e18a1f0e26a5fc6f346eaeadbe204 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 23:12:17 +0000 Subject: [PATCH 2/3] lab: ReadPolicy full read also requires spread ranges (D-DRAM-P3-1) plan_read decided on covered rows alone. In D-GLOBAL-WHERE P2 it therefore sent one contiguous 10% range to a full scan, 3.8x (1M) and 4.6x (10M) slower than the range read. A full read now also needs `ranges x page_rows >= n_rows`: the ranges must be spread widely enough to touch about every page. ReadPolicy::DEFAULT uses page_rows = 4,096, a POLICY PIN taken from the #1788 A.3 chunk size; page_rows = 0 restores the old rule. Measured with where_lance_arm at 1M (all oracle and refusal checks pass): - clustered 10% and 50% now read ranges, 1.2-7.8x faster than the old full read (10%, k=16: 103,492 -> 13,292 us); - scattered 10% still reads in full; - scattered 50% coalesces into one range covering nearly the table, at the old full-read time (56,820 vs 54,430 us). Tests (4 new): - a long run keeps the range read; - scattered ranges still read in full; - the page condition fires at the boundary and not one range below; - page_rows = 0 restores the coverage-only rule. Disable runs: with the old coverage-only rule, two tests fail; with `spread = false`, six fail. Full crate: fmt, clippy -D warnings and cargo test (63 tests) all pass. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01XAqKMNZqKXSaeb9g2ypywF --- .../examples/addressed_read.rs | 1 + crates/crepe-plan-probe/src/addressed.rs | 65 ++++++++++++++++++- crates/crepe-plan-probe/tests/pipeline.rs | 5 ++ 3 files changed, 69 insertions(+), 2 deletions(-) diff --git a/crates/crepe-plan-probe/examples/addressed_read.rs b/crates/crepe-plan-probe/examples/addressed_read.rs index 1bbf0e6d8..0ef40ea35 100644 --- a/crates/crepe-plan-probe/examples/addressed_read.rs +++ b/crates/crepe-plan-probe/examples/addressed_read.rs @@ -163,6 +163,7 @@ async fn main() { let p = ReadPolicy { coalesce_gap: g, full_permille: u32::MAX, + page_rows: 0, }; let (t, ..) = run(&rules, &d, &st, &lanes, p, k64).await; print!(" {:>8.3}", ms(t)); diff --git a/crates/crepe-plan-probe/src/addressed.rs b/crates/crepe-plan-probe/src/addressed.rs index c8065efc8..7955f7444 100644 --- a/crates/crepe-plan-probe/src/addressed.rs +++ b/crates/crepe-plan-probe/src/addressed.rs @@ -178,6 +178,12 @@ pub struct ReadPolicy { /// The table is read in full when the (merged) ranges cover at least this /// many rows per 1,000 of the table. Above 1,000, never. pub full_permille: u32, + /// Rows per storage page for the full-read rule. When non-zero, a full + /// read also needs `ranges × page_rows ≥ n_rows`: the ranges must be + /// spread widely enough to touch about every page. One long run touches + /// few pages, so a range read wins even at high coverage. 0 disables the + /// condition, so coverage alone decides. + pub page_rows: u32, } impl ReadPolicy { @@ -185,6 +191,7 @@ impl ReadPolicy { pub const EXACT: Self = Self { coalesce_gap: 0, full_permille: u32::MAX, + page_rows: 0, }; /// The measured switch (`examples/addressed_read.rs`, release, uniform /// demand, policy pins rather than derived values): @@ -194,16 +201,24 @@ impl ReadPolicy { /// the two, either choice is within run-to-run noise; above 10% the /// full read is up to 1.9× faster (50% of centers). /// - A coalescing gap of 16 rows measured neutral on uniform demand. A - /// gap of 64 lost (11.7 vs 6.9 ms at 1% of 200,000 rows). Clustered + /// gap of 64 lost (11,700 vs 6,900 µs at 1% of 200,000 rows). Clustered /// demand, where coalescing should help, is not measured. + /// - `page_rows` = 4,096, the chunk size from the #1788 A.3 analysis + /// (POLICY PIN, not a measured crossover). D-GLOBAL-WHERE P2 measured + /// the bracket it separates. Clustered 10%, one range: the coverage-only + /// rule sent it to a full read 3.8× (1M) and 4.6× (10M) slower than the + /// range read. Scattered 10%, 91,527 ranges in 1M rows: the full read + /// is right. Run lengths between those two are not measured. pub const DEFAULT: Self = Self { coalesce_gap: 16, full_permille: 100, + page_rows: 4_096, }; /// Always the whole table. pub const FULL: Self = Self { coalesce_gap: 0, full_permille: 0, + page_rows: 0, }; } @@ -220,7 +235,10 @@ pub fn plan_read(ranges: &[(u32, u32)], n_rows: u32, p: ReadPolicy) -> (Read, Ve } let span = |r: &[(u32, u32)]| -> u64 { r.iter().map(|&(lo, hi)| u64::from(hi - lo)).sum() }; let covered = span(&merged); - if n_rows > 0 && covered * 1000 >= u64::from(n_rows) * u64::from(p.full_permille) { + let dense = covered * 1000 >= u64::from(n_rows) * u64::from(p.full_permille); + let spread = + p.page_rows == 0 || merged.len() as u64 * u64::from(p.page_rows) >= u64::from(n_rows); + if n_rows > 0 && dense && spread { return (Read::Full, Vec::new()); } // Merging ranges that touch reads no extra row: still exact. @@ -474,9 +492,52 @@ mod tests { ReadPolicy { coalesce_gap, full_permille, + page_rows: 0, } } + fn paged(full_permille: u32, page_rows: u32) -> ReadPolicy { + ReadPolicy { + coalesce_gap: 0, + full_permille, + page_rows, + } + } + + /// The clustered case measured in D-GLOBAL-WHERE P2: one contiguous run + /// covering 30% of the table. Coverage alone sends it to a full read; it + /// touches 1 of 250 pages, so the page condition keeps the range read. + #[test] + fn one_long_run_keeps_the_range_read_despite_high_coverage() { + let r = [(100, 400)]; // 300 of 1,000 rows, one range + assert_eq!(plan_read(&r, 1_000, paged(100, 4)).0, Read::Ranges); + } + + /// The scattered case: as many one-row ranges as rows covered. Every page + /// is touched, so the full read still fires. + #[test] + fn many_scattered_ranges_still_read_in_full() { + let r: Vec<(u32, u32)> = (0..300).map(|i| (3 * i, 3 * i + 1)).collect(); + assert_eq!(r.len() * 4, 1_200, "ranges x page_rows must reach n_rows"); + assert_eq!(plan_read(&r, 1_000, paged(100, 4)).0, Read::Full); + } + + /// The page condition fires exactly at `ranges × page_rows = n_rows`. + #[test] + fn the_page_condition_fires_at_the_boundary_and_not_one_range_below() { + let r250: Vec<(u32, u32)> = (0..250).map(|i| (4 * i, 4 * i + 1)).collect(); + let r249 = &r250[..249]; + assert_eq!(plan_read(&r250, 1_000, paged(100, 4)).0, Read::Full); + assert_eq!(plan_read(r249, 1_000, paged(100, 4)).0, Read::Ranges); + } + + /// `page_rows = 0` disables the condition: the long run reads in full on + /// coverage alone, as before. + #[test] + fn page_rows_zero_restores_the_coverage_only_rule() { + assert_eq!(plan_read(&[(100, 400)], 1_000, paged(100, 0)).0, Read::Full); + } + #[test] fn a_gap_of_exactly_the_limit_merges_and_one_more_does_not() { let r = [(0, 2), (5, 6)]; // gap of 3 rows diff --git a/crates/crepe-plan-probe/tests/pipeline.rs b/crates/crepe-plan-probe/tests/pipeline.rs index 97c190db2..25a6ba15e 100644 --- a/crates/crepe-plan-probe/tests/pipeline.rs +++ b/crates/crepe-plan-probe/tests/pipeline.rs @@ -1634,10 +1634,12 @@ mod arm_b { ReadPolicy { coalesce_gap: 1, full_permille: u32::MAX, + page_rows: 0, }, ReadPolicy { coalesce_gap: u32::MAX, full_permille: u32::MAX, + page_rows: 0, }, ]; for x in [17, 5, 999] { @@ -1689,12 +1691,14 @@ mod arm_b { let (sparse, _, sparse_io, sparse_l) = run(ReadPolicy { coalesce_gap: 0, full_permille: 100, + page_rows: 0, }) .await .unwrap(); let (dense, _, dense_io, dense_l) = run(ReadPolicy { coalesce_gap: 0, full_permille: 1, + page_rows: 0, }) .await .unwrap(); @@ -1751,6 +1755,7 @@ mod arm_b { let (merged, _, _, ml) = run(ReadPolicy { coalesce_gap: u32::MAX, full_permille: u32::MAX, + page_rows: 0, }) .await .unwrap(); From a1ab4b67395fb736cfaed13fd2836ffbca47dbb0 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 3 Nov 2026 23:14:34 +0000 Subject: [PATCH 3/3] board: D-DRAM-1 census (D-DRAM-P3-3); mark P3-1 done Read-only census, verified in code: - ndarray ships only `half_add_mask` / `full_add_mask` (and `sum8_u8_add_masked`) for counting. - The "5x24" kernels are the `U24x5` microtile packing, in examples/ only ("Lab fixture only: no public type"); Abacus is examples/ only. - dt64 `dr.rs` owns the DR map. Every focus/lease candidate outside it (drf-focus-probe FocusKey, lance-graph-nsl FocusLease) is in an excluded lab crate, so a locator receipt has no production host. D-DRAM-2 stays gated on EXO-46/EXO-151. Also: the STATUS_BOARD and plan checklist mark P3-1 and P3-3 done, the two Haiku harvest runs are logged with their misses, and the indexes are regenerated. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01XAqKMNZqKXSaeb9g2ypywF --- .claude/board/AGENT_LOG.md | 7 ++++ .claude/board/STATUS_BOARD.md | 4 +-- .../entries/2026-11-03-d-dram-1-census.md | 36 +++++++++++++++++++ .claude/board/entries/README.md | 3 +- .claude/plans/d-dram-p3-integrated-v1.md | 4 +-- 5 files changed, 49 insertions(+), 5 deletions(-) create mode 100644 .claude/board/entries/2026-11-03-d-dram-1-census.md diff --git a/.claude/board/AGENT_LOG.md b/.claude/board/AGENT_LOG.md index 3e3ca3091..96b24d9bf 100644 --- a/.claude/board/AGENT_LOG.md +++ b/.claude/board/AGENT_LOG.md @@ -1,3 +1,10 @@ +## 2026-11-03 — D-DRAM-1 census (Haiku harvest, Opus verify) + +- Two `harvest-scout` runs: ndarray `src/benches/examples/tests` for the "5×24" spellings, `half_add_mask` / `full_add_mask` / `sum8_u8_add_masked` and `Abacus`; lance-graph `crates/` and `.claude/plans/` for focus/lease/seal/map/locator symbols. +- Checks on the main thread, all held: `half_add_mask` at `simd_masking_ops.rs:6523`, `full_add_mask` at `:6545`, the re-export at `simd.rs:1062`, `sum8_u8_add_masked` at `simd_scalar.rs:550`, `U24x5` at `examples/microtile/tile.rs:60` (plus its "lab fixture only" line); `FocusKey`/`SourceId`/`check_seal` in `drf-focus-probe`, `FocusLease` in `lance-graph-nsl`, `Focus`/`Guards` in `dt64/dr.rs`; workspace membership read in `Cargo.toml`. +- Misses: the ndarray run ran `carry` over all of `src/` instead of the matched files only and hit its cap (self-reported). The lance-graph run hit its cap on three patterns and listed substring false positives (`allocator`, `release(`, `RelationshipMapping`), all self-reported. Neither card concluded or edited. Retired: none. +- Result: `entries/2026-11-03-d-dram-1-census.md`. Written by the orchestrating main thread only. + ## 2026-10-22 — Stage B E1 probe (Opus spec, Sonnet draft) - One Sonnet drafter from a written spec (no cargo). The main thread replaced its hand-written variable expansion (a second rewriter outside egglog) with an egglog constructor cost, fixed a test that pinned egglog's cost tie-break, withheld the out-of-fold egglog timing pending an operator ruling, compiled, and ran 6 disable runs (all red). diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 95b09c021..e27abdb64 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -15,9 +15,9 @@ | D-id | Deliverable | Status | Owner / gate | | --- | --- | --- | --- | -| D-DRAM-P3-1 | `ReadPolicy` weighs range count (clustered 10% misrouted 3.8–4.6×) | In progress | this session; lab crate | +| D-DRAM-P3-1 | `ReadPolicy` weighs range count (clustered 10% misrouted 3.8–4.6×) | Done in lab: `page_rows` rule; clustered 10–50% now 1.2–7.8× faster, scattered unchanged | this session; lab crate | | D-DRAM-P3-2 | non-vacuous `project` k4/k16 sparse/mixed cases | Queued | this session; research branch | -| D-DRAM-P3-3 | = D-DRAM-1 read-only census | In progress | this session; report to EXO-46 / ndarray owner | +| D-DRAM-P3-3 | = D-DRAM-1 read-only census | Done: [entry](entries/2026-11-03-d-dram-1-census.md); only `half_add_mask`/`full_add_mask` shipped, `U24x5`/`Abacus` lab-only, no production lease | this session; report to EXO-46 / ndarray owner | | D-DRAM-P3-4 | EXISTS early exit per tile (0.16× DuckDB at 10M) | Queued | this session drafts; mask-risc owner reviews | | D-DRAM-P3-5 | = D-DRAM-3 k-of-n Abakus bench, A1/A2/A5–A7 | Queued, after P3-3 | this session; no new opcode/type | | D-DRAM-P3-6 | ordered u8-in-u32 compare proposal | Queued | this session proposes; ndarray owner | diff --git a/.claude/board/entries/2026-11-03-d-dram-1-census.md b/.claude/board/entries/2026-11-03-d-dram-1-census.md new file mode 100644 index 000000000..d7012eebd --- /dev/null +++ b/.claude/board/entries/2026-11-03-d-dram-1-census.md @@ -0,0 +1,36 @@ +# D-DRAM-1 (= D-DRAM-P3-3): census of the shipped carry words, the "5×24" kernels, and the DR lease owner + +**Status:** VERIFIED-IN-CODE, read-only. ndarray `master` at `b3d7919`, lance-graph `main` at `58e01cfab`. Owner: this session (D-DRAM-P3). Each listed item was read at the cited line. The search space was closed: ndarray `src/`, `benches/`, `examples/`, `tests/`; lance-graph `crates/` and `.claude/plans/`; each searched with named spellings. An item not found is reported as "not found under these spellings", never as absent. + +## ndarray: what a k-of-n consumer may use + +| Word | Where | Status | +|---|---|---| +| `half_add_mask(a, carry) -> (u64, u64)` | `src/simd_masking_ops.rs:6523`, re-exported `src/simd.rs:1062` | **Shipped, public** `const fn`: 64 independent lanes, carry to the next significance column in the same lane | +| `full_add_mask(a, b, c) -> (u64, u64)` | `src/simd_masking_ops.rs:6545`, re-exported `src/simd.rs:1062` | **Shipped, public** `const fn`: 3:2 carry-save compressor on 64 lanes | +| `U64x8::sum8_u8_add_masked(self, a: U8x64, keep: u64)` | per backend, e.g. `src/simd_scalar.rs:550` (read), plus avx512, avx2, avx, neon, wasm, nightly (harvested) | **Shipped**: masked vertical byte sums (#390), a different operation from bit-sliced counting | +| `U24x5` | `examples/microtile/tile.rs:60` | **Lab fixture only**: five 24-bit fields plus 8 metadata bits per 128-bit microtile. `tile.rs:23` says: "Lab fixture only: no public type, no production caller." | +| `Abacus`, `Abacus`, `AccAbacus` | `examples/u9_abacus/abacus.rs:60`, `examples/u9_domino/domino.rs:316` | **Lab fixture only**. `src/` holds one comment (`simd_masking_ops.rs:6504`) and no type. | + +The "5×24" spellings searched over `src/` (`5x24`, `5×24`, `5_24`, `24x5`, `x24`) matched only hex literals. **The only 5×24-like name is the `U24x5` microtile packing in `examples/`**, so #1949's UNVERIFIED mark stands, now with a reason: no shipped 5×24 kernel family was found under these spellings. + +**Consequence for D-DRAM-3 / P3-5:** the k-of-n bench uses only `half_add_mask` and `full_add_mask`. Importing `U24x5` or `Abacus` would hit #1949's stop condition "unverified 5×24 API claimed as shipped". Promoting either to a public type is an ndarray-owner decision and needs a real consumer. + +## lance-graph: who owns the DR map, and where a lease exists + +| Item | Where | Crate status | +|---|---|---| +| `MapTable` (versioned coordinate table), `LocalMap` (local → global identity), `Frame`, `FrameStore` | `crates/lance-graph-dt64/src/dr.rs:805`, `:556`, `:469`, `:379` | workspace **member** (`Cargo.toml:23`): the DR map owner | +| `Guards { frame, focus, scope }`, `Guards::check` | `dr.rs:509`, `:526` | member. Here `Focus` is a tile *revision*: "Which tile of the frame is read" (`dr.rs:493-498`). It is not a lease. | +| `FocusKey { source: SourceId { dataset, branch, version, index_seal }, map, scope, facet }`, `check_seal` | `crates/drf-focus-probe/src/lib.rs:62`, `:24-30`, `:132` | **excluded lab** (`Cargo.toml` exclude). Its `Focus` gathers one `i32` facet column, the D-DRF-1 §5 derived cache, not an address receipt (`lib.rs:1-4`, `:143-160`). | +| `FocusLease` (one feature table per source; `StaleSeal` refusal) | `crates/lance-graph-nsl/src/lib.rs:380-398` | **excluded lab** (`Cargo.toml` exclude) | +| `SealedFacetLane` (Lance-side lane witness used by #1633) | `crates/lance-graph-contract/src/ordered_lane.rs:180` | member (contract) | + +**Consequence for D-DRAM-2 / P3-9:** under these spellings, every focus or lease candidate outside dt64 is in an excluded lab crate. dt64's `Focus` is a revision, not a lease. So #1949's "existing focus lease to host disposable locator metadata" has no production host today. + +- The nearest key shape is `drf-focus-probe`'s `FocusKey`: full source witness, map, scope and facet. That matches the receipt key #1949 §2.1 sketches. +- Its value, however, is a gathered column, not a locator. + +A receipt therefore needs a production owner. D-DRAM-2 stays **GATED on EXO-46 / EXO-151**, and this census is the input to that decision. + +**Not done:** the harvest capped on `carry`, on the full `MapTable`/`Frame` hit list and on the plan-reference list. The items above were read; the uncapped tails were not enumerated. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 73ec68041..9a1f6ddf5 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -569 entries, 2026-08-06 .. 2026-11-03. +570 entries, 2026-08-06 .. 2026-11-03. | date | entry id | finding | file | |---|---|---|---| @@ -35,6 +35,7 @@ exactly one of them, never both. | 2026-11-03 | `D-GLOBAL-WHERE-P2` | | [2026-11-03-d-global-where-p2.md](2026-11-03-d-global-where-p2.md) | | 2026-11-03 | `D-GLOBAL-WHERE` | | [2026-11-03-d-global-where-p0.md](2026-11-03-d-global-where-p0.md) | | 2026-11-03 | `D-DRAM-0` | | [2026-11-03-d-dram-abacus-address-memoization.md](2026-11-03-d-dram-abacus-address-memoization.md) | +| 2026-11-03 | `D-DRAM-1` | | [2026-11-03-d-dram-1-census.md](2026-11-03-d-dram-1-census.md) | | 2026-11-03 | `D-DR-FOLD-MAP` | | [2026-11-03-d-dr-fold-map.md](2026-11-03-d-dr-fold-map.md) | | 2026-11-03 | `D-COGNITIVE-POLYFILL` | | [2026-11-03-d-cognitive-polyfill-p0.md](2026-11-03-d-cognitive-polyfill-p0.md) | | 2026-11-03 | `D-BAG-FOLD` | | [2026-11-03-d-bag-fold-witnessed-existential.md](2026-11-03-d-bag-fold-witnessed-existential.md) | diff --git a/.claude/plans/d-dram-p3-integrated-v1.md b/.claude/plans/d-dram-p3-integrated-v1.md index da4b82e86..7e31f0e85 100644 --- a/.claude/plans/d-dram-p3-integrated-v1.md +++ b/.claude/plans/d-dram-p3-integrated-v1.md @@ -76,9 +76,9 @@ Stop on any of: ## 5. Checklist -- [ ] P3-1 `ReadPolicy` range count, with a can-fire and a can-stay-silent test +- [x] P3-1 `ReadPolicy` range count, with a can-fire and a can-stay-silent test (disable runs: 2 and 6 tests red) - [ ] P3-2 non-vacuous `project` cases -- [ ] P3-3 D-DRAM-1 census written +- [x] P3-3 D-DRAM-1 census written (`entries/2026-11-03-d-dram-1-census.md`) - [ ] P3-4 EXISTS early exit, draft PR - [ ] P3-5 D-DRAM-3 k-of-n bench (A1, A2, A5–A7) - [ ] P3-6 ordered compare proposal