D-RFT-1 follow-up: compose reads the DR256 register (Distinct = Page::roles, Bag = bit product on the support)

.claude/board/STATUS_BOARD.md

@@ -6,7 +6,7 @@ Plans in #1667 (`d-rft-relalg-thought-git-frontier-v1.md`, `d-qsp-classview-quac
 |---|---|---|---|
 | **D-RFT-0 / D-QSP-0** | K3 normalization safety: rewrites keep SQL three-valued TRUE rows | Shipped (#1668; EXO-50 claim `session_01G22yT6htkcdyXsihxxXdrg/031`; `entries/2026-10-21-d-rft-0-kleene-normalization.md`): `normalize_kleene`; `SetNormalize` admits it under `ThreeValued`. MEASURED: PostgreSQL 16.14 = Kleene oracle = `normalize_kleene` on 2000/2000; the two-valued form is wrong on 474 | Kleene evaluator over random expressions + PostgreSQL differential; 5 disables red. Remainder open: NULL literals, `COALESCE`, nullable arithmetic, empty `SUM`, `COUNT(expr)` |
 | **D-QSP-0 remainder, slices 1–2** | slice 1: scalar aggregates under SQL NULL (empty / all-NULL `SUM` is NULL, `COUNT(x)` vs `COUNT(*)`, MIN/MAX/AVG); slice 2: NULL operands (`x = NULL`, `IN`/`NOT IN` with NULL, K3 AND/OR/NOT over TRUE/FALSE/UNKNOWN) | Shipped: slice 1 #1675 (`a55fc67`), board correction #1680 (`b306516`), slice 2 #1682 (`da9b755`; CI 18/18 green on head `96e8010`) (EXO-50 claims `session_013z2e1YXkm6o6HNrDn5BPWA/026`, `/031`; `entries/2026-10-21-d-qsp-0-scalar-aggregates-null.md`). Slice 1: Quack `lower_sum` + `sum_finish` over existing terminals. Slice 2: no `Filter` change; a NULL operand is a lane alias with an all-clear validity plane under `sql_where`. MEASURED (lab, not CI): PostgreSQL 16.15 = Quack on 3600/3600 aggregate queries and on the TRUE and FALSE planes of 2000 × 200 NULL-operand expressions. Not shipped: column-to-column comparison (G4), `COALESCE`/`NULLIF`, nullable arithmetic, subquery `NOT IN` | `tests/sql_aggregate_null.rs` (4 disables red); `tests/sql_null_operand.rs` (11 tests, 4 disables red); `examples/qsp_agg_pg_parity.rs`, `examples/qsp_null_operand_pg_parity.rs` (both SKIP without a server; not run in CI) |
-| **D-RFT-1** | typed composition: membership × role carriers over one slot coordinate; Distinct/Bag from the cardinality witness; NULL key matches nothing; XOR, seal mismatch, stale binding, bag overflow refused | Shipped, PROBE (#1681 `0f756b98`; EXO-50 claim `session_01G22yT6htkcdyXsihxxXdrg/036`; `crates/dir-rmap-probe/src/compose.rs`; `entries/2026-10-21-d-rft-1-typed-composition.md`). r2il polyfill: `IntXor`, `ANY`, `ALL` + procedures. MEASURED: PostgreSQL 16.14 lane equal over 600 users × 64 roles, bag 1421 = `count(*)`, distinct 1374 = `count(DISTINCT)`; composition 310 µs vs the same plan in direct SIMD 14 µs. Open correction #1686: on main `bool_or`/`bool_and` read NULL as FALSE and `count_distinct_sorted` counts NULL keys (review `GPT:gpt-ada-01/…-consolidated-review-01` item 2) | IAM diamond (Bag 2, Distinct 1); independent DFS oracle lane-equal; 7 compose + 5 r2il disables red |
+| **D-RFT-1** | typed composition: membership × role carriers over one slot coordinate; Distinct/Bag from the cardinality witness; NULL key matches nothing; XOR, seal mismatch, stale binding, bag overflow refused | Shipped, PROBE (#1681 `0f756b98`; follow-up #1685 In PR: reads the register — Distinct = `Page::roles`, Bag = AND + popcount on the support, 12.4 µs for 600 users, dense int8 tile GEMM 57.5 µs; EXO-50 claim `session_01G22yT6htkcdyXsihxxXdrg/036`; `crates/dir-rmap-probe/src/compose.rs`; `entries/2026-10-21-d-rft-1-typed-composition.md`). r2il polyfill: `IntXor`, `ANY`, `ALL` + procedures. MEASURED: PostgreSQL 16.14 lane equal over 600 users × 64 roles, bag 1421 = `count(*)`, distinct 1374 = `count(DISTINCT)`; composition 310 µs vs the same plan in direct SIMD 14 µs. Open correction #1686: on main `bool_or`/`bool_and` read NULL as FALSE and `count_distinct_sorted` counts NULL keys (review `GPT:gpt-ada-01/…-consolidated-review-01` item 2) | IAM diamond (Bag 2, Distinct 1); independent DFS oracle lane-equal; 7 compose + 5 r2il disables red |
 | **D-RFT-2..5, D-LFA-0** | substitution, recursion, frontier, replay; flow census | PROPOSAL (`entries/2026-10-21-d-rft-d-lfa-0-assessment.md`) | per-row falsifiers in the entry; unclaimed |
 | **D-LFA-1 / D-LFA-2** (amended 2026-10-21) | guarded-control verification of a Rubicon flow; integration with the existing drivers | PROPOSAL (plan amendment, `d-lfa-…-v1.md` § Amendment 2026-10-21; map `entries/2026-10-21-coresearch-plans-mapping-foresight.md`) | GKAT equivalence + separate no-commit-without-revision-and-MUL trace check; 3 disables; `GoTo`/`GoBack` refused; unclaimed |
 | **D-GKAT-0** | GKAT fixture probe: G1 factoring, G2 nondeterminism refusal, G5 declared atom bound | Shipped, PROBE (#1676, merged `6d7442c4`; EXO-50 claim `session_01VncKoyG6JkpsoqhqTGTv3f/026`, RESULT `/028`; `crates/gkat-probe`; `entries/2026-10-21-d-gkat-0-probe.md`). TEST-PINNED: Rubicon reference ≡ optimized, both safe; CHECK-authorized flows equivalent and unsafe; ≤ 8 tests (256 atoms) else refused. K3 guards are a leaf plus a validity bit and never read the payload of a NULL, as Quack's `sql_where` (#1679) | 3 flow disables + 15 checker disables red; independent bounded oracle; not a production verifier |

.claude/board/entries/2026-10-21-d-rft-1-typed-composition.md

@@ -1,6 +1,6 @@
 # 2026-10-21 — D-RFT-1-COMPOSE: typed composition over the DR256 slot coordinate (D-RFT-1)
 
-**Status:** PROBE (lab, `crates/dir-rmap-probe`). TEST-PINNED and MEASURED as stated below; no production ABI changes.
+**Status:** PROBE (lab, `crates/dir-rmap-probe`). TEST-PINNED and MEASURED as stated below; no production ABI changes. The execution path was replaced by a read of the register; see the last section.
 **Claim:** EXO-50 `session_01G22yT6htkcdyXsihxxXdrg/036`, status in `/038`.
 **Plan:** `.claude/plans/d-rft-relalg-thought-git-frontier-v1.md` (D-RFT-1); assessment `entries/2026-10-21-d-rft-d-lfa-0-assessment.md`.
 
@@ -109,3 +109,27 @@ Written by `session_013z2e1YXkm6o6HNrDn5BPWA` on the operator's instruction (EXO
 - **Tests.** `bool_aggregates_keep_null_apart_from_false` (the six cases TRUE/NULL, FALSE/NULL, TRUE/FALSE/NULL, NULL/NULL, empty, FALSE/FALSE, each with scrambled NULL payloads); `count_distinct_excludes_null_keys_whatever_their_payload` (first, middle and last runs; all-NULL selection). The existing primitive test's empty-selection expectation changed from `(0, 1)` to NULL. Four disables red: no `valid` in `bool_or`, in `bool_and`'s FALSE fold, in the presence fold, in `count_distinct_sorted`.
 - **PostgreSQL 16.15** (`examples/null_aggs_pg_parity.rs`, lab, SKIP without a server): 300 queries over 60 random tables, values and NULL states equal; 64 NULL bool aggregates, 12 `bool_and` TRUE over a NULL row; the disable arm (validity = every row) differs on 224.
 - **Signature change.** `bool_or`, `bool_and` and `count_distinct_sorted` take new arguments; no caller outside the crate.
+
+## Update — the composition reads the register (follow-up PR)
+
+The first version recomputed, with about 2 600 r2il replays per page, what the DR256 register already holds. That was the cost, not the algebra. It now reads the register:
+
+- **Distinct** is `Page::roles`, the role set the page resolved. Nothing is computed. Under a presence plane (`Members::with_valid`), it is the union of the present slots' role words.
+- **Bag** is one AND + popcount per supported role, over the page's membership row and the register's carrier rows (`Qr::role_slots`), through `ndarray::simd`.
+- `Members` borrows the page; the presence plane is optional, so the 8 KiB copy is gone.
+
+Measured (`examples/rft1_bench.rs`, release, median of 101 runs, 600 users / 3 pages):
+
+| arm | time | allocations per run |
+|---|---|---|
+| A — composition over the register | 12.4 µs | 6 (the answer lanes) |
+| C+ — support by union, then AND + popcount | 14 µs | 0 |
+| G — dense int8 tile product (`gemm_u8_i8`, AMX host), bits unpacked to bytes | 57.5 µs | 9 |
+| C — every coordinate | 147–151 µs | 0 |
+| B — tuple baseline, lab only | 159 µs | 10 |
+
+The dense tile product is slower than the bit product here: unpacking 0/1 bits to bytes costs more than the composition. PostgreSQL 16.14 parity unchanged.
+
+Tests: 11. A random-fixture test checks that the two Distinct routes (`Page::roles`, and the union under a full presence plane) agree page by page. Disables, all red: support ignoring `Page::roles`; presence plane ignored; Bag skipping a supported role; NULL-key support read from the raw row (the last one needed a new fixture, where the nulled key is the only path to a role).
+
+The "per-execution cost" and "no count-product terminal" open points above no longer apply to this composition: it runs no mask-risc execution.

crates/dir-rmap-probe/Cargo.toml

@@ -26,7 +26,7 @@ ogar-dir-sim = { git = "https://github.com/AdaWorldAPI/OGAR", branch = "main" }
 # The mask facade (`mask_or_assign`, `eq_u8_to_mask`, popcount), the same
 # kernels #1634's walk_mask uses. Same coordinate as mask-risc, so the graph
 # keeps one ndarray package.
-ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master", default-features = false, features = ["std"] }
+ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master", default-features = false, features = ["std", "runtime-dispatch"] }
 # Arm D, the independent oracle: crepe's generated Datalog evaluator over the
 # fixture's own integer ids (never dir-sim ordinals). AdaWorldAPI fork, by
 # branch (no pins), as in crepe-plan-probe.

crates/dir-rmap-probe/examples/rft1_bench.rs

@@ -2,18 +2,21 @@
 //! through direct security groups and role carriers) three ways, over all
 //! pages of one fixture.
 //!
-//! * A: `Composer::compose` (the typed composition: r2il procedures,
-//!   compiled once, replayed through mask-risc; support first, so only
-//!   coordinates that can hold a path are counted);
+//! * A: `Composer::compose` (the typed composition over the register:
+//!   Distinct is the page's resolved roles, Bag is one AND + popcount per
+//!   supported role, through `ndarray::simd`);
 //! * C: every coordinate's AND + popcount with `ndarray::simd` primitives on
 //!   the 4-word rows, no support pass (the brute-force SIMD reference);
-//! * C+: arm A's plan (support first) written directly with
-//!   `ndarray::simd`: the gap A − C+ is mask-risc's per-execution cost;
+//! * C+: support first (the user's present slots' role words OR'd, then
+//!   one AND + popcount per supported role), directly with `ndarray::simd`;
+//! * G: the dense int8 tile product, `ndarray::simd::gemm_u8_i8`, per
+//!   page (users × 256 slots as 0/1 bytes, times 256 slots × 64 roles); the
+//!   bits are unpacked to bytes first, and that is counted;
 //! * B: LAB-ONLY tuple baseline: every path materialized as a
 //!   `(user, role, group)` tuple, sorted, runs counted. It exists only to
 //!   show what leaving the fold costs; it is not a candidate.
 //!
-//! Bind, page preparation, program preparation (`Composer::new`) and
+//! Bind, page preparation, buffer preparation (`Composer::new`) and
 //! execution are timed separately (median of 101 runs, in ns). All three arms must agree on bag and distinct before any timing.
 //! Allocations are counted by a wrapping global allocator.
 //!
@@ -27,7 +30,7 @@ use dir_rmap_probe::fixture::{Fixture, Spec};
 use dir_rmap_probe::focus::SLOTS;
 use dir_rmap_probe::prep::{Prepared, Refusal, ones};
 use dir_rmap_probe::qr::{Page, Qr};
-use ndarray::simd::{mask_and_assign, mask_or_assign, popcount_batch_u64};
+use ndarray::simd::{gemm_u8_i8, mask_and_assign, mask_or_assign, popcount_batch_u64};
 use std::alloc::{GlobalAlloc, Layout, System};
 use std::hint::black_box;
 use std::sync::atomic::{AtomicUsize, Ordering};
@@ -132,14 +135,40 @@ fn main() {
         }
         (bag, distinct)
     };
+    // Arm G: dense int8 tile product per page.
+    let mut rmat = vec![0i8; SLOTS * 64];
+    for (sl, &roles) in qr.role.iter().enumerate() {
+        for r in ones(&[roles]) {
+            rmat[sl * 64 + r] = 1;
+        }
+    }
+    let mut a_tile = vec![0u8; SLOTS * SLOTS];
+    let mut c_tile = vec![0i32; SLOTS * 64];
+    let mut arm_g = || {
+        let (mut bag, mut distinct) = (0u64, 0u64);
+        for (m, &n) in members.iter().zip(&counts) {
+            let rows = n.div_ceil(16) * 16;
+            a_tile[..rows * SLOTS].fill(0);
+            for i in 0..n {
+                for sl in ones(&m.member[i]) {
+                    a_tile[i * SLOTS + sl] = 1;
+                }
+            }
+            gemm_u8_i8(&a_tile, &rmat, &mut c_tile, rows, 64, SLOTS);
+            for &c in &c_tile[..n * 64] {
+                bag += c as u64;
+                distinct += u64::from(c > 0);
+            }
+        }
+        (bag, distinct)
+    };
     // Arm C: direct ndarray::simd on the 4-word rows.
     let arm_c = || {
         let (mut bag, mut distinct) = (0u64, 0u64);
         for (m, &n) in members.iter().zip(&counts) {
             for i in 0..n {
                 for r in 0..64 {
                     let mut buf = m.member[i];
-                    mask_and_assign(&mut buf, &m.valid[i]);
                     mask_and_assign(&mut buf, &carriers.role_slots[r]);
                     let c = popcount_batch_u64(&buf);
                     bag += c;
@@ -154,8 +183,7 @@ fn main() {
         let (mut bag, mut distinct) = (0u64, 0u64);
         for (m, &n) in members.iter().zip(&counts) {
             for i in 0..n {
-                let mut present = m.member[i];
-                mask_and_assign(&mut present, &m.valid[i]);
+                let present = m.member[i];
                 let mut support = [0u64];
                 for s in ones(&present) {
                     mask_or_assign(&mut support, std::slice::from_ref(&qr.role[s]));
@@ -175,8 +203,7 @@ fn main() {
         let mut paths: Vec<(u32, u8, u16)> = Vec::new();
         for (p, (m, &n)) in members.iter().zip(&counts).enumerate() {
             for i in 0..n {
-                let mut present = m.member[i];
-                mask_and_assign(&mut present, &m.valid[i]);
+                let present = m.member[i];
                 for s in ones(&present) {
                     for r in 0..64u8 {
                         if qr.role[s] >> r & 1 == 1 {
@@ -203,6 +230,7 @@ fn main() {
     assert_eq!(a, c, "arm A vs arm C: (bag, distinct)");
     assert_eq!(a, arm_cp(), "arm A vs arm C+: (bag, distinct)");
     assert_eq!(a, t, "arm A vs arm B: (bag, distinct)");
+    assert_eq!(a, arm_g(), "arm A vs arm G: (bag, distinct)");
     println!(
         "users {users}, pages {}, bag {}, distinct {} (all three arms agree)",
         pages.len(),
@@ -220,19 +248,22 @@ fn main() {
     let compile_ns = median_ns(REPS, Composer::new);
     println!("bind     {bind_ns} ns");
     println!("pages    {prep_ns} ns ({} pages)", pages.len());
-    println!("programs {compile_ns} ns (Composer::new)");
+    println!("buffers  {compile_ns} ns (Composer::new)");
 
     let ta = median_ns(REPS, &mut arm_a);
     let tc = median_ns(REPS, arm_c);
     let tcp = median_ns(REPS, arm_cp);
     let tb = median_ns(REPS, arm_b);
+    let tg = median_ns(REPS, &mut arm_g);
     let (aa, ab) = allocs_of(&mut arm_a);
     let (ca, cb) = allocs_of(arm_c);
     let (cpa, cpb) = allocs_of(arm_cp);
     let (ba, bb) = allocs_of(arm_b);
-    println!("A compose (mask-risc)      {ta} ns  allocs {aa} ({ab} B)");
+    let (ga, gb) = allocs_of(&mut arm_g);
+    println!("A compose (register+bits)  {ta} ns  allocs {aa} ({ab} B)");
     println!("C ndarray::simd direct     {tc} ns  allocs {ca} ({cb} B)");
     println!("C+ support-first simd      {tcp} ns  allocs {cpa} ({cpb} B)");
+    println!("G int8 tile gemm          {tg} ns  allocs {ga} ({gb} B)");
     println!("B tuple baseline (lab-only) {tb} ns  allocs {ba} ({bb} B)");
     println!(
         "ratios: B/A {:.2}  A/C {:.2}  A/C+ {:.2}",

crates/dir-rmap-probe/examples/rft1_cost.rs

@@ -24,6 +24,11 @@
 //! ```text
 //! cargo run --release --example rft1_cost
 //! ```
+//!
+//! Since #1685, `Composer::compose` no longer runs these r2il replays: it
+//! reads `Page::roles` and does one AND + popcount per supported role. This
+//! file still measures the replayed plan through its own programs; the
+//! "arm A" timings it prints are of the new `compose`.
 
 use dir_rmap_probe::World;
 use dir_rmap_probe::compose::{Carriers, Composer, Law, Members, add_bag};
@@ -41,6 +46,12 @@ use std::hint::black_box;
 use std::sync::atomic::{AtomicUsize, Ordering};
 use std::time::Instant;
 
+/// Row `i` of the presence plane: the given plane, or `member` itself when
+/// every key is present (`Members::of`).
+fn valid_row(m: &Members<'_>, i: usize) -> [u64; 4] {
+    m.valid.map_or(m.member[i], |v| v[i])
+}
+
 struct Counting;
 static ALLOCS: AtomicUsize = AtomicUsize::new(0);
 
@@ -194,7 +205,7 @@ fn main() {
         for i in 0..n {
             present_calls += 1;
             let mut present = m.member[i];
-            mask_and_assign(&mut present, &m.valid[i]);
+            mask_and_assign(&mut present, &valid_row(m, i));
             let keys: Vec<usize> = ones(&present).collect();
             if keys.is_empty() {
                 continue;
@@ -245,7 +256,7 @@ fn main() {
         run(
             &pa,
             &mut sa,
-            &[&m0.member[i0], &m0.valid[i0], &carriers.role_slots[r0]],
+            &[&m0.member[i0], &valid_row(m0, i0), &carriers.role_slots[r0]],
             SLOTS,
             Out::None,
         )
@@ -259,7 +270,7 @@ fn main() {
         run(
             &pi,
             &mut si,
-            &[&m0.member[i0], &m0.valid[i0]],
+            &[&m0.member[i0], &valid_row(m0, i0)],
             SLOTS,
             Out::Mask(&mut present),
         )
@@ -298,7 +309,7 @@ fn main() {
     // (4) the four-word cost: the same AND, AND, popcount in direct SIMD.
     let simd_ns = per_call_ns(|| {
         let mut buf = m0.member[i0];
-        mask_and_assign(&mut buf, &m0.valid[i0]);
+        mask_and_assign(&mut buf, &valid_row(m0, i0));
         mask_and_assign(&mut buf, &carriers.role_slots[r0]);
         black_box(popcount_batch_u64(&buf));
     });
@@ -314,7 +325,7 @@ fn main() {
         run(
             &pa,
             &mut sa,
-            &[&m0.member[i0], &m0.valid[i0], &carriers.role_slots[r0]],
+            &[&m0.member[i0], &valid_row(m0, i0), &carriers.role_slots[r0]],
             SLOTS,
             Out::None,
         )