diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 805f897ba..3618adb6f 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -6,7 +6,7 @@ Plans in #1667 (`d-rft-relalg-thought-git-frontier-v1.md`, `d-qsp-classview-quac |---|---|---|---| | **D-RFT-0 / D-QSP-0** | K3 normalization safety: rewrites keep SQL three-valued TRUE rows | Shipped (#1668; EXO-50 claim `session_01G22yT6htkcdyXsihxxXdrg/031`; `entries/2026-10-21-d-rft-0-kleene-normalization.md`): `normalize_kleene`; `SetNormalize` admits it under `ThreeValued`. MEASURED: PostgreSQL 16.14 = Kleene oracle = `normalize_kleene` on 2000/2000; the two-valued form is wrong on 474 | Kleene evaluator over random expressions + PostgreSQL differential; 5 disables red. Remainder open: NULL literals, `COALESCE`, nullable arithmetic, empty `SUM`, `COUNT(expr)` | | **D-QSP-0 remainder, slices 1–2** | slice 1: scalar aggregates under SQL NULL (empty / all-NULL `SUM` is NULL, `COUNT(x)` vs `COUNT(*)`, MIN/MAX/AVG); slice 2: NULL operands (`x = NULL`, `IN`/`NOT IN` with NULL, K3 AND/OR/NOT over TRUE/FALSE/UNKNOWN) | Shipped: slice 1 #1675 (`a55fc67`), board correction #1680 (`b306516`), slice 2 #1682 (`da9b755`; CI 18/18 green on head `96e8010`) (EXO-50 claims `session_013z2e1YXkm6o6HNrDn5BPWA/026`, `/031`; `entries/2026-10-21-d-qsp-0-scalar-aggregates-null.md`). Slice 1: Quack `lower_sum` + `sum_finish` over existing terminals. Slice 2: no `Filter` change; a NULL operand is a lane alias with an all-clear validity plane under `sql_where`. MEASURED (lab, not CI): PostgreSQL 16.15 = Quack on 3600/3600 aggregate queries and on the TRUE and FALSE planes of 2000 × 200 NULL-operand expressions. Not shipped: column-to-column comparison (G4), `COALESCE`/`NULLIF`, nullable arithmetic, subquery `NOT IN` | `tests/sql_aggregate_null.rs` (4 disables red); `tests/sql_null_operand.rs` (11 tests, 4 disables red); `examples/qsp_agg_pg_parity.rs`, `examples/qsp_null_operand_pg_parity.rs` (both SKIP without a server; not run in CI) | -| **D-RFT-1** | typed composition: membership × role carriers over one slot coordinate; Distinct/Bag from the cardinality witness; NULL key matches nothing; XOR, seal mismatch, stale binding, bag overflow refused | Shipped, PROBE (#1681 `0f756b98`; EXO-50 claim `session_01G22yT6htkcdyXsihxxXdrg/036`; `crates/dir-rmap-probe/src/compose.rs`; `entries/2026-10-21-d-rft-1-typed-composition.md`). r2il polyfill: `IntXor`, `ANY`, `ALL` + procedures. MEASURED: PostgreSQL 16.14 lane equal over 600 users × 64 roles, bag 1421 = `count(*)`, distinct 1374 = `count(DISTINCT)`; composition 310 µs vs the same plan in direct SIMD 14 µs. Open correction #1686: on main `bool_or`/`bool_and` read NULL as FALSE and `count_distinct_sorted` counts NULL keys (review `GPT:gpt-ada-01/…-consolidated-review-01` item 2) | IAM diamond (Bag 2, Distinct 1); independent DFS oracle lane-equal; 7 compose + 5 r2il disables red | +| **D-RFT-1** | typed composition: membership × role carriers over one slot coordinate; Distinct/Bag from the cardinality witness; NULL key matches nothing; XOR, seal mismatch, stale binding, bag overflow refused | Shipped, PROBE (#1681 `0f756b98`; follow-up #1685 In PR: reads the register — Distinct = `Page::roles`, Bag = AND + popcount on the support, 12.4 µs for 600 users, dense int8 tile GEMM 57.5 µs; EXO-50 claim `session_01G22yT6htkcdyXsihxxXdrg/036`; `crates/dir-rmap-probe/src/compose.rs`; `entries/2026-10-21-d-rft-1-typed-composition.md`). r2il polyfill: `IntXor`, `ANY`, `ALL` + procedures. MEASURED: PostgreSQL 16.14 lane equal over 600 users × 64 roles, bag 1421 = `count(*)`, distinct 1374 = `count(DISTINCT)`; composition 310 µs vs the same plan in direct SIMD 14 µs. Open correction #1686: on main `bool_or`/`bool_and` read NULL as FALSE and `count_distinct_sorted` counts NULL keys (review `GPT:gpt-ada-01/…-consolidated-review-01` item 2) | IAM diamond (Bag 2, Distinct 1); independent DFS oracle lane-equal; 7 compose + 5 r2il disables red | | **D-RFT-2..5, D-LFA-0** | substitution, recursion, frontier, replay; flow census | PROPOSAL (`entries/2026-10-21-d-rft-d-lfa-0-assessment.md`) | per-row falsifiers in the entry; unclaimed | | **D-LFA-1 / D-LFA-2** (amended 2026-10-21) | guarded-control verification of a Rubicon flow; integration with the existing drivers | PROPOSAL (plan amendment, `d-lfa-…-v1.md` § Amendment 2026-10-21; map `entries/2026-10-21-coresearch-plans-mapping-foresight.md`) | GKAT equivalence + separate no-commit-without-revision-and-MUL trace check; 3 disables; `GoTo`/`GoBack` refused; unclaimed | | **D-GKAT-0** | GKAT fixture probe: G1 factoring, G2 nondeterminism refusal, G5 declared atom bound | Shipped, PROBE (#1676, merged `6d7442c4`; EXO-50 claim `session_01VncKoyG6JkpsoqhqTGTv3f/026`, RESULT `/028`; `crates/gkat-probe`; `entries/2026-10-21-d-gkat-0-probe.md`). TEST-PINNED: Rubicon reference ≡ optimized, both safe; CHECK-authorized flows equivalent and unsafe; ≤ 8 tests (256 atoms) else refused. K3 guards are a leaf plus a validity bit and never read the payload of a NULL, as Quack's `sql_where` (#1679) | 3 flow disables + 15 checker disables red; independent bounded oracle; not a production verifier | diff --git a/.claude/board/entries/2026-10-21-d-rft-1-typed-composition.md b/.claude/board/entries/2026-10-21-d-rft-1-typed-composition.md index b2e544442..39d8d3f00 100644 --- a/.claude/board/entries/2026-10-21-d-rft-1-typed-composition.md +++ b/.claude/board/entries/2026-10-21-d-rft-1-typed-composition.md @@ -1,6 +1,6 @@ # 2026-10-21 — D-RFT-1-COMPOSE: typed composition over the DR256 slot coordinate (D-RFT-1) -**Status:** PROBE (lab, `crates/dir-rmap-probe`). TEST-PINNED and MEASURED as stated below; no production ABI changes. +**Status:** PROBE (lab, `crates/dir-rmap-probe`). TEST-PINNED and MEASURED as stated below; no production ABI changes. The execution path was replaced by a read of the register; see the last section. **Claim:** EXO-50 `session_01G22yT6htkcdyXsihxxXdrg/036`, status in `/038`. **Plan:** `.claude/plans/d-rft-relalg-thought-git-frontier-v1.md` (D-RFT-1); assessment `entries/2026-10-21-d-rft-d-lfa-0-assessment.md`. @@ -109,3 +109,27 @@ Written by `session_013z2e1YXkm6o6HNrDn5BPWA` on the operator's instruction (EXO - **Tests.** `bool_aggregates_keep_null_apart_from_false` (the six cases TRUE/NULL, FALSE/NULL, TRUE/FALSE/NULL, NULL/NULL, empty, FALSE/FALSE, each with scrambled NULL payloads); `count_distinct_excludes_null_keys_whatever_their_payload` (first, middle and last runs; all-NULL selection). The existing primitive test's empty-selection expectation changed from `(0, 1)` to NULL. Four disables red: no `valid` in `bool_or`, in `bool_and`'s FALSE fold, in the presence fold, in `count_distinct_sorted`. - **PostgreSQL 16.15** (`examples/null_aggs_pg_parity.rs`, lab, SKIP without a server): 300 queries over 60 random tables, values and NULL states equal; 64 NULL bool aggregates, 12 `bool_and` TRUE over a NULL row; the disable arm (validity = every row) differs on 224. - **Signature change.** `bool_or`, `bool_and` and `count_distinct_sorted` take new arguments; no caller outside the crate. + +## Update — the composition reads the register (follow-up PR) + +The first version recomputed, with about 2 600 r2il replays per page, what the DR256 register already holds. That was the cost, not the algebra. It now reads the register: + +- **Distinct** is `Page::roles`, the role set the page resolved. Nothing is computed. Under a presence plane (`Members::with_valid`), it is the union of the present slots' role words. +- **Bag** is one AND + popcount per supported role, over the page's membership row and the register's carrier rows (`Qr::role_slots`), through `ndarray::simd`. +- `Members` borrows the page; the presence plane is optional, so the 8 KiB copy is gone. + +Measured (`examples/rft1_bench.rs`, release, median of 101 runs, 600 users / 3 pages): + +| arm | time | allocations per run | +|---|---|---| +| A — composition over the register | 12.4 µs | 6 (the answer lanes) | +| C+ — support by union, then AND + popcount | 14 µs | 0 | +| G — dense int8 tile product (`gemm_u8_i8`, AMX host), bits unpacked to bytes | 57.5 µs | 9 | +| C — every coordinate | 147–151 µs | 0 | +| B — tuple baseline, lab only | 159 µs | 10 | + +The dense tile product is slower than the bit product here: unpacking 0/1 bits to bytes costs more than the composition. PostgreSQL 16.14 parity unchanged. + +Tests: 11. A random-fixture test checks that the two Distinct routes (`Page::roles`, and the union under a full presence plane) agree page by page. Disables, all red: support ignoring `Page::roles`; presence plane ignored; Bag skipping a supported role; NULL-key support read from the raw row (the last one needed a new fixture, where the nulled key is the only path to a role). + +The "per-execution cost" and "no count-product terminal" open points above no longer apply to this composition: it runs no mask-risc execution. diff --git a/crates/dir-rmap-probe/Cargo.toml b/crates/dir-rmap-probe/Cargo.toml index 1dfd28932..c64946973 100644 --- a/crates/dir-rmap-probe/Cargo.toml +++ b/crates/dir-rmap-probe/Cargo.toml @@ -26,7 +26,7 @@ ogar-dir-sim = { git = "https://github.com/AdaWorldAPI/OGAR", branch = "main" } # The mask facade (`mask_or_assign`, `eq_u8_to_mask`, popcount), the same # kernels #1634's walk_mask uses. Same coordinate as mask-risc, so the graph # keeps one ndarray package. -ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master", default-features = false, features = ["std"] } +ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master", default-features = false, features = ["std", "runtime-dispatch"] } # Arm D, the independent oracle: crepe's generated Datalog evaluator over the # fixture's own integer ids (never dir-sim ordinals). AdaWorldAPI fork, by # branch (no pins), as in crepe-plan-probe. diff --git a/crates/dir-rmap-probe/examples/rft1_bench.rs b/crates/dir-rmap-probe/examples/rft1_bench.rs index af1fdd22f..7bf857ef6 100644 --- a/crates/dir-rmap-probe/examples/rft1_bench.rs +++ b/crates/dir-rmap-probe/examples/rft1_bench.rs @@ -2,18 +2,21 @@ //! through direct security groups and role carriers) three ways, over all //! pages of one fixture. //! -//! * A: `Composer::compose` (the typed composition: r2il procedures, -//! compiled once, replayed through mask-risc; support first, so only -//! coordinates that can hold a path are counted); +//! * A: `Composer::compose` (the typed composition over the register: +//! Distinct is the page's resolved roles, Bag is one AND + popcount per +//! supported role, through `ndarray::simd`); //! * C: every coordinate's AND + popcount with `ndarray::simd` primitives on //! the 4-word rows, no support pass (the brute-force SIMD reference); -//! * C+: arm A's plan (support first) written directly with -//! `ndarray::simd`: the gap A − C+ is mask-risc's per-execution cost; +//! * C+: support first (the user's present slots' role words OR'd, then +//! one AND + popcount per supported role), directly with `ndarray::simd`; +//! * G: the dense int8 tile product, `ndarray::simd::gemm_u8_i8`, per +//! page (users × 256 slots as 0/1 bytes, times 256 slots × 64 roles); the +//! bits are unpacked to bytes first, and that is counted; //! * B: LAB-ONLY tuple baseline: every path materialized as a //! `(user, role, group)` tuple, sorted, runs counted. It exists only to //! show what leaving the fold costs; it is not a candidate. //! -//! Bind, page preparation, program preparation (`Composer::new`) and +//! Bind, page preparation, buffer preparation (`Composer::new`) and //! execution are timed separately (median of 101 runs, in ns). All three arms must agree on bag and distinct before any timing. //! Allocations are counted by a wrapping global allocator. //! @@ -27,7 +30,7 @@ use dir_rmap_probe::fixture::{Fixture, Spec}; use dir_rmap_probe::focus::SLOTS; use dir_rmap_probe::prep::{Prepared, Refusal, ones}; use dir_rmap_probe::qr::{Page, Qr}; -use ndarray::simd::{mask_and_assign, mask_or_assign, popcount_batch_u64}; +use ndarray::simd::{gemm_u8_i8, mask_and_assign, mask_or_assign, popcount_batch_u64}; use std::alloc::{GlobalAlloc, Layout, System}; use std::hint::black_box; use std::sync::atomic::{AtomicUsize, Ordering}; @@ -132,6 +135,33 @@ fn main() { } (bag, distinct) }; + // Arm G: dense int8 tile product per page. + let mut rmat = vec![0i8; SLOTS * 64]; + for (sl, &roles) in qr.role.iter().enumerate() { + for r in ones(&[roles]) { + rmat[sl * 64 + r] = 1; + } + } + let mut a_tile = vec![0u8; SLOTS * SLOTS]; + let mut c_tile = vec![0i32; SLOTS * 64]; + let mut arm_g = || { + let (mut bag, mut distinct) = (0u64, 0u64); + for (m, &n) in members.iter().zip(&counts) { + let rows = n.div_ceil(16) * 16; + a_tile[..rows * SLOTS].fill(0); + for i in 0..n { + for sl in ones(&m.member[i]) { + a_tile[i * SLOTS + sl] = 1; + } + } + gemm_u8_i8(&a_tile, &rmat, &mut c_tile, rows, 64, SLOTS); + for &c in &c_tile[..n * 64] { + bag += c as u64; + distinct += u64::from(c > 0); + } + } + (bag, distinct) + }; // Arm C: direct ndarray::simd on the 4-word rows. let arm_c = || { let (mut bag, mut distinct) = (0u64, 0u64); @@ -139,7 +169,6 @@ fn main() { for i in 0..n { for r in 0..64 { let mut buf = m.member[i]; - mask_and_assign(&mut buf, &m.valid[i]); mask_and_assign(&mut buf, &carriers.role_slots[r]); let c = popcount_batch_u64(&buf); bag += c; @@ -154,8 +183,7 @@ fn main() { let (mut bag, mut distinct) = (0u64, 0u64); for (m, &n) in members.iter().zip(&counts) { for i in 0..n { - let mut present = m.member[i]; - mask_and_assign(&mut present, &m.valid[i]); + let present = m.member[i]; let mut support = [0u64]; for s in ones(&present) { mask_or_assign(&mut support, std::slice::from_ref(&qr.role[s])); @@ -175,8 +203,7 @@ fn main() { let mut paths: Vec<(u32, u8, u16)> = Vec::new(); for (p, (m, &n)) in members.iter().zip(&counts).enumerate() { for i in 0..n { - let mut present = m.member[i]; - mask_and_assign(&mut present, &m.valid[i]); + let present = m.member[i]; for s in ones(&present) { for r in 0..64u8 { if qr.role[s] >> r & 1 == 1 { @@ -203,6 +230,7 @@ fn main() { assert_eq!(a, c, "arm A vs arm C: (bag, distinct)"); assert_eq!(a, arm_cp(), "arm A vs arm C+: (bag, distinct)"); assert_eq!(a, t, "arm A vs arm B: (bag, distinct)"); + assert_eq!(a, arm_g(), "arm A vs arm G: (bag, distinct)"); println!( "users {users}, pages {}, bag {}, distinct {} (all three arms agree)", pages.len(), @@ -220,19 +248,22 @@ fn main() { let compile_ns = median_ns(REPS, Composer::new); println!("bind {bind_ns} ns"); println!("pages {prep_ns} ns ({} pages)", pages.len()); - println!("programs {compile_ns} ns (Composer::new)"); + println!("buffers {compile_ns} ns (Composer::new)"); let ta = median_ns(REPS, &mut arm_a); let tc = median_ns(REPS, arm_c); let tcp = median_ns(REPS, arm_cp); let tb = median_ns(REPS, arm_b); + let tg = median_ns(REPS, &mut arm_g); let (aa, ab) = allocs_of(&mut arm_a); let (ca, cb) = allocs_of(arm_c); let (cpa, cpb) = allocs_of(arm_cp); let (ba, bb) = allocs_of(arm_b); - println!("A compose (mask-risc) {ta} ns allocs {aa} ({ab} B)"); + let (ga, gb) = allocs_of(&mut arm_g); + println!("A compose (register+bits) {ta} ns allocs {aa} ({ab} B)"); println!("C ndarray::simd direct {tc} ns allocs {ca} ({cb} B)"); println!("C+ support-first simd {tcp} ns allocs {cpa} ({cpb} B)"); + println!("G int8 tile gemm {tg} ns allocs {ga} ({gb} B)"); println!("B tuple baseline (lab-only) {tb} ns allocs {ba} ({bb} B)"); println!( "ratios: B/A {:.2} A/C {:.2} A/C+ {:.2}", diff --git a/crates/dir-rmap-probe/examples/rft1_cost.rs b/crates/dir-rmap-probe/examples/rft1_cost.rs index 85d5b536a..e561e1caf 100644 --- a/crates/dir-rmap-probe/examples/rft1_cost.rs +++ b/crates/dir-rmap-probe/examples/rft1_cost.rs @@ -24,6 +24,11 @@ //! ```text //! cargo run --release --example rft1_cost //! ``` +//! +//! Since #1685, `Composer::compose` no longer runs these r2il replays: it +//! reads `Page::roles` and does one AND + popcount per supported role. This +//! file still measures the replayed plan through its own programs; the +//! "arm A" timings it prints are of the new `compose`. use dir_rmap_probe::World; use dir_rmap_probe::compose::{Carriers, Composer, Law, Members, add_bag}; @@ -41,6 +46,12 @@ use std::hint::black_box; use std::sync::atomic::{AtomicUsize, Ordering}; use std::time::Instant; +/// Row `i` of the presence plane: the given plane, or `member` itself when +/// every key is present (`Members::of`). +fn valid_row(m: &Members<'_>, i: usize) -> [u64; 4] { + m.valid.map_or(m.member[i], |v| v[i]) +} + struct Counting; static ALLOCS: AtomicUsize = AtomicUsize::new(0); @@ -194,7 +205,7 @@ fn main() { for i in 0..n { present_calls += 1; let mut present = m.member[i]; - mask_and_assign(&mut present, &m.valid[i]); + mask_and_assign(&mut present, &valid_row(m, i)); let keys: Vec = ones(&present).collect(); if keys.is_empty() { continue; @@ -245,7 +256,7 @@ fn main() { run( &pa, &mut sa, - &[&m0.member[i0], &m0.valid[i0], &carriers.role_slots[r0]], + &[&m0.member[i0], &valid_row(m0, i0), &carriers.role_slots[r0]], SLOTS, Out::None, ) @@ -259,7 +270,7 @@ fn main() { run( &pi, &mut si, - &[&m0.member[i0], &m0.valid[i0]], + &[&m0.member[i0], &valid_row(m0, i0)], SLOTS, Out::Mask(&mut present), ) @@ -298,7 +309,7 @@ fn main() { // (4) the four-word cost: the same AND, AND, popcount in direct SIMD. let simd_ns = per_call_ns(|| { let mut buf = m0.member[i0]; - mask_and_assign(&mut buf, &m0.valid[i0]); + mask_and_assign(&mut buf, &valid_row(m0, i0)); mask_and_assign(&mut buf, &carriers.role_slots[r0]); black_box(popcount_batch_u64(&buf)); }); @@ -314,7 +325,7 @@ fn main() { run( &pa, &mut sa, - &[&m0.member[i0], &m0.valid[i0], &carriers.role_slots[r0]], + &[&m0.member[i0], &valid_row(m0, i0), &carriers.role_slots[r0]], SLOTS, Out::None, ) diff --git a/crates/dir-rmap-probe/src/compose.rs b/crates/dir-rmap-probe/src/compose.rs index 8608430d4..f154ffe66 100644 --- a/crates/dir-rmap-probe/src/compose.rs +++ b/crates/dir-rmap-probe/src/compose.rs @@ -16,8 +16,24 @@ //! DIRECT groups through which `r` is reached. Its existence is the Distinct //! answer: whether the path mask is non-empty. There is no value-directed //! gather, no semijoin, and no tuple: the answer lives in coordinate lanes -//! indexed `i * 64 + r` and in a bit plane over the same coordinates, and -//! every count, existence and equality is a mask-risc terminal. +//! indexed `i * 64 + r` and in a bit plane over the same coordinates. +//! +//! The register already holds both answers' inputs, and the composition +//! reads them instead of recomputing them: +//! +//! - **Distinct** is the page's resolved role set ([`Page::roles`]), when +//! every key is present. Nothing is computed. +//! - **Bag** is the bit product of the page's membership matrix (users × +//! 256 slots) and the register's carrier matrix ([`Qr::role_slots`], +//! roles × 256 slots), evaluated only on the support: +//! `count(i, r) = popcount(member[i] & role_slots[r])` for each role `r` +//! in user `i`'s role set, through `ndarray::simd`. Every other +//! coordinate is zero by construction. +//! +//! The int8 tile GEMM (`ndarray::simd::gemm_u8_i8`) computes the same +//! product densely; measured at about 19 µs per page (unpack included) on an +//! AMX host, it is slower than the bit product, because the 0/1 bits must first +//! be unpacked to bytes (`examples/rft1_bench.rs` prints both). //! //! The join key is a slot. A bit set in `member` but clear in `valid` is a //! NULL join key: present payload, absent value. It matches nothing, as a @@ -29,10 +45,9 @@ //! groups reaching one role) would read zero, an answer neither stratum //! gives. -use lance_graph_mask_risc::{ - Foreign, Out, Planes, Program, Scratch, Value, execute_into, words_for, -}; +use lance_graph_mask_risc::{Foreign, Planes}; use lance_graph_r2il::{FoldDialect, Val, procs}; +use ndarray::simd::{mask_and_assign, mask_or_assign, popcount_batch_u64}; use crate::focus::{Row, SLOTS}; use crate::prep::{Refusal, Witness}; @@ -86,7 +101,8 @@ impl From for ComposeError { } } -/// Left operand: one page's membership rows plus their presence plane. +/// Left operand: one page's membership rows, the page's resolved role sets, +/// and an optional presence plane. /// /// A bit set in `member` but clear in `valid` is a NULL join key: present /// payload, absent value. It matches nothing. @@ -97,13 +113,17 @@ pub struct Members<'a> { pub users: usize, /// Row `i`: user `i`'s direct security groups as slots. pub member: &'a [Row; SLOTS], - /// Presence plane, same coordinates as `member`. - pub valid: [Row; SLOTS], + /// Row `i`: user `i`'s roles, resolved with the page. Valid only while + /// every key is present. + pub roles: &'a [u64; SLOTS], + /// Presence plane, same coordinates as `member`. `None` means every key + /// is present; nothing is copied for that case. + pub valid: Option<&'a [Row; SLOTS]>, } impl<'a> Members<'a> { - /// The page's rows with every key present (`valid == member`): 8 KiB of - /// presence plane is the only thing copied. + /// The page's rows with every key present. Borrows the page; copies + /// nothing. /// /// The witness is the page's own, never the register's: a page resolved /// against another register version then fails `compose`'s seal check @@ -115,9 +135,18 @@ impl<'a> Members<'a> { witness: &page.witness, users, member: &page.member, - valid: page.member, + roles: &page.roles, + valid: None, } } + + /// The same rows read through a presence plane: a key whose `valid` bit + /// is clear is NULL. The page's resolved roles no longer apply, so the + /// support is taken from the counts. + pub fn with_valid(mut self, valid: &'a [Row; SLOTS]) -> Self { + self.valid = Some(valid); + self + } } /// Right operand: role carriers over the same slots, held by the register @@ -129,15 +158,25 @@ pub struct Carriers<'a> { pub role_slots: &'a [Row; 64], /// Slot -> the roles its reach carries (one word per slot). pub role: &'a [u64; SLOTS], + /// The right-side cardinality witness: max carrier slots per role, + /// counted once per bind. + pub right_fan: u64, } impl<'a> Carriers<'a> { - /// The register's role carriers. + /// The register's role carriers, with the right fan counted once. pub fn of(qr: &'a Qr) -> Self { + let right_fan = qr + .role_slots + .iter() + .map(|slots| popcount_batch_u64(slots)) + .max() + .unwrap_or(0); Carriers { witness: &qr.witness, role_slots: &qr.role_slots, role: &qr.role, + right_fan, } } } @@ -155,7 +194,7 @@ pub struct Composed { pub support: Vec, /// Sum of `counts`, checked. pub bag: u64, - /// Popcount of `support` (mask-risc `Count` over the support plane). + /// Popcount of `support`. pub distinct: u64, /// The cardinality witness: max path-mask fan on the left (members per /// user row). @@ -195,166 +234,27 @@ fn take(stack: &mut Vec) -> u64 { } } -/// An r2il procedure compiled once and replayed over planes of the same -/// shape: a prepared statement. -/// -/// A [`FoldDialect`] builds the mask-risc [`Program`] once from an r2il -/// procedure over planes `0..k`; every later call runs that program over its -/// own `k` zero-copy plane slices, with one scratch reused. Per call the -/// cost is mask-risc execution alone, not program construction. -struct Replay { - k: usize, - n_rows: usize, - program: Program, - scratch: Scratch<'static>, -} - -impl Replay { - /// Compile `body` over `k` planes of `n_rows` rows. `keep` lends the - /// dialect a sink, for a body that ends in `KEEP`. - fn compile( - k: usize, - n_rows: usize, - keep: bool, - body: impl FnOnce(&mut FoldDialect<'_>, &mut Vec, &[u16]), - ) -> Self { - let zero = vec![0u64; words_for(n_rows)]; - let mut sink = vec![0u64; words_for(n_rows)]; - let masks = vec![zero.as_slice(); k]; - let planes = Planes { - n_rows, - masks: &masks, - lanes: &[], - }; - let ix: Vec = (0..k) - .map(|p| u16::try_from(p).expect("at most 256 planes")) - .collect(); - let mut d = if keep { - FoldDialect::with_keep(&planes, &Foreign::NONE, &mut sink) - } else { - FoldDialect::new(&planes, &Foreign::NONE) - }; - let mut stack = Vec::new(); - body(&mut d, &mut stack, &ix); - let program = d.last_program().expect("the body folds").clone(); - let scratch = Scratch::for_program(&program, n_rows).expect("scratch"); - Replay { - k, - n_rows, - program, - scratch, - } - } - - /// `procs::count_and` over `k` planes. - fn count_and(k: usize, n_rows: usize) -> Self { - Self::compile(k, n_rows, false, |d, stack, ix| { - procs::count_and(d, stack, ix).expect("compiles"); - take(stack); - }) - } - - /// `procs::union` of `k` planes, then `procs::keep`. - fn keep_union(k: usize, n_rows: usize) -> Self { - Self::compile(k, n_rows, true, |d, stack, ix| { - procs::union(d, stack, ix).expect("compiles"); - procs::keep(d, stack).expect("compiles"); +/// Set positions of a word, lowest first: the pointer table that inverts a +/// plane to its local positions. +fn ones(mut word: u64) -> impl Iterator { + std::iter::from_fn(move || { + (word != 0).then(|| { + let p = word.trailing_zeros() as usize; + word &= word - 1; + p }) - } - - /// `procs::intersect` of `k` planes, then `procs::keep`. - fn keep_intersect(k: usize, n_rows: usize) -> Self { - Self::compile(k, n_rows, true, |d, stack, ix| { - procs::intersect(d, stack, ix).expect("compiles"); - procs::keep(d, stack).expect("compiles"); - }) - } - - fn planes<'m>(&self, masks: &'m [&'m [u64]]) -> Planes<'m> { - assert_eq!(masks.len(), self.k, "compiled for {} planes", self.k); - Planes { - n_rows: self.n_rows, - masks, - lanes: &[], - } - } - - /// Run a compiled count over `masks`. - fn count(&mut self, masks: &[&[u64]]) -> u64 { - let planes = self.planes(masks); - match execute_into( - &self.program, - &planes, - &Foreign::NONE, - &mut self.scratch, - Out::None, - ) { - Ok(Value::Count(n)) => n as u64, - other => panic!("a count program answers a Count, got {other:?}"), - } - } - - /// Run a compiled keep over `masks`, writing the mask into `out`. - fn keep(&mut self, masks: &[&[u64]], out: &mut [u64]) { - let planes = self.planes(masks); - match execute_into( - &self.program, - &planes, - &Foreign::NONE, - &mut self.scratch, - Out::Mask(out), - ) { - Ok(Value::Mask(_)) => {} - other => panic!("a keep program answers a Mask, got {other:?}"), - } - } -} - -/// Population count of `words` over `n_rows` rows (`procs::count_and` of -/// one plane, i.e. `LOAD` then `PopCount`). -fn plane_count(words: &[u64], n_rows: usize) -> u64 { - Replay::count_and(1, n_rows).count(&[words]) -} - -/// The set positions of `words`: the pointer table that inverts a plane to -/// its local positions (the one thing the inversion materializes). -fn positions(words: &[u64], out: &mut Vec) { - out.clear(); - for (w, &word) in words.iter().enumerate() { - let mut bits = word; - while bits != 0 { - out.push(w * 64 + bits.trailing_zeros() as usize); - bits &= bits - 1; - } - } -} - -/// The composition's prepared programs: every r2il procedure it runs, -/// compiled once. Building one is the prepare step; [`Composer::compose`] -/// is execution alone and reuses them (unions are compiled once per arity, -/// on first use). -pub struct Composer { - present_of: Replay, - path: Replay, - fan: Replay, - union_of: Vec>, + }) } -impl Default for Composer { - fn default() -> Self { - Self::new() - } -} +/// The composition's executor. It holds no state; the register is the +/// prepared form. +#[derive(Default)] +pub struct Composer; impl Composer { - /// Compile the composition's procedures. + /// A composer. pub fn new() -> Self { - Composer { - present_of: Replay::keep_intersect(2, SLOTS), - path: Replay::count_and(3, SLOTS), - fan: Replay::count_and(1, SLOTS), - union_of: (0..=SLOTS).map(|_| None).collect(), - } + Composer } /// Compose membership with role carriers. @@ -384,58 +284,44 @@ impl Composer { } assert!(left.users <= SLOTS, "a page holds at most {SLOTS} users"); let users = left.users; - - // Support first. A user's roles are the union of the role words of its - // present join keys (`member & valid`), so the only coordinates that can - // hold a path are the bits of that union; every other coordinate has - // count 0 by construction and is never evaluated. Each step is an r2il - // procedure compiled once and replayed over zero-copy row slices: - // present_i = keep(intersect(member_i, valid_i)) over 256 slots - // support_i = keep(union(role[s] for s in present_i)) over 64 roles - // count(i,r) = count_and(member_i, valid_i, carriers_r) for r in support_i - // The two pointer tables (present slots, supported roles) are the - // inversion from a plane to its local positions. let mut counts = vec![0u16; users * 64]; let mut support = vec![0u64; users]; - let mut bag = 0u64; - let mut left_fan = 0u64; - let mut present = [0u64; 4]; - let (mut keys, mut roles): (Vec, Vec) = (Vec::new(), Vec::new()); - let mut words: Vec<&[u64]> = Vec::with_capacity(SLOTS); + let (mut bag, mut left_fan) = (0u64, 0u64); for i in 0..users { - let (m, v) = (&left.member[i][..], &left.valid[i][..]); - self.present_of.keep(&[m, v], &mut present); - positions(&present, &mut keys); - left_fan = left_fan.max(keys.len() as u64); - if keys.is_empty() { - continue; + // Present keys: the row itself, or the row under its presence plane. + let mut present = left.member[i]; + if let Some(valid) = left.valid { + mask_and_assign(&mut present, &valid[i]); } - words.clear(); - words.extend(keys.iter().map(|&s| std::slice::from_ref(&right.role[s]))); - self.union_of[keys.len()] - .get_or_insert_with(|| Replay::keep_union(keys.len(), 64)) - .keep(&words, &mut support[i..=i]); - positions(&support[i..=i], &mut roles); - for &r in &roles { - let c = self.path.count(&[m, v, &right.role_slots[r][..]]); - // A path mask lives over 256 slots, so its count is at most 256. + left_fan = left_fan.max(popcount_batch_u64(&present)); + // Distinct: the page's resolved roles; under a presence plane, + // the union of the present slots' role words. + support[i] = match left.valid { + None => left.roles[i], + Some(_) => { + let mut roles = [0u64]; + for (w, &word) in present.iter().enumerate() { + for b in ones(word) { + mask_or_assign( + &mut roles, + std::slice::from_ref(&right.role[w * 64 + b]), + ); + } + } + roles[0] + } + }; + // Bag: one AND + popcount per supported role. + for r in ones(support[i]) { + let mut path = present; + mask_and_assign(&mut path, &right.role_slots[r]); + let c = popcount_batch_u64(&path); counts[i * 64 + r] = u16::try_from(c).expect("a count over 256 slots fits u16"); bag = add_bag(bag, c)?; } } - let distinct = if users == 0 { - 0 - } else { - plane_count(&support, users * 64) - }; - - // Cardinality witness. Left: present join keys per user row (above). - // Right: carrier slots per role. - let mut right_fan = 0u64; - for carriers in right.role_slots.iter() { - right_fan = right_fan.max(self.fan.count(&[&carriers[..]])); - } - + let distinct = popcount_batch_u64(&support); + let right_fan = right.right_fan; let stratum = if left_fan <= 1 || right_fan <= 1 { Stratum::Distinct } else { diff --git a/crates/dir-rmap-probe/tests/compose.rs b/crates/dir-rmap-probe/tests/compose.rs index 7a3461b21..713dfa60e 100644 --- a/crates/dir-rmap-probe/tests/compose.rs +++ b/crates/dir-rmap-probe/tests/compose.rs @@ -99,8 +99,9 @@ fn null_key_matches_nothing() { Stratum::Bag, "anti-vacuity: with both keys present the left side fans out" ); - let mut m = Members::of(&qr, &page, 1); - m.valid[0][s >> 6] &= !(1u64 << (s & 63)); + let mut valid = page.member; + valid[0][s >> 6] &= !(1u64 << (s & 63)); + let m = Members::of(&qr, &page, 1).with_valid(&valid); let out = compose(&m, &c, &w.witness(), Law::Count).unwrap(); assert_eq!(out.counts[5], 1); assert_eq!(out.bag, 1); @@ -110,6 +111,38 @@ fn null_key_matches_nothing() { assert_eq!(out.stratum, Stratum::Distinct); } +/// FAILS IF: the support under a presence plane is taken from the raw +/// membership row instead of the present keys. g1 alone grants role 7, so +/// nulling g1 must drop role 7 from the support, while role 5 (reached +/// through g0 too) stays. +#[test] +fn null_key_drops_the_role_only_it_reaches() { + let mut f = diamond(vec![(0, 0), (0, 1)]); + f.grants.push((1, 7)); + let (w, qr, page) = rig(f); + let ord = Ordinals::of(&w.f, &w.view()); + let s = qr.slot(ord.group[1]).expect("g1 is a security group"); + let c = Carriers::of(&qr); + let full = compose(&Members::of(&qr, &page, 1), &c, &w.witness(), Law::Count).unwrap(); + assert_eq!( + full.support[0], + (1 << 5) | (1 << 7), + "anti-vacuity: both roles held" + ); + let mut valid = page.member; + valid[0][s >> 6] &= !(1u64 << (s & 63)); + let out = compose( + &Members::of(&qr, &page, 1).with_valid(&valid), + &c, + &w.witness(), + Law::Count, + ) + .unwrap(); + assert_eq!(out.support[0], 1 << 5); + assert_eq!(out.counts[7], 0); + assert_eq!((out.bag, out.distinct), (1, 1)); +} + /// FAILS IF: operands bound to different registers are composed. #[test] fn mismatched_seal_is_refused() { @@ -268,6 +301,20 @@ fn random_fixture_matches_independent_oracle() { let page = qr.page(&b, p).unwrap(); let m = Members::of(&qr, &page, users); let out = compose(&m, &c, &w.witness(), Law::Count).unwrap(); + // The two Distinct routes agree: the page's resolved roles (no + // presence plane) and the product's non-zero entries (a full one). + let via_product = compose( + &Members::of(&qr, &page, users).with_valid(&page.member), + &c, + &w.witness(), + Law::Count, + ) + .unwrap(); + assert!( + !differ(&out.support, &via_product.support, users * 64), + "page {p}: Page::roles vs product support" + ); + assert_eq!(out.counts, via_product.counts, "page {p}: counts"); let want = oracle_lane(&b, p * 256, users); assert_eq!(out.counts, want, "page {p}: counts lane"); let mut want_support = vec![0u64; users];