Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Plan: `.claude/plans/d-relalg-unified-v1.md`. Decisions recorded there: ADMISSIO
| **D-RU-3** | cross-rule CSE in crepe-plan-probe (256-rule benchmark, separate from the hop benchmark) | Queued (after D-RU-2) | answers == crepe and ascent; traversals and materializations measured |
| **D-RU-4** | co-addressed join operator over `JoinDomain` | Queued (after D-RU-2) | parity with the star path; no position taken from another table's values |
| **D-RU-5** | Quack physical pins (dense vs sorted, materialize vs recompute) | Queued (after D-RU-3/4) | can-fire and can-stay-silent per pin |
| **D-RU-6** | recursion driver over BIND tiles (256-hop benchmark) | Queued (slot assignment must be argued as an attested frozen-codebook lookup under ADMISSION = B) | least fixpoint == datafrog; ≥ 200 iterations asserted; cycles terminate |
| **D-RU-6** | recursion driver over BIND tiles (256-hop benchmark) | Queued/Lab (slot assignment must be argued as an attested frozen-codebook lookup under ADMISSION = B). Lab probe MEASURED (`crepe-plan-probe/examples/walk_tile.rs`, `entries/2026-10-21-d-ru-6-tile-walk-probe.md`): tuple sets equal ascent, crepe and walk_mask; execution gather-free; binding witness probe-local, so ADMISSION B is NOT demonstrated; count-only 1.0–9.3× slower than walk_mask, end to end 1.1–2.0× slower | least fixpoint == datafrog; ≥ 200 iterations asserted; cycles terminate. Benchmark requirement (operator, 2026-10-21; refs D-RMAP-2 §5b, #1642): tuple materialization, count-only evaluation, memory growth, address binding and value-derived reads by origin reported separately; tuple-set equality with crepe and ascent wherever they run |
| **D-RU-7** | prefetch and staging of Arm B ranges | Queued (after D-RU-6 or a non-recursive multi-stage plan) | critical-path overlap measured; bytes read unchanged |
| **D-RU-8** | Wikidata- and SAP-shaped fixtures with completeness refusals | Queued (after D-RU-6/7) | `!R` refused under an incomplete scope |

Expand Down
103 changes: 103 additions & 0 deletions .claude/board/entries/2026-10-21-d-ru-6-tile-walk-probe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# 2026-10-21 — D-RU-6-LAB-1: graph_walking over co-addressed RMAP tiles, measured against walk_mask

**Status:** MEASURED, lab only. D-RU-6 stays **Queued/Lab**: ADMISSION B is not demonstrated for the binding path, and passing tests is not an admission. Plan: `.claude/plans/d-relalg-unified-v1.md` §8, §10, §14 (U6). References: D-RMAP-2 §5b (`d-rmap-2-context-tile-algebra-probe-v1.md`, tile kernels consume prepared relationships), #1642 (the `walk_mask` tuple oracle and fixtures).

## Benchmark requirement (operator, 2026-10-21)

Every D-RU-6 comparison reports these separately, never folded into one number:
1. tuple materialization (decoding to the canonical `Walk` / `NoWalk` sets);
2. count-only evaluation (no tuple built);
3. memory growth (resident, per engine, in a fresh process);
4. address binding: attestation, witness validation, identity resolution and focus binding, projection;
5. value-derived reads, by origin.

Correctness is the full tuple set against crepe and ascent wherever they run; Boolean reachability alone does not count. Above the oracle size the result is marked UNCHECKED and compared engine against engine only.

## What landed

`crates/crepe-plan-probe/examples/walk_tile.rs`, a separate example; the baseline `walk_mask.rs` is compiled in as a module, and it only gains `pub(crate)` on the reused items.
- **Binding.** The node ids form a frozen sorted codebook, with one `rmap::Focus` per 256 ids. The witness is `rmap::Witness`: facts digest, codebook seal, focus generation, scope.
- **Projection.** One block matrix of 256 × 256 tiles per edge length, `rmap::tile_addr` layout.
- **Recursion.** Semi-naive over length planes, `new &= !walk; walk |= new`. The hop is a Boolean tile product in predicated form: per coordinate `y`, `C |= broadcast(row y of A) & spread(bit y of each row of B)`, as one mask-risc `Ternlog` (0xF8) program. Every address is a loop coordinate.
- **Paging.** More than 256 nodes are paged over several foci; products run over all `(I, K, J)` tile triples, and cross-page edges are counted. A focus limit that the nodes exceed is refused (`IncompleteFocus`), never truncated. A scope that misses reached ids is refused.
- **Guards.** `admit` refuses any program containing `MaskOp::Gather` before execution. A stale witness (focus generation, facts, seal) is refused.

**Tests (23, of which 8 are the baseline's own):**
- tuple-set equality with ascent, crepe and walk_mask on mixed lengths {1,2,3,5,7}, cycles, alternative path lengths, zero-length and over-length edges, ids that are not nodes, 255/256/257/300 nodes across foci, the benchmark generator over two foci, and a sparse graph over three foci;
- a chain that needs ≥ 200 iterations (230 measured);
- a zero-length 200-ring that terminates;
- the tile product against `tile_gemm::BitMat::compose`;
- refusals.

**Disable runs** (each reverted afterwards; all red):

| Disable | Red |
|---|---|
| `admit` lets `Gather` through | the gather test |
| witness check removed | the stale test |
| escape refusal removed | the scope test |
| page cap removed | the cap test |
| length bound made exclusive | 6 tests |
| cross-page products dropped | 2 tests |
| spread reads `y ^ 1` | 8+ tests |
| decode shifts a length | 8+ tests |

CI runs `cargo test --example walk_tile`; the benchmark is not run in CI.

## Measured (release, 4 vCPU, load < 1.3, medians of 7)

| fixture | nodes | foci | mask count-only ms | tile count-only ms | tile/mask | mask + decode ms | tile + decode ms | tile/mask | oracle |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---|
| edges(128) | 128 | 1 | 0.43 | 0.72 | 1.7× | 4.14 | 4.54 | 1.1× | equal (ascent, crepe) |
| edges(256) | 256 | 1 | 1.28 | 1.32 | 1.0× | 15.8 | 18.2 | 1.2× | equal |
| edges(512) | 512 | 2 | 3.07 | 8.95 | 2.9× | 68.3 | 74.7 | 1.1× | equal |
| edges(1024) | 1024 | 4 | 12.7 | 72.0 | 5.7× | 277 | 384 | 1.4× | UNCHECKED (tile = mask) |
| chain(230) | 230 | 1 | 7.43 | 27.6 | 3.7× | 16.8 | 33.6 | 2.0× | equal |
| mixed(256) | 256 | 1 | 8.65 | 80.0 | 9.3× | 298 | 568 | 1.9× | equal |

"Tile count-only" includes attestation, validation, binding, projection, recursion and counting. Its phases at edges(1024), in ms: attest 0.84, validate < 0.001, bind 0.80, project 1.74, recurse 67.6, count 0.16, decode 318.

**Memory.** Resident growth (VmHWM in a fresh process, MiB):

| | mask count-only | tile count-only | mask + decode | tile + decode |
|---|---:|---:|---:|---:|
| edges(512) | 3.61 | 0.66 | 13.9 | 10.9 |
| edges(1024) | 14.2 | 2.23 | 50.7 | 44.9 |
| mixed(256) | 0.65 | 1.30 | 57.1 | 57.5 |

At edges(1024), the mask planes take 19.1 MiB (three dense sets of 51 planes) and the tile walk takes 1.28 MiB, because only non-empty tiles exist.

**Value-derived reads at edges(1024):**

| Engine | Phase | Reads |
|---|---|---|
| walk_mask | execution (delta row at an adjacency bit) | 419,230 |
| tile | preparation (codebook lookups, 2 per edge) | 83,846 |
| tile | execution | 0 |
| tile | decoding (codebook decodes, 2 per output tuple) | 4,036,624 |

## Reading

- **Value-derived reads in execution: removed.** The tile recursion reads no position taken from a value, and `Gather` is refused before execution.
- **Speed: lost, everywhere but one cell.** Count-only, the tile path is 1.0× (edges(256), one focus, single edge length) to 9.3× slower (mixed lengths). End to end it is 1.1–2.0× slower; the gap narrows only because decoding (BTreeSet) dominates both engines.
- **Where the time goes.** The predicated product touches 3 × 1,024 words per contributing `y`, whatever the sparsity. walk_mask touches `w` words per set adjacency bit. Recursion is 94% of the tile path's count-only time at edges(1024).
- **Binding costs are small.** Attest, bind and project together are about 3.4 ms at edges(1024), about 4.7% of the count-only time. They are reported, not hidden.
- **Memory is the one gain.** Only non-empty tiles exist, so the count-only resident growth is 6.4× smaller at edges(1024). Decoding erases that.
- **Not measured:** a sparsity-aware co-addressed product (skipping whole 64-row blocks of `B` without addressing by value); AMX or BF16 tile products for this program (D-RMAP-2 §5a: the bit arm beats them on Boolean composition); identity-scale paging (65,536 nodes).

## ADMISSION B classification

| Step | Classification |
|---|---|
| attestation, codebook build | same-table sequential scans, counted |
| slot resolution | one frozen-codebook lookup per edge endpoint, keyed by the edge's own value. This is the shape the ruling admits as representation translation, but the witness is a probe-local digest (`rmap::Witness`), not the attested sorted key the ruling names. **Not demonstrated.** |
| edge tiles | projection of the same table at resolved slots |
| recursion | co-addressed. **Demonstrated** (refusal guard, disable-verified). |
| decoding | frozen-codebook decoding, counted |

**Verdict:** execution is gather-free; the complete preparation and binding path is not shown to satisfy ADMISSION B. D-RU-6 stays Queued/Lab. No production promotion.

## Open

- A contract-level attestation for the codebook (an `OrderedLaneWitness`-shaped seal), before any ADMISSION B claim.
- Whether a co-addressed product can use block sparsity of `B` to close the 1.0–9.3× count-only gap without a value-addressed read.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,11 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

397 entries, 2026-08-06 .. 2026-10-20.
398 entries, 2026-08-06 .. 2026-10-21.

| date | entry id | finding | file |
|---|---|---|---|
| 2026-10-21 | `D-RU-6-LAB-1` | | [2026-10-21-d-ru-6-tile-walk-probe.md](2026-10-21-d-ru-6-tile-walk-probe.md) |
| 2026-10-20 | `D-RU-1` | | [2026-10-20-u1-jc-batch.md](2026-10-20-u1-jc-batch.md) |
| 2026-10-20 | `thought-git-replay-api-write-rubicon` | | [2026-10-20-thought-git-replay-api-write-rubicon.md](2026-10-20-thought-git-replay-api-write-rubicon.md) |
| 2026-10-20 | `ADR-2026-10-20-SIX` | | [2026-10-20-six-decisions.md](2026-10-20-six-decisions.md) |
Expand Down
2 changes: 2 additions & 0 deletions .claude/plans/d-relalg-unified-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,8 @@ This is the `walk_mask` loop. To be lawful it needs the step "rule fragment appl

**Benchmark requirement.** At least one input must need deep propagation. A chain or a ladder of length ≥ 200 forces the loop to run that many iterations (asserted, not assumed), plus a cyclic graph to check termination.

**Benchmark requirement, D-RU-6 (operator, 2026-10-21).** Every comparison against `walk_mask` (#1642) reports separately: tuple materialization, count-only evaluation, memory growth, address binding (attestation, witness validation, identity resolution and focus binding, projection), and value-derived reads by origin. Correctness is the full `Walk` / `NoWalk` tuple sets against crepe and ascent wherever they run, never Boolean reachability alone. D-RMAP-2 §5b's split of preparation, execution and maintenance applies. First lab measurement: `entries/2026-10-21-d-ru-6-tile-walk-probe.md` (execution gather-free; binding not shown to satisfy ADMISSION B; the tile path loses on time and wins on count-only memory).

## 9. Physical planning in Quack

Quack has no cost model, and its own docs say a selectivity-only model cannot separate the measured regimes (`lib.rs:674-688`). The first step is therefore what #1639 just did for Arm B: per-decision **policy pins measured on fixtures**, each with a can-fire and a can-stay-silent test. Not a general cost model. Candidate decisions:
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/crepe-plan-probe.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ jobs:
# when named. The benchmark itself (`cargo run --release --example
# walk_mask`) is not run in CI.
- run: cargo test --example walk_mask
# D-RU-6 lab probe: the same program over RMAP tiles, its oracle
# comparisons and refusals. Its benchmark is not run in CI either.
- run: cargo test --example walk_tile
# relalg is its own workspace and patches ndarray to a sibling checkout
# (`../../../ndarray` from its directory); no other job runs its tests.
- name: relalg tests and clippy
Expand Down
40 changes: 20 additions & 20 deletions crates/crepe-plan-probe/examples/walk_mask.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ use std::time::{Duration, Instant};
use crepe::crepe;
use ndarray::simd::{mask_andnot_assign, mask_any, mask_or_assign, popcount_batch_u64};

const MAX_PATH_LEN: u32 = 50;
pub(crate) const MAX_PATH_LEN: u32 = 50;
const REPS: usize = 9;
/// Largest `n` the benchmark runs the ascent and crepe oracles for.
const ORACLE_MAX_N: usize = 512;
Expand Down Expand Up @@ -104,7 +104,7 @@ ascent::ascent! {
}

/// The benchmark's edges (every edge has length 5).
fn edges(n: i32) -> Vec<(i32, i32, u32)> {
pub(crate) fn edges(n: i32) -> Vec<(i32, i32, u32)> {
let mut e = Vec::new();
for i in 0..n {
for j in 0..n {
Expand All @@ -118,13 +118,13 @@ fn edges(n: i32) -> Vec<(i32, i32, u32)> {

/// The two output relations as canonical sets.
#[derive(Debug, PartialEq, Eq)]
struct Tuples {
walk: BTreeSet<(i32, i32, u32)>,
no_walk: BTreeSet<(i32, i32)>,
pub(crate) struct Tuples {
pub(crate) walk: BTreeSet<(i32, i32, u32)>,
pub(crate) no_walk: BTreeSet<(i32, i32)>,
}

impl Tuples {
fn counts(&self) -> (usize, usize) {
pub(crate) fn counts(&self) -> (usize, usize) {
(self.walk.len(), self.no_walk.len())
}
}
Expand All @@ -138,12 +138,12 @@ fn run_ascent(e: &[(i32, i32, u32)]) -> WalkAscent {
p
}

fn ascent_walk(e: &[(i32, i32, u32)]) -> (usize, usize) {
pub(crate) fn ascent_walk(e: &[(i32, i32, u32)]) -> (usize, usize) {
let p = run_ascent(e);
(p.walk.len(), p.no_walk.len())
}

fn ascent_tuples(e: &[(i32, i32, u32)]) -> Tuples {
pub(crate) fn ascent_tuples(e: &[(i32, i32, u32)]) -> Tuples {
let p = run_ascent(e);
Tuples {
walk: p.walk.into_iter().collect(),
Expand All @@ -158,12 +158,12 @@ fn run_crepe(e: &[(i32, i32, u32)]) -> (Vec<Walk>, Vec<NoWalk>) {
(walk.into_iter().collect(), nowalk.into_iter().collect())
}

fn crepe_walk(e: &[(i32, i32, u32)]) -> (usize, usize) {
pub(crate) fn crepe_walk(e: &[(i32, i32, u32)]) -> (usize, usize) {
let (walk, nowalk) = run_crepe(e);
(walk.len(), nowalk.len())
}

fn crepe_tuples(e: &[(i32, i32, u32)]) -> Tuples {
pub(crate) fn crepe_tuples(e: &[(i32, i32, u32)]) -> Tuples {
let (walk, nowalk) = run_crepe(e);
Tuples {
walk: walk.into_iter().map(|Walk(x, z, l)| (x, z, l)).collect(),
Expand All @@ -185,15 +185,15 @@ fn bit(row: &[u64], i: usize) -> bool {
}

/// The fixpoint of the program over bit planes.
struct Planes {
pub(crate) struct Planes {
n: usize,
w: usize,
lens: usize,
node: Vec<u64>,
/// `walk[l][x]`: the nodes `z` with `Walk(x, z, l)`.
walk: Vec<u64>,
/// Value-addressed row reads (`delta[row(l2, y)]` for a set bit `y`).
row_reads: u64,
pub(crate) row_reads: u64,
}

impl Planes {
Expand All @@ -211,7 +211,7 @@ impl Planes {
}

/// `(|Walk|, |NoWalk|)` from popcounts; no tuple is built.
fn counts(&self) -> (usize, usize) {
pub(crate) fn counts(&self) -> (usize, usize) {
let walk = popcount_batch_u64(&self.walk) as usize;
let mut reach = vec![0u64; self.w];
let mut miss = vec![0u64; self.w];
Expand All @@ -229,7 +229,7 @@ impl Planes {
}

/// Decode the planes into the canonical tuple sets.
fn tuples(&self) -> Tuples {
pub(crate) fn tuples(&self) -> Tuples {
let mut walk = BTreeSet::new();
let mut no_walk = BTreeSet::new();
for x in (0..self.n).filter(|&x| self.is_node(x)) {
Expand All @@ -249,13 +249,13 @@ impl Planes {
}

/// Bytes of one length-indexed plane set; the fixpoint holds three.
fn plane_bytes(&self) -> usize {
pub(crate) fn plane_bytes(&self) -> usize {
self.walk.len() * 8
}
}

/// The program over bit planes. Node ids must lie in `0..n`.
fn mask_walk(n: usize, e: &[(i32, i32, u32)]) -> Planes {
pub(crate) fn mask_walk(n: usize, e: &[(i32, i32, u32)]) -> Planes {
let w = words(n);
let lens = MAX_PATH_LEN as usize + 1;
let row = |plane: usize, x: usize| (plane * n + x) * w;
Expand Down Expand Up @@ -358,7 +358,7 @@ fn difference<T: Ord + std::fmt::Debug>(a: &BTreeSet<T>, b: &BTreeSet<T>) -> Str
}

/// Compare the mask tuples with one oracle's; `Err` names the difference.
fn agree(mask: &Tuples, oracle: &Tuples) -> Result<(), String> {
pub(crate) fn agree(mask: &Tuples, oracle: &Tuples) -> Result<(), String> {
if mask.walk != oracle.walk {
return Err(format!("Walk: {}", difference(&mask.walk, &oracle.walk)));
}
Expand Down Expand Up @@ -462,7 +462,7 @@ fn main() {
}

#[cfg(test)]
mod tests {
pub(crate) mod tests {
use super::*;

/// The mask tuples equal both oracles' tuples, and the count-only path
Expand Down Expand Up @@ -491,11 +491,11 @@ mod tests {
}
}

const LENGTHS: [u32; 5] = [1, 2, 3, 5, 7];
pub(crate) const LENGTHS: [u32; 5] = [1, 2, 3, 5, 7];

/// `n` nodes, `per_node` random out-edges each with a length from
/// `LENGTHS`, plus the ring `0 -> 1 -> 2 -> 0`.
fn random_graph(n: usize, per_node: usize, seed: u64) -> Vec<(i32, i32, u32)> {
pub(crate) fn random_graph(n: usize, per_node: usize, seed: u64) -> Vec<(i32, i32, u32)> {
let mut r = Rng(seed);
let mut e = vec![(0, 1, 1), (1, 2, 2), (2, 0, 3)];
for x in 0..n {
Expand Down
Loading
Loading