diff --git a/.claude/board/entries/2026-10-23-exo129-learned-lut-preregistration.md b/.claude/board/entries/2026-10-23-exo129-learned-lut-preregistration.md new file mode 100644 index 000000000..22a40dde7 --- /dev/null +++ b/.claude/board/entries/2026-10-23-exo129-learned-lut-preregistration.md @@ -0,0 +1,144 @@ +# 2026-10-23 — EXO-129: does a LUT learned from observed transitions compose? (pre-registration) + +**Status:** PRE-REGISTERED (written and committed before any probe code) +**Thread:** TORQUE T-2026-10-23-1, Linear EXO-129. Crumb trail T-CLOSURE (EXO-132). +**Owner:** CC:session_01G22yT6htkcdyXsihxxXdrg + +## Question + +P-S2 (#1726) showed that `ComposeTable` does not survive two hops. It snaps +`entry[a] ⊕ entry[b]` back to the nearest entry, and most bound pairs land +outside the codebook. This probe asks two things: + +1. Does a 256×256 table **learned from observed one-hop transitions** + (`PaletteLut`-shaped, cell = modal next state) compose over k hops where + `ComposeTable` does not? +2. **Mechanism M:** does a data-only statistic, the **purity** `P` of the + learned table (the share of training observations that equal their cell's + modal result), predict how well it composes? This is the testable form + of T-CLOSURE's "is the carrier closed under the operation". + +## Setup + +- **Operation per hop, with key index `b`:** `x ← x ⊕ entry[b]` in Base17 + (`xor_bind`), as in P-S2. +- **Truth for a chain** `(x0, b1..bk)`: `assign(x0 ⊕ e_b1 ⊕ … ⊕ e_bk)`, one + snap at the end. `x0` is a population row, not a palette entry. +- **Arms:** + - **C:** the `ComposeTable` chain from `s0 = assign(x0)`. + - **L:** the learned-table chain from `s0`. +- **Training:** 20 000 trajectories of 6 hops from population rows (seed A). + Each hop records `(assign(x_t), b, assign(x_{t+1}))`. `L[a][b]` is the + modal result. Unobserved cells fall back to `ComposeTable`, and the + unobserved fraction is reported. +- **Test:** 2 000 fresh chains (seed B) for each k = 1..6. +- **Worlds (six settings):** + - **W1 clustered** and **W2 uniform:** P-S2's generators and seed, + palette from `WeightPalette::build`. These are inherited from P-S2, not + built for this probe. + - **W3 group-closed:** 256 palette entries built directly as an XOR group. + `entry[g]` has `dims[i] = 4096` for each set bit `i < 8` of `g`, so + `entry[a] ⊕ entry[b] = entry[a ^ b]` exactly. Population: 4 members per + element, each `entry[g] + u`, with `u` non-negative per-dim noise in + `[0, n]` (non-negative so that XOR never touches the sign bit). + Noise levels: `n ∈ {0, 500, 1500, 3000}`, four settings. Larger noise + lets XOR of noise reach bit 12 and break closure gradually. + +## Measures + +- `agreement_L(k)` and `agreement_C(k)` for k = 1..6: the share of test + chains whose arm result equals the truth. +- `P` (purity of L on its training observations) and the unobserved-cell + fraction. + +## Decision rules (fixed now) + +1. **Harness control:** in W3 with n = 0, both L and C read ≥ 0.99 at every + k. Otherwise the harness is broken and nothing below is read. +2. **EXO-129 torque card:** + - **PROMOTE** if L ≥ 0.90 at k = 2 in **both** W1 and W2. + - **KILL** if L < 0.90 at k = 2 in both. + - Mixed: report, and the row stays TORQUE with the next probe named. +3. **Mechanism M:** passes if, over the six settings, the Spearman rank + correlation between `P` and `agreement_L(2)` is ≥ 0.9. The prediction + `agreement_L(k) ≈ P^k` (independent per-hop errors) is reported with its + residual, but it does not gate. +4. **Disables (both must behave, or no claim is made):** + - **D1:** shuffle L's result bytes across cells. Agreement at k = 2 must + drop to ≤ 0.10 in every setting. + - **D2:** set L := C. This must reproduce C's numbers exactly. + +## Applying GOLDSTAUB G-2026-10-23-1 to this probe + +The world that passes by construction is W3 with n = 0. That is why it is +only a harness control. The world expected to fail is W2, which has no +structure for a table to learn. So no conclusion is drawn from a single +world's pass or fail: + +- the **mechanism** verdict comes only from M across the six settings; +- the **torque** verdict comes from W1/W2, which P-S2 fixed before this + probe existed. + +## Results (2026-10-23, after the run; nothing above was changed) + +`cargo test --release --test exo129_learned_lut_probe -- --nocapture` +(bgz-tensor, debug info off). Agreement with the truth at k = 1..6: + +| setting | purity P | unobserved | L k=1 | L k=2 | C k=1 | C k=2 | L k=6 | C k=6 | +|---|---|---|---|---|---|---|---|---| +| W1 clustered | 0.471 | 0.199 | 0.242 | **0.017** | 0.579 | 0.029 | 0.005 | 0.005 | +| W2 uniform | 0.479 | 0.170 | 0.112 | **0.009** | 0.279 | 0.027 | 0.005 | 0.005 | +| W3 n=0 | 1.000 | 0.159 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | +| W3 n=500 | 1.000 | 0.159 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | +| W3 n=1500 | 1.000 | 0.159 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | +| W3 n=3000 | 0.544 | 0.340 | 0.263 | 0.145 | 0.231 | 0.260 | 0.041 | 0.248 | + +- **Rule 1 (harness control):** holds. W3 with n = 0 reads 1.000 for both + arms at every k. +- **Rule 4 (disables):** both hold. + - D1 (shuffled table) reads ≤ 0.005 at k = 2 in every setting. + - D2 (L := C) reproduces C exactly. +- **Rule 2 (torque): KILL.** L reads 0.017 / 0.009 at k = 2 in W1 / W2, + against the 0.90 bar. The learned table is worse than `ComposeTable` + there, even at k = 1. +- **Rule 3 (mechanism M): PASS by the registered rule**, Spearman ρ = 0.935, + **but the evidence is thin.** Four of the six settings tie at P = 1.0 / + agreement = 1.0, so the rank correlation rests on three distinct points. + The unregistered magnitude prediction `agreement_L(k) ≈ P^k` fails badly: + W1 k = 2 is 0.017 against P² = 0.222. +- **Pin:** both verdicts are asserted in the test. Disable run: inverting + purity (`1 − P`) makes the M assertion fail, so the pin is real. + +### What the numbers say (reading, not a ruling) + +- **Purity < 1 means the palette state is not Markov for this operation.** + The cell does not determine the next cell: members of one cell go to + different cells. In W1/W2 the purity is about 0.47. No 256×256 table + can compose there, learned or derived, because one byte of state does not + carry the information the next hop needs. In Markov-chain terms this is a + **lumpability** failure (Kemeny–Snell): the partition is not lumpable for + the operation. That is a candidate name for trail T-CLOSURE. It is not + adopted here. +- **Learning made W1/W2 worse at k = 1** (0.242 vs C 0.579). Training cells + pool starts from population rows with states reached after several hops, + and 5/6 of the observations are reached states. The modal result then + favours the reached-state distribution, while the test starts from + population rows. This is a real cost of the pre-registered design, and it + is itself a non-lumpability symptom: where in the cell a state sits + matters. +- **W3 n=3000 separates two error sources.** C stays flat at about 0.25 from + k = 1 to 6. The table is the exact group law, so composition adds no + error, and the loss comes from the start and end snaps alone, which + depend on the fixed member noise. L decays (0.263 → 0.041), because its + cells were filled from mis-snapped observations. A closed carrier + therefore composes with its exact law even when snapping is noisy. A + table learned from noisy snaps does not. + +### Closeout + +PR (this branch) | STATUS: measured | OUTCOME: a learned 256×256 table does +not compose where the codebook is not closed (KILL); purity < 1 marks those +cases, with thin rank evidence (M PASS on three distinct points) | +OPEN: whether a purity bar (for example P ≥ 0.99) predicts composition on +real Moore/DR pair data. That needs labelled data (GOLDSTAUB +G-2026-10-23-1). diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 37a43912a..c2d7d6217 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,12 +25,13 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -443 entries, 2026-08-06 .. 2026-10-23. +444 entries, 2026-08-06 .. 2026-10-23. | date | entry id | finding | file | |---|---|---|---| | 2026-10-23 | `ps2-compose-chain-preregistration` | | [2026-10-23-ps2-compose-chain-preregistration.md](2026-10-23-ps2-compose-chain-preregistration.md) | | 2026-10-23 | `ledger-revision-1635-1726` | | [2026-10-23-ledger-revision-1635-1726.md](2026-10-23-ledger-revision-1635-1726.md) | +| 2026-10-23 | `exo129-learned-lut-preregistration` | | [2026-10-23-exo129-learned-lut-preregistration.md](2026-10-23-exo129-learned-lut-preregistration.md) | | 2026-10-22 | `probes-pd4-pe2-pd3` | | [2026-10-22-probes-pd4-pe2-pd3.md](2026-10-22-probes-pd4-pe2-pd3.md) | | 2026-10-22 | `pal256-salience-mp3-probe` | | [2026-10-22-pal256-salience-mp3-probe.md](2026-10-22-pal256-salience-mp3-probe.md) | | 2026-10-22 | `pal256-nars-bayes-evidence-contract` | | [2026-10-22-pal256-nars-bayes-evidence-contract.md](2026-10-22-pal256-nars-bayes-evidence-contract.md) | diff --git a/crates/bgz-tensor/tests/exo129_learned_lut_probe.rs b/crates/bgz-tensor/tests/exo129_learned_lut_probe.rs new file mode 100644 index 000000000..c35621339 --- /dev/null +++ b/crates/bgz-tensor/tests/exo129_learned_lut_probe.rs @@ -0,0 +1,358 @@ +//! EXO-129: does a 256×256 table learned from observed one-hop transitions +//! compose over k hops, and does its purity predict how well? Pre-registered +//! in `.claude/board/entries/2026-10-23-exo129-learned-lut-preregistration.md`. +//! +//! The learned table has `PaletteLut`'s shape (row-major `a << 8 | b`, one +//! byte per cell). It is a plain array here because bgz-tensor does not +//! depend on cognitive-shader-driver. + +use bgz_tensor::attention::ComposeTable; +use bgz_tensor::palette::WeightPalette; +use bgz_tensor::projection::{Base17, BASE_DIM}; + +struct Lcg(u64); +impl Lcg { + fn next(&mut self) -> u64 { + self.0 = self + .0 + .wrapping_mul(6364136223846793005) + .wrapping_add(1442695040888963407); + self.0 >> 33 + } + fn sym(&mut self, r: i32) -> i32 { + (self.next() % (2 * r as u64 + 1)) as i32 - r + } + fn upto(&mut self, n: i32) -> i32 { + (self.next() % (n as u64 + 1)) as i32 + } + fn idx(&mut self) -> u8 { + (self.next() & 0xff) as u8 + } + fn below(&mut self, n: usize) -> usize { + (self.next() % n as u64) as usize + } +} + +fn random_row(rng: &mut Lcg, r: i32) -> Base17 { + let mut dims = [0i16; BASE_DIM]; + for d in dims.iter_mut() { + *d = rng.sym(r) as i16; + } + Base17 { dims } +} + +/// P-S2's clustered generator, unchanged. +fn clustered(rng: &mut Lcg) -> Vec { + let centres: Vec = (0..64).map(|_| random_row(rng, 1500)).collect(); + let mut rows = Vec::with_capacity(1024); + for c in ¢res { + for _ in 0..16 { + let mut dims = [0i16; BASE_DIM]; + for (d, x) in dims.iter_mut().zip(c.dims.iter()) { + *d = (i32::from(*x) + rng.sym(120)).clamp(-32768, 32767) as i16; + } + rows.push(Base17 { dims }); + } + } + rows +} + +/// P-S2's uniform generator, unchanged. +fn uniform(rng: &mut Lcg) -> Vec { + (0..1024).map(|_| random_row(rng, 1500)).collect() +} + +const BASIS: i16 = 4096; + +fn group_entry(g: u8) -> Base17 { + let mut dims = [0i16; BASE_DIM]; + for (i, d) in dims.iter_mut().enumerate().take(8) { + if g >> i & 1 == 1 { + *d = BASIS; + } + } + Base17 { dims } +} + +/// W3: the palette is the XOR group itself; members carry non-negative noise. +fn group_world(noise: i32, rng: &mut Lcg) -> (WeightPalette, Vec) { + let entries: Vec = (0..=255u8).map(group_entry).collect(); + let mut rows = Vec::with_capacity(1024); + for e in &entries { + for _ in 0..4 { + let mut dims = e.dims; + for d in dims.iter_mut() { + *d += rng.upto(noise) as i16; + } + rows.push(Base17 { dims }); + } + } + let palette = WeightPalette { + radii: vec![0; 256], + counts: vec![4; 256], + entries, + }; + (palette, rows) +} + +struct World { + name: String, + palette: WeightPalette, + rows: Vec, + compose: ComposeTable, +} + +fn worlds() -> Vec { + let mut rng = Lcg(0x0005_2C0D); + let mut out = Vec::new(); + for (name, rows) in [ + ("W1 clustered", clustered(&mut rng)), + ("W2 uniform", uniform(&mut rng)), + ] { + let palette = WeightPalette::build(&rows, 256); + assert_eq!(palette.len(), 256, "{name}: palette is not full"); + let compose = ComposeTable::build(&palette); + out.push(World { + name: name.to_string(), + palette, + rows, + compose, + }); + } + let mut grng = Lcg(0x0129_0003); + for n in [0, 500, 1500, 3000] { + let (palette, rows) = group_world(n, &mut grng); + let compose = ComposeTable::build(&palette); + out.push(World { + name: format!("W3 group n={n}"), + palette, + rows, + compose, + }); + } + out +} + +const HOPS: usize = 6; +const TRAIN: usize = 20_000; +const TEST: usize = 2_000; + +fn snap(w: &World, v: &Base17) -> u8 { + w.palette.assign(v).index +} + +struct Learned { + table: Vec, + purity: f64, + unobserved: f64, +} + +fn learn(w: &World) -> Learned { + let mut counts = vec![0u32; 65_536 * 256]; + let mut rng = Lcg(0xA129_0001); + let mut total = 0u64; + for _ in 0..TRAIN { + let mut x = w.rows[rng.below(w.rows.len())].clone(); + let mut s = snap(w, &x); + for _ in 0..HOPS { + let b = rng.idx(); + x = x.xor_bind(&w.palette.entries[b as usize]); + let t = snap(w, &x); + counts[((s as usize) << 8 | b as usize) * 256 + t as usize] += 1; + total += 1; + s = t; + } + } + let mut table = vec![0u8; 65_536]; + let mut modal_hits = 0u64; + let mut unobserved = 0usize; + for cell in 0..65_536usize { + let row = &counts[cell * 256..cell * 256 + 256]; + let (best, &n) = row + .iter() + .enumerate() + .max_by_key(|&(i, &c)| (c, std::cmp::Reverse(i))) + .unwrap(); + if n == 0 { + unobserved += 1; + table[cell] = w.compose.compose((cell >> 8) as u8, cell as u8); + } else { + table[cell] = best as u8; + modal_hits += u64::from(n); + } + } + Learned { + table, + purity: modal_hits as f64 / total as f64, + unobserved: unobserved as f64 / 65_536.0, + } +} + +/// Agreement of a table-driven chain with the truth, k = 1..HOPS. +fn agreement(w: &World, step: &dyn Fn(u8, u8) -> u8) -> Vec { + let mut rng = Lcg(0xB129_0002); + (1..=HOPS) + .map(|k| { + let hits = (0..TEST) + .filter(|_| { + let x0 = w.rows[rng.below(w.rows.len())].clone(); + let keys: Vec = (0..k).map(|_| rng.idx()).collect(); + let truth = snap( + w, + &keys.iter().fold(x0.clone(), |x, &b| { + x.xor_bind(&w.palette.entries[b as usize]) + }), + ); + let arm = keys.iter().fold(snap(w, &x0), |s, &b| step(s, b)); + arm == truth + }) + .count(); + hits as f64 / TEST as f64 + }) + .collect() +} + +fn ranks(v: &[f64]) -> Vec { + let mut idx: Vec = (0..v.len()).collect(); + idx.sort_by(|&a, &b| v[a].partial_cmp(&v[b]).unwrap()); + let mut r = vec![0.0; v.len()]; + let mut i = 0; + while i < idx.len() { + let mut j = i; + while j + 1 < idx.len() && v[idx[j + 1]] == v[idx[i]] { + j += 1; + } + let avg = (i + j) as f64 / 2.0 + 1.0; + for &t in &idx[i..=j] { + r[t] = avg; + } + i = j + 1; + } + r +} + +fn spearman(a: &[f64], b: &[f64]) -> f64 { + let (ra, rb) = (ranks(a), ranks(b)); + let n = a.len() as f64; + let (ma, mb) = (ra.iter().sum::() / n, rb.iter().sum::() / n); + let cov: f64 = ra.iter().zip(&rb).map(|(x, y)| (x - ma) * (y - mb)).sum(); + let va: f64 = ra.iter().map(|x| (x - ma).powi(2)).sum(); + let vb: f64 = rb.iter().map(|y| (y - mb).powi(2)).sum(); + if va == 0.0 || vb == 0.0 { + return 0.0; + } + cov / (va * vb).sqrt() +} + +fn fmt(v: &[f64]) -> String { + v.iter() + .map(|a| format!("{a:.3}")) + .collect::>() + .join(" ") +} + +#[test] +fn learned_lut_under_multi_hop_composition() { + let mut purities = Vec::new(); + let mut agree2 = Vec::new(); + let mut verdict_w12 = Vec::new(); + for w in worlds() { + let l = learn(&w); + let lt = l.table.clone(); + let al = agreement(&w, &|a, b| lt[(a as usize) << 8 | b as usize]); + let ac = agreement(&w, &|a, b| w.compose.compose(a, b)); + + // D1: shuffle the learned bytes across cells. + let mut shuffled = l.table.clone(); + let mut srng = Lcg(0xD129_0001); + for i in (1..shuffled.len()).rev() { + shuffled.swap(i, srng.below(i + 1)); + } + let ad1 = agreement(&w, &|a, b| shuffled[(a as usize) << 8 | b as usize]); + + println!( + "{}: purity {:.4} unobserved {:.4}", + w.name, l.purity, l.unobserved + ); + println!("{}: L k=1..6 {}", w.name, fmt(&al)); + println!("{}: C k=1..6 {}", w.name, fmt(&ac)); + println!( + "{}: P^k {}", + w.name, + fmt(&(1..=HOPS) + .map(|k| l.purity.powi(k as i32)) + .collect::>()) + ); + println!("{}: D1 shuffled k=2 {:.3}", w.name, ad1[1]); + + // Rule 4, D1: a shuffled table must not compose. + assert!( + ad1[1] <= 0.10, + "{}: D1 shuffled table still agrees {:.3}", + w.name, + ad1[1] + ); + // Rule 1: the harness control. + if w.name == "W3 group n=0" { + assert!( + al.iter().chain(&ac).all(|&a| a >= 0.99), + "harness control failed: L {} C {}", + fmt(&al), + fmt(&ac) + ); + } + if w.name.starts_with("W1") || w.name.starts_with("W2") { + verdict_w12.push(al[1]); + } + purities.push(l.purity); + agree2.push(al[1]); + } + let rho = spearman(&purities, &agree2); + println!("M: spearman(purity, agreement_L(2)) over 6 settings = {rho:.3}"); + let torque = if verdict_w12.iter().all(|&a| a >= 0.90) { + "PROMOTE" + } else if verdict_w12.iter().all(|&a| a < 0.90) { + "KILL" + } else { + "MIXED" + }; + let m = if rho >= 0.9 { "PASS" } else { "FAIL" }; + println!("EXO-129 torque verdict: {torque}; mechanism M: {m}"); + + // Pinned results (2026-10-23). A change in either fails here, which means + // the probe must be re-registered, not that the pin may be edited. + assert_eq!( + torque, "KILL", + "the learned table now composes in W1/W2; re-register EXO-129" + ); + assert_eq!( + m, "PASS", + "purity no longer ranks k=2 agreement; re-register EXO-129" + ); +} + +/// Rule 4, D2: with L := C the harness reproduces C exactly. +#[test] +fn compose_table_as_learned_reproduces_c() { + for w in worlds() { + let ac = agreement(&w, &|a, b| w.compose.compose(a, b)); + let table: Vec = (0..65_536usize) + .map(|c| w.compose.compose((c >> 8) as u8, c as u8)) + .collect(); + let al = agreement(&w, &|a, b| table[(a as usize) << 8 | b as usize]); + assert_eq!(ac, al, "{}: D2 L:=C differs from C", w.name); + } +} + +/// W3's palette really is an XOR group: the closure the control relies on. +#[test] +fn group_world_is_closed_under_bind() { + for a in 0..=255u8 { + for b in 0..=255u8 { + assert_eq!( + group_entry(a).xor_bind(&group_entry(b)).dims, + group_entry(a ^ b).dims + ); + } + } +}