Repository navigation
lab(mpf): D-MPF-1 — the Stage 1 math polyfill already exists; one expression, oracle, refusals, cost split - #1752
Conversation
…oop oracle, typed refusals Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
- examples/cost_split.rs: bind / mask-risc exec / dialect count / whole expression / keep vs count / relalg / oracle, ns per call, reported not gated; asserts oracle equality before timing. - .claude/plans/d-mpf-math-polyfill-v1.md: the Stage 1 polyfill is the shipped r2il procedure layer + checked scalar arithmetic over mask-risc; gaps G1-G4 by counterexample; next stages; open points. - .github/workflows/math-polyfill-probe.yml: fmt, clippy, tests, cost split. - Board: entry E-D-MPF-1, STATUS_BOARD D-MPF rows, INTEGRATION_PLANS entry, entries index regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…aph), oracle tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…racle) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
… takes ndarray by git) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…tives Three probe examples and a codegen witness, lab crate only. No primitive, contract or ABI changed. - fixed_cost: one question (sum x > t) through six arms, each one layer more than the last, every arm checked against the oracle at every size. At 64 rows: scalar 12 ns, ndarray floor 39, mask-risc +57, Program::new +20 with 1 allocation, dialect +50 (~172 ns). - floor_split: per row the floor is the bottleneck, and inside it ndarray::simd::masked_sum_i32, whose cost tracks popcount (1.7 / 25 / 44 us at 0 / 50 / 100 % of 65 536 rows). A branch-free sum equals the oracle and is flat in selectivity; the walk stays 7-10x faster on empty words. - codegen_probe + codegen-witness.sh (modelled on ndarray's): masked_sum is a scalar tzcnt/blsr walk with no vector register; gt_i32_to_mask is one vpcmpltd + kmov per 16 lanes; the branch-free sum vectorizes. Corrects D-MPF-1: '1.6x faster than the scalar loop' compared against the i128 filter-chain oracle; an equal-work i64 loop is ~3x faster than the fold. Plan section 7, STATUS_BOARD D-MPF-2, board entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
examples/layers.rs runs count, exists, sum, min and max over x > t through ndarray primitives, the mask-risc program and the R2IL polyfill on the same resident lane, oracle-checked at every size, and reports each layer's delta over the one below (before / after r2il) and allocations per call. Measured: mask-risc adds ~50-100 ns per call and the R2IL polyfill another ~55-90 ns plus one allocation; both are fixed and fall inside noise by 65 536 rows. Count beats the scalar loop; sum is ~3x slower (masked_sum bit walk); min/max match it; exists is not equal work (the scalar loop stops at the first hit). Plan section 8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
examples/hybrid.rs, probe crate only; ndarray and R2IL unchanged. Every configuration is oracle-checked. - Hybrid sum (skip empty words, walk sparse words, branch-free over dense ones, threshold 16 bits): matches or beats ndarray's masked_sum_i32 walk at every density measured, ~2x at 10 %, ~3x at 50 %, ~3.7x at 100 % (uniform), 4-5x on clustered masks. Branch-free alone loses 2-4x on sparse masks, so the walk stays for sparse words. - Mask reuse for count/sum/min/max: ~2x over recomputing when sparse, 20-30 % when dense; a fused scalar pass is still ~6x faster at 50 % selected because masked_min/max walk every set bit. Proposal for the ndarray owner, approved separately. Plan section 9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
classid_route resolves each ClassId's route (DIRECT / COMPOSE / LOWER / REFUSE) once in a LazyLock, copies the callable into the binding, and times it against direct ndarray::simd, prepared mask-risc, the FoldDialect's own last_program reused as a CompiledProgram, the dialect per call and the ONNX-shaped wrapper (diagnostic). Frame, focus and scope stay checked on every run with the DR lab's FrameStore::check. Asserted: one resolution per ClassId, zero route evaluations during the steady-state and multi-op loops, zero allocations on the route path, LOWER/REFUSE refused at bind, focus/scope/source changes refused at run, rebinding does not re-resolve the route. Adds no ABI, interpreter or dispatcher to any production crate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…nd measurement Plan §10 separates five things that are all called R2IL: - r2sleigh's r2il IL types; - the machine reading (r2conc / r2sym); - the ogar-r2il opcode table, with readings 0xC400 machine and 0xC401 fold; - lance-graph-r2il::FoldDialect, the fold reading only; - mask-risc / ndarray::simd / simd_runtime. The D-MPF-2 figures (94-99 ns, 171-172 ns) are recorded as the cost of choosing those paths per call, not as a cost of R2IL routing. Adds: - the recovered single-digit-ns figures, each with its exact symbol (all direct kernels, none an R2IL call); - a DIRECT / COMPOSE / LOWER / REFUSE table for the wired fold-reading ops and the machine-only cases that need R2Sleigh; - the per-call layers that existing facilities already remove; - the classid_route numbers. The prebound route equals direct ndarray. The remaining 27-28 ns is the frame/focus/scope guard: four atomic RMWs from Arc::downgrade. Renames the hybrid section to D-MPF-2b, with a correction note: D-MPF-3 is G1 on the board. Adds STATUS_BOARD rows D-MPF-2b / D-MPF-4 and a board entry, and regenerates the entries index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 46ced03f44
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| count(d, &mut s)?; | ||
| s.push(Val::Scalar(e.k)); | ||
| d.call(INT_MULT, [0; 3], &mut s)?; | ||
| d.call(INT_SUB, [0; 3], &mut s)?; |
There was a problem hiding this comment.
Check overflow in the expression's final subtraction
When the product fits i64 but the final subtraction does not, this expression silently returns a wrapped answer in release builds instead of the promised typed refusal. For example, one selected, present row with v = -2 and k = i64::MAX produces oracle = (1, None) but eval = Ok(i64::MAX) (reproduced against the release build). The existing FoldDialect::INT_SUB implementation uses unchecked x - y, unlike its checked multiplication. Ensure this subtraction is checked and add a boundary case that overflows at subtraction rather than multiplication.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed and fixed in 4dc44a4c1. FoldDialect's IntSub was an unchecked x - y. It now uses checked_sub and refuses with FoldError::Overflow(INT_SUB), the same as IntAdd and IntMult.
Your case (one row, v = -2, k = i64::MAX) returned Ok(i64::MAX) before the fix. It is now the test an_expression_that_overflows_at_the_subtraction_is_refused, paired with a version that fits and still answers. The r2il refusal table also gains two IntSub rows.
The r2il, relalg, r2il-mask-abi-probe and math-polyfill-probe test suites pass.
Generated by Claude Code
FoldDialect computed IntSub as an unchecked x - y, so in release a subtraction that leaves i64 wrapped (the polyfill expression returned i64::MAX for one row v = -2, k = i64::MAX, where the oracle says the answer does not fit). IntSub now uses checked_sub and refuses with FoldError::Overflow, like IntAdd and IntMult. Tests: the probe gains a boundary case that overflows at the subtraction with a silent twin that fits; the r2il refusal table gains two IntSub rows. Red before the fix (Ok(i64::MAX)), green after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…be-20261023 # Conflicts: # .claude/board/STATUS_BOARD.md # .claude/board/entries/README.md
…1753) The diagram labelled the ClassId-prebound router SHIPPED. Only the strategies beneath it ship; the router exists as the #1752 lab example and no production ClassId table exists (S17a). Marked supersession. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…be-20261023 # Conflicts: # .claude/board/STATUS_BOARD.md # .claude/board/entries/README.md
…marked shipped - PR_ARC_INVENTORY: compact entries for #1742–#1758, recorded from each merge commit and its changed-file list. - LATEST_STATE: "main after #1758" section. - STATUS_BOARD: D-ONNX-NATIVE-0 (#1754), D-DRF-1 (#1756), D-NFR-6 (#1755), D-MPF-1 (#1752) and D-ONNX-0 (#1744) were still "In PR" / "this branch"; now point at their merge commits. - INTEGRATION_PLANS: index row for six plans merged without one (#1745–#1749, #1751). - d-onnx-relalg-cap-v1 (#1744's S0): owner supersession note adopting D-NFC §1/§2 by reference (D-NFC §6 step 1); OP-ONNX-1 marked answered by D-NFC §4 + lance-graph-onnx. Which S0 draft carries the adapter layer stays open for the three owners. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VncKoyG6JkpsoqhqTGTv3f
Lab crate plus board records. This adds no primitive, IR or executor, and it does not touch ONNX. It is independent of the three competing S0 plans (#1744, #1745, #1750) and does not choose between them.
Finding
The Stage 1 mathematical polyfill API already ships:
lance-graph-r2il::procs: selection, comparisons, NULL presence, COUNT/SUM/MIN/MAX, set operations;FoldDialect:IntAdd,IntSub,IntMult,IntSDiv;Plan:
.claude/plans/d-mpf-math-polyfill-v1.md.Probe (
crates/math-polyfill-probe)E = SUM(v | sel) − k·COUNT(sel), withsel = valid(v) ∧ x ≥ lo ∧ y < hi. It is written only with the existing procedures and scalar calls.i128row loop that shares no code with the procedures, the dialect or mask-risc.tests/polyfill.rs(9 tests):i64, and division by zero.Disable runs, each committed before the run with its edit anchor asserted:
≥→>IntSub→IntAddCost split (reported, not gated)
examples/cost_split.rs. Median ns per call, three runs on a 4-core container:i128oracle; an equal-worki64loop is ~3× faster than the fold.)Gaps, each shown by a failing case
lt/le/gt/ge_u64_to_mask; the refusal is a mask-risc gap for U64 lanes.)The ONNX fork
AdaWorldAPI/optimum-onnx-rsis empty, andAdaWorldAPI/onnxis not accessible, so there is no ONNX integration surface to wire yet.Checks run
cargo fmt --checkcargo clippy --release --all-targets -- -D warningscargo test --release: 9/9entries_index.py --write;fruits_index.py --checkpassescitation_decay.py: 0 new decaysappend_only_gate.py: OKNew CI workflow
math-polyfill-probe.yml.D-MPF-2 — where the cost goes (commit
0a6f54c7c, measured, nothing changed)Every arm is checked against the
i128oracle at every size before timing. Host: AVX-512,target-cpu=native.Per call at 64 rows (~172 ns total):
i64loop, for comparisonProgram::newPer row at 64 K rows:
ndarray::simd::masked_sum_i32, a scalar set-bit walk.LLVM (
codegen-witness.sh, modelled on ndarray's):masked_sum_i32is atzcnt/blsrwalk with no vector register.gt_i32_to_maskis onevpcmpltd+kmovper 16 lanes.Correction: the earlier "1.6× faster than the scalar loop" was measured against the
i128filter-chain oracle. An equal-worki64loop is ~3× faster than the fold.Proposed, not made: a density-chosen
masked_sum_i32body (owner:ndarray::simd). Sum branch-free when a word has more than ~16 set bits, keep the walk otherwise.Second proposal, not made: reuse the dialect's
opsvector across folds to remove its one allocation per fold (owner: r2il). (⊘ An existing facility already removes the whole dialect pass for a fixed query: keeplast_program()as aCompiledProgram; D-MPF-4.)Still open: whether the mask form beats a scalar loop when the mask is reused or combined. (⊘ Measured in D-MPF-2b below.)
Plan §7 has the full tables.
D-MPF-2b — hybrid masked sum and mask reuse (commits
0834609fc,6efc2743e; plan §9)Those commits call this D-MPF-3. On the board that id is G1, so it is renamed D-MPF-2b.
masked_sum_i32: 2–3.7× faster on uniform masks, 4–5× on clustered ones, and never slower than the walk.ndarray::simdowner only; nothing in ndarray changed.D-MPF-4 — R2IL as a ClassId-lazily-bound router (plan §10)
Attribution. Five different things are called R2IL:
r2il: IL types only;r2conc/r2sym;ogar-r2il: the opcode table;lance-graph-r2il::FoldDialect: the fold reading;ndarray::simd/simd_runtime.The earlier single-digit-ns figures are all direct kernels. The relevant one is
ndarray::simdAND, AND, popcount over 4 words: 2.2–2.8 ns, against 54–70 ns forcount_and(3)through r2il + mask-risc (entries/2026-10-21-d-rft-1-cost-breakdown.md).Routing table (plan §10.3). Every wired fold-reading op is either DIRECT or COMPOSE onto existing
ndarray::simdentrypoints, or refused. LOWER (R2Sleigh) is needed only for machine semantics: carry/borrow flags, address-space load/store, branches, width-exact integers. Floats, atomics andCallOtherare refused byr2concitself. Certification (egglog) is separate and runs before execution, never per call.Probe
examples/classid_route.rs. ClassId →LazyLock→ callable copied into the binding → direct kernel. Frame, focus and scope are checked on every run by the DR lab'sFrameStore::check. ns per call, two runs:FrameStore::check, fromArc::downgrade.last_program()as aCompiledProgram. It equals prepared mask-risc. The remaining ~50 ns over direct ndarray is mask-risc's per-callprecheck/validate.probe_route_runis the guard followed by a tailjmpq *80(%rax)into the stored callable. There is noLazyLock,Onceor backendmatchon that path.Weak::as_ptrwithArc::as_ptrinFrameStore::check. Identity semantics are unchanged and the atomics go away.D-NFC cross-reference
This PR is layer [1] of the canonical dependency diagram in #1753. (⊘ Layer [1] is the R2IL router. The procedures and FoldDialect measured here are one strategy beneath it, not the layer itself; #1753's diagram is being corrected to match.) The PR stays in draft pending independent review. The 50 ns one-word target is not met through the mask-risc and FoldDialect paths. The fair
i64scalar baseline supersedes the earlier 1.6× claim (plan §7–§8).Generated by Claude Code
🤖 Generated with Claude Code
https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg