diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 3c99897de..918256687 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,6 @@ +## 2026-10-23 — d-onnx-native-v1 — ONNX vocabulary over existing native functions → `.claude/plans/d-onnx-native-v1.md` + +**Status:** D-ONNX-NATIVE-0 IMPLEMENTED as a draft PR (`crates/lance-graph-onnx`), not merged. The path is ONNX name → existing native function: ALIAS where the contract is identical, a small INLINE adapter where it is not, a typed `Refusal` otherwise. No graph, dispatcher or initialization. R2IL routing (D-MPF-4, #1752) is separate and is not imported. ## 2026-10-23 — d-native-repr-contract-v1 + d-native-first-dependency-correction-v1 — the native polyfill is the root → `.claude/plans/d-native-repr-contract-v1.md`, `.claude/plans/d-native-first-dependency-correction-v1.md` **Status:** D-NFR-0 / D-NFC-0 documentation (DECISION 2026-10-23: corrections to #1746–#1751 and #1750 approved). D-NFR-1..5 PROPOSED, not authorized. ONNX is an external compatibility specification; the R2IL polyfill is the native root; Quack binding + DR64³ + Palette256³ is the intended native representation, with its shipped, lab and missing seams separated. The smallest missing seam is that a Quack-bound query does not know its frame. The #1744 / #1745 / #1750 S0 dispute is not decided. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index d203743b0..4c34fb31a 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,12 @@ +## D-ONNX-NATIVE — ONNX vocabulary over existing native functions (2026-10-23) + +Plan: `.claude/plans/d-onnx-native-v1.md`. Crate: `crates/lance-graph-onnx` (ALIAS / INLINE only; no dispatch). + +| D-id | scope | status | gate / falsifier | +|---|---|---|---| +| **D-ONNX-NATIVE-0** | crosswalk + minimal facade: elementwise float arithmetic, int32 comparisons, Boolean ops, Where, ReduceSum, ReduceSum(Where(Greater)) | IMPLEMENTED, draft PR (`entries/2026-10-23-d-onnx-native-0-facade.md`) | 11 conformance tests + doctest; 4 disable runs red on their own test; alias = native symbol; asm diff | +| **D-ONNX-NATIVE-1** | the NATIVE rows: approximate transcendentals, float ReduceSum tolerance, empty-population min/max | PROPOSAL | each row names its open decision (plan §2) | + ## D-NFR / D-NFC — native representation contract + dependency correction (2026-10-23) Docs only. Plans `d-native-repr-contract-v1.md` (D-NFR) and `d-native-first-dependency-correction-v1.md` (D-NFC). Nothing implemented; no ABI or execution contract changed. diff --git a/.claude/board/TECH_DEBT.md b/.claude/board/TECH_DEBT.md index 9a8bb6a70..893433f99 100644 --- a/.claude/board/TECH_DEBT.md +++ b/.claude/board/TECH_DEBT.md @@ -24,7 +24,7 @@ ## TD-BGZ-TENSOR-INTEGRATION-TESTS-UNGATED-1 (2026-10-23) — OPEN -CI runs `bgz-tensor` with `--lib` only (`.github/workflows/rust-test.yml:274`), +CI runs `bgz-tensor` with `--lib` only (`.github/workflows/rust-test.yml`, step "Run bgz-tensor tests (workspace-excluded, ndarray sibling)"), so nothing under `crates/bgz-tensor/tests/` runs in CI. The crate is workspace-excluded, so no other step reaches it. diff --git a/.claude/board/entries/2026-10-23-d-onnx-native-0-facade.md b/.claude/board/entries/2026-10-23-d-onnx-native-0-facade.md new file mode 100644 index 000000000..1cdb2235a --- /dev/null +++ b/.claude/board/entries/2026-10-23-d-onnx-native-0-facade.md @@ -0,0 +1,12 @@ +# D-ONNX-NATIVE-0 — ONNX names over existing native functions + +**Date:** 2026-10-23 · **Status:** IMPLEMENTED (draft PR, not merged) · **Plan:** `.claude/plans/d-onnx-native-v1.md` + +`crates/lance-graph-onnx` exposes ONNX operator names over existing native functions. It has no dispatcher, no graph and no initialization, and its only normal dependency is `ndarray`. No native crate depends on it. + +- **ALIAS** (same symbol, pinned by `fn_addr_eq`): `Abs`, `Sqrt` (vml); `Greater`/`GreaterOrEqual`/`Less`/`LessOrEqual`/`Equal` on int32 vs a scalar; `And`/`Or`/`Xor`/`Not` on packed masks; `Where` int32. +- **INLINE**: float `Add`/`Sub`/`Mul`/`Div` (a broadcast check turns ndarray's panic into a refusal); `Neg`; int32 `ReduceSum` (attributes validated once, plus an overflow refusal); `ReduceSum(Where(Greater))`, the #1752 composite. +- **MEASURED** (ns per call, reported not gated): Greater direct = onnx (14.6 / 14.6 at 64 rows, 0 allocs). The composite is 38.6 direct vs 39.3 onnx vs 224 for the #1752 wrapper (3 allocs). In the asm, the composite's extra instructions are the int32 range check only. +- **FOUND:** `vml::vsneg` returns `+0` for `Neg(+0)` when n ≥ 16 (`0.0 - v` in the SIMD body), so `Neg` is INLINE, not ALIAS. `vml::vsexp` saturates +inf. Both are reported to the ndarray owner, not fixed here. +- **REFUSED:** integer arithmetic (overflow undefined), tensor-vs-tensor compares, MatMul, control flow, protobuf import, automatic Palette substitution. +- **OPEN:** independent review; the NATIVE rows in plan §2 (approximate transcendentals, float ReduceSum tolerance, empty-population min/max). diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index d43e6eb05..d36a8cb8e 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -459 entries, 2026-08-06 .. 2026-10-23. +460 entries, 2026-08-06 .. 2026-10-23. | date | entry id | finding | file | |---|---|---|---| @@ -39,6 +39,7 @@ exactly one of them, never both. | 2026-10-23 | `D-TGA-5` | | [2026-10-23-d-tga-5-squash-gate-preregistration.md](2026-10-23-d-tga-5-squash-gate-preregistration.md) | | 2026-10-23 | `D-ONNX-RELALG-CAP` | S0 result: ONNX as import membrane, DR64³ plan not found | [2026-10-23-d-onnx-relalg-cap-s0.md](2026-10-23-d-onnx-relalg-cap-s0.md) | | 2026-10-23 | `E-D-ONNX-RELALG-CAP-S0-1` | | [2026-10-23-d-onnx-relalg-cap-s0-capstone-1750.md](2026-10-23-d-onnx-relalg-cap-s0-capstone-1750.md) | +| 2026-10-23 | `D-ONNX-NATIVE-0` | | [2026-10-23-d-onnx-native-0-facade.md](2026-10-23-d-onnx-native-0-facade.md) | | 2026-10-23 | `D-NFR-0` | | [2026-10-23-d-nfr-0-native-repr-contract.md](2026-10-23-d-nfr-0-native-repr-contract.md) | | 2026-10-23 | `D-MPF-4` | | [2026-10-23-d-mpf-4-classid-route.md](2026-10-23-d-mpf-4-classid-route.md) | | 2026-10-23 | `D-MPF-2` | | [2026-10-23-d-mpf-2-fixed-cost.md](2026-10-23-d-mpf-2-fixed-cost.md) | diff --git a/.claude/plans/d-onnx-native-v1.md b/.claude/plans/d-onnx-native-v1.md new file mode 100644 index 000000000..18f582320 --- /dev/null +++ b/.claude/plans/d-onnx-native-v1.md @@ -0,0 +1,134 @@ +# D-ONNX-NATIVE — ONNX vocabulary over existing native functions + +**Status:** D-ONNX-NATIVE-0 IMPLEMENTED as a draft (`crates/lance-graph-onnx`). It does not change the production ABI and has not been merged. Everything past §5 is a proposal. + +**Rule:** ONNX is a vocabulary over our existing mathematics, not another execution system. The path is `ONNX-compatible Rust API → existing native function` and nothing else: no graph, AST, interpreter, VM, runtime dispatcher, `LazyLock`/`OnceLock`, ClassId routing or per-call backend choice. + +**Scope boundaries:** +- R2IL routing (D-MPF-4, #1752) is a separate architecture, and this crate does not import it. +- `ndarray` owns SIMD and the kernels, and it does not depend on this crate. +- Quack, DR64³ and Palette256³ are not recreated here. + +## 1. Categories + +| category | meaning | cost contract | +|---|---|---| +| **ALIAS** | The native function already has the ONNX contract, so it is re-exported under the ONNX name. | The same symbol: zero added instructions, zero allocations. | +| **INLINE** | A small adapter does only the semantic work the native function lacks, then calls it. | Only that work is added (§4). | +| **NATIVE** | The mathematics exists natively, but it needs a composition or a decision that is not made yet. | Not exported. | +| **REFUSE** | The ONNX semantics differ from what we have, or would need an execution system. | Not exported. Runtime facts are refused as `Refusal` values. | + +## 2. Crosswalk (source-backed) + +Every ALIAS and INLINE row was checked against the native source. The checks covered dtype, rank, broadcasting, axes, overflow, rounding, empty tensors, output shape and the bool representation. + +**Bool convention:** ONNX `tensor(bool)` is the native packed mask. Bit `i % 64` of word `i / 64` is element `i`, and the bits past the last element are zero. `mask_words(n) = n.div_ceil(64)`. + +### ALIAS + +| ONNX | native | checks | +|---|---|---| +| `Abs` (f32) | `ndarray::hpc::vml::vsabs` | IEEE exact on ±0, ±inf and NaN, for n = 0..70 (SIMD body and tail) | +| `Sqrt` (f32) | `ndarray::hpc::vml::vssqrt` | IEEE exact (correctly rounded sqrt); `sqrt(-x) = NaN`; `sqrt(-0) = -0` | +| `Greater`, `GreaterOrEqual`, `Less`, `LessOrEqual`, `Equal` (int32 tensor vs a scalar) | `ndarray::simd::{gt,ge,lt,le,eq}_i32_to_mask` | signed comparison; `i32::MIN`/`MAX` thresholds; n ∈ {0,1,15,16,17,63,64,65,130}; tail bits zero | +| `And`, `Or`, `Xor` (bool) | `ndarray::simd::mask_{and,or,xor}` | word-wise and exact; zero tails stay zero | +| `Not` (bool, n elements) | `ndarray::simd::mask_not(src, n, dst)` | clears the tail bits past n | +| `Where` (int32, equal shapes) | `ndarray::simd::blend_i32` | a set bit takes X, a clear bit takes Y; native asserts equal lengths | + +`aliases_are_the_native_symbols` pins each alias by `fn_addr_eq`, so an alias cannot quietly become a wrapper. That was the D4 disable. + +### INLINE + +| ONNX | native call | added work and reason | +|---|---|---| +| `Add`/`Sub`/`Mul`/`Div` (f32, f64; multidirectional broadcast) | `ndarray` `&a op &b` | `broadcastable(shapes)`. ndarray broadcasts like numpy but **panics** on a mismatch (`broadcast_with(..).unwrap()`; `co_broadcast` is `pub(crate)`). The adapter turns that panic into `Refusal::Broadcast`. The arithmetic is IEEE in both. `simd::add_f32` and its siblings were **not** used: they truncate to `min(len)`, which is not ONNX. | +| `Neg` (f32, f64) | `ndarray` element map `-v` (the body of its `Neg` impl) | none. It is INLINE, not ALIAS, because the obvious alias `vml::vsneg` is wrong: its 16-lane body computes `0.0 - v`, so `Neg(+0) = +0` for n ≥ 16, while its scalar tail computes `-v`. Pinned by `vml_vsneg_is_not_onnx_neg_on_positive_zero`. | +| `ReduceSum` (int32, rank 1) | iterator `i64` sum | attribute validation at construction, once; an exact `i64` sum; an `int32` range check (`Refusal::OutOfDtypeRange`). ONNX does not define integer overflow, so the adapter refuses rather than wraps. There is no native unmasked `i32 → i64` sum (§6). | +| `ReduceSum(Where(Greater(X, T), X, 0))` (int32) | `gt_i32_to_mask` + `masked_sum_i32` | the range check only. Scratch comes from the caller, so there is no allocation. This is the #1752 composition. | + +`ReduceSum` attributes are checked once in `ReduceSum::new(axes, keepdims, noop_with_empty_axes)`, never per call: +- `axes` ∈ {absent, `[0]`, `[-1]`} for rank 1; +- `keepdims` ∈ {0, 1}; +- `noop_with_empty_axes = 1` with absent axes is refused, because it returns a tensor, not a scalar. + +### NATIVE (exists, not exported yet) + +| ONNX | native | why not exported | +|---|---|---| +| `Exp`, `Log`, `Tanh`, `Sigmoid` | `vml::vsexp` and the related vml functions | approximate (polynomial). On ndarray master `vsexp` returns 1.7e38 for +inf, halves every result above x ≈ 88.38 and returns ~1.2e-38 for −inf (fixed in AdaWorldAPI/ndarray#368, open). Exact and approximate stay separate, so these would need an explicitly named approximate surface with ULP bounds. | +| `ReduceSum` (float) | `ArrayBase::sum` | it reassociates (`unrolled_fold`). That is allowed under ONNX tolerance but is not bit-identical to sequential order, so the tolerance contract has to be stated first. | +| `ReduceMin`/`ReduceMax` of a masked population | `masked_min_i32`/`masked_max_i32` | the empty-population result is undecided (ONNX ReduceMin over an empty set is ±inf for floats and undefined for ints) | +| comparisons on u64 | `{lt,le,gt,ge}_u64_to_mask` | ONNX has no unsigned-64 compare-with-scalar distinct from int64; needs a dtype decision | + +### REFUSE + +| ONNX | reason | +|---|---| +| integer `Add`/`Sub`/`Mul` | overflow is undefined in ONNX. ndarray `+` wraps in release and panics in debug. Needs the G1 overflow decision. | +| tensor-vs-tensor comparisons; f32 comparisons to a mask | no native kernel. Writing one here would duplicate a SIMD kernel. | +| `MatMul`/`Gemm` | out of scope for D-ONNX-NATIVE-0. The CompiledLinear identity defect has to be resolved first. | +| `Gather`, `Scatter`, `If`, `Loop`, control flow | these need graph or execution semantics, which this crate does not have | +| ONNX protobuf import | out of scope. It would be a separate optional crate. | +| Palette substitution for any op | approximate. Never automatic. | + +## 3. Cross-check against relalg and mask-risc + +The composite answer is checked against four independent computations in one test (`composite_matches_the_oracle_mask_risc_r2il_and_relalg`): + +1. a plain-iterator oracle; +2. a `lance-graph-mask-risc` program, `Pred::GtI32` + `Terminal::MaskedSumI32` via `execute_into`; +3. `lance-graph-r2il` `FoldDialect` (`LOAD`, `INT_S_LESS`, `procs::sum`); +4. `lance-graph-relalg` `run(Query{ rel: Atom::I32{ op: Gt, .. }, measure: Sum })`. + +All four are dev-dependencies only. None is on the facade's call path. + +## 4. Measurements and assembly (reported, not gated) + +Setup: `examples/alias_cost.rs`, release, AVX-512 at compile time, load ≈ 0.1–0.4. Medians over 21 reps, in ns per call; 3 runs agree. + +| rows | op | direct | onnx | #1752 wrapper (diagnostic) | allocs d/o/w | +|---:|---|---:|---:|---:|---| +| 64 | Greater (ALIAS) | 14.6 | 14.6 | — | 0/0 | +| 64 | ReduceSum(Where(Greater)) (INLINE) | 38.6 | 39.3 | 224.4 | 0/0/3 | +| 64 | Add f32 (INLINE) | 34.8 | 35.6 | — | 1/1 | +| 4096 | Greater | 311.1 | 310.7 | — | 0/0 | +| 4096 | ReduceSum(Where(Greater)) | 1807.9 | 1807.3 | 2047.1 | 0/0/3 | +| 4096 | Add f32 | 362.3 | 382.7 | — | 1/1 | +| 65536 | Greater | 6004.8 | 5796.6 | — | 0/0 | +| 65536 | ReduceSum(Where(Greater)) | 30531.1 | 28112.2 | 29773.5 | 0/0/3 | +| 65536 | Add f32 | 8003.4 | 8465.6 | — | 1/1 | + +At 65536 rows the direct/onnx differences are run-to-run noise (other runs reverse them). Add's single allocation is the output array, the same as direct `&a + &b`. + +The assembly is from the out-of-line `probe_*` functions (`cargo rustc --release --example alias_cost -- --emit=asm`): +- **Greater (ALIAS):** `fn_addr_eq(greater_i32, gt_i32_to_mask)` is true, so there is no wrapper symbol at all. +- **ReduceSum(Where(Greater)):** with `#[inline]`, the onnx probe is 163 instructions against direct's 147. Diffing the bodies, the extra instructions are the `int32` range check (`movslq`/`cmp`) and the `Result` write. Both probes carry the same `memset` (mask zero-fill in `pack`), `memcpy` and bounds-check panics. Without `#[inline]`, the onnx probe was a single tail `jmp` into the adapter. +- **Add:** direct is a 4-instruction tail jump into ndarray's `Add for &ArrayRef`. The onnx probe inlines `broadcastable` (a shape loop plus the cold `to_vec` allocation on the refusal path) and then makes the same ndarray call. +- **Scan for dispatch:** no `Lazy`, `Once`, `cpuid` or feature probe appears in any onnx probe or in `src/`. + +The old 50 ns target does not apply to aliases. An alias's cost is the native function's cost. + +## 5. Dependency audit + +- `lance-graph-onnx`'s only normal dependency is `ndarray`, on the same git coordinate as `lance-graph-mask-risc`. The root `[patch]` unifies it with the sibling checkout, so the workspace holds one ndarray. `cargo tree -p lance-graph-onnx -e normal --depth 1` shows exactly that. +- Reverse: `cargo tree -i lance-graph-onnx -e normal` lists only the crate itself. No manifest under `crates/` names it. The root `Cargo.toml` names it only as a workspace member. The ndarray sibling has no reference to it. +- The relalg, mask-risc, r2il and ogar-loco crates appear only under `[dev-dependencies]`, for the cross-checks and the diagnostic baseline. + +## 6. Proposed (not implemented) + +- An ndarray owner fix for `vml::vsneg` (+0) and `vml::vsexp` (±inf, overflow halving): AdaWorldAPI/ndarray#368, open. After it lands, `Neg` could become an ALIAS. +- A native unmasked `i32 → i64` sum in `ndarray::simd`, so that `ReduceSum::sum_i32` becomes a direct call. +- The NATIVE rows in §2, each with its open decision named. +- `#1752` `onnx_compat` stays as the independent probe. This crate is the facade, and #1752's measurements are kept as the diagnostic baseline. + +## 7. Gates + +- [x] ALIAS = native symbol (`fn_addr_eq`), zero allocations (measured) +- [x] INLINE adds only its semantic work (asm diff) +- [x] no dispatch, no initialization, no per-loop decision (source grep + asm) +- [x] agreement with independent references (11 tests + 1 doctest) +- [x] explicit refusals (broadcast, axes, keepdims, noop, overflow) +- [x] 4 disable runs, each red on its own test only +- [x] clippy `-D warnings`, fmt +- [ ] independent review +- [ ] operator authorization before merge diff --git a/.github/workflows/rust-test.yml b/.github/workflows/rust-test.yml index 45af511c6..69d573ec3 100644 --- a/.github/workflows/rust-test.yml +++ b/.github/workflows/rust-test.yml @@ -187,6 +187,14 @@ jobs: run: cargo clippy --manifest-path crates/lance-graph-r2il/Cargo.toml --all-targets -- -D warnings - name: Run r2il-mask-abi-probe mask-ABI differential (excluded tier, OGAR + ndarray siblings) run: cargo test --manifest-path crates/r2il-mask-abi-probe/Cargo.toml + # lance-graph-onnx (D-ONNX-NATIVE-0): ONNX names over existing native + # functions. A workspace member, and `cargo build --workspace` does not + # run a member's tests. Its conformance tests cross-check against + # mask-risc, r2il (OGAR sibling) and relalg, so it runs here beside them. + - name: Run lance-graph-onnx conformance tests + run: cargo test --release --manifest-path crates/lance-graph-onnx/Cargo.toml + - name: Clippy lance-graph-onnx + run: cargo clippy --manifest-path crates/lance-graph-onnx/Cargo.toml --all-targets -- -D warnings # deepnsm: standalone 0-dep codec crate, workspace-excluded, so the # lance-graph test steps above never reached it. ~217 lib + integration + # doctests, fast (no lance/datafusion/ndarray deps). Gating. diff --git a/Cargo.toml b/Cargo.toml index 049d569b9..369cc5ad9 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -13,6 +13,9 @@ members = [ # `ogar-r2il`). The contract may never depend on OGAR (`ogar-loco` depends on # the contract), but members other than the contract may. "crates/lance-graph-r2il", + # ONNX mathematical compatibility facade over existing ndarray functions + # (D-ONNX-NATIVE-0). Depends on ndarray only; nothing native depends on it. + "crates/lance-graph-onnx", # EXPERIMENTAL, routed nowhere: Cypher (upstream front end, read-only) → # bound LogicalOperator → Quack, with typed refusals. "crates/lance-graph-cypher-quack", diff --git a/crates/lance-graph-onnx/Cargo.toml b/crates/lance-graph-onnx/Cargo.toml new file mode 100644 index 000000000..beb2fe2ee --- /dev/null +++ b/crates/lance-graph-onnx/Cargo.toml @@ -0,0 +1,21 @@ +[package] +name = "lance-graph-onnx" +version = "0.1.0" +edition = "2021" +publish = false +description = "ONNX mathematical compatibility facade: ONNX operator names and semantics over existing native ndarray functions. Aliases where the contract is identical, minimal adapters where it is not, typed refusals otherwise. No graph, interpreter, dispatcher or runtime initialization." + +[dependencies] +# The one native dependency, on the coordinate lance-graph-mask-risc uses. The +# workspace root's `[patch."https://github.com/AdaWorldAPI/ndarray"]` points it +# at the sibling checkout, so the workspace holds one ndarray package. +ndarray = { git = "https://github.com/AdaWorldAPI/ndarray", branch = "master", default-features = false, features = ["std"] } + +# Cross-checks and the diagnostic baseline only. The facade itself never calls +# them: the normal path is "ONNX-compatible Rust API -> existing native function". +[dev-dependencies] +lance-graph-mask-risc = { path = "../lance-graph-mask-risc" } +lance-graph-r2il = { path = "../lance-graph-r2il" } +lance-graph-relalg = { path = "../lance-graph-relalg" } +# The trait that carries `FoldDialect::call`, on the coordinate lance-graph-r2il uses. +ogar-loco = { git = "https://github.com/AdaWorldAPI/OGAR", branch = "main" } diff --git a/crates/lance-graph-onnx/examples/alias_cost.rs b/crates/lance-graph-onnx/examples/alias_cost.rs new file mode 100644 index 000000000..4bf04ea14 --- /dev/null +++ b/crates/lance-graph-onnx/examples/alias_cost.rs @@ -0,0 +1,210 @@ +//! D-ONNX-NATIVE-0: what the facade costs over the native call. Reported, +//! not gated. +//! +//! Arms, same inputs, outputs and CPU configuration: +//! - `direct`: the native function, called by its ndarray name. +//! - `onnx`: the facade under its ONNX name (the same symbol for an ALIAS). +//! - `wrapper (#1752)`: diagnostic baseline only. It reproduces the #1752 +//! `onnx_compat::reduce_sum_where_greater` flow: attributes checked and a +//! `FoldDialect` built on every call. +//! +//! Every arm is checked against the oracle before timing. Allocations per call +//! are counted by a global allocator wrapper. `ONNX_REPS` (default 21) and +//! `ONNX_ITERS` (default 4000) set the sampling. The `probe_*` functions are +//! out of line so their assembly can be read on its own: +//! `cargo rustc --release -p lance-graph-onnx --example alias_cost -- --emit=asm`. + +use std::alloc::{GlobalAlloc, Layout, System}; +use std::hint::black_box; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::time::Instant; + +use lance_graph_mask_risc::{Foreign, LaneRef, Planes}; +use lance_graph_onnx as onnx; +use lance_graph_r2il::{FoldDialect, Val, INT_S_LESS, LOAD}; +use ndarray::Array1; +use ogar_loco::Dialect; + +struct Counting; +static ALLOCS: AtomicUsize = AtomicUsize::new(0); + +// SAFETY: forwards every call to `System` unchanged; only counts. +unsafe impl GlobalAlloc for Counting { + unsafe fn alloc(&self, l: Layout) -> *mut u8 { + ALLOCS.fetch_add(1, Ordering::Relaxed); + // SAFETY: same contract as the caller's. + unsafe { System.alloc(l) } + } + unsafe fn dealloc(&self, p: *mut u8, l: Layout) { + // SAFETY: same contract as the caller's. + unsafe { System.dealloc(p, l) } + } +} + +#[global_allocator] +static A: Counting = Counting; + +fn env(name: &str, d: usize) -> usize { + std::env::var(name) + .ok() + .and_then(|v| v.parse().ok()) + .unwrap_or(d) +} + +/// Median ns per call, plus heap allocations per call. +fn measure(reps: usize, iters: usize, mut f: impl FnMut()) -> (f64, f64) { + for _ in 0..100 { + f(); + } + let before = ALLOCS.load(Ordering::Relaxed); + for _ in 0..100 { + f(); + } + let allocs = (ALLOCS.load(Ordering::Relaxed) - before) as f64 / 100.0; + let mut v: Vec = (0..reps) + .map(|_| { + let t = Instant::now(); + for _ in 0..iters { + f(); + } + t.elapsed().as_nanos() as f64 / iters as f64 + }) + .collect(); + v.sort_by(f64::total_cmp); + (v[v.len() / 2], allocs) +} + +// ── out-of-line call shapes for the assembly reading ────────────────────── + +#[inline(never)] +pub fn probe_direct_add(a: &Array1, b: &Array1) -> Array1 { + a + b +} + +#[inline(never)] +pub fn probe_onnx_add(a: &Array1, b: &Array1) -> Result, onnx::Refusal> { + onnx::add(a, b) +} + +#[inline(never)] +pub fn probe_direct_sum_where(x: &[i32], t: i32, m: &mut [u64]) -> i64 { + ndarray::simd::gt_i32_to_mask(x, t, m); + ndarray::simd::masked_sum_i32(x, m) +} + +#[inline(never)] +pub fn probe_onnx_sum_where( + rs: onnx::ReduceSum, + x: &[i32], + t: i32, + m: &mut [u64], +) -> Result { + rs.sum_where_greater_i32(x, t, m) +} + +/// The #1752 wrapper's work per call: attribute checks, a dialect, the +/// procedure, the range check. Diagnostic only. +fn wrapper_1752(x: &[i32], t: i32, keepdims: i64) -> Option { + if keepdims != 0 && keepdims != 1 { + return None; + } + let lanes = [LaneRef::I32(x)]; + let planes = Planes { + n_rows: x.len(), + masks: &[], + lanes: &lanes, + }; + let mut d = FoldDialect::new(&planes, &Foreign::NONE); + let mut stack = vec![Val::Scalar(i64::from(t)), Val::Scalar(0)]; + d.call(LOAD, [1, 0, 0], &mut stack).ok()?; + d.call(INT_S_LESS, [0; 3], &mut stack).ok()?; + lance_graph_r2il::procs::sum(&mut d, &mut stack, 0).ok()?; + match stack.pop() { + Some(Val::Scalar(s)) => i32::try_from(s).ok(), + _ => None, + } +} + +fn main() { + let reps = env("ONNX_REPS", 21); + let iters0 = env("ONNX_ITERS", 4000); + println!( + "alias identity: greater_i32 == gt_i32_to_mask: {}", + std::ptr::fn_addr_eq( + onnx::greater_i32 as fn(&[i32], i32, &mut [u64]), + ndarray::simd::gt_i32_to_mask as fn(&[i32], i32, &mut [u64]) + ) + ); + println!( + "avx512f compile-time: {}\n", + cfg!(target_feature = "avx512f") + ); + println!("| rows | op | direct | onnx | wrapper (#1752) | allocs direct/onnx/wrapper |"); + println!("|---:|---|---:|---:|---:|---|"); + for n in [64usize, 4_096, 65_536] { + let iters = (iters0 * 64 / n).max(50); + let x: Vec = (0..n).map(|i| ((i * 13) % 97) as i32 - 48).collect(); + let t = 0; + let want: i64 = x.iter().filter(|&&v| v > t).map(|&v| i64::from(v)).sum(); + let mut m = vec![0u64; onnx::mask_words(n)]; + let rs = onnx::ReduceSum::new(None, 0, 0).unwrap(); + assert_eq!(probe_direct_sum_where(&x, t, &mut m), want); + assert_eq!( + probe_onnx_sum_where(rs, &x, t, &mut m).unwrap().value as i64, + want + ); + assert_eq!(wrapper_1752(&x, t, 0).map(i64::from), Some(want)); + + // Greater: an ALIAS, so both arms call one symbol. + let g = [ + measure(reps, iters, || { + ndarray::simd::gt_i32_to_mask(black_box(&x), black_box(t), &mut m) + }), + measure(reps, iters, || { + onnx::greater_i32(black_box(&x), black_box(t), &mut m) + }), + ]; + println!( + "| {n} | Greater (ALIAS) | {:.1} | {:.1} | — | {:.0}/{:.0}/— |", + g[0].0, g[1].0, g[0].1, g[1].1 + ); + + let s = [ + measure(reps, iters, || { + black_box(probe_direct_sum_where(black_box(&x), black_box(t), &mut m)); + }), + measure(reps, iters, || { + let _ = black_box(probe_onnx_sum_where( + rs, + black_box(&x), + black_box(t), + &mut m, + )); + }), + measure(reps, iters, || { + black_box(wrapper_1752(black_box(&x), black_box(t), 0)); + }), + ]; + println!( + "| {n} | ReduceSum(Where(Greater)) (INLINE) | {:.1} | {:.1} | {:.1} | {:.0}/{:.0}/{:.0} |", + s[0].0, s[1].0, s[2].0, s[0].1, s[1].1, s[2].1 + ); + + let a = Array1::from_iter((0..n).map(|i| i as f32 * 0.5)); + let b = Array1::from_iter((0..n).map(|i| 1.0 - i as f32)); + assert_eq!(probe_onnx_add(&a, &b).unwrap(), probe_direct_add(&a, &b)); + let ad = [ + measure(reps, iters, || { + black_box(probe_direct_add(black_box(&a), black_box(&b))); + }), + measure(reps, iters, || { + let _ = black_box(probe_onnx_add(black_box(&a), black_box(&b))); + }), + ]; + println!( + "| {n} | Add f32 (INLINE) | {:.1} | {:.1} | — | {:.0}/{:.0}/— |", + ad[0].0, ad[1].0, ad[0].1, ad[1].1 + ); + } + println!("\nns per call, median of ONNX_REPS. Reported, not gated."); +} diff --git a/crates/lance-graph-onnx/src/lib.rs b/crates/lance-graph-onnx/src/lib.rs new file mode 100644 index 000000000..b9a313f87 --- /dev/null +++ b/crates/lance-graph-onnx/src/lib.rs @@ -0,0 +1,276 @@ +//! `lance-graph-onnx` — ONNX mathematical compatibility over existing native +//! functions (D-ONNX-NATIVE-0, `.claude/plans/d-onnx-native-v1.md`). +//! +//! ONNX is a vocabulary over our existing mathematics, not another execution +//! system. The normal path is +//! +//! ```text +//! ONNX-compatible Rust API → existing native function +//! ``` +//! +//! and nothing more. This crate has no graph, AST, interpreter, virtual +//! machine, runtime dispatcher, `LazyLock`/`OnceLock`, ClassId routing or +//! per-call backend choice. It depends on `ndarray` only, and no native crate +//! depends on it. +//! +//! Each operator is one of: +//! +//! - **ALIAS**: the native function has the ONNX contract. It is re-exported +//! under the ONNX name; the symbol is the native one. +//! - **INLINE**: a small adapter does only the semantic work the native +//! function does not (a shape check before broadcasting, an `int32` range +//! check), then calls it. +//! - Anything else has no function here. The crosswalk in the plan lists each +//! such operator as NATIVE (a native composition still to be built) or REFUSE. +//! +//! # Conventions +//! +//! - **Tensors** are `ndarray` arrays or slices. Rank-1 `int32` operators take +//! `&[i32]`, as the native kernels do. +//! - **`tensor(bool)`** is a packed mask: bit `i % 64` of word `i / 64` is +//! element `i`, and bits past the last element are zero. That is the +//! representation of every `ndarray::simd` mask kernel. Use [`mask_words`] +//! to size one. +//! - **Exact versus approximate**: every operator here is exact (IEEE-754 for +//! floats, exact integers). Polynomial `Exp`/`Log`/`Tanh`/`Sigmoid` and +//! palette operations are approximate and are not exported. +//! +//! # Example +//! +//! ``` +//! use lance_graph_onnx as onnx; +//! +//! // Greater(X, 0) → tensor(bool), then ReduceSum(Where(·, X, 0)). +//! let x = [3, -1, 4, -1, 5]; +//! let mut cond = vec![0u64; onnx::mask_words(x.len())]; +//! onnx::greater_i32(&x, 0, &mut cond); +//! assert_eq!(cond[0], 0b10101); +//! +//! let rs = onnx::ReduceSum::new(None, 0, 0).unwrap(); +//! let mut scratch = vec![0u64; onnx::mask_words(x.len())]; +//! assert_eq!(rs.sum_where_greater_i32(&x, 0, &mut scratch), Ok(onnx::Reduced { value: 12, keepdims: false })); +//! ``` + +use std::ops::{Add, Div, Mul, Neg, Sub}; + +use ndarray::{Array, ArrayBase, ArrayRef, Data, DimMax, Dimension}; + +// ── ALIAS: identical contracts, re-exported under the ONNX name ──────────── + +/// `Abs` (`float`). Alias of `ndarray::hpc::vml::vsabs`. +pub use ndarray::hpc::vml::vsabs as abs_f32; +/// `Sqrt` (`float`). Alias of `ndarray::hpc::vml::vssqrt`. +pub use ndarray::hpc::vml::vssqrt as sqrt_f32; + +/// `Equal(X, T)`, scalar `T`. Alias of `ndarray::simd::eq_i32_to_mask`. +pub use ndarray::simd::eq_i32_to_mask as equal_i32; +/// `GreaterOrEqual(X, T)`, scalar `T`. Alias of `ndarray::simd::ge_i32_to_mask`. +pub use ndarray::simd::ge_i32_to_mask as greater_or_equal_i32; +/// `Greater(X, T)`, `int32` `X`, scalar `T`. Alias of `ndarray::simd::gt_i32_to_mask`. +pub use ndarray::simd::gt_i32_to_mask as greater_i32; +/// `LessOrEqual(X, T)`, scalar `T`. Alias of `ndarray::simd::le_i32_to_mask`. +pub use ndarray::simd::le_i32_to_mask as less_or_equal_i32; +/// `Less(X, T)`, scalar `T`. Alias of `ndarray::simd::lt_i32_to_mask`. +pub use ndarray::simd::lt_i32_to_mask as less_i32; + +/// `Where(C, X, Y)`, `int32`, equal shapes. Alias of `ndarray::simd::blend_i32` +/// (a set bit takes `X`). +pub use ndarray::simd::blend_i32 as where_i32; +/// `And` on `tensor(bool)`. Alias of `ndarray::simd::mask_and`. +pub use ndarray::simd::mask_and as and; +/// `Not` on `tensor(bool)` of `n` elements. Alias of `ndarray::simd::mask_not`. +pub use ndarray::simd::mask_not as not; +/// `Or` on `tensor(bool)`. Alias of `ndarray::simd::mask_or`. +pub use ndarray::simd::mask_or as or; +/// `Xor` on `tensor(bool)`. Alias of `ndarray::simd::mask_xor`. +pub use ndarray::simd::mask_xor as xor; + +/// Words of a packed `tensor(bool)` of `n` elements. +pub const fn mask_words(n: usize) -> usize { + n.div_ceil(64) +} + +// ── refusals ─────────────────────────────────────────────────────────────── + +/// Why an operator does not answer. Unsupported dtypes are refused by type +/// (there is no function for them), so only runtime facts appear here. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Refusal { + /// The shapes are not multidirectionally broadcastable. + Broadcast { lhs: Vec, rhs: Vec }, + /// `axes` other than absent, `[0]` or `[-1]` for a rank-1 input. + Axes(Vec), + /// `keepdims` other than 0 or 1. + KeepDims(i64), + /// `noop_with_empty_axes = 1` with absent axes returns the input + /// unchanged, a tensor result outside the scalar reductions here. + NoopWithEmptyAxes, + /// The exact result does not fit `int32`. ONNX does not define integer + /// overflow; this crate refuses instead of wrapping. + OutOfDtypeRange(i64), +} + +// ── INLINE: element-wise float arithmetic with broadcasting ──────────────── + +/// The float types whose arithmetic is exact IEEE-754 in both ONNX and +/// `ndarray`. Integer arithmetic is not admitted: ONNX leaves overflow +/// undefined and `ndarray`'s `+` wraps in release and panics in debug. +pub trait Ieee: + Copy + + Add + + Sub + + Mul + + Div + + Neg + + sealed::Sealed +{ +} +impl Ieee for f32 {} +impl Ieee for f64 {} +mod sealed { + pub trait Sealed {} + impl Sealed for f32 {} + impl Sealed for f64 {} +} + +/// ONNX multidirectional (numpy) broadcasting: shapes align from the right, +/// and each aligned pair is equal or contains a 1. `ndarray`'s arithmetic +/// broadcasts the same way but panics on a mismatch, so this check turns the +/// panic into a [`Refusal`]. It reads only the two shapes. +#[inline] +pub fn broadcastable(lhs: &[usize], rhs: &[usize]) -> Result<(), Refusal> { + let ok = lhs + .iter() + .rev() + .zip(rhs.iter().rev()) + .all(|(&a, &b)| a == b || a == 1 || b == 1); + if ok { + Ok(()) + } else { + Err(Refusal::Broadcast { + lhs: lhs.to_vec(), + rhs: rhs.to_vec(), + }) + } +} + +macro_rules! binary { + ($(#[$doc:meta])* $name:ident, $op:tt) => { + $(#[$doc])* + #[inline] + pub fn $name( + a: &ArrayRef, + b: &ArrayRef, + ) -> Result>::Output>, Refusal> + where + T: Ieee, + D: Dimension + DimMax, + E: Dimension, + { + broadcastable(a.shape(), b.shape())?; + Ok(a $op b) + } + }; +} + +binary!( + /// `Add`. INLINE: [`broadcastable`], then `ndarray`'s `&a + &b`. + add, + +); +binary!( + /// `Sub`. INLINE: [`broadcastable`], then `ndarray`'s `&a - &b`. + sub, - +); +binary!( + /// `Mul`. INLINE: [`broadcastable`], then `ndarray`'s `&a * &b`. + mul, * +); +binary!( + /// `Div` (float). INLINE: [`broadcastable`], then `ndarray`'s `&a / &b`. + /// Division by zero follows IEEE-754 (`±inf`, `NaN`) in both. + div, / +); + +/// `Neg`. INLINE: `ndarray`'s element map with `-v`, the same body as its +/// `Neg for &ArrayRef` impl. +/// +/// Not an alias of `ndarray::hpc::vml::vsneg`: its 16-lane body computes +/// `0.0 - v`, so `Neg(+0.0)` gives `+0.0` instead of `-0.0` whenever the input +/// has at least 16 elements (its scalar tail computes `-v` and is correct). +#[inline] +pub fn neg, D: Dimension>(a: &ArrayBase) -> Array { + a.map(|&v| -v) +} + +// ── INLINE: ReduceSum, attributes validated once ─────────────────────────── + +/// `ReduceSum` over a rank-1 `int32` input, with its attributes validated +/// once. Every reduction axis of a rank-1 input is axis 0, so absent `axes` +/// (with `noop_with_empty_axes = 0`), `[0]` and `[-1]` are the same reduction. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ReduceSum { + keepdims: bool, +} + +/// A rank-1 reduction's result. `keepdims` decides the ONNX output shape: +/// `[1]` when true, `[]` when false. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Reduced { + pub value: i32, + pub keepdims: bool, +} + +impl ReduceSum { + /// Validate `axes`, `keepdims` and `noop_with_empty_axes` (opset 18+, + /// where `axes` is an input). + pub fn new( + axes: Option<&[i64]>, + keepdims: i64, + noop_with_empty_axes: i64, + ) -> Result { + match axes { + None if noop_with_empty_axes == 1 => return Err(Refusal::NoopWithEmptyAxes), + None => {} + Some(a) if a == [0] || a == [-1] => {} + Some(a) => return Err(Refusal::Axes(a.to_vec())), + } + let keepdims = match keepdims { + 0 => false, + 1 => true, + k => return Err(Refusal::KeepDims(k)), + }; + Ok(Self { keepdims }) + } + + #[inline] + fn fit(self, s: i64) -> Result { + let value = i32::try_from(s).map_err(|_| Refusal::OutOfDtypeRange(s))?; + Ok(Reduced { + value, + keepdims: self.keepdims, + }) + } + + /// `ReduceSum(X)`. INLINE: an exact `i64` sum and the `int32` range check. + /// `ndarray` has no unmasked `i32 → i64` sum, so this is a plain iterator + /// reduction, which LLVM vectorizes (plan §4). + #[inline] + pub fn sum_i32(self, x: &[i32]) -> Result { + self.fit(x.iter().map(|&v| i64::from(v)).sum()) + } + + /// `ReduceSum(Where(Greater(X, T), X, 0))`, the composition #1752 probes. + /// INLINE: `ndarray::simd::gt_i32_to_mask` into the caller's `scratch` + /// (at least [`mask_words`]`(x.len())` words), `masked_sum_i32`, then the + /// `int32` range check. No allocation. + #[inline] + pub fn sum_where_greater_i32( + self, + x: &[i32], + t: i32, + scratch: &mut [u64], + ) -> Result { + ndarray::simd::gt_i32_to_mask(x, t, scratch); + self.fit(ndarray::simd::masked_sum_i32(x, scratch)) + } +} diff --git a/crates/lance-graph-onnx/tests/conformance.rs b/crates/lance-graph-onnx/tests/conformance.rs new file mode 100644 index 000000000..e4a004080 --- /dev/null +++ b/crates/lance-graph-onnx/tests/conformance.rs @@ -0,0 +1,404 @@ +//! Conformance: every admitted operator against an independent per-element +//! reference written from the ONNX operator definitions, every refusal, the +//! alias identities, and a cross-check of the composed reduction against +//! mask-risc, the r2il fold reading and relalg. + +// The operator tables below pair a name, the facade function and its +// reference; spelling the tuple types out is the point. +#![allow(clippy::type_complexity)] + +use lance_graph_onnx as onnx; +use lance_graph_onnx::{ReduceSum, Reduced, Refusal}; +use ndarray::{arr0, Array1, Array2, ArrayD, IxDyn}; + +/// Values that separate an exact IEEE operator from a careless one. +const SPECIAL: [f32; 12] = [ + 0.0, + -0.0, + 1.0, + -1.5, + f32::NAN, + f32::INFINITY, + f32::NEG_INFINITY, + f32::MIN_POSITIVE, + 1.0e-40, // subnormal + -1.0e-40, + f32::MAX, + 3.25, +]; + +fn samples(n: usize) -> Vec { + (0..n) + .map(|i| SPECIAL[i % SPECIAL.len()] * if i % 5 == 0 { 7.0 } else { 1.0 }) + .collect() +} + +/// Bit equality, so `-0.0 != 0.0` and NaN payload classes count; any NaN +/// equals any NaN (ONNX does not fix NaN payloads). +fn same(a: f32, b: f32) -> bool { + (a.is_nan() && b.is_nan()) || a.to_bits() == b.to_bits() +} + +#[test] +fn unary_aliases_are_exact_on_special_values_and_every_tail_length() { + for n in 0..70 { + let x = Array1::from(samples(n)); + let mut out = Array1::::zeros(n); + let cases: [( + &str, + fn(ndarray::ArrayView1, ndarray::ArrayViewMut1), + fn(f32) -> f32, + ); 2] = [ + ("Abs", onnx::abs_f32, f32::abs), + ("Sqrt", onnx::sqrt_f32, f32::sqrt), + ]; + for (name, op, reference) in cases { + op(x.view(), out.view_mut()); + for i in 0..n { + assert!( + same(out[i], reference(x[i])), + "{name}({}) = {} at n = {n}", + x[i], + out[i] + ); + } + } + let y = onnx::neg(&x); + for i in 0..n { + assert!(same(y[i], -x[i]), "Neg({}) = {} at n = {n}", x[i], y[i]); + } + } +} + +/// The aliases are the native items, not wrappers around them. +/// Why `Neg` is not an alias: the vml kernel's SIMD body loses the sign of +/// zero. If ndarray fixes it, this test fails and `Neg` can become an alias. +#[test] +fn vml_vsneg_is_not_onnx_neg_on_positive_zero() { + let x = Array1::::zeros(16); + let mut out = Array1::::zeros(16); + ndarray::hpc::vml::vsneg(x.view(), out.view_mut()); + assert_eq!( + out[0].to_bits(), + 0.0f32.to_bits(), + "vsneg(+0) is +0 on the SIMD path" + ); + assert_eq!(onnx::neg(&x)[0].to_bits(), (-0.0f32).to_bits()); +} + +#[test] +fn aliases_are_the_native_symbols() { + type Cmp = fn(&[i32], i32, &mut [u64]); + assert!(std::ptr::fn_addr_eq( + onnx::greater_i32 as Cmp, + ndarray::simd::gt_i32_to_mask as Cmp + )); + assert!(std::ptr::fn_addr_eq( + onnx::less_i32 as Cmp, + ndarray::simd::lt_i32_to_mask as Cmp + )); + assert!(std::ptr::fn_addr_eq( + onnx::equal_i32 as Cmp, + ndarray::simd::eq_i32_to_mask as Cmp + )); + type Bin = fn(&[u64], &[u64], &mut [u64]); + assert!(std::ptr::fn_addr_eq( + onnx::and as Bin, + ndarray::simd::mask_and as Bin + )); + type Un = fn(ndarray::ArrayView1, ndarray::ArrayViewMut1); + assert!(std::ptr::fn_addr_eq( + onnx::abs_f32 as Un, + ndarray::hpc::vml::vsabs as Un + )); +} + +/// Reference ONNX broadcasting: right-aligned, a 1 stretches. +fn reference_binary(a: &ArrayD, b: &ArrayD, f: fn(f32, f32) -> f32) -> ArrayD { + let r = a.ndim().max(b.ndim()); + let pad = |s: &[usize]| { + let mut v = vec![1; r - s.len()]; + v.extend_from_slice(s); + v + }; + let (sa, sb) = (pad(a.shape()), pad(b.shape())); + let out: Vec = sa.iter().zip(&sb).map(|(&x, &y)| x.max(y)).collect(); + let a = a.view().into_shape_with_order(IxDyn(&sa)).unwrap(); + let b = b.view().into_shape_with_order(IxDyn(&sb)).unwrap(); + ArrayD::from_shape_fn(IxDyn(&out), |idx| { + let ia: Vec = (0..r) + .map(|k| if sa[k] == 1 { 0 } else { idx[k] }) + .collect(); + let ib: Vec = (0..r) + .map(|k| if sb[k] == 1 { 0 } else { idx[k] }) + .collect(); + f(a[IxDyn(&ia)], b[IxDyn(&ib)]) + }) +} + +#[test] +fn binary_arithmetic_matches_onnx_broadcasting() { + let shapes: [(&[usize], &[usize]); 7] = [ + (&[37], &[37]), + (&[3, 1], &[4]), + (&[2, 3], &[3]), + (&[], &[5]), + (&[5], &[]), + (&[2, 1, 4], &[3, 1]), + (&[1], &[6, 1]), + ]; + for (sa, sb) in shapes { + let na: usize = sa.iter().product(); + let nb: usize = sb.iter().product(); + let a = ArrayD::from_shape_vec(IxDyn(sa), samples(na)).unwrap(); + let b = ArrayD::from_shape_vec(IxDyn(sb), samples(nb + 3)[3..].to_vec()).unwrap(); + type Op = fn( + &ndarray::ArrayRef, + &ndarray::ArrayRef, + ) -> Result, Refusal>; + let ops: [(&str, Op, fn(f32, f32) -> f32); 4] = [ + ("Add", onnx::add, |x, y| x + y), + ("Sub", onnx::sub, |x, y| x - y), + ("Mul", onnx::mul, |x, y| x * y), + ("Div", onnx::div, |x, y| x / y), + ]; + for (name, op, f) in ops { + let got = op(&a, &b).unwrap(); + let want = reference_binary(&a, &b, f); + assert_eq!(got.shape(), want.shape(), "{name} {sa:?} {sb:?}"); + for (g, w) in got.iter().zip(want.iter()) { + assert!(same(*g, *w), "{name} {sa:?} {sb:?}: {g} vs {w}"); + } + } + } + // Static ranks broadcast too: a 0-d scalar against a matrix. + let m = Array2::from_shape_vec((2, 2), vec![1.0f32, 2.0, 3.0, 4.0]).unwrap(); + assert_eq!(onnx::mul(&m, &arr0(2.0f32)).unwrap(), m.mapv(|v| v * 2.0)); +} + +#[test] +fn incompatible_shapes_are_refused_not_panicked() { + let a = ArrayD::::zeros(IxDyn(&[3])); + let b = ArrayD::::zeros(IxDyn(&[4])); + assert_eq!( + onnx::add(&a, &b), + Err(Refusal::Broadcast { + lhs: vec![3], + rhs: vec![4] + }) + ); + let c = ArrayD::::zeros(IxDyn(&[2, 3])); + let d = ArrayD::::zeros(IxDyn(&[2])); + assert!(matches!(onnx::div(&c, &d), Err(Refusal::Broadcast { .. }))); + // The check is silent on every admitted pair (it must not refuse what + // ndarray would compute). + assert_eq!(onnx::broadcastable(&[2, 1, 4], &[3, 1]), Ok(())); + assert_eq!(onnx::broadcastable(&[], &[7, 7]), Ok(())); +} + +fn pack(bits: impl Iterator, n: usize) -> Vec { + let mut w = vec![0u64; onnx::mask_words(n)]; + for (i, b) in bits.enumerate() { + if b { + w[i / 64] |= 1 << (i % 64); + } + } + w +} + +fn lane(n: usize) -> Vec { + (0..n) + .map(|i| match i % 9 { + 0 => i32::MIN, + 1 => i32::MAX, + _ => (i as i32 * 37) % 23 - 11, + }) + .collect() +} + +#[test] +fn comparisons_against_a_scalar_match_the_reference() { + type Cmp = fn(&[i32], i32, &mut [u64]); + let ops: [(&str, Cmp, fn(i32, i32) -> bool); 5] = [ + ("Greater", onnx::greater_i32, |x, t| x > t), + ("GreaterOrEqual", onnx::greater_or_equal_i32, |x, t| x >= t), + ("Less", onnx::less_i32, |x, t| x < t), + ("LessOrEqual", onnx::less_or_equal_i32, |x, t| x <= t), + ("Equal", onnx::equal_i32, |x, t| x == t), + ]; + for n in [0, 1, 15, 16, 17, 63, 64, 65, 130] { + let x = lane(n); + for t in [i32::MIN, -3, 0, 5, i32::MAX] { + for (name, op, f) in ops { + let mut got = vec![u64::MAX; onnx::mask_words(n)]; + op(&x, t, &mut got); + assert_eq!( + got, + pack(x.iter().map(|&v| f(v, t)), n), + "{name} n={n} t={t}" + ); + } + } + } +} + +#[test] +fn boolean_ops_and_where_match_the_reference() { + for n in [1, 63, 64, 65, 200] { + let p: Vec = (0..n).map(|i| i % 3 == 0).collect(); + let q: Vec = (0..n).map(|i| i % 5 < 2).collect(); + let (a, b) = (pack(p.iter().copied(), n), pack(q.iter().copied(), n)); + let mut out = vec![0u64; a.len()]; + onnx::and(&a, &b, &mut out); + assert_eq!(out, pack((0..n).map(|i| p[i] & q[i]), n)); + onnx::or(&a, &b, &mut out); + assert_eq!(out, pack((0..n).map(|i| p[i] | q[i]), n)); + onnx::xor(&a, &b, &mut out); + assert_eq!(out, pack((0..n).map(|i| p[i] ^ q[i]), n)); + onnx::not(&a, n, &mut out); + assert_eq!( + out, + pack((0..n).map(|i| !p[i]), n), + "Not keeps the tail zero" + ); + + let x = lane(n); + let y: Vec = (0..n as i32).collect(); + let mut dst = vec![0i32; n]; + onnx::where_i32(&a, &x, &y, &mut dst); + let want: Vec = (0..n).map(|i| if p[i] { x[i] } else { y[i] }).collect(); + assert_eq!(dst, want); + } +} + +#[test] +fn reduce_sum_attributes_are_validated_once() { + assert!(ReduceSum::new(None, 0, 0).is_ok()); + assert!(ReduceSum::new(Some(&[0]), 1, 0).is_ok()); + assert!(ReduceSum::new(Some(&[-1]), 0, 1).is_ok()); + assert_eq!( + ReduceSum::new(Some(&[1]), 0, 0), + Err(Refusal::Axes(vec![1])) + ); + assert_eq!( + ReduceSum::new(Some(&[0, 0]), 0, 0), + Err(Refusal::Axes(vec![0, 0])) + ); + assert_eq!(ReduceSum::new(None, 2, 0), Err(Refusal::KeepDims(2))); + assert_eq!(ReduceSum::new(None, 0, 1), Err(Refusal::NoopWithEmptyAxes)); +} + +#[test] +fn reduce_sum_is_exact_and_refuses_int32_overflow() { + let rs = ReduceSum::new(None, 1, 0).unwrap(); + assert_eq!( + rs.sum_i32(&[]), + Ok(Reduced { + value: 0, + keepdims: true + }) + ); + let x: Vec = (-50..70).collect(); + assert_eq!(rs.sum_i32(&x).unwrap().value, x.iter().sum::()); + assert_eq!( + rs.sum_i32(&[i32::MAX, 1]), + Err(Refusal::OutOfDtypeRange(i64::from(i32::MAX) + 1)) + ); + // An intermediate sum may leave int32 as long as the result fits. + assert_eq!(rs.sum_i32(&[i32::MAX, 1, -1]).unwrap().value, i32::MAX); +} + +fn oracle_sum_where_greater(x: &[i32], t: i32) -> i64 { + x.iter().filter(|&&v| v > t).map(|&v| i64::from(v)).sum() +} + +#[test] +fn composite_matches_the_oracle_mask_risc_r2il_and_relalg() { + use lance_graph_mask_risc::exec::{execute_into, Scratch}; + use lance_graph_mask_risc::ir::{MaskOp, Operand, Pred, Program, Terminal}; + use lance_graph_mask_risc::{Foreign, LaneRef, Out, Planes, Value}; + use lance_graph_r2il::{FoldDialect, Val, INT_S_LESS, LOAD}; + use lance_graph_relalg::{run, Answer, Atom, Cmp, Measure, Query, Rel}; + + let rs = ReduceSum::new(None, 0, 0).unwrap(); + for n in [0usize, 1, 64, 65, 1000, 4097] { + let x: Vec = (0..n).map(|i| ((i * 13) % 97) as i32 - 48).collect(); + let lanes = [LaneRef::I32(&x)]; + let planes = Planes { + n_rows: n, + masks: &[], + lanes: &lanes, + }; + for t in [-49, -1, 0, 30, 48] { + let want = oracle_sum_where_greater(&x, t); + let mut scratch = vec![0u64; onnx::mask_words(n)]; + assert_eq!( + rs.sum_where_greater_i32(&x, t, &mut scratch).unwrap().value as i64, + want, + "facade n={n} t={t}" + ); + + // mask-risc: Pred::GtI32 → Terminal::MaskedSumI32. + let p = Program::new( + vec![MaskOp::Pred { + pred: Pred::GtI32 { lane: 0, t }, + under: None, + dst: 0, + }], + Terminal::MaskedSumI32 { + mask: Operand::Scratch(0), + lane: 0, + }, + ); + let mut buf = vec![0u64; lance_graph_mask_risc::exec::tile_words_for(n).max(1) * 2]; + let mut s = Scratch::over_for_program(&mut buf, &p, n).unwrap(); + assert_eq!( + execute_into(&p, &planes, &Foreign::NONE, &mut s, Out::None), + Ok(Value::SumI64(want)), + "mask-risc n={n} t={t}" + ); + + // The r2il fold reading: t < x, then SUM. + let mut d = FoldDialect::new(&planes, &Foreign::NONE); + let mut stack = vec![Val::Scalar(i64::from(t)), Val::Scalar(0)]; + use ogar_loco::Dialect as _; + d.call(LOAD, [1, 0, 0], &mut stack).unwrap(); + d.call(INT_S_LESS, [0; 3], &mut stack).unwrap(); + lance_graph_r2il::procs::sum(&mut d, &mut stack, 0).unwrap(); + assert_eq!(stack.pop(), Some(Val::Scalar(want)), "r2il n={n} t={t}"); + + // relalg: σ(x > t), Σ x. + let q = Query { + rel: Rel::Atom(Atom::I32 { + lane: 0, + op: Cmp::Gt, + t, + }), + measure: Measure::Sum(0), + }; + assert_eq!( + run(&q, &planes, None).unwrap().answer, + Answer::Sum(want), + "relalg n={n} t={t}" + ); + } + } +} + +#[test] +fn composite_refuses_int32_overflow_and_keeps_keepdims() { + let rs = ReduceSum::new(Some(&[0]), 1, 0).unwrap(); + let x = [i32::MAX, i32::MAX, -5]; + let mut scratch = [0u64; 1]; + assert_eq!( + rs.sum_where_greater_i32(&x, 0, &mut scratch), + Err(Refusal::OutOfDtypeRange(2 * i64::from(i32::MAX))) + ); + assert_eq!( + rs.sum_where_greater_i32(&[1, 2, 3], 1, &mut scratch), + Ok(Reduced { + value: 5, + keepdims: true + }) + ); +}