Skip to content

U2 (D-RU-2): let-bound planes over the contract set algebra - #1649

Merged
AdaWorldAPI merged 4 commits into
mainfrom
ccr-d3ce1263-ied9yh
Oct 21, 2026
Merged

AdaWorldAPI merged 4 commits into
mainfrom
ccr-d3ce1263-ied9yh

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

What this adds

lance_graph_contract::set_program

  • SetProgram { version, resident, lets, queries }:
    • Sym(i) with i < resident is a resident plane;
    • Sym(resident + k) is lets[k];
    • a let reads only earlier lets.
  • validate() refuses forward and unknown references.
  • share(version, resident, queries) turns a junction subset held by two or more junctions of one kind into a let, largest first.
    • It judges after normalize. normalize flattens nested junctions, so whole identical subtrees are rare and sharing has to work on subsets.

lance_graph_relalg::program

  • run_program evaluates each let once into a scratch plane, then runs every query over the resident planes plus those planes.
    • A let's plane acts as one more resident plane, so every fold stays a tree.
  • run_recompute is the baseline: each query with its lets inlined.
  • Both refuse:
    • a program built for another table version;
    • a malformed program;
    • the wrong number of resident planes;
    • a measure count that does not match the queries.
  • ProgramCost counts programs, ops, lane reads, plane reads, planes materialized and their bytes, and the most scratch slots any one fold used.

Measured

examples/program_bench: 16 queries sharing one three-comparison conjunction. Release build, medians of 9 runs.

rows program recompute lane reads materialized
100,000 0.35 ms 1.15 ms 14 vs 59 25 kB
1,000,000 4.77 ms 14.87 ms 14 vs 59 250 kB
4,000,000 20.98 ms 69.35 ms 14 vs 59 1 MB
  • Each lane read is one full pass over n_rows values.
  • Both paths use at most 2 scratch slots per fold.

Correctness

  • The program, recompute and a row-at-a-time oracle give the same answers and the same kept masks. Fixtures:
    • hand-built sets: one shared conjunction; nested and distinct lets; nothing shared;
    • 60 generated sets.
  • With nothing shared, the program costs exactly what recompute costs.
  • Eight disable runs, all red, including "wrong let evaluated" and "lets cannot read earlier lets". Both of those were green at first, because no fixture had more than one let. nested_and_distinct_lets_agree_with_recompute_and_the_oracle was added for them.

Also

  • The crepe-plan-probe workflow's relalg step now runs the benchmark once at n = 100003.
    • At first, the benchmark set bits past the last row in its resident plane, and mask-risc refused them (PlaneTail).
  • Board:
    • STATUS_BOARD D-RU-2 is MEASURED;
    • new entry entries/2026-10-21-u2-let-planes.md;
    • plan stage note;
    • entries index regenerated.

Open

  • share is greedy and does not weigh one plane's materialization against the reads it saves. It is measured here only where sharing wins.
  • U3 and U4 are unblocked.

Generated by Claude Code

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6dd42f90-eab1-483e-bede-ca85ea420a6a
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review October 21, 2026 01:34
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T01:36:30.090347Z 30087cc Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30087ccaaf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/lance-graph-contract/src/set_program.rs Outdated
lance-graph-contract::set_program: SetProgram { version, resident, lets,
queries }. A let is referenced as Sym(resident + k), only from later lets
and from queries; validate() refuses forward and unknown references.
share() lifts every junction subset held by two or more junctions of one
kind into a let, largest first, judged after normalize().

lance-graph-relalg::program: run_program evaluates each let once into a
scratch plane, then every query over the resident planes plus those
planes; run_recompute runs each query with its lets inlined. Both refuse
a program built for another table version. ProgramCost counts programs,
ops, lane reads, plane reads, planes materialized and their bytes.

Tests: program, recompute and a row oracle agree on answers and kept
masks, for hand-built and 60 generated query sets; lane reads, planes
and bytes are pinned for the shared case.

Benchmark (examples/program_bench, n = 1M, 16 queries, 2 lets, release):
program 4.63 ms, 14 lane reads, 250000 bytes materialized;
recompute 14.49 ms, 59 lane reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VncKoyG6JkpsoqhqTGTv3f
The benchmark set bits past the last row in its resident plane, which
mask-risc refuses (PlaneTail) for n not a multiple of 64. The plane tail
is now cleared, and CI runs the benchmark once at n = 100003.

Board: STATUS_BOARD D-RU-2 MEASURED, entry
2026-10-21-u2-let-planes.md, plan stage note, entries index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VncKoyG6JkpsoqhqTGTv3f
validate() and let_sym() added `resident + lets.len()` unchecked; with
resident = u32::MAX that panics in debug and wraps in release. Now:
- let_sym returns Option<u32>; validate refuses with TooManyIds;
- share stops lifting before `resident + lets.len()` would not fit, so
  every program it builds still validates;
- relalg checks its u16 plane limit before validate, so an oversized
  program is refused as TooManyPlanes.

The benchmark also prints which SIMD arm it was compiled for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VncKoyG6JkpsoqhqTGTv3f
@AdaWorldAPI
AdaWorldAPI merged commit 6a6aa8d into main Oct 21, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants