Skip to content

D-RFT-1 typed composition + r2il PostgreSQL primitive polyfill (IntXor/ANY/ALL/KEY_RUNS, checked scalar arithmetic, 19 procedures) - #1681

Merged
AdaWorldAPI merged 10 commits into
mainfrom
ccr-0455e606-rft1
Oct 21, 2026
Merged

AdaWorldAPI merged 10 commits into
mainfrom
ccr-0455e606-rft1

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

What

D-RFT-1, PROBE (crates/dir-rmap-probe/src/compose.rs). It composes user → group membership with the role carriers over the DR256 register's shared 256-slot coordinate, under one Witness.

  • Path mask: per (user i, role r), member_i ∧ valid_i ∧ carriers_r.
  • Bag is the path count and Distinct is its existence. They are two readings of one computation.
  • Stratum comes from the measured cardinality witness: the left fan (present keys per user) and the right fan (carrier slots per role). The caller does not choose it.
  • NULL key: payload set, presence clear. It matches nothing, the same model as Quack's validity plane.
  • Refused: XOR (an even number of parallel paths reads 0), seal mismatch, stale binding, and bag overflow (checked add).

r2il polyfill (lance-graph-r2il):

  • IntXor (R2IL ordinal 22) lowers to MaskOp::Xor, or to BoolExpr::Xor when fused.
  • ANY and ALL lower to Terminal::Any/All and push 1 or 0. They were Unimplemented before.
  • New procedures: exists, all, differ (IntXor then ANY, mask equality as a fold), intersect, union, keep, count_and.

How it executes. Every count, existence and mask equality in the composition is one of those procedures. Each is compiled once by FoldDialect (Composer::new, a prepared statement) and replayed through mask-risc execute_into over zero-copy row slices. The probe has no counting loop of its own.

Support first. A user's roles are the union of its present keys' role words. Only coordinates in that union are counted; the others have no path by construction.

Evidence

  • Tests (tests/compose.rs):
    • IAM diamond: Bag 2, Distinct 1.
    • Refusals: XOR, NULL key, seal mismatch, stale binding, overflow.
    • Functional stratum.
    • A random fixture against an independent oracle: a DFS over the Prepared CSRs, not Qr's closure. Compared lane to lane, with supports compared by differ; anti-vacuity on counts 1 and ≥ 2.
  • Disables: 7 on compose and 5 on r2il, all red.
  • PostgreSQL 16.14 (examples/rft1_pg_parity.rs, WITH RECURSIVE):
    • the per-coordinate lane is equal over 600 users × 64 roles;
    • bag 1421 = count(*);
    • distinct 1374 = count(DISTINCT (u, role));
    • NULL-group membership rows match nothing in either engine.
  • Bench (examples/rft1_bench.rs, release, median of 101 runs, 600 users):
step / arm time allocations per run
bind 46 µs
pages 39 µs
program prep 1.4 µs
A — composition (r2il → mask-risc) 310 µs 39
C — every coordinate, direct ndarray::simd 131 µs 0
C+ — A's plan, direct ndarray::simd 14 µs 0
B — tuple baseline, lab only, not a candidate 134–146 µs 10

The first wiring ran one fold per (user, role) coordinate and took 17 ms with 78 210 allocations. Program replay plus support-first brought it to 310 µs.

Open

  • Per-execution cost. A − C+ is about 115 ns of fixed cost per mask-risc execute_into on 4-word planes (about 2 600 executions). Not profiled.
  • No count-product terminal. mask-risc has no terminal that produces many coordinates' counts in one execution (a bit-matrix count product or a masked positional popcount), and ndarray has no positional popcount.
  • Presence plane is a copy. Members::of copies member into an owned valid plane (8 KiB per page). A real source would supply its own presence plane.
  • Fixture scale. Spec::iam(8000) exceeds 256 security slots, so the bench runs Spec::small().

Manifests

  • dir-rmap-probe adds lance-graph-r2il and moves lance-graph-mask-risc from a dev-dependency to a dependency.
  • OGAR patches are added for ogar-loco and ogar-r2il, plus a lance-graph patch so the contract resolves once.
  • The CI workflow paths now include r2il and the contract.

No production ABI changes.

Board: entries/2026-10-21-d-rft-1-typed-composition.md, and the STATUS_BOARD D-RFT-1 row (In PR).


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

@codex review


Generated by Claude Code

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0aff59e5-3ccf-42b7-b95d-d51672e94dd2
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T20:48:12.587945Z f67e9c5 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

…dures

The D-RFT-1 composition needs three things FoldDialect refused as
Unimplemented: a Boolean reduction terminal (ANY, ALL) and IntXor. They
lower to mask-risc Terminal::Any/All and MaskOp::Xor (BoolExpr::Xor when
fused); ANY/ALL push 1 or 0. The new procedures are call sequences over
those, so mask equality (differ = IntXor then ANY) and a composed path
count (count_and = IntAnd chain then PopCount) are folds, not row loops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…torical

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
count_and is now intersect + PopCount. union is the IntOr chain; keep
writes the mask on top of the stack into the dialect's KEEP sink. Tested
against the word-wise OR, compared by differ.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
User -> group membership composed with role carriers over one shared slot
coordinate, bound by one Qr witness. Bag (count of paths) and Distinct
(existence) are two readings of one computation; Xor is refused (an even
number of parallel paths reads zero). A key with payload set and presence
clear is NULL and matches nothing. The stratum comes from the measured
cardinality witness (left fan of present keys, right fan of carriers),
never from the caller. Bag sums are checked; a wrap is refused.

Every count, existence and mask equality is an r2il procedure (intersect,
union, keep, count_and, differ) compiled once and replayed through
mask-risc over zero-copy row slices. Support first: a user's roles are the
union of its present keys' role words, so only coordinates in that union
are counted.

Tests: the IAM diamond (Bag 2, Distinct 1), Xor, NULL key, seal mismatch,
stale binding, overflow, functional stratum, and a random fixture against
an independent DFS over Prepared ordinals (lane compared, supports by
differ). examples/rft1_pg_parity: PostgreSQL 16 recursive CTE, count(*) vs
count(DISTINCT) and the per-coordinate multiplicities, with NULL-key rows.
examples/rft1_bench: bind, pages, program prep and execution separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3a5d41f786

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/dir-rmap-probe/src/compose.rs
Comment thread crates/dir-rmap-probe/src/compose.rs
…tation

The entry's heading id collided with the D-RFT-1..5 assessment entry
(entries_index DUPLICATE ID). The metta-datalog plan cited FoldDialect by
line number, which the r2il additions moved; it now cites the symbols.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
…IntSDiv, 12 procedures)

Dialect: KEY_RUNS lowers to CountKeyRunsU32 (COUNT(DISTINCT key) on a
key-ordered lane; an unordered lane is refused by mask-risc). IntAdd,
IntMult and IntSDiv combine fold results, checked: overflow and division by
zero are refused (new FoldError::Overflow / DivisionByZero), as PostgreSQL
raises for bigint.

Procedures: is_null, is_not_null, and_between_i32, in_u32 (IN () keeps
nothing), count_valid (COUNT(x) vs COUNT(*)), avg_parts (SUM and COUNT;
avg(int) is numeric, so the quotient is the caller's), bool_or, bool_and,
not_exists, except, count_distinct_sorted, count_union_all. Each WHERE-shaped
procedure leaves a mask; KEEP it and it is a Lance 13 row-address
prefilter (with_row_addr_prefilter, as crepe-plan-probe #1633 does).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
… in the D-RFT-1 entry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
@AdaWorldAPI AdaWorldAPI changed the title D-RFT-1: typed composition (Distinct/Bag, NULL key, IAM diamond) over r2il procedures; r2il gains IntXor, ANY, ALL D-RFT-1 typed composition + r2il PostgreSQL primitive polyfill (IntXor/ANY/ALL/KEY_RUNS, checked scalar arithmetic, 19 procedures) Oct 21, 2026

Copy link
Copy Markdown
Owner Author

Rebased onto main and fixed the two red checks. Both were mine:

  • an entries-index duplicate id: the entry's heading id is now D-RFT-1-COMPOSE;
  • citation decay: my r2il insertions moved pub struct FoldDialect, and the metta-datalog plan now cites it by symbol instead of by line number.

New in this push: the PostgreSQL primitive card in lance-graph-r2il. The full table is in the D-RFT-1 entry's update section.

Dialect:

  • KEY_RUNS lowers to CountKeyRunsU32, i.e. COUNT(DISTINCT key) on a key-ordered lane. An unordered lane is refused.
  • Scalar IntAdd, IntMult and IntSDiv are checked. Overflow and division by zero are refused as FoldError::Overflow and FoldError::DivisionByZero.

Procedures: is_null, is_not_null, and_between_i32, in_u32, count_valid, avg_parts, bool_or, bool_and, not_exists, except, count_distinct_sorted, count_union_all. A kept WHERE mask is the Lance 13 with_row_addr_prefilter input, as in #1633.

Checks: 6 more disables, all red. r2il has 28 tests; clippy -D warnings and fmt are clean; the r2il-mask-abi-probe and the dir-rmap compose suites are green.

@codex review


Generated by Claude Code

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review October 21, 2026 20:43
…ts own witness

Two Codex findings on #1681:
- procs::union/intersect/in_u32 now chain through FoldDialect::call_owned,
  so a chain over k planes reuses two scratch slots instead of k-1. A user
  with >= 9 present groups no longer panics with ScratchBufferTooSmall.
  Fused mode still refuses a 24-leaf chain (mask-risc fuse gives each
  ternlog group its own slot); pinned as a refusal, recorded as open.
- Page now carries the Witness it was resolved against, and Members::of
  stamps that witness, not the register's. An old page paired with a newly
  bound register fails compose's seal check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg

Copy link
Copy Markdown
Owner Author

@codex review


Generated by Claude Code

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: f67e9c5a7f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants