Skip to content

D-QSP-0 slice 1: scalar SUM is NULL on an empty or all-NULL selection - #1675

Merged
AdaWorldAPI merged 1 commit into
mainfrom
claude/eager-albattani-iufugq
Oct 21, 2026
Merged

AdaWorldAPI merged 1 commit into
mainfrom
claude/eager-albattani-iufugq

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

This PR implements D-QSP-0 remainder slice 1 (track C in EXO-50 /033, claimed as session_013z2e1YXkm6o6HNrDn5BPWA/026). It closes gap G7 from entries/2026-10-21-pg-classview-polyfill-map.md.

Defect. Scalar Agg::SumI32 lowers to MaskedSumI32, which starts from 0. A selection with no non-NULL x therefore answered 0, where PostgreSQL answers NULL. Values that really sum to zero also answer 0, so the two cases could not be told apart.

Change (crates/lance-graph-quack/src/lib.rs, next to AvgPlan):

  • lower_sum(filter, val, valid) lowers the existing MaskedSumI32 and Count terminals, both over filter ∧ valid(val). Because the function takes the validity plane itself, the NULL gate cannot end up on one half and not the other.
  • sum_finish(sum, count) -> Option<i64> returns NULL when count == 0.
  • SumPlan is a type alias of AvgPlan.
  • There is no new terminal, operator, rule or registry. set_algebra, set_rules, relalg and mask-risc are untouched, and so is every Semijoin path (ADMISSION B).

Supported SQL subset, as pinned. One table with two nullable int4 columns.

  • WHERE: any tree, depth ≤ 3, of comparisons (= <> < <= > >=) against an integer constant, IS [NOT] NULL, AND, OR and NOT, read through sql_where.
  • Aggregates: count(*), count(x), sum(x), min(x), max(x) and avg(x), ungrouped.
  • P-EMPTYSUM: an empty WHERE gives one NULL ungrouped and zero groups grouped (GroupReduce { SumI32 } after normalize_group_sink).

Differential results

  • PostgreSQL 16.15 (examples/qsp_agg_pg_parity.rs, 400 tables, 3600 queries): Quack, PostgreSQL and the Kleene oracle agree on all six aggregates, with 0 mismatches.
    • The corpus contains 1455 empty selections, 507 all-NULL selections and 64 real zero sums.
    • The naive scalar SUM differs on exactly 1962 = 1455 + 507 queries, and the example asserts this.
    • AVG is compared with a relative tolerance of 1e-12, because PostgreSQL's avg(int) is numeric.
    • Without a server the example prints SKIP and exits 0.
  • tests/sql_aggregate_null.rs compares the same six aggregates against a column-lane Kleene oracle (folds over positions, no row objects) on 1200 random WHERE clauses. It asserts that the corpus contains empty, all-NULL and real-zero cases. It also has targeted NULL-vs-zero cases and the P-EMPTYSUM pin.

Disable runs (each red at an assertion line):

  • D1 (sum_finish always returns Some): both sum tests fail. The PostgreSQL example reports 1962 sum(x) mismatches and exits 101.
  • D2 (lower_sum ignores valid): both sum tests fail.
  • D3 (only the sum half is gated): both sum tests fail.
  • D4 (normalize_group_sink marks every slot present): only the P-EMPTYSUM test fails.

Unsupported, listed in the entry:

  • grouped Agg::GroupSumI32, which still coalesces an unreached group to 0 (documented);
  • a reached group whose x are all NULL (T4), and NULL group keys (G2), which belong to the GROUP BY slice;
  • NULL literals and NOT IN (…, NULL) (G1);
  • COALESCE and NULLIF;
  • nullable arithmetic;
  • int4 sum overflow, and numeric AVG.

There is no performance claim. SUM uses two passes, the same as AVG.

Board: entry entries/2026-10-21-d-qsp-0-scalar-aggregates-null.md, the D-QSP STATUS_BOARD row, and the regenerated entries index. The quack crate passes every suite, and clippy -D warnings is clean (Rust 1.99.0).


Generated by Claude Code

…election

Scalar Agg::SumI32 (MaskedSumI32) starts from the additive identity, so a
selection with no non-NULL x answered 0 where PostgreSQL answers NULL.

lower_sum(filter, val, valid) lowers the existing MaskedSumI32 and Count
terminals, both over filter AND valid(val); sum_finish(sum, count) returns
None when no value contributed. SumPlan is an alias of AvgPlan. No new
terminal, operator or rule; set_algebra, set_rules, relalg and mask-risc are
unchanged.

Evidence:
- tests/sql_aggregate_null.rs: COUNT(*), COUNT(x), SUM, MIN, MAX, AVG
  against a column-lane Kleene oracle over 1200 random WHERE clauses, with
  empty, all-NULL and real-zero cases asserted present; targeted
  NULL-vs-zero cases; P-EMPTYSUM (ungrouped one NULL, grouped zero groups).
  Disables D1-D4 each red.
- examples/qsp_agg_pg_parity.rs: PostgreSQL 16.15, 3600 queries, Quack =
  PostgreSQL = oracle on all six aggregates; the naive scalar SUM differs on
  exactly the 1455 empty + 507 all-NULL queries. Prints SKIP without a server.

Board: entry 2026-10-21-d-qsp-0-scalar-aggregates-null.md, STATUS_BOARD row,
regenerated entries index. EXO-50 claim session_013z2e1YXkm6o6HNrDn5BPWA/026.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013z2e1YXkm6o6HNrDn5BPWA
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e19edb55-904e-409d-b1e2-8992b1330516
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review October 21, 2026 19:50
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T19:50:56.147925Z 0d2d5b1 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@AdaWorldAPI
AdaWorldAPI merged commit a55fc67 into main Oct 21, 2026
19 checks passed
AdaWorldAPI pushed a commit that referenced this pull request Oct 21, 2026
STATUS_BOARD D-TGB-4 row to Shipped, PR_ARC_INVENTORY and LATEST_STATE
entries for #1674. Also regenerates the entries index, whose count went
stale when #1674 and #1675 merged one after the other (421 -> 422).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019fbZ92dFKQ5BJHrA3YcH9A
AdaWorldAPI pushed a commit that referenced this pull request Oct 21, 2026
…1680/#1682, boundaries

Only this session's rows and entry: the STATUS_BOARD "D-QSP-0 remainder"
row (In PR -> Shipped, slice 2 added), PR_ARC entries for #1675, #1680 and
#1682 prepended, and the entry now states that Quack has no NULL-literal
Filter variant, that the PostgreSQL examples are lab evidence and not CI,
and #1682's final CI (18/18 green on 96e8010).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013z2e1YXkm6o6HNrDn5BPWA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants