Repository navigation
Description
Problem, by example
The workload of #902 (three SQL queries every 10 s, phase 0, each the p90 of value over the last 5 minutes):
- Q1: overall
- Q2:
… AND city = 'NYC' - Q3:
… GROUP BY city
On main, Q2 reads its group from Q3's per-city summary (a Filter(city = 'NYC') over the grouped result), which closed #902.
On the #1207 stack (Pass 2 rewritten in #1234), Q2's predicate city = 'NYC' stays in Q2's own Scan, so Pass 2 offers no candidate that filters Q3's per-city result: Q2 builds its own summary. This was found in the Pass 2 prune traces printed by #1239 (stack head refactor/1207-tools).
What should happen: Q2's p90 group has a candidate Filter(city = 'NYC') over Q3's GROUP BY city summary group (key-only filter, β = 1), and Stage 3 can select it.
Why it happens
The general rollup rule of #1234 (stage1-pass2-reuse-candidates/src/rules/rollup.rs) builds the finest common group G* only over members whose rows already have the union selection (decision R5 / PR 4 Q5, approved 2026-11-09). It builds no union Scan, so the old scan-widening step (R7) is not folded in, and a member whose selection is narrower (Q2: city = 'NYC') never joins the cluster of the unselected members (Q1, Q3).
Proposal
Let the rollup rule widen a member's selection when another member groups by the selected column: a member with selection c ∈ S joins a cluster whose G* groups by c (or a finer key set) over the wider selection, and its readout becomes a key-only Filter(c ∈ S) over G* — using SourceColumn lineage to match c. Alternatives: a separate widening rule before rollup (the old R7), kept apart from G*.
Scope and acceptance
- In: SQL and PromQL equality / set selections on a group key column.
- Out: range predicates on non-key columns.
- Acceptance: a Pass 2 test on the workload above where Q2's group has a
Filter(city = 'NYC')candidate over Q3's group, and the A filtered query (city = 'NYC') doesn't read its group from a grouped summary another query builds #902 regression test on the stack selects it.
Status and related work
- Regression on the Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 stack relative to main, where A filtered query (city = 'NYC') doesn't read its group from a grouped summary another query builds #902 is closed. Deferred out of the stack by decision on 2026-11-09 (kept separate from refactor(stage1-pass2): rollup from the finest common group, readout requirements, dead-candidate elimination, chain groups (#1207 PR 4) #1234).
- Related: A filtered query (city = 'NYC') doesn't read its group from a grouped summary another query builds #902, Reuse a finer aggregate through a compensating filter and a rollup of values or states (view matching) #950 (finer-aggregate reuse), Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207, refactor(stage1-pass2): rollup from the finest common group, readout requirements, dead-candidate elimination, chain groups (#1207 PR 4) #1234, refactor: tools and snapshots over the #1207 memo, edges and contexts (#1207 PR 9) #1239.
Activity