Skip to content

Rollup rule: a filtered query (city = 'NYC') does not read the per-city group another query builds (#902 regression on the #1207 stack) #1252

Open

Description

@zzylol

Problem, by example

The workload of #902 (three SQL queries every 10 s, phase 0, each the p90 of value over the last 5 minutes):

  • Q1: overall
  • Q2: … AND city = 'NYC'
  • Q3: … GROUP BY city

On main, Q2 reads its group from Q3's per-city summary (a Filter(city = 'NYC') over the grouped result), which closed #902.

On the #1207 stack (Pass 2 rewritten in #1234), Q2's predicate city = 'NYC' stays in Q2's own Scan, so Pass 2 offers no candidate that filters Q3's per-city result: Q2 builds its own summary. This was found in the Pass 2 prune traces printed by #1239 (stack head refactor/1207-tools).

What should happen: Q2's p90 group has a candidate Filter(city = 'NYC') over Q3's GROUP BY city summary group (key-only filter, β = 1), and Stage 3 can select it.

Why it happens

The general rollup rule of #1234 (stage1-pass2-reuse-candidates/src/rules/rollup.rs) builds the finest common group G* only over members whose rows already have the union selection (decision R5 / PR 4 Q5, approved 2026-11-09). It builds no union Scan, so the old scan-widening step (R7) is not folded in, and a member whose selection is narrower (Q2: city = 'NYC') never joins the cluster of the unselected members (Q1, Q3).

Proposal

Let the rollup rule widen a member's selection when another member groups by the selected column: a member with selection c ∈ S joins a cluster whose G* groups by c (or a finer key set) over the wider selection, and its readout becomes a key-only Filter(c ∈ S) over G* — using SourceColumn lineage to match c. Alternatives: a separate widening rule before rollup (the old R7), kept apart from G*.

Scope and acceptance

Status and related work

Activity

  1. added
    bugSomething isn't working
    sqlSQL front-end lowering (DataFusion → L2)
    optimizerWorkload optimization and plan selection
    rollupFine-to-coarse aggregation and summary reuse
    on Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingoptimizerWorkload optimization and plan selectionrollupFine-to-coarse aggregation and summary reusesqlSQL front-end lowering (DataFusion → L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions