Skip to content

refactor(stage3): constraint (K), accuracy at each query root (#1207 PR 8) - #1238

Draft
zzylol wants to merge 8 commits into
refactor/1207-contextsfrom
refactor/1207-accuracy
Draft

zzylol wants to merge 8 commits into
refactor/1207-contextsfrom
refactor/1207-accuracy

Conversation

@zzylol

@zzylol zzylol commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1237 (PR 7, execution contexts). Part of #1207 (PR 8 of the order), #1204 item 3, #1203. Draft; not to be merged on its own (one merge unit, PRs 0–9).

Why

Until this PR nothing checked a query's composed accuracy: Stage 3 selected the cheapest sizes the edges' parameter floors allowed, and a floor is only a pruning cap (Pass 2 takes the largest cap over the queries, and a target whose metric the family does not state leaves the input unbounded). PR 6 showed the result (handoff H1): quantile_over_time(0.99, latency_ms[60m]) with a rank target of 0.01 selected six DDSketch windows at alpha 0.1, a relative error of 10 % that says nothing about rank. The concepts doc §4 and #1204 item 3 define constraint (K): per-candidate error bounds $E_c$, $D_c$ composed over the selected edges with big-M, checked at each query root.

What

  • Constraint (K) in the selection MILP: $E_{\kappa,c,m}$ per execution and error metric, $D_{\kappa,c}$, $F$/$H$ per input, big-M only on selected edges ($y_{\kappa,u,c,i}$), root checks $E_v \le \varepsilon_q$, $D_v \le \delta_q / n$ when $z_{q,v} = 1$.
  • The metric rule on the selected path: an input error in a metric the consumer's contract does not carry must be zero on a selected edge, and a root accepts only the metrics its target accepts (a rank target rejects a DDSketch readout; a relative-error ratio rejects a KLL operand). Validity caps $b_{c,i}$ (a ratio's denominator) are enforced on selected edges.
  • Read errors $\varepsilon^{\mathrm{loc}}$, $\delta^{\mathrm{loc}}$: where a consumer reads a state (a readout's statistic, or a merge), from the producer's $\boldsymbol\theta$ (asap_accuracy::estimators::local_guarantee), a constant per edge $(u, c)$, with the conditions PR 4 removed with node-tree propagation brought back: a DDSketch interior quantile and a Count-Min count hold only over values proven non-negative; a windowed or merged state keeps a guarantee only when its family's merged error does; a distinct count over a source's own rows takes its failure probability at the source's declared distinct bound (classic HLL, distinct-count UnivMon: the PR 4 dead code, now used).
  • Root check after the solve: each query's composed guarantee in the selected DAG is checked again with the deployment's accuracy model and evidence; an output none accepts is ErrorSelectingPlan::SelectedPlanFailedCheck (the PR 0 variant, now constructed).
  • Accuracy policy: (K) applies to every query unless the deployment's model may accept an output regardless of its guarantee or its evidence may meet a target (may_accept_regardless_of_guarantee, may_provide_evidence; e.g. IgnoreAccuracyGuarantees): the traits' contract is that plan selection then keeps every candidate, and the check after the solve decides. A whole-result failure probability is divided over $G_q$ values, read from the root the front ends lowered (AccuracyCheckInputs::query_roots, new).
  • Every todo!("#1207 PR 8 …") is implemented (accuracy_constraints); the dead_code allowances for PR 8 are gone (AccuracyVariables, accuracy_constraints, HLL classic_guarantee, UnivMon distinct_guarantee, incomplete_top_layer_statistic).

Not here: tools and snapshots (PR 9); latency (L) and resource budgets (B) are out of scope (#1204 items 5, 6).

Representation, rules, enumeration (#1207 (a)/(b)/(c))

  • (a) Representation: no memo change. MILP variables for accuracy: $E_{\kappa,c,m}$, $F_{\kappa,c,i,m}$, $D_{\kappa,c}$, $H_{\kappa,c,i}$, and $U_{\kappa,c}$ (part of the output without a guarantee).
  • (b) Rules: none.
  • (c) Enumeration: the MILP's feasible set shrinks to plans meeting (K). The reference enumerator (Appendix A) now also filters DAGs by each root's composed error (CandidateErrorModel::composed_errors) and agrees with the MILP on Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207's problem example under 12 cost/target sets (targets that admit a plan and targets that admit none).

Notation → code

Notation Code
$\varepsilon^{\mathrm{loc}}$, $\delta^{\mathrm{loc}}$ CandidateErrorModel::read_error_on_edge(u, c): ErrorBounds::{error_bound, failure_probability, without_guarantee} per metric, from read_bounds / read_guarantee
$\beta_{c,i}$, $b_{c,i}$ CandidateErrorModel::{factor, validity_cap} (from LogicalRequirements::error_propagation, per metric)
$M_{c,i}$, $M_q$ CandidateErrorModel::{error_bound_ceiling, failure_probability_ceiling} (the producer's / output's ceiling)
$E_{\kappa,c}$, $F_{\kappa,c,i}$, $D_{\kappa,c}$, $H_{\kappa,c,i}$ AccuracyVariables::{error_bound, error_bound_of_input, failure_probability, failure_probability_of_input} (keys: Execution, input, metric index)
(K) accuracy_constraints (model.rs)
$\varepsilon_q$, $\delta_q$ at the root RootAccuracyLimits (root_accuracy_limits, per query; None when (K) does not apply)
$G_q$ (values a whole-result δ covers) values_per_result_per_query over AccuracyCheckInputs::query_roots
$U_{\kappa,c}$ (not in the doc) AccuracyVariables::without_guarantee
metric rule accuracy_constraints: $E_{u,m} \le M(1-y)$ when $c$'s contract lacks $m$; RootAccuracyLimits::error_bound_limit[m] = 0 for a metric the target does not accept

How

  1. select_plan builds CandidateErrorModel::new(physical, data, executions): per physical candidate its error contracts in the five metrics; per admissible edge on which a consumer reads a state a build builds, the read error of the consumer's statistic(s) on the producer's θ; and ceilings (rounds of $\bar E_{c,m} = \sum_i \max_{u \in P_{c,i}} (\beta_{c,i,m} \bar E_{u,m} + \varepsilon^{\mathrm{loc}}_{u,c,m})$ until stable, at most one round per execution: a valid $M$ for every acyclic selection).
  2. root_accuracy_limits(entries, G, model, evidence): per query $\varepsilon_q$ in the metrics its target accepts (metric_can_meet of the built-in rule), $\delta_q / n$ with $n$ = max_values_covered(scope, G_q); all None when the model or evidence may accept outputs regardless. A scalar root that combines outputs (not scalar(v)) requires exact references.
  3. solve_selection: when (K) applies, accuracy_variables and accuracy_constraints add it per execution and $y_{\kappa,u,c,i}$ (a row whose big-M side holds at its ceiling is left out), with HiGHS's feasibility tolerances at $10^{-9}$.
  4. After realization, accepted_accuracy composes each query's error over the selected DAG (composed_errors, no big-M), states it over the target's scope with the same $G_q$, and checks it with the model, then the evidence, then the accept-regardless rule.

Key code interfaces added / modified

models.rs, pass.rs

/// Inputs of the composed-accuracy check, constraint (K). // stays
pub struct AccuracyCheckInputs<'a> {
    /// Decides whether a guarantee meets a target, and whether an output is accepted regardless. // unchanged
    pub model: &'a dyn AccuracyModel,
    /// Deployment facts the guarantees need (operand domains, observed guarantees). // unchanged
    pub evidence: &'a dyn AccuracyEvidenceProvider,
    /// Each query's root as the front ends lowered it, by query: Stage 3 reads G_q from it. // new
    pub query_roots: &'a [QueryRoot],
}

impl<'a> PlanningModels<'a> {
    /// Stage 3's accuracy check inputs over the front ends' roots. // was accuracy_check_inputs(&self)
    pub fn accuracy_check_inputs<'b>(&self, query_roots: &'b [QueryRoot]) -> AccuracyCheckInputs<'b>
    where
        'a: 'b;
}

estimators/mod.rs

/// A distinct count read from a classic HLL or distinct-count UnivMon over one source's own rows,
/// its failure probability bound at the source's max distinct items per evaluation; None for other
/// parameters. Stage 3 calls it for a Cardinality read when the source declares the bound. // new
pub fn distinct_count_guarantee_over_source(
    parameters: &SketchParams,
    source: &str,
    max_distinct: u64,
) -> Option<AccuracyGuarantee>;

report.rs

pub enum ErrorSelectingPlan {
    // other variants unchanged
    /// The selected plan's composed guarantee misses a query's target by the deployment's model, no
    /// evidence meets it and the model does not accept it regardless: a bug in (K)'s error model or
    /// a model stricter than the built-in rule. Now constructed. // doc changed (was budget shares)
    SelectedPlanFailedCheck { cause: RejectionCause, reason: String },
}
pub enum RejectionCause {
    // other variants unchanged
    /// No plan meets (A)–(I), (R) and every query's accuracy target (K): the MILP is infeasible. // doc changed
    Infeasible,
}

error_bounds.rs, model.rs (crate-private, pending approval)

/// An error bound per metric, a failure probability, or no guarantee: a read's error on an edge,
/// or a selected node's composed error. // new
pub(crate) struct ErrorBounds {
    /// Error bound per metric, by index into METRICS. // new
    pub(crate) error_bound: [f64; METRIC_COUNT],
    /// Failure probability; None when unknown ((K) counts 1). // new
    pub(crate) failure_probability: Option<f64>,
    /// Part of the output has no guarantee the planner can state. // new
    pub(crate) without_guarantee: bool,
}
/// Each physical candidate's error contracts, read errors per edge and ceilings (M). // new
pub(crate) struct CandidateErrorModel { /* private fields */ }
impl CandidateErrorModel {
    /// Built once per select_plan; longest_path = the executions, for the ceilings. // new
    pub(crate) fn new(physical: &PhysicalASAPMemoCandidates, data: &DataWorkload, longest_path: usize) -> Self;
    /// eps_loc, delta_loc of consumer c reading producer u (exact unless c reads a state u builds). // new
    pub(crate) fn read_error_on_edge(&self, u: usize, c: usize) -> &ErrorBounds;
    /// beta_{c,i} and b_{c,i} in metric m. // new
    pub(crate) fn factor(&self, c: usize, input: usize, m: usize) -> Option<f64>;
    pub(crate) fn validity_cap(&self, c: usize, input: usize, m: usize) -> Option<f64>;
    /// M of E_{c,m} and of D_c. // new
    pub(crate) fn error_bound_ceiling(&self, c: usize, m: usize) -> f64;
    pub(crate) fn failure_probability_ceiling(&self, c: usize) -> f64;
    /// The composed error of every selected node, no big-M: the check after the solve. // new
    pub(crate) fn composed_errors(&self, selected: &DagOfSelectedCandidates) -> Vec<ErrorBounds>;
}
/// One query's limits at its root: eps_q per accepted metric (0 otherwise), delta_q / n. // new
pub(crate) struct RootAccuracyLimits {
    pub(crate) error_bound_limit: [f64; METRIC_COUNT],
    pub(crate) failure_probability_limit: Option<f64>,
}
/// Per query; all None when the model or evidence may accept outputs regardless. // new
pub(crate) fn root_accuracy_limits(entries: &[QueryWorkloadEntry], values_per_result: &[Option<u64>],
    model: &dyn AccuracyModel, evidence: &dyn AccuracyEvidenceProvider) -> Vec<Option<RootAccuracyLimits>>;

/// The accuracy variables of (K), per execution (and metric for E, F). // stays; keys changed
pub(crate) struct AccuracyVariables {
    /// E_{kappa,c,m}. // was error_bound: Vec<Variable> by PhysicalCandidateId
    error_bound: HashMap<(Execution, usize), Variable>,
    /// F_{kappa,c,i,m}. // was keyed (PhysicalCandidateId, usize)
    error_bound_of_input: HashMap<(Execution, usize, usize), Variable>,
    /// D_{kappa,c}. // was Vec<Variable>
    failure_probability: HashMap<Execution, Variable>,
    /// H_{kappa,c,i}. // was keyed (PhysicalCandidateId, usize)
    failure_probability_of_input: HashMap<(Execution, usize), Variable>,
    /// U_{kappa,c} in [0, 1]: part of the output without a guarantee. // new
    without_guarantee: HashMap<Execution, Variable>,
}
/// (K): read errors, big-M on selected edges, metric rule, validity caps, union bound, root checks. // was todo!()
fn accuracy_constraints(inputs: &SelectionInputs<'_>, selection: &SelectionVariables,
    variables: &ExecutionContextVariables, accuracy: &AccuracyVariables) -> Vec<Constraint>;
// was accuracy_constraints(physical, selection, accuracy)

Usage (end to end; crates/planner/tests/accuracy_at_query_roots.rs at 55a1384):

// The user's workload: #1207's problem example, every 10 s at phase 0, over stored samples of
// `latency` (1,000 series every 15 s, values declared non-negative).
let queries = [
    // A: rank error 0.01.
    ("quantile_over_time(0.95, latency[1m])",
     AccuracyTarget::max_error(0.01).accuracy_metric(AccuracyMetric::Rank)),
    // B: relative error 0.03.
    ("quantile_over_time(0.99, latency[1m]) / quantile_over_time(0.5, latency[1m])",
     AccuracyTarget::max_error(0.03).accuracy_metric(AccuracyMetric::Relative)),
];
let output = plan(&queries, /* ignore_accuracy_guarantees */ false).await;
//   e2e_plan → run_planning → select_plan(.., &models.accuracy_check_inputs(&roots), ..):
//     CandidateErrorModel::new: KLL readouts carry rank errors, DDSketch readouts relative ones
//       (p50 interior: valid because `latency` is declared non-negative)
//     root_accuracy_limits: built-in model, no evidence → A: rank ≤ 0.01; B: relative ≤ 0.03
//     solve_selection + accuracy_constraints: A cannot read DDSketch (relative ≠ rank); B's Div has
//       no rank contract, so its operands read DDSketch, and (α + α)/(1 − 0.1) ≤ 0.03
//     accepted_accuracy: composed guarantees rechecked with BuiltinAccuracyModel::satisfies
let selection = output.selection.as_ref().expect("a plan is selected");
// A accepted in rank through a KLL; B in relative through one DDSketch read by both operands.
assert_eq!(selection.accepted_accuracy_per_query[0].sketches, ["Kll"]);
assert_eq!(selection.accepted_accuracy_per_query[1].accepted_metric, Some(AccuracyMetric::Relative));
// The selected DDSketch alpha is any ≤ 0.0135 (sizes above the floor tie in the objective, spec question 5).

Major algorithm changes

Stage 3: constraint (K)

  • Problem. Sizes were constrained only by per-edge floors. quantile_over_time(0.99, latency_ms[60m]), rank 0.01, selected DDSketch alpha 0.1 windows (relative error, not rank); the ratio p99/p50 at relative 0.03 could take alpha 0.02 for both operands (each within its floor, composed $(0.02+0.02)/0.9 = 0.044 &gt; 0.03$).
  • Before / after. Before: (A)–(I), (R); accuracy recorded, never checked. After: How 1–4.
  • Correctness. Soundness: for a selected acyclic DAG, $E$ at each selected execution is at least $\varepsilon^{\mathrm{loc}}$ plus $\beta$ times its selected producers' $E$ (the big-M is slack exactly on unselected edges since $M \ge$ the producer's largest $E$); the root check bounds it by the target; the post-check recomputes it without big-M and rejects anything over the target. Conservative only (never accepts a plan missing its target): a producer reached by two paths and its failure event count twice; a merge's read error is the largest over the statistics read from its group. Tests: the_problem_example_meets_each_query_target_at_its_root, a_ddsketch_interior_quantile_needs_non_negative_values, a_read_without_a_guarantee_does_not_affect_another_read_of_the_same_state, failure_probabilities_add_along_the_selected_edges, a_producer_s_error_counts_only_on_its_selected_edge, the_milp_agrees_with_the_reference_enumeration_under_accuracy, end to end each_query_meets_its_target_at_its_root.
  • Complexity. Variables: $\le 5$ $E$ per execution that may carry an error, one $D$, one $U$; $F$/$H$ per (execution, input) with an approximate producer. Constraints: $O(5 \cdot |y|)$ plus one root check per (output, metric). Measured (After, below): 117/181 vs 78/110 variables/constraints for the p99, 583/958 vs 462/579 for Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207's example; solve 21–160 ms. Ceilings: $O(\text{executions} \cdot |\mathcal E| \cdot 5)$ worst case; in every workload of the test suite they settle within 8 rounds (instrumented run).
  • Assumptions and limits. Cost model 1 only (as PR 6/7). The MILP encodes the built-in target rule (metric_can_meet, no cross-metric conversion). A ceiling grows with the rounds on a cycle that carries error (bounded by the round count; none in the tests). Latency (L) and budgets (B) out of scope.

Deleted (replaced in this PR)

Deleted Replaced by
todo!("#1207 PR 8: constraint (K)") and the dead_code allowances on AccuracyVariables, accuracy_constraints implemented
#[allow(dead_code)] on HLL classic_guarantee, UnivMon distinct_guarantee, incomplete_top_layer_statistic read by distinct_count_guarantee_over_source
accepted_accuracy that accepted every sketch output in its target's metric without a check accepted_accuracy over the composed guarantee (model, evidence, accept-regardless), composed_guarantees
AccuracyVariables keyed by PhysicalCandidateId (Vec) keyed by execution and metric (HashMap)
accuracy_constraints(physical, selection, accuracy) accuracy_constraints(inputs, selection, variables, accuracy)

ErrorSelectingPlan::SelectedPlanFailedCheck (PR 0) is now constructed; its doc no longer mentions budget shares.

New abstractions not in #1207 or the doc (pending approval)

  • AccuracyCheckInputs::query_roots: &[QueryRoot] (pub; PlanningModels::accuracy_check_inputs now takes the roots): Stage 3 reads $G_q$ (values_per_result); a length other than the entries' is ErrorSelectingPlan::EntryCount.
  • asap_accuracy::estimators::distinct_count_guarantee_over_source(parameters, source, max_distinct) (pub): classic HLL / distinct-count UnivMon guarantee with δ bound at the source's declared distinct bound.
  • Stage 3 private (milp/error_bounds.rs): CandidateErrorModel (contracts per metric, read_error_on_edge, ceilings, composed_errors), ErrorBounds (+ EXACT), RootAccuracyLimits (+ EXACT), root_accuracy_limits, values_per_result_per_query, returns_its_output_unchanged, METRICS/METRIC_COUNT; private helpers read_bounds, statistics_read, read_guarantee, proven_non_negative, raw_data_samples_of_metric, source_distinct_bound.
  • AccuracyVariables beyond the doc's $E_c, F_{c,i}, D_c, H_{c,i}$: a metric index on $E$/$F$ (metric rule on selected paths) and $U_{\kappa,c}$ (without_guarantee, part of the output without a guarantee), plus per-execution keys (doc §4: $E_{\kappa,c}$ under contexts).
  • accuracy_variables (creates the family; PR 0 had only the struct and accuracy_constraints).
  • Rules Stage 3 states: (K) applies to all queries or none (the model/evidence may_… contract); a scalar root other than scalar(v) / a scalar subquery requires exact references; unknown δ counts as 1; read errors sit on edges (spec question 2).

Decisions taken conservatively (please confirm)

  1. IgnoreAccuracyGuarantees (PR 4 question). Ignore mode (and any model or evidence provider whose may_accept_regardless_of_guarantee / may_provide_evidence is true, including a custom model that keeps the trait defaults) turns off only (K) for every query; the check after the solve then decides, and an output nothing accepts is SelectedPlanFailedCheck (no re-solve). Pass 2's $\alpha^{\log}$ stays: parameter floors from the query targets, heap-size floors (heap ≥ k, a correctness need, not accuracy) and the metric rule's dropped edges ($\alpha^{\log}=0$) still apply, since Pass 2 and Stage 2 do not know the policy. So in ignore mode a plan meets every per-edge floor and only the composed root check is skipped.
  2. No per-query exemption. accepts_output_regardless_of_guarantee needs the realized output, so it is asked only after the solve; before it, only the model-wide may_… methods decide.
  3. DDSketch interior quantile / Count-Min (the PR 4 handoff's sign condition): a read has a guarantee only when the update's weight domain, confirmed by the data workload, is non-negative, or its raw samples' metric is declared non-negative (counter or values_non_negative); otherwise no guarantee ($U = 1$).
  4. A merge's read error is the largest per metric (and largest δ) over every statistic read from the merge's group, directly or through further merges (a readout's is its own statistic's).

Spec questions

  1. Whole-result failure probabilities are unreachable with the lists (PR 1). Every listed θ fails with probability 0.01 per value, so a whole-result target δ = 0.01 over 1,000 series allows $10^{-5}$ per value: no KLL/CMS meets it. Before Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207, Pass 1 sized δ per value (with_failure_probability). Options: (a) add δ variants to the lists (KLL failure_probability, CMS depth); (b) keep as is (such queries are exact or infeasible). 4 planner tests now fail for this (below); I did not change their workloads.
  2. Read error placement. The doc puts $\varepsilon^{\mathrm{loc}}c$ on the candidate; a readout has no θ and a state serves reads of different statistics, so the error is a constant per edge $(u, c)$: $E_c \ge \sum_u \varepsilon^{\mathrm{loc}}{u,c} y_{u,c,i} + \sum_i \beta_{c,i} F_{c,i}$ (linear: one $y$ per input). Update the doc's formula?
  3. Error metrics in $E$. The doc's $E_c$ is one number; a value-preserving consumer (projection, Concat) can carry a rank error from one producer and a relative error from another, so $E$ is per metric here. OK?
  4. Evidence and custom models turn (K) off for all queries (their may_… defaults are true): a deployment that supplies evidence for one query loses (K) for all, and the post-solve check may then reject the cheapest plan with no re-solve. Keep, or should (K) stay on and evidence only be read after the solve?
  5. Sizes tie in the objective. Above the floor, sizes differ in cost only through memory held during an evaluation, which the objective leaves out (PR 7 spec question 2): e.g. p99+p50 selects KLL k 1404 and Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207's example DDSketch alpha 0.002 where k 269 / alpha 0.01 meet the targets at the same objective.

Lower-PR items (not changed)

  • RejectionCause::CountMinWeightsNotProvenNonNegative is never constructed: PR 6 deleted the executability check (nodes_with_proven_non_negative_weights); a Count-Min over unproven weights is now only "no guarantee" under (K), and in ignore mode it can be selected and the executor refuses it (PR 5/6).
  • RejectionCause::{LatencyBoundExceeded, LatencyOverRepeatInterval, OverClusterCapacity} never constructed ((L), (B) out of scope; PR 6).
  • raw_data_samples_of_metric is duplicated: Pass 1's (realization.rs) is crate-private; Stage 3 needs the same walk and cannot depend on Pass 1. Moving it to asap-types changes PR 3.
  • docs/design_docs/proposals/planner-layering.md lines ~277, 313, 372–421 (memo concepts table and the ratio-of-quantiles example) still describe root_node, input_groups, accuracy_budget_parts and pre-Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 counts (PR 0/PR 3 doc; numbers come from PR 9's re-baselined stage_pipeline). crates/devtools/src/bin/asap_planner.rs:90 comment names accuracy_budget_parts (PR 9 tool).
  • PR 4: local_guarantee gives a Count-Min TopK read a frequency bound (old code had none for top-k membership); unchanged.

Before this PR

No query's composed accuracy was checked: sizes were bounded only by the per-edge parameter floors (pruning caps), a target whose metric a family does not state left its inputs unbounded, and every sketch output was reported as accepted in its target's metric. p99 over 60m at rank 0.01 selected DDSketch alpha 0.1 windows; #1207's ratio could compose to 0.044 against a 0.03 target.

After this PR

(K) holds for every selected plan unless the deployment's model or evidence may accept outputs regardless; the composed guarantee is rechecked after the solve. Representative plans (asap-planner --output text stops at PR 9 todo!()s, so these come from the same plan_installable call through a scratch test; cost model 1 defaults; fixed_interval_at 10 min; 1,000 series every 1 s, values_non_negative; "without (K)" is the same build with ignore_accuracy_guarantees, which selects exactly as PR 7):

Workload With (K) Without (K) (PR 7)
p99 over 60m, rank 0.01, δ 0.01 whole result, no raw data infeasible (spec question 1) 6 ingestion DDSketch alpha 0.1 windows, 3.352/s
same, raw data retained exact plan (scan, time range, aggregate), 4.336e-2/s query-time DDSketch alpha 0.005, 3.844e-2/s
p99 over 60m, rank 0.01, δ 0.01 per value (or no δ), no raw data 6 ingestion KLL k 269 windows, 7.825/s; MILP 96 var / 136 constr, 21 ms DDSketch alpha 0.1, 3.352/s; 78 / 110, 24 ms
p99 and p50 over 10m, rank 0.01 per value, raw data one shared KLL (k 2863), 7.235e-3/s; 488 / 617, 64 ms one DDSketch alpha 0.005, 6.249e-3/s; 452 / 565, 182 ms
#1207 example: A p95 rank 0.01; B p99/p50 relative 0.03, raw data A: KLL (k 2863); B: one DDSketch (alpha 0.001) read by both operands; 1.265e-3/s; 498 / 633, 140 ms one DDSketch alpha 0.005 for A and B, 8.403e-4/s; 462 / 579, 235 ms
sum by job of rate 5m, exact unchanged (exact), 3.905e-3/s; 60 / 78 same

Sizes above the floor tie in the objective (memory held during an evaluation is not in it, PR 7 spec question 2), so k 2863 / alpha 0.001 are as good as k 269 / alpha 0.01 to the MILP; plan_cost differs.

Verification

  • cargo fmt --all; cargo clippy --workspace --all-features --all-targets -- -D warnings clean; crate_graph passes.
  • New tests. stage3 (model): the_problem_example_meets_each_query_target_at_its_root (Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 Problem: A via KLL, B via one shared DDSketch at alpha 0.01 where 0.02 composes to 0.044, metric rule at the root; without (K) the weakest sizes), a_ddsketch_interior_quantile_needs_non_negative_values (infeasible undeclared, feasible declared), failure_probabilities_add_along_the_selected_edges (δ sums, a producer reached twice counted twice: 1.5δ infeasible, 2δ met), a_producer_s_error_counts_only_on_its_selected_edge (big-M only on selected edges), a_read_without_a_guarantee_does_not_affect_another_read_of_the_same_state (regression for review finding 1), the_milp_agrees_with_the_reference_enumeration_under_accuracy (reference over DAGs filtered by composed errors; 12 cost/target sets incl. infeasible ones). stage3 (error_bounds): root_limits_accept_the_target_s_metric_and_divide_delta_over_its_scope, a_read_s_error_is_its_statistic_s_on_the_producer_s_parameters. accuracy: a_distinct_count_over_a_source_binds_its_failure_probability. planner (end to end): each_query_meets_its_target_at_its_root, ignoring_accuracy_guarantees_drops_k, a_whole_result_failure_probability_is_divided_over_the_values.
  • Independent review (fresh agent) of the diff. Fixed: read errors per edge and statistic instead of per state (one query's read changed another's); (K) for all queries or none, following the model/evidence may_… contract (per-query exemptions left per-edge rows on exempt paths, and evidence could never apply); one $G_q$ for (K) and the check; a short query_roots is an error; no ceiling cap (it could make unrelated plans infeasible); ceilings bounded by executions, not candidates; HiGHS feasibility tolerances $10^{-9}$ under (K); renamed OwnGuarantee → ErrorBounds; stronger tests. Not taken: ceilings over the SCC condensation (no error-carrying cycle in any tested workload: instrumented run, all settle within 8 rounds); a per-(merge, statistic) variable family (merges take the largest over their group's statistics; rollup merges are offered only for reads whose merged guarantee is kept); validity-cap and scalar-root tests (no listed size reaches the cap $b = 0.1$ with error above it).

Test table (cargo test -p <crate> --all-features --no-fail-fast, at 55a1384; PR 7: 1341/94/9):

crate passed failed ignored
asap-executor 281 0 1
asap-types 197 0 0
asap-frontend-promql 218 0 0
asap-frontend-common 49 0 0
asap-planner 25 9 1
asap-accuracy 55 0 0
asap-cost 37 0 0
asap-frontend-metricsql 15 0 0
asap-frontend-sql 175 0 0
asap-sql-function-catalog 10 0 0
asap-stage0-memo 13 0 0
asap-stage1-pass1-per-computation-candidates 33 0 0
asap-stage1-pass2-reuse-candidates 20 0 0
asap-stage2-physical-optimization 15 0 0
asap-stage3-plan-selection 28 0 0
metricsql_common 3 0 0
asap-devtools 42 52 3
asap-integration-tests 134 36 4
total 1350 97 9

Compared with PR 7: fixed joint_search_equivalence::custom_accuracy_policy_is_respected (the policy hook: satisfies is called on the composed guarantee) and production_rollup_workloads::production_sql_distinct_region_service_rollup. New failures, all stopping at spec question 1 (whole-result δ 0.01 over 1,000 series, lists fail 0.01 per value; workloads unchanged): planner columnar_batches::a_tumbling_window_is_built_from_arrow_columns_with_one_label_entry_per_series, runtime_install::{the_plan_carries_stage3s_predicted_usage, the_plan_file_carries_theta_and_coverage_and_reads_back, tumbling_windows_of_an_hour_share_one_definition_with_distinct_times}; and sharing_issue_regressions::grouped_filtered_quantile_plans (δ 0.001 targets → exact plans; the exact plans tie in the objective, 0.196/s, and plan_cost of the one picked is 0.508/s vs 0.216/s: PR 7's inexact objective; it failed in PR 6 and passed in PR 7 by the same tie). Other failures stop where PR 7's did: devtools 37 at PR 9 todo!()s, 13 fuzz inconclusive (witness with ASAP operators through Pass 1, PR 9), 2 CLI JSON (PR 9); planner 5 as PR 7 (Pass 2 metric rule / raw data, PR 4; Stage 2 executor-support gap, PR 5; ignoring_accuracy_guarantees_plans_the_undeclared_p99_query is infeasible with (K) off, Stage 2/PR 5); integration 35 as PR 7 (Stage 2 infeasible outputs, PR 5/Pass 1; pinned plan shapes and θ-less snapshots, PR 9; joint_search_equivalence::rootless_request_overhead_respects_capacity_and_latency needs latency (L), out of scope; different_windows_and_raw_retention_policies_produce_valid_plans Stage 2 infeasible with raw data off, PR 5; avg-over-quantile / sum contracts stop at the PR 4 decision). No failure stops at a PR 8 todo!(); none is SelectedPlanFailedCheck.

…ailure probability at the source's distinct bound (#1207 PR 8)

distinct_count_guarantee_over_source reads classic HLL's and a distinct-count
UnivMon's guarantees (kept dead since PR 4 for this) and binds their failure
probability at SourceFacts::max_distinct_per_evaluation, for Stage 3's (K).
CandidateErrorModel (milp/error_bounds.rs): each physical candidate's own
guarantee (a sketch state's: the largest error per metric and failure
probability of the statistics read from its group; none for a read whose
bound needs values not proven non-negative, a windowed or merged state that
does not keep the guarantee, or a family without an error model), its error
contract per metric (error_propagation), and ceilings of E and D used as M.
AccuracyVariables E/F per execution and metric, D/H, U (no guarantee);
accuracy_constraints: E >= eps_loc + sum beta F, F >= E_u - M(1-y), the
metric rule and validity caps on selected edges, D by a union bound, and the
root checks E_v <= eps_q, D_v <= delta_q / n, U_v = 0 when z = 1, for every
query the accuracy model does not accept regardless of its guarantee.
After the solve, each query's composed guarantee is checked with the model
and evidence; an output none accepts is SelectedPlanFailedCheck.
AccuracyCheckInputs carries the front ends' query roots (G_q and the
accept-regardless policy). Tests: #1207's problem example, the DDSketch sign
condition, failure probabilities adding, big-M only on selected edges, and
the reference enumeration under (K).
…ee per metric; an exact target's limits are exact (#1207 PR 8)
…ignore mode; a whole-result failure probability over 1,000 series (#1207 PR 8)
@zzylol zzylol added accuracy Accuracy requirements, guarantees, and propagation optimizer Workload optimization and plan selection refactor labels Nov 7, 2026
…its edge; (K) applies unless the model or evidence may accept outputs regardless; one G per query for (K) and the check; no ceiling cap; tighter HiGHS tolerances under (K) (#1207 PR 8)

- The error of reading a state is a constant per edge (u, c) for the
  consumer's statistic(s) on the producer's theta (E_c >= sum eps_uc y + ...),
  so one query's read no longer changes another's (a DDSketch read at the
  maximum keeps its guarantee when a p50 read of the same state has none).
- root_accuracy_limits follows the AccuracyModel/AccuracyEvidenceProvider
  contract (may_accept_regardless_of_guarantee, may_provide_evidence): (K)
  applies to every query or none; the per-query pre-ASAP accept-regardless
  call is gone.
- values_per_result is computed once from the lowered roots and used by
  both (K) and the check after the solve; a short query_roots slice is an
  EntryCount error.
- Ceilings stop after one round per execution (a valid M for every acyclic
  selection) with no cap, which could make unrelated plans infeasible.
- OwnGuarantee renamed ErrorBounds (read errors and composed errors).
- Tests: the regression test above; the DDSketch sign test checks the
  declared case too; the big-M test's description matches it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

accuracy Accuracy requirements, guarantees, and propagation optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant