Skip to content

Latest commit

 

History

History
138 lines (124 loc) · 8.26 KB

asap-aware-mapping-contracts.md

File metadata and controls

138 lines (124 loc) · 8.26 KB

ASAP-Aware Mapping contracts

Audience: developers changing the planner. This reference states the rules the stage code relies on: what each stage's input and output may contain, and which component owns each decision. Read the architecture first; use the extension guide when changing one. Field-level details are in the rustdoc (cargo doc -p asap-stage0-memo -p asap-stage1-pass1-per-computation-candidates -p asap-stage1-pass2-reuse-candidates -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-types --no-deps).

1. IR at each stage

After The DAG holds Timing
Stage 0 (frontends) NonASAPOp nodes only; aggregates carry AggIntent measures with each query's AccuracyTarget as written none
Stage 1 (Pass 1 and Pass 2 rules) ASAPOp nodes where a candidate chose an exact accumulator or a summary (SummaryAgg, SummaryTumblingWindowChain, SummaryMerge, SummaryEstimate, FinalizeExactAccumulator); everything else stays NonASAPOp none: Stage 1 never decides when anything runs
Stage 2 (generate_physical_asap_memo_candidates) and Stage 3 (select_plan, which builds the plan) the same operators, with a timing and a Lifetime on each summary build every node of the plan timed from the physical candidate it was built from (selected_physical_candidate)
  • Structurally equal sub-DAGs are interned before Stage 1 (cse.rs in asap_stage0_memo), so Stage 1 compares nodes by structure and Pass 2 matches readers structurally, never by pointer.
  • Timing is applied with apply_physical_properties, which validates every data-state edge. A violation is an ExecutionDataStateError: for example QueryTimeValueUnderIngestionSummary (a query-time value beneath an ingestion-time build), ConflictingTiming (one shared node needs two timings; split_shared_by_timing copies it for one side first) or IllegalChildDataState. These are internal errors in a candidate the stages built, not bad user input.
  • A summary build's AggregateCoverage (which rows and time range its state represents) is a memo-group property, derived by the memo, never declared; a merge's inputs must be disjoint as its family's MergeInputRule requires, or coverage fails (InputsOverlap).

2. Stage 1 candidates

  • Targets. ComputationTarget is one measure of one Aggregate. Its ways_to_compute list the exact option first (ExactAccumulator when the intent has a mergeable exact state, else Exact), then each Summary(family) from summary_realization::summary_candidates. Candidates are unranked: Pass 1 never compares cost.
  • Pruning. A candidate that cannot meet the target is moved to the inventory's pruned_ways_to_compute (with its target) with a PruneReason (RequiresExact, NullableAvgInput, UnsupportedByDeployment, QuantileOutsideUnitInterval); it is never silently dropped. Pass 1 prunes only what it can prove; accuracy that depends on data facts is left to Stage 3.
  • Fused computations. FusedComputation { absorbed_targets, ways_to_compute } replaces the targets beneath (the whole-expression top-k, the nested sum). It absorbs the target directly beneath and, for a heap over the raw data samples beneath a grouped sum of per-series sums or counts, the per-series target too. A whole-workload choice keeps one digit per target; a fused pick with a non-first digit on an absorbed target is skipped, so each plan is enumerated once.
  • Pass 2. Each rule adds candidates; none removes a Pass 1 candidate, adds a summary family, or adds a readout. So a deployment that does not support a family never sees it again after Pass 1.
  • Sizing. A summary shared by several readers is sized with each parameter's largest value among them. A summary is sized for ε in Stage 1; a δ that depends on data facts (a distinct-count bound, series counts) is left symbolic in its guarantee for Stage 3 to bind.

3. Accuracy

  • A summary's guarantee is the deployment's measurement (AccuracyEvidence::measured_guarantee) when there is one, else its algorithm's error formula (theoretical_guarantee); Pass 2 compares it with each readout's RequiredAccuracy (producer_satisfies_logical_requirements). It fails closed: no rule means no guarantee, and the edge is not admissible.
  • A target with no accuracy_metric accepts a guarantee in any metric that meets its ε, δ and scope; Stage 3 reports the metric it accepted (AcceptedAccuracy).
  • An approximate input split across several inputs gets its share through accuracy::budget::per_input_target.
  • Facts come only from the declared DataWorkload; an undeclared fact is unknown, never zero.

4. Stage 2

  • Only summary builds (SummaryAgg, SummaryTumblingWindowChain) get materialization options, each a timing and a lifetime (PhysicalProperties): not materialized always; materialized at ingestion time, AcrossEvaluations(Memory), when the build is eligible to run as data arrives; materialized at query time for OneEvaluation(Memory) when two or more queries read it (reused within one evaluation); materialized at query time AcrossEvaluations(Memory) for a tumbling-window chain that builds one new tumbling window per evaluation, when the deployment declares query_time_results_retained.
  • One PhysicalCandidate per assignment, the all-not-materialized one first (P<n>, then P<n>-m<k>). An assignment that puts a build at ingestion time over one that is not is not generated. A logical candidate Stage 2 cannot implement becomes a PrunedCandidate with its reason.
  • On the memo (generate_physical_asap_memo_candidates), every physical candidate Stage 2 prunes is recorded in PhysicalASAPMemoCandidates::pruned_candidates as PrunedCandidate<PhysicalCandidate> with a PruneReason: an ingestion-time alternative without an ingestion pipeline (NoIngestionPipeline), a query-time scan of raw data the deployment does not retain (RawDataNotRetained) and a materialized query-time result the deployment does not retain (QueryTimeResultsNotRetained) are not generated; a candidate no buildable input or query output can use is removed (InputHasNoBuildableProducer, ReachesNoQueryOutput). The planner does not check executor support. Stage 2 emits one candidate per operator node, timing and lifetime; it makes no execution-scope copies.

5. Stage 3

  • select_plan validates its inputs first: ErrorSelectingPlan::InvalidSettings and EntryCount are bad input and fail the whole selection. The facade validates the deployment's capabilities before any stage (asap_planner::SelectionError::InvalidDeploymentCapabilities).
  • Each candidate is then either invalid or valid, recorded as a Rejection with its cause (RejectionCause) and reason: a failed check (AccuracyTargetMissed, CountMinWeightsNotProvenNonNegative, NoAccuracyGuarantee, LatencyBoundExceeded, LatencyOverRepeatInterval (a repeating query's latency per evaluation is over its repeat interval, checked with or without a latency bound), OverClusterCapacity under cost model 3, or CostNotRepresentable (its cost overflows)) or a loss under the cost model (Costlier, Slower, MoreMemory, TiedLostOnEnumerationOrder). Stage 3 never resizes a summary: a candidate whose Stage 1 sizes miss its composed accuracy target is rejected alone.
  • Usage (ResourceUsage) is measured without prices; the CostModel in CostInputs turns it into a choice. Ties keep enumeration order under every cost model.
  • Selection.plan is the selected candidate's roots with the weight requirements Stage 3 confirmed from the data facts.

6. Ownership

Decision Owner
What a query means the frontend (Stage 0)
Which computations exist and how each may be computed Pass 1
What is shared, split into tumbling windows or reused Pass 2
When each build runs and whether it is kept Stage 2
Which plan is valid and which is chosen Stage 3
What the executor can build, retain and how it is priced the deployment, through PlanningModels (DeploymentCapabilities, CostInputs, AccuracyEvidence)
Running the plan the executor (asap-executor), outside the planner