Skip to content

feat(analytics): sketch analytics for every sketch and framework - #262

Merged
GordonYuanyc merged 20 commits into
mainfrom
feat/sketch-analytics
Oct 30, 2026
Merged

GordonYuanyc merged 20 commits into
mainfrom
feat/sketch-analytics

Conversation

@GordonYuanyc

@GordonYuanyc GordonYuanyc commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Implements docs/sketch_analytics.md (#253): every sketch and framework reports its theoretical memory, update/query/merge ops, error bound per statistic, and the parameters that meet a target (ε, δ), for ASAPPlanner to call. ASAPPlanner pins asap_sketchlib by rev, so it moves to this API in its own PR.

One commit per step; review commit by commit.

Commits

Commit What
fix(fold)! CountMinSketchFold::new / CountSketchFold::new panic for full_cols > 65536 (MAX_FULL_COLS): a full column is a u16.
fix(bloom)! Bloom::dimensions_for returns Option, widens until the exact rate (1 − (1 − 1/c)^n)^r meets the target, and returns None past BLOOM_MAX_BITS; with_capacity panics there. predicted_fpp uses the exact rate.
fix(capability)! CountMinSketch Total is AsIfBuiltOverUnion, not Exact: under Nitro sampling it is an estimate.
docs(univmon) UnivMon's distinct bound is 1.35/(kε²), the variance the proof derives; the tests check it.
feat! rename estimated_memory_bytes → theoretical_memory_bytes everywhere.
feat(analytics) trait SketchAnalytics, ErrorBound, ErrorMetric, NormScale, StatisticKind::QuantileInterpolated, shared helpers (P_med, est(d), h(k), lg).
feat(analytics) op-count op-count feature: analytics::ops::count(f) returns the ops f performed. Off by default and free when off.
feat(analytics) × 7 One per doc family: counter matrices, heavy hitters, counter matrices with top-k, cardinality and membership, quantiles and sampling, UnivMon family, frameworks. Each adds the trait impls, the op instrumentation, and its tests in tests/analytics/.
style(analytics) clippy and rustdoc.
docs(tests) docs/tests.md rows for the new tests.
fix(analytics) × 4 Fixes from an independent review, below.

Decisions

  • CountSketch sizing. ItemFrequency takes a chosen δ: ε = √(q/β) with β the largest per-row rate where P_med(d, β) ≤ δ. params_for picks the odd d ≤ 19 and width with the fewest counters, which gives d = 5, w ≈ 9.5/ε² at δ = 0.01 (ASAPPlanner's per-row 1/3 gave d = 47, past ASAPv1's 20 rows). Contract count_sketch_l2_median_chosen_row_rate_v1; the heap, Space-Saving, fold, L2-norm and UnivMonQ variants follow it.
  • Kll Rank. rank() returns a count, so Rank is Absolute(L1) with a fit measured on asap_sketchlib: ε = 2.07·s(δ)/k^0.9646 at 99%, contract kll_rank_measured_99_v1. Measured: 99th percentile of |rank(x) − true rank|/n over 1000 seeded trials × 9 query points, n = 10^6, m = 8, k from 32 to 4096; the regression is 1.8647/k^0.9646, and 2.07 makes it an upper envelope of every measured k.
  • Statistics. The trait covers StatisticKind only. Doc rows for Min, Max and Sum (DDSketch, UnivMonQ) and for Totals without an accessor (CocoSketch, SpaceSaving, UniformSampling) are dropped, matching the capability table.

Formulas corrected by counting

The op tests count every access under op-count and check it against the formula; the doc now states what the code does. The larger corrections:

  • h(k) (a TopKHeap update): 8 + 16·⌊log2 k⌋ → 8 + 14·⌊log2 k⌋.
  • SpaceSaving: update ≤ 47, merge 44m + 2m·lg(2m) + 13.
  • Kmv: update k + 3 + 6·⌊log2 k⌋. Bloom: update r on both paths (no inserted count).
  • Kll: update average 15 + lg(c_b + 1), merge B·lg B + 16R + 8L + 40; queries count L(n) + 1 levels.
  • Fold same-level merge, UnivMon merge and query, Hydra, ExponentialHistogram and TumblingWindow costs were undercounted and are restated.
  • ElasticSketch's δ is e^−min(d, 5): light row 5 shares the heavy part's seed.

Other public API

  • memory_bytes() on the sketches and frameworks that lacked it (CHANGELOG lists them); SpaceSaving, Bloom, UnivMonTwoTier and UnivMonQ gain an inherent theoretical_memory_bytes; SetAggregator::contains.
  • Breaking: CountMinFastOps and CountSketchFastOps require memory_bytes.

Independent review fixes

Eight reviewers read the diff against the spec without the author's conclusions; the confirmed findings are fixed in the four fix(analytics) commits:

  • theoretical_memory_bytes returns Option<usize>: None when it needs an input the section gives no bound for (key, label or mean key bytes), where an unknown key length used to count as 0.
  • A merged: Option<bool> data input: SpaceSaving, ElasticSketch, the folds' TopK and Hydra's Kll readouts give a bound only for an unmerged state.
  • Sizing that met a different bound than theoretical_error reports (HLL Classic), returned parameters new rejects (fold level for W < 8), or returned None too early (CountSketchWithL2Norm under the 128-bit hash, DdSketch for ε ≥ 1).
  • Octo's τ_max/τ_min include the initial threshold; UnivMonQ's Quantile/Rank condition reads F̂₂·M < n² instead of a flag; DdSketch checks the indexable range; Kll Rank needs k ≥ 32; HLL uses Flajolet's constants below 128 registers; Hydra's and EH's HLL Distinct have no bound (a standard error mixed with a probabilistic term).
  • Op models: std BinaryHeap::pop (7 + 5⌊log2 n⌋), TopKHeap::clear (2 per resident), SpaceSaving merges of unequal capacities.
  • Op counting whose argument did work (cells()) is behind #[cfg(feature = "op-count")], so the default build does none of it.
  • Tests that could not fail (UnivMonQ ItemFrequency, Hydra Count-Min) and weak ones (UnivMon Distinct, UnivMonQ quantiles on ordered data, FastPath non-power-of-two widths) are rebuilt and mutation-checked.

Tests

tests/analytics/ (156 tests with all features, 108 without op-count): memory against memory_bytes(), counted ops against each formula at two or more sizes, error bounds as binomial failure rates over independent trials, and sizing against its own bound. Every error test is mutation-checked: breaking the sketch's update or estimator (a dropped row, an ignored sign, a stale heap, a wrong compaction coin, …) turns it red.

Local: cargo fmt --check; cargo test --test asapv1_golden; clippy with -D warnings for default, --all-features, octo-runtime and experimental; cargo doc with -D warnings; cargo test --test analytics with default and all features; cargo test --doc; the lib tests of each sketch whose source changed. The pbt and e2e suites run in CI.

Not covered

  • UnivMonQ readouts other than ItemFrequency have no op formula (they rebuild candidates through maps, sorts and isotonic regression).
  • Octo is a runtime without one sketch type: its analytics are free functions (octo_error, octo_memory_bytes, octo_update_ops).
  • Error tests cover the main readout of each family; Hydra's Kll/HLL/UnivMon cells, EH by mass and EHUnivMon's UnivMon tier have memory, ops and sizing tests only.

🤖 Generated with Claude Code

GordonYuanyc and others added 20 commits October 29, 2026 16:24
A full column is stored as u16, so a wider sketch folded distinct
columns together. CountMinSketchFold::new and CountSketchFold::new now
panic above MAX_FULL_COLS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dimensions_for sized slices from the approximation e^(-n/c) and capped
them at BLOOM_MAX_BITS, so it could return a filter above the target.
It now widens until the exact rate (1 - (1 - 1/c)^n)^r is at most the
target and returns None when that needs more than BLOOM_MAX_BITS;
with_capacity panics there. predicted_fpp uses the exact rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With Nitro sampling enabled, Total sums sampled counters and is an
unbiased estimate, so the family row marks it AsIfBuiltOverUnion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The proof bounds the heap estimator's variance by 1.35 F0^2/k and the
docs rounded it up to 2. The UnivMon and UnivMonTwoTier pages now state
1.35/(k eps^2), and the cardinality tests check it (2000-trial runs
pass).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the analytics module with the trait each sketch implements, the
error-bound types, and the shared formula helpers (P_med, est(d), h(k),
lg). StatisticKind gains QuantileInterpolated, answered by DDSketch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
analytics::ops::count(f) returns the ops f performed on this thread.
Vector2D, the fixed matrices, BitMatrix and the median helper count
their cell reads and writes; without the feature the counting compiles
to nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for CountMinSketch, CountSketch, CountSketchWithL2Norm,
and the experimental CountMinHll and MicroCm, with memory, op-count,
error and sizing tests in the analytics test target.

CountSketch's ItemFrequency bound takes a chosen delta: the largest
per-row rate beta with P_med(d, beta) <= delta gives eps = sqrt(q/beta),
and params_for picks the odd d <= 19 and width with the fewest counters
(d = 5, w ~ 9.5/eps^2 at delta = 0.01).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for SpaceSaving, ElasticSketch and CocoSketch, with
memory, op-count, error and sizing tests in the analytics test target,
and memory_bytes() on all three.

SpaceSaving: update 47 ops (from 45), merge 44m + 2m·lg(2m) + 13 (from
36m + 2m·lg(2m) + 17), TopK 1 + 7·min(k, m), memory with the index under
churn; Total has no accessor and leaves the table.
ElasticSketch: delta = e^-min(d, 5), update 2 + d, TopK H·(5 + d) +
2H·lg H, merge H·(13 + 3d) + 2dw.
CocoSketch: update d + 1, merge dw·(d + 2); Total has no accessor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for CountMinSketchWithHeap, CountSketchWithHeap,
CountMinSketchWithSpaceSaving, CountSketchWithSpaceSaving,
CountMinSketchFold and CountSketchFold, with memory, op-count, error and
sizing tests in the analytics test target, op counting in TopKHeap,
CommonHeap and the fold cells, and memory_bytes() on the Space-Saving
variants and the folds.

TopKHeap: h(k) = 8 + 14·floor(log2 k) (from 16 per level), exact for a
root-to-leaf descent.
Heap variants: merge 2dw + 3 + u·(3 + est(d) + h(k)).
Space-Saving variants: update d + 47 (from d + 45), TopK
m'·(5 + est(d)) + m'·lg m' (from 7), merge 2dw + 44m + 2m·lg(2m) + 13
(from 2dw + 36m + 2m·lg(2m) + 17).
Folds: same-level merge dW/2^F + d·min(W, D)·(2 + E) +
min(k, D)·(1 + dE + est(d) + h(k)) (from dW·2^F + k·(dE + est(d) + h(k)),
which undercounted), hierarchical merge dW + Σ(dW/2^F_i + 2e_i +
k_i·(1 + est(d) + h(k))).
TopK error multiplies the per-key δ by D (heap, folds) or min(m, D)
(Space-Saving); sizing rows added for ItemFrequency and TopK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for HyperLogLog (Classic, Ertl), HyperLogLogHip, Kmv,
SetAggregator and Bloom, with memory, op-count, error and sizing tests in
the analytics test target, op counting in the HLL, HIP, Kmv and
SetAggregator code paths, memory_bytes() on SetAggregator and Bloom,
Bloom::theoretical_memory_bytes and SetAggregator::contains.

Classic HLL with max_distinct takes a chosen delta: the smallest eps from
max_distinct/(2(m - max_distinct)) to 1 with max over every D of
lambda^t/t! at most delta. Bloom params_for minimizes r*c over r <= 20 and
power-of-two c.

HyperLogLogHip update: 6 (from 4).
Kmv: update k + 3 + 6*floor(log2 k) (from 8 per level), query 1 (from 2),
merge k*(k + 4 + 6*floor(log2 k)); D alone as data.
SetAggregator: Distinct query 0 (a length), merge 2D.
Bloom: update r on both paths (from r + 1 / 2r + 1; there is no inserted
count), merge 2r*ceil(c/64) (from + 2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for Kll, KllDynamic, DdSketch and the experimental
UniformSampling, with memory, op-count, error and sizing tests in the
analytics test target, op counting in their code paths, and
UniformSampling::memory_bytes().

Kll and KllDynamic: Quantile on the 2.296/k^0.9723 fit and Rank as
Absolute(L1) on a fit measured on asap_sketchlib, 2.07/k^0.9646, both with
a chosen delta scaled by s(delta) below 0.01 and None for m < 8. A state
holds L(n) - 1 to L(n) + 1 levels, so query and merge bounds use L(n) + 1.
Kll: update average 15 + lg(c_b + 1) (from 26 + ...), quantile query
3R + R lg R + 2L + lg R + 2 (from 3R + R lg R + lg R), rank R + 2L (from R),
merge B lg B + 16R + 8L + 40 (from B lg B + 25R - 11B + 10L + 2(L_a + L_b)
+ 74).
KllDynamic: update average 11 + 2 lg(c_b + 1) (from 5 + lg(c_b + 1) +
lg(1.5k), exceeded for small k), merge S + 16R + 8L, queries as Kll.
DdSketch: Quantile and QuantileInterpolated only (no sum, min or max),
update 4 (from 6), quantile B + 4, interpolated 2B + 7, merge
2(B_a + B_b) + 11.
UniformSampling: Sample only, update 3 + lg s + 2s, query 2s, merge
3 + 2(s_a + s_b) + (s_a + s_b) lg(s_a + s_b).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for UnivMon, UnivMonTwoTier and UnivMonQ, with op-count
instrumentation, memory/ops/error/sizing tests, and
UnivMonTwoTier::memory_bytes, UnivMonTwoTier::theoretical_memory_bytes and
UnivMonQ::theoretical_memory_bytes.

UnivMon and UnivMonTwoTier: Total exact, FrequencyL2 as
CountSketchWithL2Norm on layer 0, Distinct at a chosen delta with
eps = sqrt(1.35/(k(delta - psi))). Per-layer heap terms recounted with
h(k) = 8 + 14 floor(log2 k); queries add the total reads and the per-layer
partial sums (4 + 2L); merge counts the candidate set reads and inserts
and the flags (3dw + d + 6 + c(22 + 4d + 2d lg d + 28 floor(log2 k))).

UnivMonQ: ItemFrequency follows Count Sketch (delta chosen,
eps = sqrt(q/beta)), with q the most collision probability over the
layers (1/w_j for power-of-two widths). Quantile and Rank bounds need
UnivMonQData::heavy_set_empty. Total/Min/Max rows dropped (not in the
capability row). BinaryHeap pushes and pops are counted at their sift
worst case; update, ItemFrequency query (1 + 2d + d lg d) and a new merge
formula follow those counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SketchAnalytics for Hydra, ExponentialHistogram, EhBucketSketch,
ExponentialHistogramUnivMon (experimental), NitroBatch<S: NitroInner>,
TumblingWindow<S> and HashSketchEnsemble; Octo as functions over
OctoConfig (octo_error, octo_memory_bytes, octo_update_ops). Enum-cell
frameworks dispatch per cell kind (HydraCell, EhCell, EnsembleCell);
generic ones compose the inner trait through InnerSketch, whose C_in is
the inner heap over its cell size. Clones, clears and the frameworks'
own grids, bucket lists, pools and sketch lists are op-counted, and
memory_bytes() is added where missing.

Doc corrections: Hydra update (2^a-1)d(1+U_in), query (d+1)Q_in+2d+d lg d,
merge dw(2+M_in), group-total ops, Kll Quantile as Rank; EH bucket-list
slots, concrete average update and query ops, FrequencyL2 as the only
by-mass readout, no DdSketch Min/Max; EHUnivMon has no ItemFrequency
readout; Nitro merge is M_in; Tumbling slot memory, per-boundary
4+2*max_windows+C_in, query C_in+t(1+M_in), no Kll-window bound;
Octo parents hash on RegularPath and the HLL parent is Classic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SketchAnalytics::theoretical_memory_bytes returns Option<usize>; None
  when it needs an unknown input with no upper bound (key bytes, label
  bytes, SetAggregator D and l, Uniform n, EH and EhUnivMon bucket counts).
- Tumbling update ops are unbounded when b is unknown.
- ElasticSketch key bytes are the passed String's capacity.
- HashSketchEnsemble cells carry each matrix's counter bytes.
- UnivMonQ clones and decodes reserve the modelled heaps and tables.
- Hydra escaped labels own exactly their length.
- Costly op-count arguments compile only with the op-count feature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SpaceSaving, ElasticSketch, the folds' TopK and Hydra's Kll readouts
  take a `merged` data input and give a bound only for Some(false).
- SpaceSaving merge ops take the merged-in capacity `other_capacity`:
  34m + 10m_o + (m + m_o)·lg(m + m_o) + 13, unbounded when unknown.
- The Space-Saving matrices' TopK error and sizing use min(m, D), m when
  D is unknown.
- The spec states a unit SpaceSaving update costs at most 47 in any state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HLL Classic: the collision bound only when its ε is at most β_m/√m,
  and sizing returns a precision whose own bound meets the request.
- HLL: Flajolet's β_m for m ≤ 128, no bound below p = 4.
- Fold sizing caps the fold level at log2 W.
- Octo τ_max covers the initial threshold and the clamped band.
- UnivMonQ Quantile/Rank read F̂₂ and need F̂₂·M < n²; sizing needs n and F̂₂.
- CountSketchWithL2Norm ItemFrequency sizing searches only shapes within
  128 hash bits; regular-path Count Sketch sizing keeps w ≥ 1.
- DdSketch: α capped at 1/2, α outside (0, 1) and magnitudes outside the
  indexable range or negative/NaN have no bound.
- normal_upper_quantile for Kll's s(δ) at tiny δ.
- Kmv rejects NaN and negative δ.
- Hydra HyperLogLog Distinct has no bound or sizing.
- Kll Rank fit holds for k ≥ 32.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- UnivMonQ: a BinaryHeap pop counts 7 + 5⌊log2 n⌋, mirrored against std;
  update and merge formulas follow.
- TopKHeap::clear counts 2 per resident; heap-matrix and UnivMon merge
  formulas follow. h(k) notes the digest-collision probe.
- Octo τ_min is the smallest threshold a worker holds.
- EH HyperLogLog Distinct has no bound; EhData drops window_distinct.
- UnivMonQ Quantile/Rank ε adds 1/n for rounding; sizing follows.
- Tests: FastPath non-power-of-two collision rate and ε, FastPath sizing
  check on its own type, heavy targets for Hydra Count-Min and UnivMonQ
  ItemFrequency, exact never-evicted Elastic residents, UnivMon Distinct at
  k = 256, UnivMonQ samples on increasing data.
- Docs: Count-Min FastPath rate, Count Sketch FastPath sizing, d ≤ 20 cap;
  CHANGELOG marks the Bloom::dimensions_for and fold-width changes breaking.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GordonYuanyc
GordonYuanyc merged commit a05b35c into main Oct 30, 2026
5 checks passed
@GordonYuanyc
GordonYuanyc deleted the feat/sketch-analytics branch October 30, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant