Skip to content

sketch-bench: seeded KLL, rank error at chosen quantiles, stream statistics, top-k count error, Bloom, and lib rows for every non-windowed asap_sketchlib type - #235

Merged
GordonYuanyc merged 5 commits into
mainfrom
sketch-bench-characterization-gaps
Nov 4, 2026
Merged

GordonYuanyc merged 5 commits into
mainfrom
sketch-bench-characterization-gaps

Conversation

@GordonYuanyc

@GordonYuanyc GordonYuanyc commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Closes the sketch-bench side of the characterization gaps: G-6, G-3, G-7, the Hydra per-group statistics, and G-8 for every non-windowed SketchAnalytics type except SetAggregator.

What changed

  • G-6 seeded KLL. kll and hydra-kll take an optional seed knob (--config 'k=200 seed=7'). With it the lib rows build with with_k_and_seed; without it they build as before, unseeded. The oxide KLL takes no seed, so it refuses one. KllDynamic, Coco, UniformSampling and Nitro take the same optional seed.
  • G-3 quantile probes. --quantiles 0.5,0.99 asks a rank-error row at those fractions and reports rank_err_q* for each one. Without the flag the 101-point grid stays.
  • G-3 stream statistics. The frequency, top-k and heavy-hitter comparators report stream_items, stream_distinct and stream_l2.
  • G-7 top-k count error. The top-k comparator reports max_count_err and mean_count_err over the keys the sketch returned.
  • Per-group statistics. --per-group-out writes group_key,n_q,error,d_q,l2_q. The flag is gated on a subpopulation capability instead of the hydra- name prefix.
  • New membership comparator for Bloom. It probes up to 10k members and 100k keys that are not in the stream, then reports fpr, fnr (always 0 for a Bloom filter) and stream_distinct.
  • G-8: 29 new lib rows, written out per type like the existing wrappers:
Type Rows Comparator
HyperLogLog<Ertl> hll-ertl cardinality
Kmv kmv cardinality
Bloom bloom: cols must be a power of two, where the FPR bound holds membership
SpaceSaving space-saving, space-saving-topk frequency, topk
CountMinSketchWithSpaceSaving count-min-space-saving, count-min-space-saving-topk frequency, topk
CountSketchWithSpaceSaving count-sketch-space-saving, count-sketch-space-saving-topk frequency, topk
CountMinSketchFold count-min-fold, count-min-fold-topk frequency, topk
CountSketchFold count-sketch-fold, count-sketch-fold-topk frequency, topk
CountSketchWithL2Norm count-sketch-l2-norm keyed-l2-norm
CountMinHll count-min-hll: each record goes into every label subset subpop-cardinality
ElasticSketch elastic, elastic-topk: --dtype string only frequency, topk
CocoSketch coco: --dtype string only frequency
KllDynamic kll-dynamic (seeded) rank-error
UniformSampling uniform-sampling rank-error
UnivMonTwoTier univmon-two-tier-{cardinality,l1-norm,l2-norm,entropy} keyed-*
UnivMonQ univmon-q-{quantile,cardinality,frequency,topk}: all but quantile at --dtype f64 only, since UnivMonQ keeps values as f64 rank-error, cardinality, frequency, topk
NitroBatch nitro-count-min, nitro-count-sketch: i64 only, the whole stream in one batch, no per-item latency frequency

The top-k k is checked only when a top-k question is asked, so a frequency row builds at any capacity. topk_row takes k from the row's own wrapper.

Not included

  • SetAggregator: it is exact, so its error is always 0.
  • The windowed types: MicroCm, ExponentialHistogram, EhBucketSketch, ExponentialHistogramUnivMon and TumblingWindow.
  • docs/sketch-bench.md is not updated with the new rows or flags.

Verification

  • cargo fmt --check, cargo clippy --workspace --all-targets --all-features -D warnings and cargo test --workspace all pass.
  • Each new row ran query and merge accuracy through the release CLI, and every error was in range.
  • Bloom with r=4, c=65536 and D=27096 measured an FPR of 0.0136; the formula (1 − (1 − 1/c)^D)^r gives 0.0132.

🤖 Generated with Claude Code

GordonYuanyc and others added 2 commits November 3, 2026 17:57
…tics, top-k count error, and 27 more asap_sketchlib rows

- KLL: `seed` knob on kll and hydra-kll (lib builds with `with_k_and_seed`;
  oxide refuses a nonzero seed).
- `--quantiles q1,q2`: quantile rows asked at those fractions, one error per
  fraction (`rank_err_q*`, `relative_value_error_q*`, grouped
  `mean_/max_rank_err_q*`); omitted keeps the 101-point grid.
- frequency, top-k and heavy-hitter comparators report `stream_items`,
  `stream_distinct`, `stream_l2`; top-k also reports `max_count_err` and
  `mean_count_err` over the reported keys.
- `--per-group-out` adds `d_q` and `l2_q` per group.
- New lib rows through one generic driver (`wrappers/row_bodies.rs`):
  hll-ertl, kmv, kll-dynamic, uniform-sampling, space-saving(-topk),
  elastic(-topk), coco, count-min/count-sketch-space-saving(-topk),
  count-min/count-sketch-fold(-topk), count-sketch-l2-norm, count-min-hll,
  univmon-two-tier-*, univmon-q-*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eSaving/fold matrices

- Drop the generic row driver and the row macros; the kept rows are written
  out per type as the other wrappers and rows are.
- Drop the rows for KllDynamic, UniformSampling, Elastic, Coco,
  CountSketchWithL2Norm, CountMinHll, UnivMonTwoTier and UnivMonQ.
- Per-fraction errors only in RankErrorGT; RelativeValueErrorGT and the
  grouped comparators are back to main. `--quantiles` is refused unless the
  row is scored by rank error.
- `answered_k` parses the new vocabularies instead of reading `topk_k` out of
  raw JSON.
- The top-k k is checked only when a top-k is asked, so a frequency row
  builds at any capacity.
- `--per-group-out` is gated on a subpopulation capability, not the `hydra-`
  name prefix.
- The top-k truth keeps its exact map instead of copying it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GordonYuanyc GordonYuanyc changed the title sketch-bench: seeded KLL, configurable quantile probes, stream statistics, top-k count error, and 27 more asap_sketchlib rows sketch-bench: seeded KLL, rank error at chosen quantiles, stream statistics, top-k count error, and 11 more asap_sketchlib rows Nov 4, 2026
GordonYuanyc and others added 2 commits November 3, 2026 19:58
…ate over keys absent from the stream)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…types

count-sketch-l2-norm, count-min-hll, elastic(-topk), coco, kll-dynamic,
uniform-sampling, univmon-two-tier-{cardinality,l1-norm,l2-norm,entropy},
univmon-q-{quantile,cardinality,frequency,topk}, nitro-count-min and
nitro-count-sketch, each written out per type as the other wrappers are.
Elastic and Coco build at --dtype string only, NitroBatch at i64 only, as
their APIs take. SetAggregator (exact) and the windowed types are left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GordonYuanyc GordonYuanyc changed the title sketch-bench: seeded KLL, rank error at chosen quantiles, stream statistics, top-k count error, and 11 more asap_sketchlib rows sketch-bench: seeded KLL, rank error at chosen quantiles, stream statistics, top-k count error, Bloom, and lib rows for every non-windowed asap_sketchlib type Nov 4, 2026
- count-sketch-l2-norm refuses a shape past the 128-bit key hash instead of
  letting the library panic; bloom refuses a non-power-of-two `cols` and
  `rows * cols` past BLOOM_MAX_BITS.
- univmon-q cardinality, frequency and top-k build at --dtype f64 only:
  UnivMonQ keeps values as f64, so an integer key past 2^53 came back as
  another key.
- `topk_row` takes k from the row's wrapper; `answered_k` is back to main.
- Elastic's String clones and CountMinHll's subset keys are made outside the
  timed loops (`write_subset_key` reuses one buffer).
- Nitro rows offer no latency: per item they would run a different path
  from the one-batch insert. The refusal for a metric other than accuracy
  now says the row does not measure it.
- `seed` is optional everywhere (KLL, hydra-kll, KllDynamic, Coco,
  UniformSampling, Nitro): absent keeps the library's own unseeded
  construction, so existing KLL runs behave as before.
- The new rows live in files named by family (count_matrix, nitro,
  string_keyed, uniform_sampling, kll, univmon).
- Tests for the shape refusals and the KLL seed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GordonYuanyc
GordonYuanyc merged commit f31707d into main Nov 4, 2026
2 checks passed
@GordonYuanyc
GordonYuanyc deleted the sketch-bench-characterization-gaps branch November 4, 2026 01:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant