Skip to content

sketch-bench: asap_sketchlib a05b35c, the library's memory_bytes, and the rows table moved in from aqpbm-cli - #234

Merged
GordonYuanyc merged 2 commits into
mainfrom
sketchlib-a05b35c
Nov 3, 2026
Merged

GordonYuanyc merged 2 commits into
mainfrom
sketchlib-a05b35c

Conversation

@GordonYuanyc

@GordonYuanyc GordonYuanyc commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Dependency. The workspace pins asap_sketchlib to git rev a05b35c, which is 0.4.0 plus main through #262. aqpbm-planeval takes the same workspace dependency, so Cargo.lock holds two copies, ours and ASAPPlanner's own 21b6357, instead of three. To move to another sketchlib commit, edit rev in the root Cargo.toml.

  • 0.4.0 names.

    • Every write is now update (UnivMon's is update_weighted).
    • Type renames: CountMinSketch, CountSketch, CountMinSketchWithHeap, CountSketchWithHeap, HeapEntry, Kll (with_k, Cdf::value_at), DdSketch, Ertl, HyperLogLogHip, HllRegistersP*.
    • UnivMon: new, layer_heaps, estimate_*.
    • The HydraCounter variants follow the new names.
    • The registry descriptions and existing comments are renamed to match.
  • Memory. Every sketchlib row now reports the library's own memory_bytes(). The parallel-insert rows report workers × the library's theoretical_memory_bytes for the shape each worker uses. Three hand-written formulas copied 0.3's internals and are deleted, along with the tests that asserted them:

    • aqpbm-core's heap_footprint (the 0.3 HHHeap layout)
    • the KLL slot count
    • the Hydra grid overhead

    The DDSketch formula and the oxide rows' formulas stay.

  • rows/ moves from aqpbm-cli to sketch-bench. aqpbm-cli/src/rows is now sketch-bench/src/rows, and the CLI calls sketch_bench::rows::measurements. aqpbm-cli's source no longer names any asap_sketchlib type; it still links the library through sketch-bench. The re-export of sketchlib types in wrappers/mod.rs is gone.

Numbers that change

  • Memory for every sketchlib row now comes from the library. It includes the struct, and for heaps and KLL it is the state at the moment of measurement, not a modelled steady state.
  • Accuracy changes because 0.4.0 fixed the UnivMon cardinality and L2 estimators.
  • Throughput may change from #245 (heap rescoring), #246 (KLL compaction moved out of the insert path) and #247 (Count Sketch medians taken on the stack).

Earlier results should be re-baselined rather than compared directly.

Docs

  • docs/rqe_optimizer_hydra.md points at sketch-bench/src/rows/mod.rs, drops the 0.3.0 mentions, and states the footprint as Hydra::memory_bytes.
  • docs/e2e_planning_flow.md drops the 0.3.0 mention.
  • README.md shows the hll sample's memory_bytes as the library reports it (16400 at lg_k = 14).

The results READMEs under rqe-optimizer/results/ still say 0.3.0, because those runs were measured on it.

Test plan

  • cargo build --release
  • cargo test --workspace --no-fail-fast: all pass
  • cargo fmt --all --check
  • approxbench sketchbench --variant cms-heap-topk-fastpath-vector2d --library lib --config 'rows=5 cols=2048 heap=128' on 1M Zipf(1.1) keys: insert 30.3 M items/s, memory_bytes 58,904 (40,960 bytes of counters plus the struct and a 128-entry heap)

🤖 Generated with Claude Code

GordonYuanyc and others added 2 commits November 3, 2026 13:05
… the rows table moved in from aqpbm-cli

- The workspace pins asap_sketchlib to git rev a05b35c (0.4.0 plus main
  through #262); aqpbm-planeval takes the same workspace dependency, so the
  lockfile carries one copy of ours instead of the crates.io 0.3.0 one.
- The wrappers follow the 0.4.0 names: update for insert, CountMinSketch,
  CountSketch, CountMinSketchWithHeap, CountSketchWithHeap, HeapEntry, Kll,
  DdSketch, Ertl, HyperLogLogHip, HllRegistersP*, UnivMon::new,
  layer_heaps, estimate_*, and the HydraCounter variants.
- Every sketchlib row reports the library's own memory_bytes. The formulas
  that reproduced 0.3 internals (aqpbm-core's heap_footprint, the KLL slot
  count, the Hydra grid overhead) and the tests that asserted them are gone.
  The oxide rows keep their formulas.
- aqpbm-cli/src/rows moves to sketch-bench/src/rows, so aqpbm-cli names no
  asap_sketchlib type and calls sketch_bench::rows::measurements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntions, and show HLL memory as the library reports it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GordonYuanyc
GordonYuanyc merged commit 1ed93d7 into main Nov 3, 2026
2 checks passed
@GordonYuanyc
GordonYuanyc deleted the sketchlib-a05b35c branch November 3, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant