Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
UddSketchfor reproducible quantile aggregation over finite signedf64values, with weighted updates, merging, and serialization.Design
Based on UDDSketch (2020) and its mergeability analysis (2021), the implementation uses logarithmic buckets and uniformly coarsens them to stay within a configured bucket budget.
f64domain with a minimum budget of three.Guarantees
For identical parameters and mapping versions, states built through updates, merges, and serialization roundtrips produce identical answers and bytes across input orders, partitions, and merge trees. Every build reaches the same smallest collapse level that fits the budget; integer parent mapping and exact count/extrema combination preserve this property during merging.
Queries approximate the lower discrete quantile at
floor(q * (n - 1)), with exact endpoints and exact zero results.relative_error()reports the final bound onabs(estimate - value) / abs(value)for nonzero quantiles. The configured initial accuracy guides bucket spacing; uniform collapse can increase the final bound.Validation and performance
cargo x check,cargo x test(800 tests including doctests), andcargo x lintpass. MSRV and release-mode checks also passed. Regression tests cover mapping boundaries, subnormals, reciprocal correction, bucket lookup, large weights, merging, error bounds, and serialization.Across seven distributions, 128 input/partition/merge layouts, and six query ranks, the 100,001-row experiment produced identical UDDSketch states and answers, with every measured error within its reported bound.
Local release benchmarks on Apple M4 Max, macOS 26.6.2, rustc 1.101.0-nightly; medians of 15 measured rounds after two warmup rounds:
The included comparison harness reports timing, size, accuracy, and merge stability: