Skip to content

refactor: simplify xor construction and local naming - #326

Merged
tisonkun merged 6 commits into
apache:mainfrom
tisonkun:codex/xor-scopedb-review
Nov 5, 2026
Merged

tisonkun merged 6 commits into
apache:mainfrom
tisonkun:codex/xor-scopedb-review

Conversation

@tisonkun

@tisonkun tisonkun commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Simplify the unreleased XOR API to XorFilter::from_values, XorFilter::from_hashes, and XorFilter::from_parts. from_values hashes and deduplicates ordinary values; from_hashes accepts distinct hashes as &[u64] and returns an error for duplicates. Fingerprint width and construction seed are explicit arguments.

Keep serialization beside the filter implementation so fields and serialization helpers remain private. Remove the builder, raw-parts wrapper, and ownership-transfer method; application-defined codecs import raw fields and encode through borrowed accessors. Use concise local names where the surrounding operation already supplies the context.

Update documentation and existing coverage for value deduplication, invalid hash inputs, bounded retries, Java serialization, and xorf interoperability. Repeated-hash benchmarks now include explicit caller-side deduplication for both implementations.

Validation: cargo x check, cargo x test (814 tests including doctests), cargo x lint, and the optimized XOR benchmark smoke suite with ThinLTO.

Batch construction at e3b508e, Apple M4 Max / macOS ARM64, nightly-2026-08-28, ThinLTO, three repetitions. Each round builds 32 independent Xor8 filters from identical distinct inputs with initial seed 1. Times include setup and cleanup, reported as median [min–max] milliseconds:

Keys per filter Rounds DataSketches xorf 0.13.0
128 10,000 271 [266–280] 417 [410–420]
16,384 40 142 [140–142] 359 [357–365]
50,000 12 167 [167–168] 337 [337–338]

Command: CARGO_PROFILE_BENCH_LTO=thin cargo +nightly-2026-08-28 bench -p benchmarks --bench benchmarks -- xor::build_batches --repeat 3 --raw. These compare the current implementations; the API simplification does not change the construction algorithm.

@tisonkun tisonkun changed the title refactor(xor): simplify raw parts and construction refactor(xor): expose direct filter constructors Nov 5, 2026
@tisonkun tisonkun changed the title refactor(xor): expose direct filter constructors refactor: simplify xor construction and local naming Nov 5, 2026
@tisonkun
tisonkun merged commit 4ba130e into apache:main Nov 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant