[tisonkun]

XOR membership queries were slower than xorf 0.13.0, and constructing filters over independent input sets spent substantial time in conditional singleton enqueues. This change improves both paths and adds benchmarks for batches of independent builds and queries rotating across imported filters.

• Inline contains_hash across crate boundaries and expose the validated 32-bit segment-length bound to the index reductions.
• Apply the known-empty check as a result mask rather than branching on the fingerprint comparison.
• Enqueue singleton candidates without a conditional branch. One spare slot per segment makes speculative writes safe even when a neighbor queue is full, adding three u32 slots to construction scratch space.
• Add a regression test for full neighbor queues and benchmarks for independent input sets and imported-filter queries.

Public APIs, construction seed selection, and serialization remain compatible. Known-empty filters still return false, but now perform the fingerprint probe before applying the empty mask. Imported filters with unknown cardinality retain the raw fingerprint comparison.

Measurements

The complete benchmark report includes all 69 scenarios in both the default bench profile (five repetitions) and fat LTO (seven repetitions), every raw sample, and all remaining gaps against xorf. In the complete LTO run, six query medians were 0.4%–2.2% higher than xorf, with overlapping observed ranges.

Apple M4 Max, macOS arm64; rustc 1.101.0-nightly (8006091eb 2026-10-17). All measurements use optimized Cargo bench builds and include fixture construction and cleanup.

The full XOR matrix covers 69 scenarios and 159 scenario/implementation combinations, with five repetitions each. The following default-profile build results compare the library at 33ec5c9 with this change using identical benchmark definitions and rounds. Each round builds 32 independent Xor8 filters; times are medians in milliseconds.

Keys per filter	Rounds	Before	After	xorf
16,384	40	294	151	359
50,000	12	296	172	332

All 24 single-filter and imported-filter query scenarios had lower measured medians than xorf in the default bench profile.

A separate Xor8 run uses fat LTO and one codegen unit, with seven repetitions for each of 25 scenarios. Imported-filter queries use identical payloads and runtime width dispatch on both sides. Each query round performs 256 lookups on each of 32 filters.

Workload	Rounds	DataSketches	xorf
Build batches, 16,384 keys/filter	40	149 ms	353 ms
Build batches, 50,000 keys/filter	12	169 ms	325 ms
Imported queries, 16,384 keys, present	20,000	335 ms	340 ms
Imported queries, 16,384 keys, absent	20,000	336 ms	336 ms

These query results indicate approximate parity under LTO, not a statistically established speedup. Results are library microbenchmarks on one architecture.

The default-seed single-set 16,384-key build remains slower: base seed 0 needs two attempts for that input, while xorf's base seed 1 needs one. The deduplicating entry point also remains slower than xorf's distinct-input construction. Both cases remain in the benchmark matrix.

Reproduce with:

cargo x bench -- xor --repeat 5 --raw
CARGO_PROFILE_BENCH_LTO=fat CARGO_PROFILE_BENCH_CODEGEN_UNITS=1 cargo x bench -- xor ::8 --repeat 7 --raw

Validation

• cargo x check
• cargo x test: 817 tests passed
• cargo x lint: passed on an identical clean source copy
• cargo x bench -- --test: 312 scenario/implementation combinations passed
• cargo +1.86.0 check --package benchmarks --all-targets --locked
• cargo x bench --no-run

[tisonkun]

XOR benchmark results

DataSketches has lower measured medians in most comparisons. Matched-seed construction and independent-filter batch construction are ahead of xorf in both profiles. Queries are ahead in the ordinary bench run; with LTO, imported-filter queries are close to parity.

Each profile covers 69 scenarios and 90 comparisons: some scenarios test two DataSketches entry points. Counts below compare medians, not statistical significance.

Workload	Ordinary bench: lower / equal / higher	Fat LTO: lower / equal / higher
Distinct build, default seeds	15 / 0 / 3	15 / 0 / 3
Distinct build, matched seeds	24 / 0 / 0	24 / 0 / 0
Independent-filter batch builds	6 / 0 / 0	6 / 0 / 0
Single-filter queries	12 / 0 / 0	10 / 0 / 2
Imported-filter shard queries	12 / 0 / 0	7 / 1 / 4
Build with deduplication	0 / 0 / 18	0 / 0 / 18
Total	69 / 0 / 21	62 / 1 / 27

Example: batch-building 32 Xor8 filters with 16K keys/filter takes 151 ms vs xorf’s 359 ms (ordinary) and 151 ms vs 352 ms (LTO). Ordinary-profile time before this PR was 294 ms for the same workload.

What changed: inline queries across crates, expose the validated 32-bit index bound, and avoid short-circuiting the empty-filter check; replace conditional construction-queue enqueues with branchless updates, using three extra scratch slots.

Remaining gaps:

• 18 deduplicating builds per profile: DataSketches also collects, sorts, and deduplicates; xorf expects distinct input. This extra work is absent from the distinct-build rows.
• 3 default-seed builds per profile: the 16K input needs two attempts for DataSketches versus one for xorf. All matched-seed cases are ahead.
• 6 LTO queries: 0.4%–2.2% higher medians, with overlapping sample ranges. Four imported-filter and two single-filter cases; all remain marked as slower measurements.

Optimized Cargo bench builds throughout: ordinary 5 repeats, fat LTO / one codegen unit 7 repeats. Apple M4 Max, macOS arm64, rustc 1.101.0-nightly; xorf 0.13.0; code a683250. These are microbenchmarks, not ScopeDB end-to-end results.

Delta vs xorf: negative = less time, positive = more time; this compares libraries, not before/after revisions.

Remaining gaps: exact cases and measured ranges

Default bench: every median above xorf

Scenario	DS entry point	DS median	DS range	xorf median	xorf range	Delta	Ranges overlap
xor::build::8/1000	deduplicate	567	566–569	459	457–478	+23.5%	no
xor::build::8/16384	distinct	361	358–363	280	276–282	+28.9%	no
xor::build::8/16384	deduplicate	555	553–560	280	276–282	+98.2%	no
xor::build::8/49999	deduplicate	453	447–458	333	325–359	+36.0%	no
xor::build::8/50000	deduplicate	451	450–457	328	323–337	+37.5%	no
xor::build::8/50001	deduplicate	455	451–456	328	324–335	+38.7%	no
xor::build::8/100000	deduplicate	340	335–345	294	291–305	+15.6%	no
xor::build::16/1000	deduplicate	571	567–579	476	474–480	+20.0%	no
xor::build::16/16384	distinct	363	361–364	278	276–284	+30.6%	no
xor::build::16/16384	deduplicate	562	557–565	278	276–284	+102.2%	no
xor::build::16/49999	deduplicate	460	458–465	325	321–336	+41.5%	no
xor::build::16/50000	deduplicate	460	456–462	335	327–358	+37.3%	no
xor::build::16/50001	deduplicate	455	455–465	333	321–347	+36.6%	no
xor::build::16/100000	deduplicate	358	351–376	314	299–328	+14.0%	no
xor::build::32/1000	deduplicate	589	587–594	476	469–477	+23.7%	no
xor::build::32/16384	distinct	366	363–370	280	276–289	+30.7%	no
xor::build::32/16384	deduplicate	567	560–572	280	276–289	+102.5%	no
xor::build::32/49999	deduplicate	461	456–465	346	340–360	+33.2%	no
xor::build::32/50000	deduplicate	460	457–469	356	348–362	+29.2%	no
xor::build::32/50001	deduplicate	460	458–470	362	353–368	+27.1%	no
xor::build::32/100000	deduplicate	352	345–357	301	294–316	+16.9%	no

Fat LTO, 1 codegen unit: every median above xorf

Scenario	DS entry point	DS median	DS range	xorf median	xorf range	Delta	Ranges overlap
xor::build::8/1000	deduplicate	575	570–584	451	449–457	+27.5%	no
xor::build::8/16384	distinct	355	352–363	285	282–303	+24.6%	no
xor::build::8/16384	deduplicate	556	551–568	285	282–303	+95.1%	no
xor::build::8/49999	deduplicate	451	444–477	323	303–403	+39.6%	no
xor::build::8/50000	deduplicate	449	446–455	312	309–347	+43.9%	no
xor::build::8/50001	deduplicate	459	450–475	330	311–355	+39.1%	no
xor::build::8/100000	deduplicate	351	346–370	316	312–319	+11.1%	no
xor::build::16/1000	deduplicate	595	592–622	462	459–476	+28.8%	no
xor::build::16/16384	distinct	357	356–376	289	283–294	+23.5%	no
xor::build::16/16384	deduplicate	560	552–582	289	283–294	+93.8%	no
xor::build::16/49999	deduplicate	457	453–463	325	317–348	+40.6%	no
xor::build::16/50000	deduplicate	458	455–467	331	313–349	+38.4%	no
xor::build::16/50001	deduplicate	456	451–476	322	318–335	+41.6%	no
xor::build::16/100000	deduplicate	341	334–359	299	293–317	+14.0%	no
xor::build::32/1000	deduplicate	581	576–585	468	463–472	+24.1%	no
xor::build::32/16384	distinct	360	356–362	287	284–292	+25.4%	no
xor::build::32/16384	deduplicate	559	553–564	287	284–292	+94.8%	no
xor::build::32/49999	deduplicate	459	455–474	344	322–358	+33.4%	no
xor::build::32/50000	deduplicate	461	456–469	337	323–354	+36.8%	no
xor::build::32/50001	deduplicate	459	457–471	344	333–357	+33.4%	no
xor::build::32/100000	deduplicate	348	340–356	309	299–316	+12.6%	no
xor::query::16_present/50000	DataSketches	237	235–243	232	230–235	+2.2%	yes
xor::query::16_absent/50000	DataSketches	236	236–241	231	229–238	+2.2%	yes
xor::query_shards::8_present/16384	DataSketches	346	342–353	343	341–351	+0.9%	yes
xor::query_shards::16_present/128	DataSketches	280	279–281	279	277–284	+0.4%	yes
xor::query_shards::16_present/16384	DataSketches	356	352–360	351	349–359	+1.4%	yes
xor::query_shards::16_absent/16384	DataSketches	351	348–359	349	344–361	+0.6%	yes

Reproduction commands and workload definitions

Environment: Apple M4 Max, 36 GiB RAM, macOS 26.6.2 (arm64); rustc 1.101.0-nightly (8006091eb 2026-10-17), LLVM 23.1.4; xorf 0.13.0. Both profiles are optimized Cargo bench builds. The LTO run explicitly sets fat LTO and one codegen unit.

cargo x bench -- xor --repeat 5 --raw
CARGO_PROFILE_BENCH_LTO=fat CARGO_PROFILE_BENCH_CODEGEN_UNITS=1 cargo x bench -- xor --repeat 7 --raw

No round overrides or calibration were used. Each repetition includes fixture construction, input preparation, allocations, validation, result destruction, and cleanup. Peers use the same rounds and input parameters, are interleaved, and rotate execution order. Tables retain every sample in measurement order, including first runs and outliers. Times are integer milliseconds for the complete fixed workload, not per-operation latency; different scenario rows may have different work sizes.

Every percentage below uses xorf as the denominator: (DataSketches median / xorf median - 1) * 100, calculated from the displayed integer medians. Negative means less elapsed time; bold positive values mean the DataSketches median is still above xorf. This differs from the runner's vs ref column, whose denominator is the first implementation. Equal displayed medians only mean parity at the reported precision. Neither medians nor min–max overlap establish statistical significance or end-to-end application performance.

Workload definitions

• build: one filter per round. distinct borrows distinct hashes, deduplicate collects and deduplicates the same inputs, and xorf consumes a distinct-hash iterator. These use their respective default construction seeds.
• build_batches: 32 independent Xor8 input sets per round. default_seed uses base seed 0; matched_seed uses base seed 1, as does xorf. Both DataSketches rows use the distinct-hash entry point.
• matched_build: base seed 1 on both sides, with the expected successful seed checked on every build. Input offset 0 succeeds on the first attempt; offset 917,504 requires the second. Eight-million-key cases retain one full build per round.
• query: each round visits every key in the present or absent set using one prepared filter. DataSketches has known cardinality; xorf uses its concrete fingerprint-width type.
• query_shards: 256 lookups across each of 32 filters per round, rotating filters on each lookup. Both sides dispatch on width at runtime and use identical xorf-generated fingerprint payloads. DataSketches imports them without cardinality metadata. Present keys belong to each filter's own input; absent keys are disjoint. Both sides include the common reference construction in setup, so this group measures imported-filter query behavior, not DataSketches build speed.

The original workloads retain their round counts; the two new groups were sized against library revision 33ec5c9. Scenario-specific assumptions and results belong here or beside the benchmark code; CONTRIBUTING.md now retains only general benchmark guidance.

Complete results: both profiles, all 318 implementation rows and 1,908 raw samples

All measurements

Each section gives the full scenario prefix; append the Case column to obtain the runner's exact scenario name. All implementation rows and raw samples are included.

Default bench — 5 repeats

xor::build:: — 18 scenarios, 54 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8/1000	distinct	60000	306	302–308	308, 306, 307, 302, 303	-33.3%
8/1000	deduplicate	60000	567	566–569	569, 567, 566, 566, 569	+23.5%
8/1000	xorf	60000	459	457–478	459, 457, 478, 458, 464	reference
8/16384	distinct	2000	361	358–363	358, 363, 359, 362, 361	+28.9%
8/16384	deduplicate	2000	555	553–560	559, 555, 553, 553, 560	+98.2%
8/16384	xorf	2000	280	276–282	276, 280, 282, 279, 281	reference
8/49999	distinct	600	246	245–247	245, 247, 247, 246, 245	-26.1%
8/49999	deduplicate	600	453	447–458	456, 453, 458, 447, 447	+36.0%
8/49999	xorf	600	333	325–359	359, 333, 327, 325, 338	reference
8/50000	distinct	600	246	244–248	248, 246, 247, 246, 244	-25.0%
8/50000	deduplicate	600	451	450–457	457, 450, 451, 451, 453	+37.5%
8/50000	xorf	600	328	323–337	330, 328, 325, 323, 337	reference
8/50001	distinct	600	247	245–250	250, 245, 248, 247, 247	-24.7%
8/50001	deduplicate	600	455	451–456	455, 451, 455, 452, 456	+38.7%
8/50001	xorf	600	328	324–335	335, 324, 326, 328, 332	reference
8/100000	distinct	200	184	182–184	184, 182, 184, 184, 183	-37.4%
8/100000	deduplicate	200	340	335–345	341, 337, 345, 335, 340	+15.6%
8/100000	xorf	200	294	291–305	305, 291, 291, 294, 298	reference
16/1000	distinct	60000	308	307–310	308, 308, 310, 307, 308	-35.3%
16/1000	deduplicate	60000	571	567–579	571, 571, 572, 567, 579	+20.0%
16/1000	xorf	60000	476	474–480	476, 475, 474, 476, 480	reference
16/16384	distinct	2000	363	361–364	364, 362, 364, 363, 361	+30.6%
16/16384	deduplicate	2000	562	557–565	558, 565, 564, 562, 557	+102.2%
16/16384	xorf	2000	278	276–284	278, 278, 277, 284, 276	reference
16/49999	distinct	600	252	250–255	252, 250, 253, 252, 255	-22.5%
16/49999	deduplicate	600	460	458–465	465, 460, 460, 461, 458	+41.5%
16/49999	xorf	600	325	321–336	336, 325, 321, 323, 334	reference
16/50000	distinct	600	253	251–263	263, 255, 251, 253, 253	-24.5%
16/50000	deduplicate	600	460	456–462	461, 462, 460, 456, 459	+37.3%
16/50000	xorf	600	335	327–358	327, 335, 334, 358, 337	reference
16/50001	distinct	600	254	252–257	252, 257, 256, 253, 254	-23.7%
16/50001	deduplicate	600	455	455–465	458, 455, 465, 455, 455	+36.6%
16/50001	xorf	600	333	321–347	323, 333, 347, 321, 342	reference
16/100000	distinct	200	191	188–193	188, 192, 193, 191, 188	-39.2%
16/100000	deduplicate	200	358	351–376	357, 376, 358, 360, 351	+14.0%
16/100000	xorf	200	314	299–328	309, 318, 328, 314, 299	reference
32/1000	distinct	60000	319	317–322	320, 317, 322, 319, 318	-33.0%
32/1000	deduplicate	60000	589	587–594	594, 587, 589, 592, 588	+23.7%
32/1000	xorf	60000	476	469–477	476, 471, 477, 477, 469	reference
32/16384	distinct	2000	366	363–370	363, 366, 364, 367, 370	+30.7%
32/16384	deduplicate	2000	567	560–572	567, 572, 560, 561, 567	+102.5%
32/16384	xorf	2000	280	276–289	289, 279, 283, 276, 280	reference
32/49999	distinct	600	259	256–260	260, 256, 259, 259, 259	-25.1%
32/49999	deduplicate	600	461	456–465	463, 456, 461, 465, 460	+33.2%
32/49999	xorf	600	346	340–360	354, 340, 346, 344, 360	reference
32/50000	distinct	600	257	257–261	257, 261, 257, 257, 258	-27.8%
32/50000	deduplicate	600	460	457–469	460, 460, 460, 457, 469	+29.2%
32/50000	xorf	600	356	348–362	348, 362, 356, 355, 359	reference
32/50001	distinct	600	257	255–264	257, 257, 260, 255, 264	-29.0%
32/50001	deduplicate	600	460	458–470	466, 460, 458, 460, 470	+27.1%
32/50001	xorf	600	362	353–368	362, 368, 358, 365, 353	reference
32/100000	distinct	200	190	189–191	190, 191, 189, 190, 191	-36.9%
32/100000	deduplicate	200	352	345–357	352, 352, 345, 348, 357	+16.9%
32/100000	xorf	200	301	294–316	299, 309, 294, 301, 316	reference

xor::build_batches:: — 3 scenarios, 9 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8/128	default_seed	10000	268	266–269	269, 266, 268, 268, 269	-32.3%
8/128	matched_seed	10000	265	262–266	265, 262, 265, 266, 262	-33.1%
8/128	xorf	10000	396	395–400	396, 395, 397, 400, 395	reference
8/16384	default_seed	40	151	148–151	151, 148, 151, 151, 150	-57.9%
8/16384	matched_seed	40	142	139–146	146, 139, 140, 142, 143	-60.4%
8/16384	xorf	40	359	355–363	359, 355, 357, 363, 359	reference
8/50000	default_seed	12	172	171–196	171, 196, 172, 171, 172	-48.2%
8/50000	matched_seed	12	167	166–168	166, 166, 168, 168, 167	-49.7%
8/50000	xorf	12	332	330–333	330, 331, 333, 333, 332	reference

xor::matched_build:: — 24 scenarios, 48 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8/(1000, 0)	DataSketches	60000	304	303–307	307, 303, 304, 304, 307	-34.5%
8/(1000, 0)	xorf	60000	464	454–476	464, 466, 459, 454, 476	reference
8/(16384, 0)	DataSketches	3000	311	308–315	315, 312, 309, 311, 308	-26.0%
8/(16384, 0)	xorf	3000	420	410–428	420, 410, 424, 415, 428	reference
8/(16384, 917504)	DataSketches	2000	364	362–368	367, 368, 362, 363, 364	-27.5%
8/(16384, 917504)	xorf	2000	502	487–508	508, 502, 495, 487, 504	reference
8/(49999, 0)	DataSketches	600	246	244–248	248, 244, 245, 247, 246	-24.8%
8/(49999, 0)	xorf	600	327	313–356	331, 319, 356, 313, 327	reference
8/(50000, 0)	DataSketches	600	249	245–252	249, 249, 252, 245, 245	-25.7%
8/(50000, 0)	xorf	600	335	316–343	332, 343, 335, 316, 335	reference
8/(50001, 0)	DataSketches	600	248	244–248	244, 247, 248, 248, 248	-26.4%
8/(50001, 0)	xorf	600	337	322–337	337, 322, 337, 337, 337	reference
8/(100000, 0)	DataSketches	200	184	182–185	184, 185, 183, 184, 182	-37.2%
8/(100000, 0)	xorf	200	293	288–307	307, 296, 291, 293, 288	reference
8/(8000000, 0)	DataSketches	1	291	275–301	289, 292, 275, 301, 291	-15.2%
8/(8000000, 0)	xorf	1	343	317–517	517, 382, 322, 317, 343	reference
16/(1000, 0)	DataSketches	60000	300	297–321	321, 300, 300, 297, 301	-36.4%
16/(1000, 0)	xorf	60000	472	468–473	468, 473, 472, 473, 469	reference
16/(16384, 0)	DataSketches	3000	315	313–323	323, 315, 317, 313, 315	-24.5%
16/(16384, 0)	xorf	3000	417	409–424	416, 421, 417, 409, 424	reference
16/(16384, 917504)	DataSketches	2000	365	363–371	365, 366, 371, 363, 364	-24.9%
16/(16384, 917504)	xorf	2000	486	481–502	486, 481, 502, 486, 498	reference
16/(49999, 0)	DataSketches	600	252	250–255	252, 250, 255, 250, 255	-23.4%
16/(49999, 0)	xorf	600	329	319–343	335, 329, 343, 327, 319	reference
16/(50000, 0)	DataSketches	600	251	250–255	251, 250, 251, 254, 255	-22.8%
16/(50000, 0)	xorf	600	325	310–354	322, 325, 334, 310, 354	reference
16/(50001, 0)	DataSketches	600	252	250–255	255, 252, 251, 253, 250	-22.9%
16/(50001, 0)	xorf	600	327	320–336	327, 320, 336, 336, 321	reference
16/(100000, 0)	DataSketches	200	189	187–191	187, 189, 189, 191, 187	-35.7%
16/(100000, 0)	xorf	200	294	292–313	294, 294, 313, 306, 292	reference
16/(8000000, 0)	DataSketches	1	289	282–306	306, 282, 288, 289, 293	-13.7%
16/(8000000, 0)	xorf	1	335	328–375	375, 332, 328, 335, 340	reference
32/(1000, 0)	DataSketches	60000	311	308–316	309, 313, 308, 311, 316	-34.5%
32/(1000, 0)	xorf	60000	475	469–483	475, 478, 472, 469, 483	reference
32/(16384, 0)	DataSketches	3000	318	317–325	318, 317, 319, 317, 325	-24.1%
32/(16384, 0)	xorf	3000	419	417–422	417, 419, 422, 421, 419	reference
32/(16384, 917504)	DataSketches	2000	368	366–376	368, 367, 366, 374, 376	-24.9%
32/(16384, 917504)	xorf	2000	490	478–508	508, 483, 490, 478, 508	reference
32/(49999, 0)	DataSketches	600	255	254–261	255, 254, 261, 255, 256	-26.9%
32/(49999, 0)	xorf	600	349	324–359	349, 324, 351, 359, 349	reference
32/(50000, 0)	DataSketches	600	257	255–259	259, 257, 258, 256, 255	-27.2%
32/(50000, 0)	xorf	600	353	330–360	352, 355, 353, 330, 360	reference
32/(50001, 0)	DataSketches	600	256	255–259	255, 259, 257, 255, 256	-25.6%
32/(50001, 0)	xorf	600	344	335–375	345, 344, 344, 335, 375	reference
32/(100000, 0)	DataSketches	200	192	189–193	192, 189, 189, 193, 192	-37.5%
32/(100000, 0)	xorf	200	307	288–319	314, 299, 307, 288, 319	reference
32/(8000000, 0)	DataSketches	1	304	290–336	336, 299, 290, 333, 304	-17.8%
32/(8000000, 0)	xorf	1	370	353–572	572, 387, 353, 370, 355	reference

xor::query:: — 12 scenarios, 24 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8_present/16384	DataSketches	10000	271	269–279	271, 271, 269, 279, 270	-2.9%
8_present/16384	xorf	10000	279	278–280	280, 279, 279, 278, 280	reference
8_present/50000	DataSketches	3000	246	244–249	244, 246, 249, 245, 246	-3.9%
8_present/50000	xorf	3000	256	250–259	257, 250, 259, 256, 250	reference
8_absent/16384	DataSketches	10000	264	262–267	264, 267, 264, 262, 262	-5.0%
8_absent/16384	xorf	10000	278	276–281	279, 281, 278, 276, 277	reference
8_absent/50000	DataSketches	3000	237	236–242	237, 242, 240, 236, 236	-6.0%
8_absent/50000	xorf	3000	252	250–255	252, 254, 255, 251, 250	reference
16_present/16384	DataSketches	10000	268	266–270	266, 268, 267, 268, 270	-3.6%
16_present/16384	xorf	10000	278	277–282	279, 278, 277, 277, 282	reference
16_present/50000	DataSketches	3000	261	258–262	262, 261, 262, 259, 258	-1.1%
16_present/50000	xorf	3000	264	262–268	266, 268, 264, 263, 262	reference
16_absent/16384	DataSketches	10000	268	268–272	268, 271, 272, 268, 268	-4.3%
16_absent/16384	xorf	10000	280	280–282	280, 280, 282, 280, 280	reference
16_absent/50000	DataSketches	3000	261	259–262	261, 261, 259, 261, 262	-0.8%
16_absent/50000	xorf	3000	263	262–268	263, 267, 262, 262, 268	reference
32_present/16384	DataSketches	10000	263	262–267	267, 262, 262, 264, 263	-6.1%
32_present/16384	xorf	10000	280	277–283	280, 279, 277, 280, 283	reference
32_present/50000	DataSketches	3000	279	278–281	278, 280, 281, 279, 278	-7.6%
32_present/50000	xorf	3000	302	301–306	306, 302, 303, 302, 301	reference
32_absent/16384	DataSketches	10000	262	261–277	262, 261, 261, 277, 269	-9.0%
32_absent/16384	xorf	10000	288	279–307	288, 279, 307, 282, 289	reference
32_absent/50000	DataSketches	3000	289	283–292	285, 283, 292, 289, 289	-5.6%
32_absent/50000	xorf	3000	306	300–312	312, 306, 301, 300, 307	reference

xor::query_shards:: — 12 scenarios, 24 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8_present/128	DataSketches	20000	276	274–287	287, 276, 277, 274, 274	-15.6%
8_present/128	xorf	20000	327	320–338	338, 325, 327, 328, 320	reference
8_absent/128	DataSketches	20000	276	272–278	276, 278, 272, 278, 272	-17.9%
8_absent/128	xorf	20000	336	318–342	318, 336, 342, 336, 341	reference
8_present/16384	DataSketches	20000	350	346–354	354, 346, 350, 354, 348	-16.1%
8_present/16384	xorf	20000	417	410–433	417, 411, 423, 433, 410	reference
8_absent/16384	DataSketches	20000	354	349–379	350, 354, 379, 349, 361	-14.9%
8_absent/16384	xorf	20000	416	410–518	410, 413, 518, 439, 416	reference
16_present/128	DataSketches	20000	283	280–290	289, 280, 282, 290, 283	-12.1%
16_present/128	xorf	20000	322	321–324	324, 322, 322, 321, 323	reference
16_absent/128	DataSketches	20000	280	276–282	281, 282, 277, 276, 280	-13.3%
16_absent/128	xorf	20000	323	318–329	323, 323, 329, 323, 318	reference
16_present/16384	DataSketches	20000	357	350–358	358, 352, 350, 358, 357	-12.7%
16_present/16384	xorf	20000	409	408–424	414, 408, 408, 409, 424	reference
16_absent/16384	DataSketches	20000	359	358–362	362, 358, 359, 359, 358	-12.4%
16_absent/16384	xorf	20000	410	406–419	419, 414, 406, 409, 410	reference
32_present/128	DataSketches	20000	275	272–281	276, 274, 272, 275, 281	-16.2%
32_present/128	xorf	20000	328	327–331	328, 328, 327, 331, 329	reference
32_absent/128	DataSketches	20000	274	273–279	279, 278, 274, 274, 273	-16.7%
32_absent/128	xorf	20000	329	327–339	330, 329, 339, 329, 327	reference
32_present/16384	DataSketches	20000	369	362–395	395, 369, 362, 369, 364	-14.4%
32_present/16384	xorf	20000	431	425–440	431, 425, 429, 436, 440	reference
32_absent/16384	DataSketches	20000	364	353–368	364, 353, 364, 368, 366	-14.6%
32_absent/16384	xorf	20000	426	421–430	426, 421, 423, 427, 430	reference

Fat LTO, 1 codegen unit — 7 repeats

xor::build:: — 18 scenarios, 54 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8/1000	distinct	60000	317	306–329	329, 328, 317, 308, 307, 306, 317	-29.7%
8/1000	deduplicate	60000	575	570–584	573, 573, 583, 575, 570, 584, 581	+27.5%
8/1000	xorf	60000	451	449–457	449, 451, 457, 453, 451, 451, 455	reference
8/16384	distinct	2000	355	352–363	353, 352, 352, 355, 363, 363, 356	+24.6%
8/16384	deduplicate	2000	556	551–568	558, 564, 556, 551, 553, 568, 555	+95.1%
8/16384	xorf	2000	285	282–303	282, 285, 294, 282, 303, 289, 282	reference
8/49999	distinct	600	242	240–244	243, 240, 240, 240, 242, 242, 244	-25.1%
8/49999	deduplicate	600	451	444–477	444, 449, 448, 451, 469, 451, 477	+39.6%
8/49999	xorf	600	323	303–403	316, 310, 303, 333, 323, 325, 403	reference
8/50000	distinct	600	243	240–249	247, 248, 242, 240, 241, 249, 243	-22.1%
8/50000	deduplicate	600	449	446–455	455, 452, 448, 448, 449, 446, 454	+43.9%
8/50000	xorf	600	312	309–347	311, 328, 332, 309, 312, 309, 347	reference
8/50001	distinct	600	247	243–256	243, 253, 256, 245, 249, 246, 247	-25.2%
8/50001	deduplicate	600	459	450–475	459, 463, 459, 475, 461, 453, 450	+39.1%
8/50001	xorf	600	330	311–355	311, 325, 345, 354, 330, 355, 314	reference
8/100000	distinct	200	185	183–191	185, 184, 183, 183, 191, 188, 189	-41.5%
8/100000	deduplicate	200	351	346–370	346, 370, 354, 353, 346, 348, 351	+11.1%
8/100000	xorf	200	316	312–319	319, 317, 316, 316, 312, 313, 312	reference
16/1000	distinct	60000	316	313–322	318, 316, 314, 315, 322, 318, 313	-31.6%
16/1000	deduplicate	60000	595	592–622	592, 605, 595, 603, 592, 592, 622	+28.8%
16/1000	xorf	60000	462	459–476	462, 476, 476, 461, 475, 460, 459	reference
16/16384	distinct	2000	357	356–376	376, 358, 357, 356, 362, 357, 357	+23.5%
16/16384	deduplicate	2000	560	552–582	582, 560, 560, 557, 560, 557, 552	+93.8%
16/16384	xorf	2000	289	283–294	291, 286, 283, 289, 292, 285, 294	reference
16/49999	distinct	600	249	246–253	252, 251, 246, 247, 253, 247, 249	-23.4%
16/49999	deduplicate	600	457	453–463	463, 456, 458, 455, 460, 453, 457	+40.6%
16/49999	xorf	600	325	317–348	343, 325, 329, 348, 322, 324, 317	reference
16/50000	distinct	600	248	246–251	247, 247, 248, 251, 250, 250, 246	-25.1%
16/50000	deduplicate	600	458	455–467	458, 458, 467, 462, 460, 457, 455	+38.4%
16/50000	xorf	600	331	313–349	338, 324, 313, 318, 349, 331, 335	reference
16/50001	distinct	600	250	247–251	250, 250, 248, 247, 247, 251, 250	-22.4%
16/50001	deduplicate	600	456	451–476	454, 457, 451, 462, 453, 476, 456	+41.6%
16/50001	xorf	600	322	318–335	322, 335, 334, 320, 320, 323, 318	reference
16/100000	distinct	200	184	182–187	185, 183, 185, 187, 184, 182, 183	-38.5%
16/100000	deduplicate	200	341	334–359	359, 340, 345, 355, 337, 334, 341	+14.0%
16/100000	xorf	200	299	293–317	313, 296, 293, 304, 317, 299, 296	reference
32/1000	distinct	60000	311	309–318	309, 315, 309, 318, 311, 309, 313	-33.5%
32/1000	deduplicate	60000	581	576–585	581, 576, 577, 585, 577, 584, 582	+24.1%
32/1000	xorf	60000	468	463–472	472, 471, 466, 466, 468, 463, 469	reference
32/16384	distinct	2000	360	356–362	356, 358, 362, 362, 360, 356, 362	+25.4%
32/16384	deduplicate	2000	559	553–564	553, 559, 564, 562, 554, 562, 558	+94.8%
32/16384	xorf	2000	287	284–292	289, 291, 284, 286, 292, 287, 287	reference
32/49999	distinct	600	253	252–262	253, 252, 255, 253, 256, 252, 262	-26.5%
32/49999	deduplicate	600	459	455–474	455, 459, 474, 456, 455, 464, 464	+33.4%
32/49999	xorf	600	344	322–358	349, 338, 358, 322, 356, 344, 341	reference
32/50000	distinct	600	253	252–255	253, 252, 253, 255, 252, 253, 252	-24.9%
32/50000	deduplicate	600	461	456–469	457, 461, 469, 461, 459, 456, 464	+36.8%
32/50000	xorf	600	337	323–354	329, 337, 323, 354, 329, 348, 347	reference
32/50001	distinct	600	253	251–254	254, 254, 253, 251, 253, 252, 252	-26.5%
32/50001	deduplicate	600	459	457–471	466, 458, 458, 465, 471, 457, 459	+33.4%
32/50001	xorf	600	344	333–357	344, 333, 356, 336, 345, 333, 357	reference
32/100000	distinct	200	189	186–197	191, 187, 187, 186, 197, 191, 189	-38.8%
32/100000	deduplicate	200	348	340–356	350, 348, 340, 341, 356, 348, 340	+12.6%
32/100000	xorf	200	309	299–316	306, 303, 313, 309, 316, 315, 299	reference

xor::build_batches:: — 3 scenarios, 9 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8/128	default_seed	10000	263	260–268	268, 263, 266, 263, 261, 260, 263	-32.4%
8/128	matched_seed	10000	267	265–275	266, 266, 275, 270, 267, 267, 265	-31.4%
8/128	xorf	10000	389	386–390	390, 389, 389, 389, 386, 386, 390	reference
8/16384	default_seed	40	151	150–152	152, 150, 151, 151, 151, 150, 152	-57.1%
8/16384	matched_seed	40	142	140–144	144, 141, 142, 140, 140, 143, 144	-59.7%
8/16384	xorf	40	352	350–355	354, 355, 355, 350, 351, 352, 352	reference
8/50000	default_seed	12	172	169–177	169, 170, 177, 172, 172, 171, 172	-47.6%
8/50000	matched_seed	12	168	164–169	164, 165, 169, 168, 167, 169, 168	-48.8%
8/50000	xorf	12	328	321–508	321, 324, 330, 345, 508, 328, 327	reference

xor::matched_build:: — 24 scenarios, 48 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8/(1000, 0)	DataSketches	60000	313	309–327	311, 309, 313, 327, 323, 313, 324	-33.4%
8/(1000, 0)	xorf	60000	470	456–485	461, 456, 471, 468, 470, 471, 485	reference
8/(16384, 0)	DataSketches	3000	307	306–310	309, 307, 307, 307, 306, 308, 310	-28.9%
8/(16384, 0)	xorf	3000	432	429–439	436, 432, 439, 429, 429, 435, 431	reference
8/(16384, 917504)	DataSketches	2000	359	357–360	358, 360, 359, 360, 358, 357, 359	-28.6%
8/(16384, 917504)	xorf	2000	503	499–508	508, 503, 503, 502, 499, 505, 506	reference
8/(49999, 0)	DataSketches	600	243	242–247	243, 247, 243, 242, 244, 243, 243	-24.1%
8/(49999, 0)	xorf	600	320	310–334	334, 333, 327, 310, 320, 310, 320	reference
8/(50000, 0)	DataSketches	600	245	243–251	243, 246, 245, 245, 244, 251, 244	-24.1%
8/(50000, 0)	xorf	600	323	312–365	319, 327, 365, 315, 360, 312, 323	reference
8/(50001, 0)	DataSketches	600	245	244–250	244, 246, 244, 247, 245, 244, 250	-26.6%
8/(50001, 0)	xorf	600	334	312–343	334, 313, 343, 328, 340, 312, 341	reference
8/(100000, 0)	DataSketches	200	184	181–186	184, 184, 181, 184, 186, 184, 186	-40.6%
8/(100000, 0)	xorf	200	310	294–327	304, 294, 327, 310, 315, 301, 318	reference
8/(8000000, 0)	DataSketches	1	279	261–307	297, 281, 261, 307, 278, 271, 279	-12.5%
8/(8000000, 0)	xorf	1	319	316–474	474, 355, 316, 329, 317, 317, 319	reference
16/(1000, 0)	DataSketches	60000	310	308–314	311, 309, 310, 310, 314, 308, 308	-33.6%
16/(1000, 0)	xorf	60000	467	464–469	467, 469, 469, 466, 467, 466, 464	reference
16/(16384, 0)	DataSketches	3000	312	309–321	321, 311, 313, 315, 309, 310, 312	-27.8%
16/(16384, 0)	xorf	3000	432	425–441	441, 438, 427, 430, 432, 425, 435	reference
16/(16384, 917504)	DataSketches	2000	366	360–372	360, 361, 366, 372, 370, 370, 363	-28.7%
16/(16384, 917504)	xorf	2000	513	505–531	511, 505, 523, 513, 531, 506, 518	reference
16/(49999, 0)	DataSketches	600	252	251–254	253, 251, 252, 254, 251, 251, 252	-22.5%
16/(49999, 0)	xorf	600	325	316–345	325, 316, 340, 325, 339, 319, 345	reference
16/(50000, 0)	DataSketches	600	251	249–252	249, 251, 251, 250, 250, 252, 252	-24.9%
16/(50000, 0)	xorf	600	334	317–354	335, 343, 354, 332, 334, 317, 324	reference
16/(50001, 0)	DataSketches	600	252	251–256	251, 252, 252, 253, 252, 256, 251	-25.9%
16/(50001, 0)	xorf	600	340	333–357	357, 339, 333, 344, 345, 340, 337	reference
16/(100000, 0)	DataSketches	200	186	184–189	187, 189, 186, 186, 184, 188, 185	-39.8%
16/(100000, 0)	xorf	200	309	299–326	326, 309, 315, 299, 305, 302, 313	reference
16/(8000000, 0)	DataSketches	1	279	275–286	286, 275, 279, 276, 283, 277, 283	-15.7%
16/(8000000, 0)	xorf	1	331	322–357	357, 337, 331, 333, 322, 326, 325	reference
32/(1000, 0)	DataSketches	60000	308	306–309	307, 309, 307, 308, 309, 306, 308	-33.8%
32/(1000, 0)	xorf	60000	465	463–468	464, 465, 466, 465, 466, 468, 463	reference
32/(16384, 0)	DataSketches	3000	318	314–325	314, 318, 316, 318, 325, 323, 314	-27.4%
32/(16384, 0)	xorf	3000	438	436–440	437, 436, 438, 439, 440, 436, 438	reference
32/(16384, 917504)	DataSketches	2000	367	365–408	367, 365, 367, 367, 408, 369, 366	-29.8%
32/(16384, 917504)	xorf	2000	523	509–557	537, 538, 557, 515, 522, 509, 523	reference
32/(49999, 0)	DataSketches	600	256	254–260	260, 255, 256, 254, 258, 256, 254	-25.8%
32/(49999, 0)	xorf	600	345	319–370	357, 339, 349, 326, 345, 319, 370	reference
32/(50000, 0)	DataSketches	600	256	254–261	255, 259, 255, 256, 261, 254, 256	-25.8%
32/(50000, 0)	xorf	600	345	321–352	348, 333, 352, 323, 349, 321, 345	reference
32/(50001, 0)	DataSketches	600	257	254–265	260, 265, 257, 257, 256, 258, 254	-26.1%
32/(50001, 0)	xorf	600	348	333–397	348, 343, 351, 337, 333, 362, 397	reference
32/(100000, 0)	DataSketches	200	195	190–222	195, 190, 190, 195, 222, 195, 192	-41.1%
32/(100000, 0)	xorf	200	331	316–367	333, 316, 317, 328, 367, 342, 331	reference
32/(8000000, 0)	DataSketches	1	303	288–328	316, 306, 288, 294, 292, 303, 328	-14.6%
32/(8000000, 0)	xorf	1	355	348–481	481, 377, 354, 355, 348, 350, 366	reference

xor::query:: — 12 scenarios, 24 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8_present/16384	DataSketches	10000	251	248–272	251, 248, 248, 272, 255, 253, 248	-1.6%
8_present/16384	xorf	10000	255	254–263	259, 255, 254, 254, 263, 255, 255	reference
8_present/50000	DataSketches	3000	229	224–318	224, 229, 231, 230, 318, 225, 224	-0.9%
8_present/50000	xorf	3000	231	229–236	230, 235, 235, 231, 236, 231, 229	reference
8_absent/16384	DataSketches	10000	247	246–252	247, 249, 248, 246, 246, 252, 247	-3.1%
8_absent/16384	xorf	10000	255	251–309	309, 257, 255, 254, 251, 254, 260	reference
8_absent/50000	DataSketches	3000	225	222–301	225, 226, 222, 225, 228, 225, 301	-2.6%
8_absent/50000	xorf	3000	231	227–233	233, 229, 228, 227, 231, 231, 233	reference
16_present/16384	DataSketches	10000	251	249–255	255, 253, 255, 251, 249, 249, 249	-1.6%
16_present/16384	xorf	10000	255	252–258	258, 256, 255, 256, 252, 255, 254	reference
16_present/50000	DataSketches	3000	237	235–243	237, 235, 239, 236, 235, 238, 243	+2.2%
16_present/50000	xorf	3000	232	230–235	233, 231, 235, 230, 232, 230, 234	reference
16_absent/16384	DataSketches	10000	254	249–260	258, 253, 254, 260, 257, 249, 251	-1.9%
16_absent/16384	xorf	10000	259	252–260	260, 259, 259, 259, 256, 254, 252	reference
16_absent/50000	DataSketches	3000	236	236–241	236, 236, 241, 237, 236, 236, 236	+2.2%
16_absent/50000	xorf	3000	231	229–238	233, 230, 230, 238, 235, 229, 231	reference
32_present/16384	DataSketches	10000	241	239–248	241, 241, 242, 239, 241, 245, 248	-4.0%
32_present/16384	xorf	10000	251	250–258	251, 250, 253, 251, 251, 252, 258	reference
32_present/50000	DataSketches	3000	246	244–249	246, 246, 244, 246, 244, 248, 249	-5.7%
32_present/50000	xorf	3000	261	256–265	265, 260, 256, 257, 261, 263, 264	reference
32_absent/16384	DataSketches	10000	242	239–250	242, 248, 241, 239, 250, 243, 240	-5.8%
32_absent/16384	xorf	10000	257	253–262	257, 262, 258, 257, 258, 253, 256	reference
32_absent/50000	DataSketches	3000	249	248–251	249, 248, 248, 249, 251, 251, 248	-3.9%
32_absent/50000	xorf	3000	259	257–261	258, 260, 259, 257, 258, 261, 261	reference

xor::query_shards:: — 12 scenarios, 24 implementation rows

Case	Implementation	Rounds	Median	Min–max	Samples (ms)	Delta vs xorf
8_present/128	DataSketches	20000	277	273–307	274, 275, 307, 277, 278, 279, 273	-1.4%
8_present/128	xorf	20000	281	274–281	281, 274, 278, 281, 279, 281, 281	reference
8_absent/128	DataSketches	20000	275	273–279	279, 275, 276, 273, 274, 274, 277	-1.4%
8_absent/128	xorf	20000	279	278–335	335, 282, 279, 278, 278, 279, 278	reference
8_present/16384	DataSketches	20000	346	342–353	346, 353, 345, 342, 346, 347, 344	+0.9%
8_present/16384	xorf	20000	343	341–351	343, 351, 342, 341, 342, 347, 343	reference
8_absent/16384	DataSketches	20000	341	337–355	341, 337, 348, 337, 337, 343, 355	-0.3%
8_absent/16384	xorf	20000	342	339–345	342, 339, 343, 340, 341, 343, 345	reference
16_present/128	DataSketches	20000	280	279–281	280, 279, 279, 281, 281, 279, 281	+0.4%
16_present/128	xorf	20000	279	277–284	279, 279, 278, 284, 277, 279, 279	reference
16_absent/128	DataSketches	20000	279	277–283	278, 280, 278, 279, 277, 283, 280	0.0%
16_absent/128	xorf	20000	279	279–285	279, 279, 281, 279, 279, 283, 285	reference
16_present/16384	DataSketches	20000	356	352–360	352, 360, 358, 352, 356, 356, 356	+1.4%
16_present/16384	xorf	20000	351	349–359	349, 349, 354, 349, 359, 355, 351	reference
16_absent/16384	DataSketches	20000	351	348–359	350, 349, 351, 354, 351, 348, 359	+0.6%
16_absent/16384	xorf	20000	349	344–361	361, 349, 347, 344, 350, 345, 351	reference
32_present/128	DataSketches	20000	276	274–283	277, 276, 274, 274, 283, 278, 274	-1.4%
32_present/128	xorf	20000	280	278–284	282, 284, 279, 278, 280, 281, 279	reference
32_absent/128	DataSketches	20000	273	271–277	273, 271, 271, 275, 277, 273, 274	-1.8%
32_absent/128	xorf	20000	278	276–288	278, 276, 276, 276, 281, 288, 279	reference
32_present/16384	DataSketches	20000	358	355–364	364, 360, 357, 358, 358, 358, 355	-1.1%
32_present/16384	xorf	20000	362	357–370	359, 362, 364, 362, 357, 358, 370	reference
32_absent/16384	DataSketches	20000	350	348–355	355, 350, 351, 348, 354, 348, 348	-1.1%
32_absent/16384	xorf	20000	354	349–360	357, 351, 354, 349, 360, 353, 356	reference