diff --git a/README.md b/README.md index a2b7f2b..dee34c2 100644 --- a/README.md +++ b/README.md @@ -93,39 +93,41 @@ each of the 30 queries run in the benchmark. ## High-level results - Query | neo4j-2025.12.1 (ms) | kuzu-0.11.3 (ms) | ladybug-0.14.1 (ms) | lance-graph-0.5.1 (ms) | +| Query | neo4j-2025.12.1 (ms) | kuzu-0.11.3 (ms) | ladybug-0.14.1 (ms) | lance-graph-0.5.1 (ms) | | --- | --- | --- | --- | --- | -| q1 | 6ms | 2ms (2.4x) | 2ms (2.7x) | 24ms (0.2x) | -| q2 | 6ms | 1ms (4.4x) | 2ms (4.2x) | 25ms (0.3x) | -| q3 | 3ms | 1ms (2.1x) | 1ms (2.2x) | 26ms (0.1x) | -| q4 | 5ms | 1ms (4.6x) | 1ms (4.3x) | 25ms (0.2x) | -| q5 | 5ms | 4ms (1.3x) | 4ms (1.4x) | 24ms (0.2x) | -| q6 | 4ms | 1ms (5.1x) | 1ms (4.6x) | 23ms (0.2x) | -| q7 | 2ms | 31ms (0.1x) | 31ms (0.1x) | 35ms (0.1x) | -| q8 | 16ms | 3ms (5.6x) | 3ms (5.5x) | 23ms (0.7x) | -| q9 | 3ms | 2ms (1.3x) | 2ms (1.2x) | 25ms (0.1x) | -| q10 | 5ms | 2ms (2.7x) | 2ms (2.6x) | 48ms (0.1x) | -| q11 | 13ms | 8ms (1.6x) | 8ms (1.5x) | 26ms (0.5x) | -| q12 | 5ms | 18ms (0.3x) | 18ms (0.3x) | 40ms (0.1x) | -| q13 | 9ms | 50ms (0.2x) | 52ms (0.2x) | 31ms (0.3x) | -| q14 | 2ms | 2ms (1.0x) | 2ms (0.9x) | 25ms (0.1x) | -| q15 | 3ms | 3ms (1.2x) | 3ms (1.2x) | 25ms (0.1x) | -| q16 | 2ms | 2ms (1.0x) | 2ms (1.0x) | 27ms (0.1x) | -| q17 | 5ms | 3ms (1.8x) | 3ms (1.8x) | 25ms (0.2x) | -| q18 | 4ms | 2ms (2.3x) | 2ms (2.3x) | 25ms (0.2x) | -| q19 | 6ms | 13ms (0.5x) | 14ms (0.5x) | 42ms (0.1x) | -| q20 | 422ms | 12ms (35.2x) | 11ms (38.1x) | 25ms (16.9x) | -| q21 | 2ms | 1ms (2.7x) | 1ms (2.7x) | 24ms (0.1x) | -| q22 | 4ms | 20ms (0.2x) | 21ms (0.2x) | 35ms (0.1x) | -| q23 | 4ms | 1ms (2.9x) | 1ms (2.8x) | 25ms (0.2x) | -| q24 | 2ms | 2ms (1.0x) | 2ms (1.0x) | 25ms (0.1x) | -| q25 | 3ms | 2ms (1.9x) | 2ms (1.8x) | 24ms (0.1x) | -| q26 | 2ms | 4ms (0.5x) | 4ms (0.5x) | 26ms (0.1x) | -| q27 | 3ms | 13ms (0.3x) | 16ms (0.2x) | 46ms (0.1x) | -| q28 | 4ms | 2ms (2.6x) | 2ms (2.3x) | 26ms (0.2x) | -| q29 | 3ms | 1ms (2.4x) | 1ms (2.4x) | 26ms (0.1x) | - -Q30 is the most expensive query of all, and was not run for this stage of the benchmark, because lance-graph's query planner doesn't yet support it. Stay tuned as more updates come along! +| q1 | 6ms | 2ms (2.4x) | 2ms (2.7x) | 1ms (3.9x) | +| q2 | 6ms | 1ms (4.4x) | 2ms (4.2x) | 2ms (2.8x) | +| q3 | 3ms | 1ms (2.1x) | 1ms (2.2x) | 3ms (0.8x) | +| q4 | 5ms | 1ms (4.6x) | 1ms (4.3x) | 3ms (1.5x) | +| q5 | 5ms | 4ms (1.3x) | 4ms (1.4x) | 2ms (2.4x) | +| q6 | 4ms | 1ms (5.1x) | 1ms (4.6x) | 1ms (5.0x) | +| q7 | 2ms | 31ms (0.1x) | 31ms (0.1x) | 13ms (0.2x) | +| q8 | 16ms | 3ms (5.6x) | 3ms (5.5x) | 1ms (12.4x) | +| q9 | 3ms | 2ms (1.3x) | 2ms (1.2x) | 2ms (1.2x) | +| q10 | 5ms | 2ms (2.7x) | 2ms (2.6x) | 25ms (0.2x) | +| q11 | 13ms | 8ms (1.6x) | 8ms (1.5x) | 3ms (4.0x) | +| q12 | 5ms | 18ms (0.3x) | 18ms (0.3x) | 18ms (0.3x) | +| q13 | 9ms | 50ms (0.2x) | 52ms (0.2x) | 9ms (1.0x) | +| q14 | 2ms | 2ms (1.0x) | 2ms (0.9x) | 3ms (0.6x) | +| q15 | 3ms | 3ms (1.2x) | 3ms (1.2x) | 2ms (1.4x) | +| q16 | 2ms | 2ms (1.0x) | 2ms (1.0x) | 4ms (0.5x) | +| q17 | 5ms | 3ms (1.8x) | 3ms (1.8x) | 3ms (2.0x) | +| q18 | 4ms | 2ms (2.3x) | 2ms (2.3x) | 2ms (2.0x) | +| q19 | 6ms | 13ms (0.5x) | 14ms (0.5x) | 19ms (0.3x) | +| q20 | 422ms | 12ms (35.2x) | 11ms (38.1x) | 2ms (175.8x) | +| q21 | 2ms | 1ms (2.7x) | 1ms (2.7x) | 2ms (0.9x) | +| q22 | 4ms | 20ms (0.2x) | 21ms (0.2x) | 12ms (0.3x) | +| q23 | 4ms | 1ms (2.9x) | 1ms (2.8x) | 3ms (1.5x) | +| q24 | 2ms | 2ms (1.0x) | 2ms (1.0x) | 2ms (0.8x) | +| q25 | 3ms | 2ms (1.9x) | 2ms (1.8x) | 1ms (2.1x) | +| q26 | 2ms | 4ms (0.5x) | 4ms (0.5x) | 3ms (0.6x) | +| q27 | 3ms | 13ms (0.3x) | 16ms (0.2x) | 23ms (0.1x) | +| q28 | 4ms | 2ms (2.6x) | 2ms (2.3x) | 3ms (1.5x) | +| q29 | 3ms | 1ms (2.4x) | 1ms (2.4x) | 3ms (1.2x) | + +The fastest and slowest systems for each query can be visualized via the following heatmap. + +![](./results/benchmark_heatmap.png) > [!NOTE] > The lance-graph query processor is undergoing significant optimization and a new version of the query planner is on the way soon. The numbers here are artificially large because several queries had to be rewritten to materialize results prematurely, with aggregations done externally so as to bypass (as yet) unsupported Cypher syntax in lance-graph. As a result, these results should be treated as a moving target -- follow this repo and check back in soon! \ No newline at end of file diff --git a/lance_graph/README.md b/lance_graph/README.md index dc069a0..8bd1c5d 100644 --- a/lance_graph/README.md +++ b/lance_graph/README.md @@ -41,54 +41,54 @@ uv run query.py "1,2,6" ```bash ❯ uv run pytest benchmark_query.py --benchmark-min-rounds=5 --benchmark-warmup-iterations=5 --benchmark-disable-gc --benchmark-sort=fullname -======================================= test session starts ======================================= +============================================ test session starts ============================================= platform darwin -- Python 3.13.7, pytest-9.0.2, pluggy-1.6.0 benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=True min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=5) rootdir: /Users/prrao/code/graph-benchmark-ldbc-snb configfile: pyproject.toml plugins: anyio-4.12.1, benchmark-5.2.3, asyncio-1.3.0, Faker-40.1.2 asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function -collected 29 items - -benchmark_query.py ............................. [100%] - - ----------------------------------------------------------------------------------- benchmark: 29 tests ----------------------------------------------------------------------------------- -Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -test_benchmark_query1 23.0999 (1.03) 31.7585 (1.35) 23.9868 (1.05) 1.8277 (10.09) 23.5428 (1.03) 0.3856 (2.23) 1;3 41.6897 (0.95) 21 1 -test_benchmark_query10 47.1176 (2.09) 48.2161 (2.06) 47.6572 (2.08) 0.3497 (1.93) 47.5551 (2.08) 0.6016 (3.47) 8;0 20.9832 (0.48) 21 1 -test_benchmark_query11 25.3632 (1.13) 26.5225 (1.13) 25.8498 (1.13) 0.2844 (1.57) 25.8189 (1.13) 0.3637 (2.10) 13;0 38.6850 (0.88) 37 1 -test_benchmark_query12 39.7474 (1.77) 40.7568 (1.74) 40.1831 (1.76) 0.2553 (1.41) 40.1617 (1.76) 0.3421 (1.97) 8;0 24.8861 (0.57) 24 1 -test_benchmark_query13 30.6525 (1.36) 31.6718 (1.35) 31.1661 (1.36) 0.2219 (1.22) 31.1685 (1.37) 0.2735 (1.58) 9;0 32.0862 (0.73) 32 1 -test_benchmark_query14 24.9189 (1.11) 25.7928 (1.10) 25.2736 (1.11) 0.2192 (1.21) 25.2234 (1.10) 0.2905 (1.68) 14;0 39.5670 (0.90) 40 1 -test_benchmark_query15 24.4846 (1.09) 25.9639 (1.11) 24.8480 (1.09) 0.3059 (1.69) 24.7728 (1.09) 0.2651 (1.53) 7;4 40.2447 (0.92) 40 1 -test_benchmark_query16 26.4125 (1.17) 28.7315 (1.22) 26.9644 (1.18) 0.4993 (2.76) 26.8477 (1.18) 0.4448 (2.57) 10;2 37.0859 (0.85) 38 1 -test_benchmark_query17 24.8586 (1.11) 25.9273 (1.11) 25.1584 (1.10) 0.2070 (1.14) 25.1088 (1.10) 0.2497 (1.44) 10;1 39.7482 (0.91) 40 1 -test_benchmark_query18 24.1813 (1.08) 25.1143 (1.07) 24.5338 (1.07) 0.2151 (1.19) 24.5376 (1.07) 0.3178 (1.83) 13;0 40.7601 (0.93) 40 1 -test_benchmark_query19 40.9274 (1.82) 42.1138 (1.80) 41.5658 (1.82) 0.2597 (1.43) 41.5231 (1.82) 0.3043 (1.76) 6;1 24.0582 (0.55) 24 1 -test_benchmark_query2 24.0586 (1.07) 25.6033 (1.09) 24.5185 (1.07) 0.3828 (2.11) 24.3870 (1.07) 0.4703 (2.71) 11;1 40.7855 (0.93) 40 1 -test_benchmark_query20 24.7223 (1.10) 25.5913 (1.09) 25.0012 (1.09) 0.2271 (1.25) 24.9686 (1.09) 0.3116 (1.80) 12;1 39.9980 (0.91) 39 1 -test_benchmark_query21 23.9809 (1.07) 24.8590 (1.06) 24.3109 (1.06) 0.2306 (1.27) 24.2767 (1.06) 0.4152 (2.40) 18;0 41.1339 (0.94) 40 1 -test_benchmark_query22 34.5507 (1.54) 35.4196 (1.51) 35.0123 (1.53) 0.1942 (1.07) 35.0202 (1.53) 0.1733 (1.0) 10;3 28.5614 (0.65) 29 1 -test_benchmark_query23 25.0542 (1.11) 25.7579 (1.10) 25.3481 (1.11) 0.1861 (1.03) 25.3184 (1.11) 0.3040 (1.75) 12;0 39.4507 (0.90) 39 1 -test_benchmark_query24 24.2572 (1.08) 24.9892 (1.07) 24.6704 (1.08) 0.1812 (1.0) 24.6422 (1.08) 0.2831 (1.63) 14;0 40.5345 (0.93) 40 1 -test_benchmark_query25 23.6439 (1.05) 24.3974 (1.04) 24.0185 (1.05) 0.1917 (1.06) 24.0060 (1.05) 0.3119 (1.80) 12;0 41.6346 (0.95) 41 1 -test_benchmark_query26 25.3782 (1.13) 26.7081 (1.14) 25.8477 (1.13) 0.2269 (1.25) 25.8677 (1.13) 0.2506 (1.45) 8;1 38.6881 (0.88) 38 1 -test_benchmark_query27 45.2601 (2.01) 47.5899 (2.03) 46.1629 (2.02) 0.7840 (4.33) 45.8066 (2.01) 1.2243 (7.07) 6;0 21.6624 (0.50) 22 1 -test_benchmark_query28 25.3215 (1.13) 26.6050 (1.13) 25.9303 (1.13) 0.3192 (1.76) 25.9396 (1.14) 0.5335 (3.08) 13;0 38.5650 (0.88) 37 1 -test_benchmark_query29 25.0946 (1.12) 26.3178 (1.12) 25.5235 (1.12) 0.2618 (1.44) 25.4866 (1.12) 0.2401 (1.39) 11;4 39.1795 (0.90) 39 1 -test_benchmark_query3 25.0039 (1.11) 26.0560 (1.11) 25.5028 (1.12) 0.2508 (1.38) 25.5228 (1.12) 0.4286 (2.47) 12;0 39.2114 (0.90) 38 1 -test_benchmark_query4 24.7615 (1.10) 25.7094 (1.10) 25.1573 (1.10) 0.1994 (1.10) 25.1555 (1.10) 0.2443 (1.41) 11;1 39.7500 (0.91) 40 1 -test_benchmark_query5 23.8402 (1.06) 25.2271 (1.08) 24.4019 (1.07) 0.3319 (1.83) 24.3533 (1.07) 0.4441 (2.56) 13;0 40.9805 (0.94) 41 1 -test_benchmark_query6 22.4930 (1.0) 23.4594 (1.0) 22.8720 (1.0) 0.2377 (1.31) 22.8306 (1.0) 0.3382 (1.95) 14;0 43.7215 (1.0) 43 1 -test_benchmark_query7 34.7297 (1.54) 35.7101 (1.52) 35.3026 (1.54) 0.2547 (1.41) 35.2882 (1.55) 0.3002 (1.73) 9;0 28.3265 (0.65) 28 1 -test_benchmark_query8 23.0910 (1.03) 23.9041 (1.02) 23.4985 (1.03) 0.2386 (1.32) 23.4764 (1.03) 0.4099 (2.37) 15;0 42.5560 (0.97) 42 1 -test_benchmark_query9 24.1168 (1.07) 32.9910 (1.41) 24.7383 (1.08) 1.3370 (7.38) 24.5148 (1.07) 0.3144 (1.81) 1;1 40.4232 (0.92) 41 1 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +collected 29 items + +benchmark_query.py ............................. [100%] + + +------------------------------------------------------------------------------------ benchmark: 29 tests ------------------------------------------------------------------------------------ +Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +test_benchmark_query1 1.3767 (1.95) 1.7772 (1.68) 1.4724 (1.77) 0.0525 (1.37) 1.4636 (1.77) 0.0596 (1.18) 54;6 679.1737 (0.56) 232 1 +test_benchmark_query10 24.5189 (34.65) 26.1044 (24.74) 25.2219 (30.36) 0.3681 (9.62) 25.2228 (30.42) 0.5567 (11.05) 13;0 39.6482 (0.03) 39 1 +test_benchmark_query11 2.7414 (3.87) 4.0430 (3.83) 3.2230 (3.88) 0.2305 (6.02) 3.2217 (3.89) 0.2930 (5.82) 75;2 310.2685 (0.26) 229 1 +test_benchmark_query12 17.1755 (24.27) 18.3755 (17.41) 17.7318 (21.35) 0.2483 (6.49) 17.6685 (21.31) 0.2672 (5.30) 14;4 56.3959 (0.05) 58 1 +test_benchmark_query13 8.3278 (11.77) 9.9391 (9.42) 8.6597 (10.43) 0.2016 (5.27) 8.6350 (10.41) 0.2352 (4.67) 22;1 115.4775 (0.10) 110 1 +test_benchmark_query14 2.5917 (3.66) 3.2937 (3.12) 2.7956 (3.37) 0.1052 (2.75) 2.7737 (3.34) 0.1003 (1.99) 61;17 357.7025 (0.30) 318 1 +test_benchmark_query15 2.1405 (3.02) 2.8080 (2.66) 2.2776 (2.74) 0.0736 (1.92) 2.2651 (2.73) 0.0787 (1.56) 84;13 439.0494 (0.36) 403 1 +test_benchmark_query16 3.9124 (5.53) 5.2454 (4.97) 4.2132 (5.07) 0.1204 (3.14) 4.2151 (5.08) 0.1292 (2.56) 45;3 237.3504 (0.20) 234 1 +test_benchmark_query17 2.4369 (3.44) 2.9189 (2.77) 2.5873 (3.11) 0.0704 (1.84) 2.5801 (3.11) 0.0785 (1.56) 82;13 386.5047 (0.32) 339 1 +test_benchmark_query18 1.8815 (2.66) 2.3139 (2.19) 2.0325 (2.45) 0.0625 (1.63) 2.0295 (2.45) 0.0839 (1.67) 131;7 492.0153 (0.41) 455 1 +test_benchmark_query19 18.4428 (26.06) 19.6378 (18.61) 18.9444 (22.81) 0.2700 (7.05) 18.9392 (22.84) 0.3506 (6.96) 16;1 52.7861 (0.04) 52 1 +test_benchmark_query2 2.0713 (2.93) 2.5650 (2.43) 2.3162 (2.79) 0.0817 (2.13) 2.3036 (2.78) 0.1143 (2.27) 121;2 431.7420 (0.36) 389 1 +test_benchmark_query20 2.1316 (3.01) 2.9786 (2.82) 2.3992 (2.89) 0.1500 (3.92) 2.3583 (2.84) 0.1600 (3.18) 85;26 416.8106 (0.35) 368 1 +test_benchmark_query21 1.5456 (2.18) 1.9687 (1.87) 1.6820 (2.02) 0.0697 (1.82) 1.6773 (2.02) 0.0838 (1.66) 103;14 594.5466 (0.49) 398 1 +test_benchmark_query22 12.1362 (17.15) 12.8211 (12.15) 12.4648 (15.01) 0.1629 (4.26) 12.4526 (15.02) 0.2373 (4.71) 26;0 80.2257 (0.07) 78 1 +test_benchmark_query23 2.4816 (3.51) 3.0308 (2.87) 2.6765 (3.22) 0.0765 (2.00) 2.6667 (3.22) 0.0908 (1.80) 96;7 373.6270 (0.31) 332 1 +test_benchmark_query24 1.8614 (2.63) 2.2366 (2.12) 1.9925 (2.40) 0.0576 (1.50) 1.9885 (2.40) 0.0692 (1.37) 124;9 501.8797 (0.42) 440 1 +test_benchmark_query25 1.3479 (1.90) 1.8455 (1.75) 1.4648 (1.76) 0.0465 (1.22) 1.4623 (1.76) 0.0584 (1.16) 175;4 682.6643 (0.57) 611 1 +test_benchmark_query26 2.8840 (4.08) 3.4320 (3.25) 3.1116 (3.75) 0.0938 (2.45) 3.1040 (3.74) 0.1305 (2.59) 93;3 321.3729 (0.27) 316 1 +test_benchmark_query27 22.4221 (31.69) 23.7178 (22.48) 23.0234 (27.72) 0.3001 (7.84) 23.0482 (27.79) 0.4085 (8.11) 14;0 43.4341 (0.04) 42 1 +test_benchmark_query28 2.6678 (3.77) 3.3802 (3.20) 2.8639 (3.45) 0.1074 (2.80) 2.8417 (3.43) 0.1035 (2.05) 62;16 349.1685 (0.29) 301 1 +test_benchmark_query29 2.4007 (3.39) 3.0182 (2.86) 2.6757 (3.22) 0.0928 (2.42) 2.6743 (3.23) 0.1226 (2.43) 118;5 373.7399 (0.31) 383 1 +test_benchmark_query3 3.1942 (4.51) 3.8568 (3.65) 3.3852 (4.08) 0.0895 (2.34) 3.3731 (4.07) 0.1102 (2.19) 75;4 295.4012 (0.25) 255 1 +test_benchmark_query4 2.8370 (4.01) 3.6791 (3.49) 3.1119 (3.75) 0.1136 (2.97) 3.1145 (3.76) 0.1467 (2.91) 96;2 321.3500 (0.27) 309 1 +test_benchmark_query5 1.8461 (2.61) 2.3278 (2.21) 2.1083 (2.54) 0.0858 (2.24) 2.1098 (2.54) 0.1243 (2.47) 128;3 474.3254 (0.39) 403 1 +test_benchmark_query6 0.7076 (1.0) 1.0553 (1.0) 0.8306 (1.0) 0.0383 (1.0) 0.8292 (1.0) 0.0504 (1.0) 273;10 1,203.8883 (1.0) 944 1 +test_benchmark_query7 12.7920 (18.08) 13.8115 (13.09) 13.2379 (15.94) 0.2134 (5.57) 13.1735 (15.89) 0.2816 (5.59) 21;1 75.5406 (0.06) 73 1 +test_benchmark_query8 1.1148 (1.58) 1.4808 (1.40) 1.2611 (1.52) 0.0517 (1.35) 1.2631 (1.52) 0.0723 (1.44) 191;4 792.9300 (0.66) 602 1 +test_benchmark_query9 2.0159 (2.85) 2.5005 (2.37) 2.1874 (2.63) 0.0663 (1.73) 2.1839 (2.63) 0.0880 (1.75) 127;3 457.1621 (0.38) 423 1 +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Legend: Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. OPS: Operations Per Second, computed as 1 / Mean -======================================= 29 passed in 31.56s ======================================= +============================================ 29 passed in 27.64s ============================================= ``` \ No newline at end of file diff --git a/results/benchmark_heatmap.png b/results/benchmark_heatmap.png index 394b9bc..3b9de45 100644 Binary files a/results/benchmark_heatmap.png and b/results/benchmark_heatmap.png differ diff --git a/results/lance-graph-0.5.1.txt b/results/lance-graph-0.5.1.txt index 3a65398..5278c62 100644 --- a/results/lance-graph-0.5.1.txt +++ b/results/lance-graph-0.5.1.txt @@ -1,33 +1,33 @@ ----------------------------------------------------------------------------------- benchmark: 29 tests ----------------------------------------------------------------------------------- -Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -test_benchmark_query1 23.0999 (1.03) 31.7585 (1.35) 23.9868 (1.05) 1.8277 (10.09) 23.5428 (1.03) 0.3856 (2.23) 1;3 41.6897 (0.95) 21 1 -test_benchmark_query10 47.1176 (2.09) 48.2161 (2.06) 47.6572 (2.08) 0.3497 (1.93) 47.5551 (2.08) 0.6016 (3.47) 8;0 20.9832 (0.48) 21 1 -test_benchmark_query11 25.3632 (1.13) 26.5225 (1.13) 25.8498 (1.13) 0.2844 (1.57) 25.8189 (1.13) 0.3637 (2.10) 13;0 38.6850 (0.88) 37 1 -test_benchmark_query12 39.7474 (1.77) 40.7568 (1.74) 40.1831 (1.76) 0.2553 (1.41) 40.1617 (1.76) 0.3421 (1.97) 8;0 24.8861 (0.57) 24 1 -test_benchmark_query13 30.6525 (1.36) 31.6718 (1.35) 31.1661 (1.36) 0.2219 (1.22) 31.1685 (1.37) 0.2735 (1.58) 9;0 32.0862 (0.73) 32 1 -test_benchmark_query14 24.9189 (1.11) 25.7928 (1.10) 25.2736 (1.11) 0.2192 (1.21) 25.2234 (1.10) 0.2905 (1.68) 14;0 39.5670 (0.90) 40 1 -test_benchmark_query15 24.4846 (1.09) 25.9639 (1.11) 24.8480 (1.09) 0.3059 (1.69) 24.7728 (1.09) 0.2651 (1.53) 7;4 40.2447 (0.92) 40 1 -test_benchmark_query16 26.4125 (1.17) 28.7315 (1.22) 26.9644 (1.18) 0.4993 (2.76) 26.8477 (1.18) 0.4448 (2.57) 10;2 37.0859 (0.85) 38 1 -test_benchmark_query17 24.8586 (1.11) 25.9273 (1.11) 25.1584 (1.10) 0.2070 (1.14) 25.1088 (1.10) 0.2497 (1.44) 10;1 39.7482 (0.91) 40 1 -test_benchmark_query18 24.1813 (1.08) 25.1143 (1.07) 24.5338 (1.07) 0.2151 (1.19) 24.5376 (1.07) 0.3178 (1.83) 13;0 40.7601 (0.93) 40 1 -test_benchmark_query19 40.9274 (1.82) 42.1138 (1.80) 41.5658 (1.82) 0.2597 (1.43) 41.5231 (1.82) 0.3043 (1.76) 6;1 24.0582 (0.55) 24 1 -test_benchmark_query2 24.0586 (1.07) 25.6033 (1.09) 24.5185 (1.07) 0.3828 (2.11) 24.3870 (1.07) 0.4703 (2.71) 11;1 40.7855 (0.93) 40 1 -test_benchmark_query20 24.7223 (1.10) 25.5913 (1.09) 25.0012 (1.09) 0.2271 (1.25) 24.9686 (1.09) 0.3116 (1.80) 12;1 39.9980 (0.91) 39 1 -test_benchmark_query21 23.9809 (1.07) 24.8590 (1.06) 24.3109 (1.06) 0.2306 (1.27) 24.2767 (1.06) 0.4152 (2.40) 18;0 41.1339 (0.94) 40 1 -test_benchmark_query22 34.5507 (1.54) 35.4196 (1.51) 35.0123 (1.53) 0.1942 (1.07) 35.0202 (1.53) 0.1733 (1.0) 10;3 28.5614 (0.65) 29 1 -test_benchmark_query23 25.0542 (1.11) 25.7579 (1.10) 25.3481 (1.11) 0.1861 (1.03) 25.3184 (1.11) 0.3040 (1.75) 12;0 39.4507 (0.90) 39 1 -test_benchmark_query24 24.2572 (1.08) 24.9892 (1.07) 24.6704 (1.08) 0.1812 (1.0) 24.6422 (1.08) 0.2831 (1.63) 14;0 40.5345 (0.93) 40 1 -test_benchmark_query25 23.6439 (1.05) 24.3974 (1.04) 24.0185 (1.05) 0.1917 (1.06) 24.0060 (1.05) 0.3119 (1.80) 12;0 41.6346 (0.95) 41 1 -test_benchmark_query26 25.3782 (1.13) 26.7081 (1.14) 25.8477 (1.13) 0.2269 (1.25) 25.8677 (1.13) 0.2506 (1.45) 8;1 38.6881 (0.88) 38 1 -test_benchmark_query27 45.2601 (2.01) 47.5899 (2.03) 46.1629 (2.02) 0.7840 (4.33) 45.8066 (2.01) 1.2243 (7.07) 6;0 21.6624 (0.50) 22 1 -test_benchmark_query28 25.3215 (1.13) 26.6050 (1.13) 25.9303 (1.13) 0.3192 (1.76) 25.9396 (1.14) 0.5335 (3.08) 13;0 38.5650 (0.88) 37 1 -test_benchmark_query29 25.0946 (1.12) 26.3178 (1.12) 25.5235 (1.12) 0.2618 (1.44) 25.4866 (1.12) 0.2401 (1.39) 11;4 39.1795 (0.90) 39 1 -test_benchmark_query3 25.0039 (1.11) 26.0560 (1.11) 25.5028 (1.12) 0.2508 (1.38) 25.5228 (1.12) 0.4286 (2.47) 12;0 39.2114 (0.90) 38 1 -test_benchmark_query4 24.7615 (1.10) 25.7094 (1.10) 25.1573 (1.10) 0.1994 (1.10) 25.1555 (1.10) 0.2443 (1.41) 11;1 39.7500 (0.91) 40 1 -test_benchmark_query5 23.8402 (1.06) 25.2271 (1.08) 24.4019 (1.07) 0.3319 (1.83) 24.3533 (1.07) 0.4441 (2.56) 13;0 40.9805 (0.94) 41 1 -test_benchmark_query6 22.4930 (1.0) 23.4594 (1.0) 22.8720 (1.0) 0.2377 (1.31) 22.8306 (1.0) 0.3382 (1.95) 14;0 43.7215 (1.0) 43 1 -test_benchmark_query7 34.7297 (1.54) 35.7101 (1.52) 35.3026 (1.54) 0.2547 (1.41) 35.2882 (1.55) 0.3002 (1.73) 9;0 28.3265 (0.65) 28 1 -test_benchmark_query8 23.0910 (1.03) 23.9041 (1.02) 23.4985 (1.03) 0.2386 (1.32) 23.4764 (1.03) 0.4099 (2.37) 15;0 42.5560 (0.97) 42 1 -test_benchmark_query9 24.1168 (1.07) 32.9910 (1.41) 24.7383 (1.08) 1.3370 (7.38) 24.5148 (1.07) 0.3144 (1.81) 1;1 40.4232 (0.92) 41 1 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- \ No newline at end of file +------------------------------------------------------------------------------------ benchmark: 29 tests ------------------------------------------------------------------------------------ +Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +test_benchmark_query1 1.3767 (1.95) 1.7772 (1.68) 1.4724 (1.77) 0.0525 (1.37) 1.4636 (1.77) 0.0596 (1.18) 54;6 679.1737 (0.56) 232 1 +test_benchmark_query10 24.5189 (34.65) 26.1044 (24.74) 25.2219 (30.36) 0.3681 (9.62) 25.2228 (30.42) 0.5567 (11.05) 13;0 39.6482 (0.03) 39 1 +test_benchmark_query11 2.7414 (3.87) 4.0430 (3.83) 3.2230 (3.88) 0.2305 (6.02) 3.2217 (3.89) 0.2930 (5.82) 75;2 310.2685 (0.26) 229 1 +test_benchmark_query12 17.1755 (24.27) 18.3755 (17.41) 17.7318 (21.35) 0.2483 (6.49) 17.6685 (21.31) 0.2672 (5.30) 14;4 56.3959 (0.05) 58 1 +test_benchmark_query13 8.3278 (11.77) 9.9391 (9.42) 8.6597 (10.43) 0.2016 (5.27) 8.6350 (10.41) 0.2352 (4.67) 22;1 115.4775 (0.10) 110 1 +test_benchmark_query14 2.5917 (3.66) 3.2937 (3.12) 2.7956 (3.37) 0.1052 (2.75) 2.7737 (3.34) 0.1003 (1.99) 61;17 357.7025 (0.30) 318 1 +test_benchmark_query15 2.1405 (3.02) 2.8080 (2.66) 2.2776 (2.74) 0.0736 (1.92) 2.2651 (2.73) 0.0787 (1.56) 84;13 439.0494 (0.36) 403 1 +test_benchmark_query16 3.9124 (5.53) 5.2454 (4.97) 4.2132 (5.07) 0.1204 (3.14) 4.2151 (5.08) 0.1292 (2.56) 45;3 237.3504 (0.20) 234 1 +test_benchmark_query17 2.4369 (3.44) 2.9189 (2.77) 2.5873 (3.11) 0.0704 (1.84) 2.5801 (3.11) 0.0785 (1.56) 82;13 386.5047 (0.32) 339 1 +test_benchmark_query18 1.8815 (2.66) 2.3139 (2.19) 2.0325 (2.45) 0.0625 (1.63) 2.0295 (2.45) 0.0839 (1.67) 131;7 492.0153 (0.41) 455 1 +test_benchmark_query19 18.4428 (26.06) 19.6378 (18.61) 18.9444 (22.81) 0.2700 (7.05) 18.9392 (22.84) 0.3506 (6.96) 16;1 52.7861 (0.04) 52 1 +test_benchmark_query2 2.0713 (2.93) 2.5650 (2.43) 2.3162 (2.79) 0.0817 (2.13) 2.3036 (2.78) 0.1143 (2.27) 121;2 431.7420 (0.36) 389 1 +test_benchmark_query20 2.1316 (3.01) 2.9786 (2.82) 2.3992 (2.89) 0.1500 (3.92) 2.3583 (2.84) 0.1600 (3.18) 85;26 416.8106 (0.35) 368 1 +test_benchmark_query21 1.5456 (2.18) 1.9687 (1.87) 1.6820 (2.02) 0.0697 (1.82) 1.6773 (2.02) 0.0838 (1.66) 103;14 594.5466 (0.49) 398 1 +test_benchmark_query22 12.1362 (17.15) 12.8211 (12.15) 12.4648 (15.01) 0.1629 (4.26) 12.4526 (15.02) 0.2373 (4.71) 26;0 80.2257 (0.07) 78 1 +test_benchmark_query23 2.4816 (3.51) 3.0308 (2.87) 2.6765 (3.22) 0.0765 (2.00) 2.6667 (3.22) 0.0908 (1.80) 96;7 373.6270 (0.31) 332 1 +test_benchmark_query24 1.8614 (2.63) 2.2366 (2.12) 1.9925 (2.40) 0.0576 (1.50) 1.9885 (2.40) 0.0692 (1.37) 124;9 501.8797 (0.42) 440 1 +test_benchmark_query25 1.3479 (1.90) 1.8455 (1.75) 1.4648 (1.76) 0.0465 (1.22) 1.4623 (1.76) 0.0584 (1.16) 175;4 682.6643 (0.57) 611 1 +test_benchmark_query26 2.8840 (4.08) 3.4320 (3.25) 3.1116 (3.75) 0.0938 (2.45) 3.1040 (3.74) 0.1305 (2.59) 93;3 321.3729 (0.27) 316 1 +test_benchmark_query27 22.4221 (31.69) 23.7178 (22.48) 23.0234 (27.72) 0.3001 (7.84) 23.0482 (27.79) 0.4085 (8.11) 14;0 43.4341 (0.04) 42 1 +test_benchmark_query28 2.6678 (3.77) 3.3802 (3.20) 2.8639 (3.45) 0.1074 (2.80) 2.8417 (3.43) 0.1035 (2.05) 62;16 349.1685 (0.29) 301 1 +test_benchmark_query29 2.4007 (3.39) 3.0182 (2.86) 2.6757 (3.22) 0.0928 (2.42) 2.6743 (3.23) 0.1226 (2.43) 118;5 373.7399 (0.31) 383 1 +test_benchmark_query3 3.1942 (4.51) 3.8568 (3.65) 3.3852 (4.08) 0.0895 (2.34) 3.3731 (4.07) 0.1102 (2.19) 75;4 295.4012 (0.25) 255 1 +test_benchmark_query4 2.8370 (4.01) 3.6791 (3.49) 3.1119 (3.75) 0.1136 (2.97) 3.1145 (3.76) 0.1467 (2.91) 96;2 321.3500 (0.27) 309 1 +test_benchmark_query5 1.8461 (2.61) 2.3278 (2.21) 2.1083 (2.54) 0.0858 (2.24) 2.1098 (2.54) 0.1243 (2.47) 128;3 474.3254 (0.39) 403 1 +test_benchmark_query6 0.7076 (1.0) 1.0553 (1.0) 0.8306 (1.0) 0.0383 (1.0) 0.8292 (1.0) 0.0504 (1.0) 273;10 1,203.8883 (1.0) 944 1 +test_benchmark_query7 12.7920 (18.08) 13.8115 (13.09) 13.2379 (15.94) 0.2134 (5.57) 13.1735 (15.89) 0.2816 (5.59) 21;1 75.5406 (0.06) 73 1 +test_benchmark_query8 1.1148 (1.58) 1.4808 (1.40) 1.2611 (1.52) 0.0517 (1.35) 1.2631 (1.52) 0.0723 (1.44) 191;4 792.9300 (0.66) 602 1 +test_benchmark_query9 2.0159 (2.85) 2.5005 (2.37) 2.1874 (2.63) 0.0663 (1.73) 2.1839 (2.63) 0.0880 (1.75) 127;3 457.1621 (0.38) 423 1 +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- \ No newline at end of file