Skip to content

Commit caa18a9

Browse files
lluisemperlluisemper
authored
docs: add comprehensive library comparison document (#109)
Add BENCHMARK_COMPARISSON.md with detailed performance comparisons between bench-node, benchmark.js, mitata, and tinybench across Node.js versions 20, 22, and 24. The document includes: - Cross-library performance results for multiple test scenarios - String processing, CPU intensive, and JIT optimization benchmarks - Performance data across different Node.js runtime versions - Comparative analysis for algorithm-specific operations This comparison helps identify potential measurement errors and provides insights for specific algorithm performance evaluation. Fixes: #108
1 parent bf119c7 commit caa18a9

File tree

1 file changed

+107
-0
lines changed

1 file changed

+107
-0
lines changed

‎BENCHMARK_COMPARISSON.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Benchmark Library Comparison
2+
3+
This document provides a comprehensive comparison between `bench-node` and other popular Node.js benchmarking libraries based on real benchmark results across different Node.js versions. This comparison helps identify potential errors in our benchmarking approach and provides insights for specific algorithm comparisons.
4+
5+
## Overview
6+
7+
The following benchmarking libraries are compared in this document:
8+
9+
- **bench-node** - Our library with V8 optimization control
10+
- **benchmark.js** - The most popular JavaScript benchmarking library
11+
- **mitata** - Modern, fast benchmarking library
12+
- **tinybench** - Lightweight benchmarking tool
13+
14+
## Test Environment
15+
16+
All benchmarks were executed on:
17+
- **CPU**: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz
18+
- **Platform**: Linux x64
19+
- **Node.js versions**: 20.19.3, 22.17.0, 24.3.0
20+
21+
22+
23+
## Detailed Benchmark Results
24+
25+
### Simple Operations - JIT Optimizable
26+
27+
This test measures a simple operation that V8 can heavily optimize:
28+
29+
#### Node.js 24.3.0
30+
| Library | ops/sec | Margin | Relative Performance |
31+
|---------|---------|--------|---------------------|
32+
| benchmark.js | 141,211,777 | ±5.18% | **1.00x** (baseline) |
33+
| bench-node | 125,902,288 | N/A | 0.89x |
34+
| mitata | 37,131,930 | ±5.00% | 0.26x |
35+
| tinybench | 28,207,575 | ±0.10% | 0.20x |
36+
37+
#### Node.js 22.17.0
38+
| Library | ops/sec | Margin | Relative Performance |
39+
|---------|---------|--------|---------------------|
40+
| benchmark.js | 142,037,530 | ±3.88% | **1.00x** (baseline) |
41+
| bench-node | 141,504,337 | N/A | 1.00x |
42+
| tinybench | 17,040,169 | ±1.40% | 0.12x |
43+
| mitata | 12,382,061 | ±5.00% | 0.09x |
44+
45+
#### Node.js 20.19.3
46+
| Library | ops/sec | Margin | Relative Performance |
47+
|---------|---------|--------|---------------------|
48+
| **benchmark.js** | **843,084,082** | ±0.25% | **1.00x** (baseline) |
49+
| bench-node | 136,801,267 | N/A | 0.16x |
50+
| tinybench | 24,275,731 | ±0.11% | 0.03x |
51+
| mitata | 18,411,459 | ±5.00% | 0.02x |
52+
53+
54+
55+
### String Processing - Regex Test
56+
57+
This test performs regex matching on strings:
58+
59+
#### Cross-Version Comparison
60+
| Node.js Version | bench-node | benchmark.js | mitata | tinybench |
61+
|-----------------|------------|--------------|--------|-----------|
62+
| **24.3.0** | 4,352,775 | 4,299,609 (±0.58%) | 3,543,737 (±5.00%) | 3,654,837 (±0.40%) |
63+
| **22.17.0** | 4,349,482 | 4,382,198 (±0.49%) | 3,933,431 (±5.00%) | 3,526,797 (±0.83%) |
64+
| **20.19.3** | 4,419,500 | 4,320,038 (±0.40%) | 3,475,360 (±5.00%) | 3,698,242 (±1.85%) |
65+
66+
67+
68+
### CPU Intensive - Fibonacci Calculations
69+
70+
#### Fibonacci(10) - Light CPU Load
71+
72+
| Node.js Version | benchmark.js | bench-node | mitata | tinybench |
73+
|-----------------|--------------|------------|--------|-----------|
74+
| **24.3.0** | 77,510,123 (±2.38%) | 69,120,722 | 42,308,343 (±5.00%) | 26,206,646 (±0.09%) |
75+
| **22.17.0** | 56,097,219 (±1.68%) | 54,623,064 | 51,268,905 (±5.00%) | 15,451,469 (±0.09%) |
76+
| **20.19.3** | 73,590,107 (±0.23%) | 57,284,808 | 10,910,003 (±5.00%) | 22,108,774 (±0.11%) |
77+
78+
#### Fibonacci(30) - Medium CPU Load
79+
80+
| Node.js Version | benchmark.js | bench-node | mitata | tinybench |
81+
|-----------------|--------------|------------|--------|-----------|
82+
| **24.3.0** | 43,276,919 (±1.26%) | 40,936,562 | 30,824,240 (±5.00%) | 18,752,734 (±0.11%) |
83+
| **22.17.0** | 22,187,269 (±0.53%) | 21,316,196 | 14,024,655 (±5.00%) | 11,373,272 (±1.98%) |
84+
| **20.19.3** | 22,078,755 (±0.27%) | 21,783,805 | 21,708,455 (±5.00%) | 14,162,915 (±0.11%) |
85+
86+
#### Fibonacci(40) - Heavy CPU Load
87+
88+
| Node.js Version | benchmark.js | bench-node | mitata | tinybench |
89+
|-----------------|--------------|------------|--------|-----------|
90+
| **24.3.0** | 35,115,900 (±1.30%) | 33,956,245 | 26,752,990 (±5.00%) | 18,719,206 (±0.10%) |
91+
| **22.17.0** | 16,511,904 (±0.91%) | 16,112,632 | 15,599,164 (±5.00%) | 9,496,167 (±4.91%) |
92+
| **20.19.3** | 16,726,334 (±0.31%) | 16,504,271 | 16,264,129 (±5.00%) | 11,755,080 (±3.32%) |
93+
94+
### Fibonacci Recursive(10) - Algorithm Comparison
95+
96+
| Node.js Version | bench-node | mitata | benchmark.js | tinybench |
97+
|-----------------|------------|--------|--------------|-----------|
98+
| **24.3.0** | **999,165** | 992,635 (±5.00%) | 986,251 (±0.31%) | 977,228 (±0.09%) |
99+
| **22.17.0** | **978,652** | 972,507 (±5.00%) | 969,517 (±0.30%) | 938,449 (±0.07%) |
100+
| **20.19.3** | **989,935** | 939,466 (±5.00%) | 973,359 (±0.19%) | 965,775 (±0.07%) |
101+
102+
103+
---
104+
105+
*Last updated: August 2025 | Based on bench-node v2.x, benchmark.js v2.1.4, mitata v1.x, tinybench v2.x*
106+
*Benchmark workflow: [benchmark-comparison.yml](https://github.com/RafaelGSS/bench-node/actions/workflows/benchmark-comparison.yml)*
107+
*Test results from Node.js versions 20.19.3, 22.17.0, and 24.3.0*

0 commit comments

Comments
 (0)