From 38db31726e0ae676053e0c084e7b2a20d2e3dddb Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 14:30:01 -0400 Subject: [PATCH 1/8] release: correct the Lua.new speedup figures in the 1.0.2 changelog Full-mode benchee measurement of benchmarks/vm_new.exs across v0.4.0, v1.0.0 and the 1.0.2 release commit puts the default Lua.new/1 at 36.7us -> 0.6us median (~60x, allocation ~92KB -> <1KB) and the custom-sandbox path at 36.0us -> 6.5us (~5.5x), measured under mix run. See bench_results/versions-2026-07-28.md. --- CHANGELOG.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ea34f27..ba7df75 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,11 +52,14 @@ is in the [`1.0.0-rc.0`](#100-rc0---2026-05-26) entry below. ## [1.0.2] - 2026-07-28 ### Changed -- `Lua.new/1` is ~100x faster (roughly 40µs down to 0.4µs) for the default and - fully-custom-sandbox configurations. Installing the standard library is pure - and deterministic, so the boot-time VM template is now built once per node - and memoized in `:persistent_term`; every later `Lua.new/1` starts from the - shared template copy-on-write. In `:interactive` mode (dev, IEx, tests) the +- `Lua.new/1` is ~60x faster for the default configuration — 36.7µs down to + 0.6µs median, with per-call allocation down from ~92KB to under 1KB — and + ~5.5x faster when a custom sandbox is passed (36.0µs down to 6.5µs), as + measured by `benchmarks/vm_new.exs` under `mix run` + ([full figures](bench_results/versions-2026-07-28.md)). Installing the + standard library is pure and deterministic, so the boot-time VM template is + now built once per node and memoized in `:persistent_term`; every later + `Lua.new/1` starts from the shared template copy-on-write. In `:interactive` mode (dev, IEx, tests) the cache self-invalidates when the modules that built it are recompiled; hosts that hot-load new code in `:embedded` mode (releases) can force a rebuild with `Lua.VM.Bootstrap.reset/0` (#398). From 42f567ff3fc934165c4e7c56819a216ed979d987 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 14:30:01 -0400 Subject: [PATCH 2/8] bench: widen the suite and fix chunk loading on pre-1.0 refs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New workloads closing representativeness gaps: patterns (string.find/ match/gsub — the pattern engine had zero coverage), metamethods (colon-syntax self-dispatch, multi-level __index chains, arithmetic/ relational metamethods), pcall_varargs (protected calls, varargs, multiple returns), and vm_new (VM instantiation incl. cold vs steady-state template cost). Existing workloads now thread the state returned by Lua.load_chunk!: on v0.4.0 the chunk ref is only valid in the state that loaded it, so discarding it broke the chunk jobs there; on 1.x the change is a no-op. Also corrects a stale oop.exs header comment (colon method syntax works). --- benchmarks/closures.exs | 6 +- benchmarks/fibonacci.exs | 6 +- benchmarks/metamethods.exs | 193 +++++++++++++++++++++++++++++++++++ benchmarks/oop.exs | 25 +++-- benchmarks/patterns.exs | 156 ++++++++++++++++++++++++++++ benchmarks/pcall_varargs.exs | 161 +++++++++++++++++++++++++++++ benchmarks/string_format.exs | 10 +- benchmarks/string_ops.exs | 8 +- benchmarks/table_ops.exs | 51 ++++----- benchmarks/vm_new.exs | 94 +++++++++++++++++ 10 files changed, 667 insertions(+), 43 deletions(-) create mode 100644 benchmarks/metamethods.exs create mode 100644 benchmarks/patterns.exs create mode 100644 benchmarks/pcall_varargs.exs create mode 100644 benchmarks/vm_new.exs diff --git a/benchmarks/closures.exs b/benchmarks/closures.exs index 8910e60..3e6c350 100644 --- a/benchmarks/closures.exs +++ b/benchmarks/closures.exs @@ -53,9 +53,13 @@ end call_closures = "return run_closures(100)" # --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. lua = Lua.new() {_, lua} = Lua.eval!(lua, closure_def) -{closure_chunk, _} = Lua.load_chunk!(lua, call_closures) +{closure_chunk, lua} = Lua.load_chunk!(lua, call_closures) # --- Luerl --- luerl_state = :luerl.init() diff --git a/benchmarks/fibonacci.exs b/benchmarks/fibonacci.exs index 502f889..88ee743 100644 --- a/benchmarks/fibonacci.exs +++ b/benchmarks/fibonacci.exs @@ -27,9 +27,13 @@ end call_fib = "return fib(30)" # --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. lua = Lua.new() {_, lua} = Lua.eval!(lua, fib_def) -{fib_chunk, _} = Lua.load_chunk!(lua, call_fib) +{fib_chunk, lua} = Lua.load_chunk!(lua, call_fib) # --- Luerl --- luerl_state = :luerl.init() diff --git a/benchmarks/metamethods.exs b/benchmarks/metamethods.exs new file mode 100644 index 0000000..57f5480 --- /dev/null +++ b/benchmarks/metamethods.exs @@ -0,0 +1,193 @@ +# Run with: mix run benchmarks/metamethods.exs +# +# Benchmarks metatable-driven dispatch. oop.exs already covers the shallow +# case — one `setmetatable` + a single-level `__index` table lookup, with +# methods invoked as plain field calls (`Animal.speak(a)`). This script covers +# the three things that shape real Lua OOP code and that the shallow case +# leaves untouched: +# +# - methods: method calls written with the `:` sugar (`v:len2()`), so the +# receiver is threaded as an implicit `self` argument, plus a +# chained call on a freshly constructed receiver +# (`v:scaled(2):len2()`). This is the self-call path; the plain +# `T.f(obj)` form in oop.exs does not reach it. +# - inherit: a three-level prototype chain (Rect -> Polygon -> Shape) built +# the idiomatic way, with `setmetatable` on the class tables +# themselves. Each call resolves at a different depth: one hit on +# the leaf, one two hops up, one three hops up — so the cost of +# walking a chain is separated from the cost of a single hit. +# - arith: arithmetic and relational metamethods (`__add`, `__sub`, +# `__lt`, `__eq`) plus `__tostring`. These route through the +# metamethod fallback in the arithmetic/comparison opcodes rather +# than through table indexing, which is a different VM path from +# `__index` entirely. +# +# Each workload runs n=200 iterations per invocation. +# +# Compares: +# - This Lua implementation (eval with string, eval with pre-compiled chunk) +# - Luerl (Erlang-based Lua 5.3 implementation) +# - C Lua 5.4 via luaport (port-based; results include IPC overhead) +# +# NOTE: luaport requires C Lua 5.4 development headers and a small in-tree +# patch (its 1.6.3 release defaults to LuaJIT and uses LUA_GLOBALSINDEX which +# was removed in Lua 5.2). On macOS: +# brew install lua@5.4 +# ./benchmarks/setup_luaport.sh # idempotent; patches + builds +# MIX_ENV=benchmark mix run benchmarks/metamethods.exs +# If luaport fails to start, the benchmark prints a notice and skips it. +# +# Run modes (see benchmarks/helpers.exs): +# default — quick mode (~4 s per Benchee.run) +# LUA_BENCH_MODE=full — long windows + memory_time, for publishable numbers + +Code.require_file("helpers.exs", __DIR__) + +Application.ensure_all_started(:luerl) + +meta_def = """ +-- Self-call dispatch via the `:` sugar, including a chained call. +Vec = {} +Vec.__index = Vec + +function Vec.new(x, y) + return setmetatable({ x = x, y = y }, Vec) +end + +function Vec:len2() + return self.x * self.x + self.y * self.y +end + +function Vec:scaled(k) + return Vec.new(self.x * k, self.y * k) +end + +function run_methods(n) + local v = Vec.new(3, 4) + local acc = 0 + for i = 1, n do + acc = acc + v:len2() + acc = acc + v:scaled(2):len2() + end + return acc +end + +-- Three-level prototype chain; the three calls below resolve at depth 1, 2 +-- and 3 respectively, so a chain walk is measured alongside a direct hit. +Shape = {} +Shape.__index = Shape + +function Shape:kind() return "shape" end +function Shape:area() return 0 end + +Polygon = setmetatable({}, { __index = Shape }) +Polygon.__index = Polygon + +function Polygon:sides() return 0 end + +Rect = setmetatable({}, { __index = Polygon }) +Rect.__index = Rect + +function Rect.new(w, h) + return setmetatable({ w = w, h = h }, Rect) +end + +function Rect:sides() return 4 end + +function run_inherit(n) + local r = Rect.new(3, 4) + local acc = 0 + for i = 1, n do + acc = acc + r:sides() + acc = acc + r:area() + acc = acc + #r:kind() + end + return acc +end + +-- Arithmetic / relational / tostring metamethods. +Money = {} +Money.__index = Money +Money.__add = function(a, b) return Money.new(a.cents + b.cents) end +Money.__sub = function(a, b) return Money.new(a.cents - b.cents) end +Money.__lt = function(a, b) return a.cents < b.cents end +Money.__eq = function(a, b) return a.cents == b.cents end +Money.__tostring = function(m) return "$" .. tostring(m.cents) end + +function Money.new(cents) + return setmetatable({ cents = cents }, Money) +end + +function run_arith(n) + local acc = Money.new(0) + local flags = 0 + local last = "" + for i = 1, n do + acc = acc + Money.new(i) + acc = acc - Money.new(1) + if Money.new(i) < Money.new(i + 1) then flags = flags + 1 end + if Money.new(i) == Money.new(i) then flags = flags + 1 end + last = tostring(acc) + end + return acc.cents, flags, #last +end +""" + +call_methods = "return run_methods(200)" +call_inherit = "return run_inherit(200)" +call_arith = "return run_arith(200)" + +# --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. +lua = Lua.new() +{_, lua} = Lua.eval!(lua, meta_def) +{methods_chunk, lua} = Lua.load_chunk!(lua, call_methods) +{inherit_chunk, lua} = Lua.load_chunk!(lua, call_inherit) +{arith_chunk, lua} = Lua.load_chunk!(lua, call_arith) + +# --- Luerl --- +luerl_state = :luerl.init() +{:ok, _, luerl_state} = :luerl.do(meta_def, luerl_state) + +# --- C Lua via luaport (optional) --- +{c_lua, c_lua_cleanup} = + case Application.ensure_all_started(:luaport) do + {:ok, _} -> + scripts_dir = Path.join(__DIR__, "scripts") + {:ok, port_pid, _} = :luaport.spawn(:meta_bench, to_charlist(scripts_dir)) + :luaport.load(port_pid, meta_def) + + { + fn func -> %{"C Lua (luaport)" => fn -> :luaport.call(port_pid, func, [200]) end} end, + fn -> :luaport.despawn(:meta_bench) end + } + + {:error, reason} -> + IO.puts("luaport not available (#{inspect(reason)}) — skipping C Lua benchmarks") + {fn _func -> %{} end, fn -> :ok end} + end + +bench = fn name, call_str, chunk, c_lua_func -> + Bench.banner(name) + + Benchee.run( + Map.merge( + %{ + "lua (eval)" => fn -> Lua.eval!(lua, call_str) end, + "lua (chunk)" => fn -> Lua.eval!(lua, chunk) end, + "luerl" => fn -> :luerl.do(call_str, luerl_state) end + }, + c_lua.(c_lua_func) + ), + Bench.opts() + ) +end + +bench.("metamethods: self-call method dispatch (n=200)", call_methods, methods_chunk, :run_methods) +bench.("metamethods: 3-level __index chain (n=200)", call_inherit, inherit_chunk, :run_inherit) +bench.("metamethods: arithmetic/relational metamethods (n=200)", call_arith, arith_chunk, :run_arith) + +c_lua_cleanup.() diff --git a/benchmarks/oop.exs b/benchmarks/oop.exs index e6790e5..2cd4be3 100644 --- a/benchmarks/oop.exs +++ b/benchmarks/oop.exs @@ -1,14 +1,21 @@ # Run with: mix run benchmarks/oop.exs # -# Benchmarks object-oriented patterns using Lua tables and metatables. -# Uses assignment-style method definitions (e.g. Animal.speak = function(self) ... end) -# which are compatible with this Lua implementation's current feature set. -# Creates 50 Animal instances per iteration and calls a method on each. +# Benchmarks the shallow, construction-dominated end of Lua OOP: 50 instances +# per iteration, each built with two field writes and a `setmetatable`, then +# sent one method whose body concatenates two of its fields. +# +# Methods are defined and invoked in assignment/field-call style +# (`Animal.speak = function(self) ... end`, called as `Animal.speak(a)`) rather +# than with the `:` sugar. That is not a language limitation — `:` method +# definitions and self-calls work — it holds dispatch to its simplest form so +# this workload stays dominated by construction cost. The dispatch-heavy +# counterpart (`:` self-calls, multi-level `__index` chains, arithmetic +# metamethods) is benchmarks/metamethods.exs. # # Patterns tested: # - Table creation and field assignment -# - setmetatable / __index prototype chain lookup -# - Closure creation per object (factory pattern variant) +# - setmetatable, plus __index lookup one level up the prototype chain +# - String concatenation and tostring inside a method body # # Compares: # - This Lua implementation (eval with string, eval with pre-compiled chunk) @@ -60,9 +67,13 @@ end call_oop = "return run_oop(50)" # --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. lua = Lua.new() {_, lua} = Lua.eval!(lua, oop_def) -{oop_chunk, _} = Lua.load_chunk!(lua, call_oop) +{oop_chunk, lua} = Lua.load_chunk!(lua, call_oop) # --- Luerl --- luerl_state = :luerl.init() diff --git a/benchmarks/patterns.exs b/benchmarks/patterns.exs new file mode 100644 index 0000000..05ffd11 --- /dev/null +++ b/benchmarks/patterns.exs @@ -0,0 +1,156 @@ +# Run with: mix run benchmarks/patterns.exs +# +# Benchmarks Lua's string-pattern engine — the part of the string library that +# compiles and matches Lua patterns, as opposed to the byte-copying and +# formatting paths covered by string_ops.exs / string_format.exs. +# +# - scan: repeated `string.find` + `string.match` over one log line. Drives +# character classes (%a, %d, %w), quantifiers, escaped magic +# characters (%[ %]) and single-capture extraction. This is the +# shape of nearly every "pull fields out of a line" script. +# - split: tokenises a comma-separated list with `string.find(s, "[^,]+", pos)` +# advanced by an explicit init offset. Exercises the anchor-free +# restart path — the matcher is re-entered once per token with a +# fresh start position. +# - gsub: template substitution in three passes: `%${(%w+)}` with a table +# replacement, `%s+` whitespace squeezing with a string +# replacement, and `(%a+)` with a *function* replacement. Covers +# all three gsub replacement kinds plus capture-driven lookup. +# +# `split` deliberately uses `string.find` with an init offset rather than +# `string.gmatch`: `gmatch` raises `badarg` in Luerl 1.5.x, so a gmatch-based +# tokeniser could not be measured against the Luerl reference on the same Lua +# source. Keeping the source identical across VMs is the fairness contract of +# this suite, so the iterator-free idiom is used instead. +# +# Each workload runs n=200 pattern-heavy iterations per invocation so the +# per-match cost is visible above harness overhead. +# +# Compares: +# - This Lua implementation (eval with string, eval with pre-compiled chunk) +# - Luerl (Erlang-based Lua 5.3 implementation) +# - C Lua 5.4 via luaport (port-based; results include IPC overhead) +# +# NOTE: luaport requires C Lua 5.4 development headers and a small in-tree +# patch (its 1.6.3 release defaults to LuaJIT and uses LUA_GLOBALSINDEX which +# was removed in Lua 5.2). On macOS: +# brew install lua@5.4 +# ./benchmarks/setup_luaport.sh # idempotent; patches + builds +# MIX_ENV=benchmark mix run benchmarks/patterns.exs +# If luaport fails to start, the benchmark prints a notice and skips it. +# +# Run modes (see benchmarks/helpers.exs): +# default — quick mode (~4 s per Benchee.run) +# LUA_BENCH_MODE=full — long windows + memory_time, for publishable numbers + +Code.require_file("helpers.exs", __DIR__) + +Application.ensure_all_started(:luerl) + +pattern_def = """ +local LOG = "2024-05-01 12:34:56 [warn] request_id=a1b2c3 latency=42ms status=503 path=/api/v1/items" + +-- Field extraction: an escaped-magic-character find plus three captures. +function run_scan(n) + local hits = 0 + for i = 1, n do + local s, e = string.find(LOG, "%[%a+%]") + if s then hits = hits + (e - s) end + local status = string.match(LOG, "status=(%d+)") + local id = string.match(LOG, "request_id=(%w+)") + local lat = string.match(LOG, "latency=(%d+)ms") + if status and id and lat then hits = hits + #status + #id + #lat end + end + return hits +end + +local CSV = "alpha,beta,gamma,delta,epsilon,zeta,eta,theta,iota,kappa" + +-- Tokenise by re-entering the matcher at an advancing init offset. +function run_split(n) + local total = 0 + for i = 1, n do + local pos = 1 + while true do + local s, e = string.find(CSV, "[^,]+", pos) + if not s then break end + total = total + (e - s + 1) + pos = e + 1 + end + end + return total +end + +local TEMPLATE = "Hello ${name}, you have ${count} new ${kind} since ${when}. Visit ${url} for details." +local VALUES = { name = "Ada", count = "7", kind = "messages", when = "Tuesday", url = "/inbox" } + +-- All three gsub replacement kinds: table, string, function. +function run_gsub(n) + local last = "" + for i = 1, n do + local filled = string.gsub(TEMPLATE, "%${(%w+)}", VALUES) + local squeezed = string.gsub(filled, "%s+", " ") + last = string.gsub(squeezed, "(%a+)", function(w) return w end) + end + return #last +end +""" + +call_scan = "return run_scan(200)" +call_split = "return run_split(200)" +call_gsub = "return run_gsub(200)" + +# --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. +lua = Lua.new() +{_, lua} = Lua.eval!(lua, pattern_def) +{scan_chunk, lua} = Lua.load_chunk!(lua, call_scan) +{split_chunk, lua} = Lua.load_chunk!(lua, call_split) +{gsub_chunk, lua} = Lua.load_chunk!(lua, call_gsub) + +# --- Luerl --- +luerl_state = :luerl.init() +{:ok, _, luerl_state} = :luerl.do(pattern_def, luerl_state) + +# --- C Lua via luaport (optional) --- +{c_lua, c_lua_cleanup} = + case Application.ensure_all_started(:luaport) do + {:ok, _} -> + scripts_dir = Path.join(__DIR__, "scripts") + {:ok, port_pid, _} = :luaport.spawn(:pattern_bench, to_charlist(scripts_dir)) + :luaport.load(port_pid, pattern_def) + + { + fn func -> %{"C Lua (luaport)" => fn -> :luaport.call(port_pid, func, [200]) end} end, + fn -> :luaport.despawn(:pattern_bench) end + } + + {:error, reason} -> + IO.puts("luaport not available (#{inspect(reason)}) — skipping C Lua benchmarks") + {fn _func -> %{} end, fn -> :ok end} + end + +bench = fn name, call_str, chunk, c_lua_func -> + Bench.banner(name) + + Benchee.run( + Map.merge( + %{ + "lua (eval)" => fn -> Lua.eval!(lua, call_str) end, + "lua (chunk)" => fn -> Lua.eval!(lua, chunk) end, + "luerl" => fn -> :luerl.do(call_str, luerl_state) end + }, + c_lua.(c_lua_func) + ), + Bench.opts() + ) +end + +bench.("patterns: find/match field extraction (n=200)", call_scan, scan_chunk, :run_scan) +bench.("patterns: find-based tokenizer (n=200)", call_split, split_chunk, :run_split) +bench.("patterns: gsub template substitution (n=200)", call_gsub, gsub_chunk, :run_gsub) + +c_lua_cleanup.() diff --git a/benchmarks/pcall_varargs.exs b/benchmarks/pcall_varargs.exs new file mode 100644 index 0000000..75ba319 --- /dev/null +++ b/benchmarks/pcall_varargs.exs @@ -0,0 +1,161 @@ +# Run with: mix run benchmarks/pcall_varargs.exs +# +# Benchmarks the call protocol: protected calls and variadic/multiple-return +# argument handling. Both are pervasive in embedded Lua — host integrations +# routinely wrap every script entry point in `pcall`, and `...`/multiple +# returns are how Lua code passes argument lists around — and neither appears +# in the comparative workloads otherwise. +# +# - pcall_ok: n protected calls that all succeed. Isolates the cost of +# entering and leaving a protected frame from the cost of +# actually raising, which is the common case in production. +# - pcall_raise: n protected calls that all raise a string error and are +# caught. Drives error-value construction, stack unwinding +# and the return of `false, err` to the caller. +# - varargs: variadic collection (`select("#", ...)`), positional +# variadic access (`select(i, ...)`), variadic forwarding +# (`f(...)` in tail position), `table.pack`/`table.unpack` +# round-tripping, and multiple-return destructuring +# (`local a, b, c = triple(i)`). +# +# Each workload runs n=500 iterations per invocation. +# +# Compares: +# - This Lua implementation (eval with string, eval with pre-compiled chunk) +# - Luerl (Erlang-based Lua 5.3 implementation) +# - C Lua 5.4 via luaport (port-based; results include IPC overhead) +# +# NOTE: luaport requires C Lua 5.4 development headers and a small in-tree +# patch (its 1.6.3 release defaults to LuaJIT and uses LUA_GLOBALSINDEX which +# was removed in Lua 5.2). On macOS: +# brew install lua@5.4 +# ./benchmarks/setup_luaport.sh # idempotent; patches + builds +# MIX_ENV=benchmark mix run benchmarks/pcall_varargs.exs +# If luaport fails to start, the benchmark prints a notice and skips it. +# +# Run modes (see benchmarks/helpers.exs): +# default — quick mode (~4 s per Benchee.run) +# LUA_BENCH_MODE=full — long windows + memory_time, for publishable numbers + +Code.require_file("helpers.exs", __DIR__) + +Application.ensure_all_started(:luerl) + +call_def = """ +-- Raises for negative input, returns normally otherwise, so the same callee +-- drives both the success and the error path below. +function classify(v) + if v < 0 then + error("negative") + end + return v * 2 +end + +function run_pcall_ok(n) + local acc = 0 + for i = 1, n do + local ok, v = pcall(classify, i) + if ok then acc = acc + v end + end + return acc +end + +function run_pcall_raise(n) + local caught = 0 + for i = 1, n do + local ok, err = pcall(classify, -i) + if not ok and type(err) == "string" then caught = caught + 1 end + end + return caught +end + +-- Variadic collection and positional access. +function tally(...) + local count = select("#", ...) + local acc = 0 + for i = 1, count do + acc = acc + select(i, ...) + end + return acc, count +end + +-- Variadic forwarding in tail position. +function forward(...) + return tally(...) +end + +function triple(i) + return i, i + 1, i + 2 +end + +function run_varargs(n) + local acc = 0 + for i = 1, n do + local a, b, c = triple(i) + local sum, count = forward(a, b, c, i, i * 2) + acc = acc + sum + local packed = table.pack(a, b, c) + acc = acc + tally(table.unpack(packed, 1, packed.n)) + end + return acc +end +""" + +call_pcall_ok = "return run_pcall_ok(500)" +call_pcall_raise = "return run_pcall_raise(500)" +call_varargs = "return run_varargs(500)" + +# --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. +lua = Lua.new() +{_, lua} = Lua.eval!(lua, call_def) +{pcall_ok_chunk, lua} = Lua.load_chunk!(lua, call_pcall_ok) +{pcall_raise_chunk, lua} = Lua.load_chunk!(lua, call_pcall_raise) +{varargs_chunk, lua} = Lua.load_chunk!(lua, call_varargs) + +# --- Luerl --- +luerl_state = :luerl.init() +{:ok, _, luerl_state} = :luerl.do(call_def, luerl_state) + +# --- C Lua via luaport (optional) --- +{c_lua, c_lua_cleanup} = + case Application.ensure_all_started(:luaport) do + {:ok, _} -> + scripts_dir = Path.join(__DIR__, "scripts") + {:ok, port_pid, _} = :luaport.spawn(:call_bench, to_charlist(scripts_dir)) + :luaport.load(port_pid, call_def) + + { + fn func -> %{"C Lua (luaport)" => fn -> :luaport.call(port_pid, func, [500]) end} end, + fn -> :luaport.despawn(:call_bench) end + } + + {:error, reason} -> + IO.puts("luaport not available (#{inspect(reason)}) — skipping C Lua benchmarks") + {fn _func -> %{} end, fn -> :ok end} + end + +bench = fn name, call_str, chunk, c_lua_func -> + Bench.banner(name) + + Benchee.run( + Map.merge( + %{ + "lua (eval)" => fn -> Lua.eval!(lua, call_str) end, + "lua (chunk)" => fn -> Lua.eval!(lua, chunk) end, + "luerl" => fn -> :luerl.do(call_str, luerl_state) end + }, + c_lua.(c_lua_func) + ), + Bench.opts() + ) +end + +bench.("call protocol: pcall, success path (n=500)", call_pcall_ok, pcall_ok_chunk, :run_pcall_ok) +bench.("call protocol: pcall, raise + catch (n=500)", call_pcall_raise, pcall_raise_chunk, :run_pcall_raise) +bench.("call protocol: varargs + multiple returns (n=500)", call_varargs, varargs_chunk, :run_varargs) + +c_lua_cleanup.() diff --git a/benchmarks/string_format.exs b/benchmarks/string_format.exs index 0dd1e16..70cde33 100644 --- a/benchmarks/string_format.exs +++ b/benchmarks/string_format.exs @@ -78,11 +78,15 @@ call_width = "return run_width_format(1000)" call_many = "return run_many_specs(1000)" # --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. lua = Lua.new() {_, lua} = Lua.eval!(lua, string_def) -{long_chunk, _} = Lua.load_chunk!(lua, call_long) -{width_chunk, _} = Lua.load_chunk!(lua, call_width) -{many_chunk, _} = Lua.load_chunk!(lua, call_many) +{long_chunk, lua} = Lua.load_chunk!(lua, call_long) +{width_chunk, lua} = Lua.load_chunk!(lua, call_width) +{many_chunk, lua} = Lua.load_chunk!(lua, call_many) # --- Luerl --- luerl_state = :luerl.init() diff --git a/benchmarks/string_ops.exs b/benchmarks/string_ops.exs index 50e6688..2c6431d 100644 --- a/benchmarks/string_ops.exs +++ b/benchmarks/string_ops.exs @@ -43,10 +43,14 @@ call_concat = "return run_concat(100)" call_format = "return run_format(100)" # --- This Lua implementation --- +# The state returned by `load_chunk!/2` is threaded through each call rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. Threading it +# is correct on every release and costs nothing. lua = Lua.new() {_, lua} = Lua.eval!(lua, string_def) -{concat_chunk, _} = Lua.load_chunk!(lua, call_concat) -{format_chunk, _} = Lua.load_chunk!(lua, call_format) +{concat_chunk, lua} = Lua.load_chunk!(lua, call_concat) +{format_chunk, lua} = Lua.load_chunk!(lua, call_format) # --- Luerl --- luerl_state = :luerl.init() diff --git a/benchmarks/table_ops.exs b/benchmarks/table_ops.exs index 684bf37..94deb47 100644 --- a/benchmarks/table_ops.exs +++ b/benchmarks/table_ops.exs @@ -97,37 +97,30 @@ lua = Lua.new() # Pre-compile chunks per (operation, n) pair so the chunk path doesn't # pay the compile cost during measurement. Inputs ship through Benchee's # `inputs:` mechanism so all sizes share warmup/measurement state. +# +# The state returned by `load_chunk!/2` is threaded through every load rather +# than discarded: a loaded chunk may be a reference *into* the state it was +# loaded against, so dropping that state can invalidate the chunk. All five +# chunk maps are therefore built against one accumulating state, and that final +# state is the one the benchmarks below evaluate against. sizes = Bench.table_inputs() -build_chunks = - Map.new(sizes, fn {label, n} -> - {chunk, _} = Lua.load_chunk!(lua, "return run_table_build(#{n})") - {label, {chunk, "return run_table_build(#{n})", n}} - end) - -sort_chunks = - Map.new(sizes, fn {label, n} -> - {chunk, _} = Lua.load_chunk!(lua, "return run_table_sort(#{n})") - {label, {chunk, "return run_table_sort(#{n})", n}} - end) - -sum_chunks = - Map.new(sizes, fn {label, n} -> - {chunk, _} = Lua.load_chunk!(lua, "return run_table_sum(#{n})") - {label, {chunk, "return run_table_sum(#{n})", n}} - end) - -map_reduce_chunks = - Map.new(sizes, fn {label, n} -> - {chunk, _} = Lua.load_chunk!(lua, "return run_table_map_reduce(#{n})") - {label, {chunk, "return run_table_map_reduce(#{n})", n}} - end) - -pairs_hash_chunks = - Map.new(sizes, fn {label, n} -> - {chunk, _} = Lua.load_chunk!(lua, "return run_table_pairs_hash(#{n})") - {label, {chunk, "return run_table_pairs_hash(#{n})", n}} - end) +load_chunks = fn lua, func -> + {loaded, lua} = + Enum.map_reduce(sizes, lua, fn {label, n}, acc -> + call = "return #{func}(#{n})" + {chunk, acc} = Lua.load_chunk!(acc, call) + {{label, {chunk, call, n}}, acc} + end) + + {Map.new(loaded), lua} +end + +{build_chunks, lua} = load_chunks.(lua, "run_table_build") +{sort_chunks, lua} = load_chunks.(lua, "run_table_sort") +{sum_chunks, lua} = load_chunks.(lua, "run_table_sum") +{map_reduce_chunks, lua} = load_chunks.(lua, "run_table_map_reduce") +{pairs_hash_chunks, lua} = load_chunks.(lua, "run_table_pairs_hash") # --- Luerl --- luerl_state = :luerl.init() diff --git a/benchmarks/vm_new.exs b/benchmarks/vm_new.exs new file mode 100644 index 0000000..10d0ad1 --- /dev/null +++ b/benchmarks/vm_new.exs @@ -0,0 +1,94 @@ +# Run with: mix run benchmarks/vm_new.exs +# +# Benchmarks VM instantiation — `Lua.new/1` — the cost an embedding host pays +# before a single line of Lua runs. Hosts that build a fresh sandbox per +# request (the recommended isolation model) pay this on every request, so it +# sits directly in the request path and is worth measuring separately from +# script execution. +# +# Three instantiation shapes are measured, chosen because all three are valid +# on every release of this library (`:sandboxed` and `:exclude` are the only +# `new/1` options that exist across the whole history; the limit options +# `:max_call_depth` / `:max_string_bytes` / `:max_instructions` / `:debug` are +# newer and would raise on older releases): +# +# - new — `Lua.new()`. The default deny-list sandbox. This is +# what >90% of embedders call. +# - new, no sandbox — `Lua.new(sandboxed: [])`. Standard library +# installed, zero sandbox passes. This is the closest +# like-for-like analogue of a bare `:luerl.init()`, +# which also performs no sandboxing — compare the +# luerl row against *this* row, not against `new`. +# - new, custom exclude — `Lua.new(exclude: [[:require]])`. The default +# deny-list minus one entry. Represents "I want the +# sandbox but need one thing back", and on releases +# that memoize instantiation it is the shape that +# still pays a per-call sandbox pass. +# +# --------------------------------------------------------------------------- +# Cold vs steady state +# --------------------------------------------------------------------------- +# Newer releases memoize the boot-time VM template in `:persistent_term`, +# written once per node. That makes the *first* `Lua.new()` on a node more +# expensive than every subsequent one, so a benchmark could mislead in either +# direction: measuring only the first call would report a cost no real +# workload repeats, while reporting only the steady state would hide a +# one-time cost that does exist. +# +# Both are therefore reported. The script prints an explicitly-labelled cold +# and second-call timing for `Lua.new()` before Benchee starts — taken as the +# very first thing that touches the library, so nothing has warmed the cache — +# and then Benchee measures steady state, which is what a host serving its +# second and subsequent request sees. On releases with no memoization the two +# figures converge, which is itself the interesting signal. +# +# The cold figure is an upper bound: under `mix run` it also absorbs first-time +# code loading of the stdlib modules, which a release has already done at boot. +# +# Note also that `mix run` puts the VM in `:interactive` code-loading mode. +# Releases run `:embedded`, where a memoizing implementation can skip the +# module-reload staleness check a cache hit otherwise performs — so the steady +# state measured here is, if anything, pessimistic relative to production. +# +# Compares: +# - This Lua implementation (three instantiation shapes) +# - Luerl (`:luerl.init/0`, the Erlang-based Lua 5.3 implementation) +# +# There is no C Lua row: `:luaport` instantiation means spawning an OS process +# and handshaking over a port, which measures process spawn and IPC rather than +# VM construction. The two numbers would not mean the same thing. +# +# Run modes (see benchmarks/helpers.exs): +# default — quick mode (~4 s per Benchee.run) +# LUA_BENCH_MODE=full — long windows + memory_time, for publishable numbers + +Code.require_file("helpers.exs", __DIR__) + +Application.ensure_all_started(:luerl) + +Bench.banner("VM instantiation: Lua.new/1 vs :luerl.init/0") + +# Taken before anything else touches the library, so this really is the cold +# path — on a memoizing release it is the call that populates the template. +{cold_us, _} = :timer.tc(fn -> Lua.new() end) +{second_us, _} = :timer.tc(fn -> Lua.new() end) + +IO.puts(""" +Lua.new() one-time vs repeat cost (single samples, informational): + first call on this node : #{:erlang.float_to_binary(cold_us / 1, decimals: 1)} us + second call : #{:erlang.float_to_binary(second_us / 1, decimals: 1)} us + +The first figure includes any one-time template build and first-time module +loading. Benchee's steady-state numbers below are the per-request cost after +that point. +""") + +Benchee.run( + %{ + "lua (new)" => fn -> Lua.new() end, + "lua (new, no sandbox)" => fn -> Lua.new(sandboxed: []) end, + "lua (new, custom exclude)" => fn -> Lua.new(exclude: [[:require]]) end, + "luerl (init)" => fn -> :luerl.init() end + }, + Bench.opts() +) From 4f6f779f1bc5bb720cfd8af6a377d164cb29390e Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 14:30:02 -0400 Subject: [PATCH 3/8] bench: record cross-version results for v0.4.0, v1.0.0 and v1.0.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Full-mode serial runs of all 11 workloads on the same machine in the same sitting, with Luerl 1.5.1 as a same-run control in every table (control spread across the three runs: <1%). One directory per released version — the convention going forward is that every release gets a directory here. Report: versions-2026-07-28.md. Supersedes the 1.0.0-era numbers in benchmarks/BASELINE.md. Refs #267 --- bench_results/README.md | 113 +++ bench_results/v0.4.0/closures.txt | 58 ++ bench_results/v0.4.0/cpu.txt | 1 + bench_results/v0.4.0/encode_decode.txt | 143 +++ bench_results/v0.4.0/environment.md | 55 + bench_results/v0.4.0/fibonacci.txt | 58 ++ bench_results/v0.4.0/metamethods.txt | 151 +++ bench_results/v0.4.0/oop.txt | 58 ++ bench_results/v0.4.0/patterns.txt | 154 +++ bench_results/v0.4.0/pcall_varargs.txt | 151 +++ bench_results/v0.4.0/string_format.txt | 154 +++ bench_results/v0.4.0/string_ops.txt | 106 ++ bench_results/v0.4.0/summary.json | 1257 +++++++++++++++++++++++ bench_results/v0.4.0/table_ops.txt | 476 +++++++++ bench_results/v0.4.0/timestamp.txt | 1 + bench_results/v0.4.0/versions.txt | 3 + bench_results/v0.4.0/vm_new.txt | 72 ++ bench_results/v1.0.0/closures.txt | 54 + bench_results/v1.0.0/cpu.txt | 1 + bench_results/v1.0.0/encode_decode.txt | 136 +++ bench_results/v1.0.0/environment.md | 36 + bench_results/v1.0.0/fibonacci.txt | 51 + bench_results/v1.0.0/metamethods.txt | 144 +++ bench_results/v1.0.0/oop.txt | 51 + bench_results/v1.0.0/patterns.txt | 147 +++ bench_results/v1.0.0/pcall_varargs.txt | 144 +++ bench_results/v1.0.0/string_format.txt | 147 +++ bench_results/v1.0.0/string_ops.txt | 99 ++ bench_results/v1.0.0/summary.json | 1290 ++++++++++++++++++++++++ bench_results/v1.0.0/table_ops.txt | 469 +++++++++ bench_results/v1.0.0/timestamp.txt | 1 + bench_results/v1.0.0/versions.txt | 3 + bench_results/v1.0.0/vm_new.txt | 65 ++ bench_results/v1.0.2/closures.txt | 54 + bench_results/v1.0.2/commit.txt | 1 + bench_results/v1.0.2/cpu.txt | 1 + bench_results/v1.0.2/encode_decode.txt | 136 +++ bench_results/v1.0.2/environment.md | 36 + bench_results/v1.0.2/fibonacci.txt | 51 + bench_results/v1.0.2/metamethods.txt | 144 +++ bench_results/v1.0.2/oop.txt | 51 + bench_results/v1.0.2/patterns.txt | 144 +++ bench_results/v1.0.2/pcall_varargs.txt | 144 +++ bench_results/v1.0.2/string_format.txt | 144 +++ bench_results/v1.0.2/string_ops.txt | 99 ++ bench_results/v1.0.2/summary.json | 1282 +++++++++++++++++++++++ bench_results/v1.0.2/table_ops.txt | 469 +++++++++ bench_results/v1.0.2/timestamp.txt | 1 + bench_results/v1.0.2/versions.txt | 3 + bench_results/v1.0.2/vm_new.txt | 65 ++ bench_results/versions-2026-07-28.md | 574 +++++++++++ 51 files changed, 9248 insertions(+) create mode 100644 bench_results/README.md create mode 100644 bench_results/v0.4.0/closures.txt create mode 100644 bench_results/v0.4.0/cpu.txt create mode 100644 bench_results/v0.4.0/encode_decode.txt create mode 100644 bench_results/v0.4.0/environment.md create mode 100644 bench_results/v0.4.0/fibonacci.txt create mode 100644 bench_results/v0.4.0/metamethods.txt create mode 100644 bench_results/v0.4.0/oop.txt create mode 100644 bench_results/v0.4.0/patterns.txt create mode 100644 bench_results/v0.4.0/pcall_varargs.txt create mode 100644 bench_results/v0.4.0/string_format.txt create mode 100644 bench_results/v0.4.0/string_ops.txt create mode 100644 bench_results/v0.4.0/summary.json create mode 100644 bench_results/v0.4.0/table_ops.txt create mode 100644 bench_results/v0.4.0/timestamp.txt create mode 100644 bench_results/v0.4.0/versions.txt create mode 100644 bench_results/v0.4.0/vm_new.txt create mode 100644 bench_results/v1.0.0/closures.txt create mode 100644 bench_results/v1.0.0/cpu.txt create mode 100644 bench_results/v1.0.0/encode_decode.txt create mode 100644 bench_results/v1.0.0/environment.md create mode 100644 bench_results/v1.0.0/fibonacci.txt create mode 100644 bench_results/v1.0.0/metamethods.txt create mode 100644 bench_results/v1.0.0/oop.txt create mode 100644 bench_results/v1.0.0/patterns.txt create mode 100644 bench_results/v1.0.0/pcall_varargs.txt create mode 100644 bench_results/v1.0.0/string_format.txt create mode 100644 bench_results/v1.0.0/string_ops.txt create mode 100644 bench_results/v1.0.0/summary.json create mode 100644 bench_results/v1.0.0/table_ops.txt create mode 100644 bench_results/v1.0.0/timestamp.txt create mode 100644 bench_results/v1.0.0/versions.txt create mode 100644 bench_results/v1.0.0/vm_new.txt create mode 100644 bench_results/v1.0.2/closures.txt create mode 100644 bench_results/v1.0.2/commit.txt create mode 100644 bench_results/v1.0.2/cpu.txt create mode 100644 bench_results/v1.0.2/encode_decode.txt create mode 100644 bench_results/v1.0.2/environment.md create mode 100644 bench_results/v1.0.2/fibonacci.txt create mode 100644 bench_results/v1.0.2/metamethods.txt create mode 100644 bench_results/v1.0.2/oop.txt create mode 100644 bench_results/v1.0.2/patterns.txt create mode 100644 bench_results/v1.0.2/pcall_varargs.txt create mode 100644 bench_results/v1.0.2/string_format.txt create mode 100644 bench_results/v1.0.2/string_ops.txt create mode 100644 bench_results/v1.0.2/summary.json create mode 100644 bench_results/v1.0.2/table_ops.txt create mode 100644 bench_results/v1.0.2/timestamp.txt create mode 100644 bench_results/v1.0.2/versions.txt create mode 100644 bench_results/v1.0.2/vm_new.txt create mode 100644 bench_results/versions-2026-07-28.md diff --git a/bench_results/README.md b/bench_results/README.md new file mode 100644 index 0000000..7afbdc3 --- /dev/null +++ b/bench_results/README.md @@ -0,0 +1,113 @@ +# `bench_results/` + +Recorded benchmark runs and the reports written from them. + +The benchmark **scripts** live in [`benchmarks/`](../benchmarks/). This +directory holds their **output**: raw stdout, a parsed JSON summary, environment +probes, and the human-readable report for each measurement campaign. Nothing +here is used by the library at runtime, and none of it ships in the Hex package. + +## Contents + +| Path | What it is | +|---|---| +| [`versions-2026-07-28.md`](./versions-2026-07-28.md) | Cross-version comparison — v0.4.0 vs v1.0.0 vs 1.0.2 (`main` @ `3a0d392`), with Luerl 1.5.1 as a same-run control in every table. Supersedes the 1.0.0-era numbers in [`benchmarks/BASELINE.md`](../benchmarks/BASELINE.md). | +| [`v0.4.0/`](./v0.4.0/), [`v1.0.0/`](./v1.0.0/), [`v1.0.2/`](./v1.0.2/) | The data behind that report — one directory per released version. This is the ongoing convention: each release gets its own directory here, measured with the full suite of its day. | + +### Layout of a version directory + +``` +/ + environment.md # ref, commit, mode, CPU, OTP/Elixir, timestamp, command form + summary.json # parsed results: workload -> case -> jobs + comparison lines + .txt # verbatim stdout of one `mix run benchmarks/.exs` + cpu.txt versions.txt timestamp.txt commit.txt +``` + +`summary.json` schema, as produced for the 2026-07-28 run: + +- Top level is keyed by **workload** (`fibonacci`, `table_ops`, `vm_new`, …). +- Each workload maps to its **case banners** (`"default"` for single-case + workloads, otherwise the banner the script printed, e.g. + `"patterns: gsub template substitution (n=200)"`). +- Each case has `jobs` — a list of `{name, ips, average, deviation, median, + p99, memory}` — plus the verbatim Benchee `comparison` lines and, when memory + measurement was on, `memory_comparison` or `memory_note`. +- `table_ops` cases nest one level deeper under `by_input` (or `inputs` on the + v0.4.0 run) keyed by input label — `small (n=10)`, `medium (n=100)`, + `large (n=1000)`. +- `vm_new` carries `cold_call` and `second_call` alongside its jobs: the + first-ever and second `Lua.new()` on the node, measured before Benchee starts. +- `encode_decode` is `{"raw": "..."}`. That script uses its own `:timer.tc` + harness rather than Benchee and prints a per-element-nanoseconds table, so it + is stored unparsed. + +Case-name keys are **not** byte-identical across refs: the v0.4.0 run appends +`" (mode: full)"` to banners and uses `"(single case)"` where later runs use +`"default"`. Normalise by stripping the mode suffix before joining across refs. + +## Reproducing a run + +Full per-ref instructions — including the three adaptations v0.4.0 needs and +the language constraints a cross-version workload must respect — are in the +[Reproduction section of the report](./versions-2026-07-28.md#reproduction). +The short version: + +```sh +MIX_ENV=benchmark mix deps.get +for w in fibonacci closures oop string_ops string_format table_ops \ + patterns metamethods pcall_varargs vm_new encode_decode; do + LUA_BENCH_MODE=full MIX_ENV=benchmark mix run "benchmarks/$w.exs" +done +``` + +Two rules are not optional: + +- **Serially, one `mix run` at a time, on a quiet machine.** Concurrent load + inflates deviation badly — the table and OOP cases swing enough to flip + orderings. +- **`LUA_BENCH_MODE=full` for anything published.** The default `quick` mode + uses short windows, skips memory measurement, and collapses the table + workloads to a single input size. It is for "did my change move the needle" + iteration, not for numbers anyone reads. + +### Older refs + +Each ref is measured in a throwaway detached worktree so the main checkout is +never modified: + +```sh +git worktree add --detach /tmp/lua- +``` + +- **v1.0.0** — copy in the four workloads that postdate it + (`patterns`, `metamethods`, `pcall_varargs`, `vm_new`), then + `MIX_ENV=benchmark mix deps.get`. +- **v0.4.0** — copy in the whole `benchmarks/` directory (the tag has none) and + add `{:benchee, "~> 1.3", only: :benchmark}` to `deps/0`. `luerl` is already + an unconditional dependency there, so the control rows need nothing. + +Remove the worktree when done (`git worktree remove --force /tmp/lua-`). + +## Benchmarking a new release + +The convention: **every released version gets a directory here**, so the +series grows one column per release. + +1. After tagging, run the full suite against the tag (serially, full mode, + quiet machine — see above) and put the outputs in + `bench_results//` with the same file layout as the existing + directories (`environment.md`, `summary.json`, one `.txt` per workload, + plus the env probes). +2. Include the Luerl control rows — they are what make the new column + comparable to the old ones despite machine/OTP drift between sittings. +3. If the suite gained workloads since the last release, note in + `environment.md` which workloads are new (older version directories will + simply lack those files). +4. Write or extend a report named `-.md` quoting + **medians**, not averages — several workloads have allocation-driven GC + pauses that pull the mean around. +5. Add a row to the Contents table above, and update the hosted page at + `website/priv/static/benchmarks.html` if the headline story changed. +6. Do not edit `benchmarks/BASELINE.md`. It is the historical 1.0.0 gate + record; a newer report supersedes it by saying so. diff --git a/bench_results/v0.4.0/closures.txt b/bench_results/v0.4.0/closures.txt new file mode 100644 index 0000000..dac5991 --- /dev/null +++ b/bench_results/v0.4.0/closures.txt @@ -0,0 +1,58 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 2.69 K 372.35 μs ±7.42% 368.92 μs 507.15 μs +lua (eval) 2.52 K 397.20 μs ±8.30% 390.13 μs 515.81 μs +luerl 2.50 K 400.26 μs ±9.75% 391.42 μs 536.69 μs + +Comparison: +lua (chunk) 2.69 K +lua (eval) 2.52 K - 1.07x slower +24.85 μs +luerl 2.50 K - 1.07x slower +27.91 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1.89 MB +lua (eval) 1.90 MB - 1.01x memory usage +0.0101 MB +luerl 1.90 MB - 1.00x memory usage +0.00886 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/cpu.txt b/bench_results/v0.4.0/cpu.txt new file mode 100644 index 0000000..de5c8ad --- /dev/null +++ b/bench_results/v0.4.0/cpu.txt @@ -0,0 +1 @@ +Apple M4 diff --git a/bench_results/v0.4.0/encode_decode.txt b/bench_results/v0.4.0/encode_decode.txt new file mode 100644 index 0000000..32d5f5a --- /dev/null +++ b/bench_results/v0.4.0/encode_decode.txt @@ -0,0 +1,143 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +lua 0.4.0 — encode!/decode! decomposition +(decode+deep_cast column: enabled) +================================================================================== +op shape N total_us per_elem_ns +encode int_list 8 0.27 33.3 +decode int_list 8 0.09 11.2 +dec+cast int_list 8 0.11 13.7 +encode int_list 64 1.61 25.1 +decode int_list 64 0.54 8.5 +dec+cast int_list 64 0.96 15.0 +encode int_list 512 26.38 51.5 +decode int_list 512 5.11 10.0 +dec+cast int_list 512 8.60 16.8 +encode int_list 4096 226.97 55.4 +decode int_list 4096 43.47 10.6 +dec+cast int_list 4096 88.67 21.6 +---------------------------------------------------------------------------------- +encode float_list 8 0.22 27.7 +decode float_list 8 0.09 11.2 +dec+cast float_list 8 0.11 13.2 +encode float_list 64 1.64 25.7 +decode float_list 64 0.55 8.5 +dec+cast float_list 64 0.94 14.7 +encode float_list 512 27.00 52.7 +decode float_list 512 5.85 11.4 +dec+cast float_list 512 9.00 17.6 +encode float_list 4096 239.42 58.5 +decode float_list 4096 64.74 15.8 +dec+cast float_list 4096 98.40 24.0 +---------------------------------------------------------------------------------- +encode bool_list 8 0.21 26.4 +decode bool_list 8 0.08 10.0 +dec+cast bool_list 8 0.10 12.6 +encode bool_list 64 1.60 25.0 +decode bool_list 64 0.56 8.7 +dec+cast bool_list 64 0.97 15.1 +encode bool_list 512 25.68 50.1 +decode bool_list 512 4.80 9.4 +dec+cast bool_list 512 8.38 16.4 +encode bool_list 4096 224.95 54.9 +decode bool_list 4096 38.99 9.5 +dec+cast bool_list 4096 84.05 20.5 +---------------------------------------------------------------------------------- +encode short_string_list 8 0.21 26.7 +decode short_string_list 8 0.08 10.5 +dec+cast short_string_list 8 0.10 12.9 +encode short_string_list 64 1.62 25.3 +decode short_string_list 64 0.55 8.6 +dec+cast short_string_list 64 0.98 15.4 +encode short_string_list 512 26.45 51.7 +decode short_string_list 512 4.86 9.5 +dec+cast short_string_list 512 9.05 17.7 +encode short_string_list 4096 202.93 49.5 +decode short_string_list 4096 43.46 10.6 +dec+cast short_string_list 4096 73.06 17.8 +---------------------------------------------------------------------------------- +encode long_string_list 8 0.22 27.2 +decode long_string_list 8 0.08 10.4 +dec+cast long_string_list 8 0.10 12.7 +encode long_string_list 64 1.67 26.1 +decode long_string_list 64 0.59 9.2 +dec+cast long_string_list 64 1.03 16.1 +encode long_string_list 512 28.42 55.5 +decode long_string_list 512 5.16 10.1 +dec+cast long_string_list 512 9.09 17.7 +encode long_string_list 4096 465.96 113.8 +decode long_string_list 4096 54.39 13.3 +dec+cast long_string_list 4096 108.40 26.5 +---------------------------------------------------------------------------------- +encode string_map 8 0.52 65.3 +decode string_map 8 0.07 8.2 +dec+cast string_map 8 0.17 21.1 +encode string_map 64 6.56 102.5 +decode string_map 64 0.40 6.2 +dec+cast string_map 64 3.21 50.2 +encode string_map 512 113.10 220.9 +decode string_map 512 4.27 8.3 +dec+cast string_map 512 39.39 76.9 +encode string_map 4096 1295.09 316.2 +decode string_map 4096 42.02 10.3 +dec+cast string_map 4096 367.76 89.8 +---------------------------------------------------------------------------------- +encode int_map 8 0.25 31.2 +decode int_map 8 0.09 10.7 +dec+cast int_map 8 0.10 12.9 +encode int_map 64 3.56 55.6 +decode int_map 64 0.57 8.9 +dec+cast int_map 64 0.97 15.1 +encode int_map 512 51.12 99.8 +decode int_map 512 4.79 9.4 +dec+cast int_map 512 8.76 17.1 +encode int_map 4096 443.17 108.2 +decode int_map 4096 40.30 9.8 +dec+cast int_map 4096 87.24 21.3 +---------------------------------------------------------------------------------- +encode record_list 8 2.07 258.7 +decode record_list 8 0.40 49.9 +dec+cast record_list 8 0.80 100.1 +encode record_list 64 31.54 492.8 +decode record_list 64 4.54 70.9 +dec+cast record_list 64 7.32 114.4 +encode record_list 512 287.96 562.4 +decode record_list 512 52.67 102.9 +dec+cast record_list 512 77.64 151.6 +encode record_list 4096 2346.38 572.8 +decode record_list 4096 406.88 99.3 +dec+cast record_list 4096 658.70 160.8 +---------------------------------------------------------------------------------- +nested chain (depth sweep) — isolates recursion/traversal from fan-out +op shape N total_us per_elem_ns +encode nested_chain 4 0.51 257.3 +decode nested_chain 4 0.12 61.7 +dec+cast nested_chain 4 0.24 118.7 +encode nested_chain 16 2.06 1032.0 +decode nested_chain 16 0.63 314.9 +dec+cast nested_chain 16 1.10 550.2 +encode nested_chain 64 8.53 4265.3 +decode nested_chain 64 4.16 2079.7 +dec+cast nested_chain 64 6.63 3317.0 +encode nested_chain 256 34.34 17170.4 +decode nested_chain 256 37.82 18909.7 +dec+cast nested_chain 256 47.43 23716.3 +================================================================================== +composite anchor — the PR's `original_nested` (matches the 18us/108us figure) +op shape N total_us per_elem_ns +encode original_nested 75 3.48 870.0 +decode original_nested 75 0.58 145.4 +dec+cast original_nested 75 1.32 329.3 diff --git a/bench_results/v0.4.0/environment.md b/bench_results/v0.4.0/environment.md new file mode 100644 index 0000000..2d7f888 --- /dev/null +++ b/bench_results/v0.4.0/environment.md @@ -0,0 +1,55 @@ +# Benchmark environment — lua v0.4.0 (full mode) + +- **Ref**: `v0.4.0` (tag), commit `5bf2069` +- **CPU**: Apple M4 +- **OTP**: Erlang/OTP 29 [erts-17.0] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] +- **Elixir**: 1.20.0 (compiled with Erlang/OTP 29) +- **Mode**: `LUA_BENCH_MODE=full MIX_ENV=benchmark` +- **Setup timestamp**: 2026-07-28T13:59:36Z (UTC) +- **luaport (C Lua)**: not installed — skipped by all workloads as expected +- **luerl**: 1.5.1 (unconditional dep at this tag) + +## Worktree setup + +``` +git -C /Users/dave/code/tvlabs/lua worktree add --detach \ + /private/tmp/claude-501/-Users-dave-code-tvlabs-lua/baeced00-0f95-4dc6-8d8e-c16607ffb143/scratchpad/bench-v0.4.0 v0.4.0 + +cp -R /Users/dave/code/tvlabs/lua/benchmarks /benchmarks + +# mix.exs deps edit (worktree only): added +# {:benchee, "~> 1.3", only: :benchmark} +# alongside the existing {:luerl, "~> 1.5.1"} unconditional dep. + +MIX_ENV=benchmark mix deps.get +MIX_ENV=benchmark mix compile +``` + +No lockfile/dep conflicts were encountered; `deps.get` resolved cleanly (benchee 1.5.1, deep_merge 1.0.2, statistex 1.1.1 added; luerl/ex_doc/dialyxir unchanged). + +## Measurement commands + +Each workload run strictly serially, one at a time, via: + +``` +(cd && LUA_BENCH_MODE=full MIX_ENV=benchmark mix run benchmarks/.exs) \ + > /.txt 2>&1 +``` + +Order: fibonacci, closures, oop, string_ops, string_format, table_ops, patterns, metamethods, pcall_varargs, vm_new, encode_decode. + +Excluded: `dispatcher_vs_interpreter.exs` (crashes on v0.4.0), `array_vs_map_probe.exs` (runs no Lua). + +`encode_decode.exs` uses its own `:timer.tc` harness (not Benchee) and prints its own table directly. + +## Cleanup + +``` +git -C /Users/dave/code/tvlabs/lua worktree remove --force \ + /private/tmp/claude-501/-Users-dave-code-tvlabs-lua/baeced00-0f95-4dc6-8d8e-c16607ffb143/scratchpad/bench-v0.4.0 +``` + +## Results + +Parsed into `summary.json` in this directory. Raw Benchee/harness stdout for each +workload is preserved verbatim in `.txt`. diff --git a/bench_results/v0.4.0/fibonacci.txt b/bench_results/v0.4.0/fibonacci.txt new file mode 100644 index 0000000..1927342 --- /dev/null +++ b/bench_results/v0.4.0/fibonacci.txt @@ -0,0 +1,58 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 1.42 703.34 ms ±1.00% 702.84 ms 715.45 ms +lua (eval) 1.40 712.63 ms ±1.13% 711.69 ms 732.64 ms +luerl 1.38 724.80 ms ±3.25% 720.40 ms 801.56 ms + +Comparison: +lua (chunk) 1.42 +lua (eval) 1.40 - 1.01x slower +9.29 ms +luerl 1.38 - 1.03x slower +21.46 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 2.45 GB +lua (eval) 2.45 GB - 1.00x memory usage +0.00001 GB +luerl 2.45 GB - 1.00x memory usage +0.00001 GB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/metamethods.txt b/bench_results/v0.4.0/metamethods.txt new file mode 100644 index 0000000..38564c2 --- /dev/null +++ b/bench_results/v0.4.0/metamethods.txt @@ -0,0 +1,151 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== metamethods: self-call method dispatch (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 2.04 K 489.21 μs ±10.11% 476.58 μs 672.87 μs +lua (eval) 1.99 K 503.66 μs ±9.40% 488.92 μs 678.61 μs +luerl 1.98 K 505.65 μs ±10.20% 492.79 μs 680.89 μs + +Comparison: +lua (chunk) 2.04 K +lua (eval) 1.99 K - 1.03x slower +14.45 μs +luerl 1.98 K - 1.03x slower +16.44 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1.42 MB +lua (eval) 1.43 MB - 1.01x memory usage +0.0105 MB +luerl 1.43 MB - 1.01x memory usage +0.0103 MB + +**All measurements for memory usage were the same** + +=== metamethods: 3-level __index chain (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 4.58 K 218.46 μs ±4.01% 217.92 μs 233.19 μs +lua (chunk) 4.57 K 219.03 μs ±10.32% 216.42 μs 254.20 μs +lua (eval) 4.52 K 221.04 μs ±5.51% 219.33 μs 245.33 μs + +Comparison: +luerl 4.58 K +lua (chunk) 4.57 K - 1.00x slower +0.57 μs +lua (eval) 4.52 K - 1.01x slower +2.57 μs + +Memory usage statistics: + +Name Memory usage +luerl 668.59 KB +lua (chunk) 658.30 KB - 0.98x memory usage -10.28906 KB +lua (eval) 668.80 KB - 1.00x memory usage +0.22 KB + +**All measurements for memory usage were the same** + +=== metamethods: arithmetic/relational metamethods (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 519.76 1.92 ms ±9.38% 1.88 ms 2.25 ms +lua (chunk) 513.47 1.95 ms ±15.04% 1.88 ms 2.30 ms +lua (eval) 508.61 1.97 ms ±10.60% 1.89 ms 2.93 ms + +Comparison: +luerl 519.76 +lua (chunk) 513.47 - 1.01x slower +0.0236 ms +lua (eval) 508.61 - 1.02x slower +0.0422 ms + +Memory usage statistics: + +Name Memory usage +luerl 6.24 MB +lua (chunk) 6.23 MB - 1.00x memory usage -0.01392 MB +lua (eval) 6.24 MB - 1.00x memory usage -0.00303 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/oop.txt b/bench_results/v0.4.0/oop.txt new file mode 100644 index 0000000..580705e --- /dev/null +++ b/bench_results/v0.4.0/oop.txt @@ -0,0 +1,58 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 9.11 K 109.76 μs ±13.66% 106.50 μs 179.25 μs +lua (eval) 8.85 K 113.01 μs ±18.72% 108.58 μs 193.98 μs +lua (chunk) 8.80 K 113.70 μs ±14.10% 110.88 μs 195.87 μs + +Comparison: +luerl 9.11 K +lua (eval) 8.85 K - 1.03x slower +3.25 μs +lua (chunk) 8.80 K - 1.04x slower +3.94 μs + +Memory usage statistics: + +Name Memory usage +luerl 381.45 KB +lua (eval) 382.52 KB - 1.00x memory usage +1.07 KB +lua (chunk) 372.03 KB - 0.98x memory usage -9.42188 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/patterns.txt b/bench_results/v0.4.0/patterns.txt new file mode 100644 index 0000000..bc1ea94 --- /dev/null +++ b/bench_results/v0.4.0/patterns.txt @@ -0,0 +1,154 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== patterns: find/match field extraction (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 947.52 1.06 ms ±3.75% 1.06 ms 1.18 ms +luerl 937.83 1.07 ms ±6.93% 1.05 ms 1.22 ms +lua (chunk) 919.82 1.09 ms ±4.39% 1.07 ms 1.22 ms + +Comparison: +lua (eval) 947.52 +luerl 937.83 - 1.01x slower +0.0109 ms +lua (chunk) 919.82 - 1.03x slower +0.0318 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 6.70 MB +luerl 6.70 MB - 1.00x memory usage -0.00021 MB +lua (chunk) 6.69 MB - 1.00x memory usage -0.00996 MB + +**All measurements for memory usage were the same** + +=== patterns: find-based tokenizer (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 731.44 1.37 ms ±3.23% 1.36 ms 1.51 ms +lua (chunk) 721.24 1.39 ms ±4.36% 1.37 ms 1.57 ms +lua (eval) 715.30 1.40 ms ±4.16% 1.39 ms 1.49 ms + +Comparison: +luerl 731.44 +lua (chunk) 721.24 - 1.01x slower +0.0193 ms +lua (eval) 715.30 - 1.02x slower +0.0308 ms + +Memory usage statistics: + +Name Memory usage +luerl 6.65 MB +lua (chunk) 6.64 MB - 1.00x memory usage -0.00977 MB +lua (eval) 6.65 MB - 1.00x memory usage +0.00024 MB + +**All measurements for memory usage were the same** + +=== patterns: gsub template substitution (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 452.66 2.21 ms ±4.59% 2.21 ms 2.50 ms +lua (eval) 446.40 2.24 ms ±8.19% 2.19 ms 2.75 ms +lua (chunk) 436.67 2.29 ms ±10.44% 2.21 ms 3.18 ms + +Comparison: +luerl 452.66 +lua (eval) 446.40 - 1.01x slower +0.0310 ms +lua (chunk) 436.67 - 1.04x slower +0.0809 ms + +Memory usage statistics: + +Name average deviation median 99th % +luerl 11.75 MB ±0.00% 11.75 MB 11.75 MB +lua (eval) 11.75 MB ±0.00% 11.75 MB 11.75 MB +lua (chunk) 11.74 MB ±0.00% 11.74 MB 11.74 MB + +Comparison: +luerl 11.75 MB +lua (eval) 11.75 MB - 1.00x memory usage +0.00021 MB +lua (chunk) 11.74 MB - 1.00x memory usage -0.00975 MB diff --git a/bench_results/v0.4.0/pcall_varargs.txt b/bench_results/v0.4.0/pcall_varargs.txt new file mode 100644 index 0000000..c6e6759 --- /dev/null +++ b/bench_results/v0.4.0/pcall_varargs.txt @@ -0,0 +1,151 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== call protocol: pcall, success path (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 3.44 K 291.08 μs ±3.72% 289.67 μs 313 μs +lua (eval) 3.42 K 292.45 μs ±3.54% 291.17 μs 315.44 μs +lua (chunk) 3.40 K 293.76 μs ±7.51% 290.50 μs 338.21 μs + +Comparison: +luerl 3.44 K +lua (eval) 3.42 K - 1.00x slower +1.36 μs +lua (chunk) 3.40 K - 1.01x slower +2.68 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.11 MB +lua (eval) 1.11 MB - 1.00x memory usage +0.00021 MB +lua (chunk) 1.10 MB - 0.99x memory usage -0.00993 MB + +**All measurements for memory usage were the same** + +=== call protocol: pcall, raise + catch (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 2.13 K 468.84 μs ±5.39% 466.38 μs 505.90 μs +lua (eval) 2.13 K 470.00 μs ±3.58% 467.50 μs 498.92 μs +lua (chunk) 2.11 K 474.55 μs ±9.26% 469.29 μs 583.95 μs + +Comparison: +luerl 2.13 K +lua (eval) 2.13 K - 1.00x slower +1.15 μs +lua (chunk) 2.11 K - 1.01x slower +5.70 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.55 MB +lua (eval) 1.55 MB - 1.00x memory usage +0.00021 MB +lua (chunk) 1.54 MB - 0.99x memory usage -0.01002 MB + +**All measurements for memory usage were the same** + +=== call protocol: varargs + multiple returns (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 418.17 2.39 ms ±4.44% 2.36 ms 2.70 ms +lua (eval) 400.02 2.50 ms ±11.76% 2.39 ms 3.48 ms +lua (chunk) 391.27 2.56 ms ±14.33% 2.40 ms 3.52 ms + +Comparison: +luerl 418.17 +lua (eval) 400.02 - 1.05x slower +0.109 ms +lua (chunk) 391.27 - 1.07x slower +0.164 ms + +Memory usage statistics: + +Name Memory usage +luerl 8.73 MB +lua (eval) 8.73 MB - 1.00x memory usage -0.00005 MB +lua (chunk) 8.72 MB - 1.00x memory usage -0.00979 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/string_format.txt b/bench_results/v0.4.0/string_format.txt new file mode 100644 index 0000000..79888e7 --- /dev/null +++ b/bench_results/v0.4.0/string_format.txt @@ -0,0 +1,154 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== string.format: long literal-heavy format string (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 249.24 4.01 ms ±6.43% 3.91 ms 4.54 ms +luerl 242.35 4.13 ms ±6.17% 4.25 ms 4.54 ms +lua (eval) 187.95 5.32 ms ±9.84% 5.58 ms 6.77 ms + +Comparison: +lua (chunk) 249.24 +luerl 242.35 - 1.03x slower +0.114 ms +lua (eval) 187.95 - 1.33x slower +1.31 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 20.36 MB +luerl 22.59 MB - 1.11x memory usage +2.23 MB +lua (eval) 20.38 MB - 1.00x memory usage +0.0172 MB + +**All measurements for memory usage were the same** + +=== string.format: width-flagged specifiers (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 582.97 1.72 ms ±3.52% 1.72 ms 1.86 ms +lua (eval) 576.12 1.74 ms ±8.52% 1.70 ms 2.41 ms +luerl 572.77 1.75 ms ±8.30% 1.73 ms 2.14 ms + +Comparison: +lua (chunk) 582.97 +lua (eval) 576.12 - 1.01x slower +0.0204 ms +luerl 572.77 - 1.02x slower +0.0306 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 7.53 MB +lua (eval) 7.55 MB - 1.00x memory usage +0.0192 MB +luerl 7.55 MB - 1.00x memory usage +0.0161 MB + +**All measurements for memory usage were the same** + +=== string.format: many specifiers (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 370.09 2.70 ms ±4.67% 2.78 ms 2.89 ms +lua (chunk) 367.47 2.72 ms ±4.88% 2.75 ms 2.99 ms +luerl 362.27 2.76 ms ±3.93% 2.80 ms 2.99 ms + +Comparison: +lua (eval) 370.09 +lua (chunk) 367.47 - 1.01x slower +0.0193 ms +luerl 362.27 - 1.02x slower +0.0584 ms + +Memory usage statistics: + +Name average deviation median 99th % +lua (eval) 13.76 MB ±0.00% 13.76 MB 13.76 MB +lua (chunk) 13.73 MB ±0.00% 13.73 MB 13.73 MB +luerl 13.76 MB ±0.00% 13.76 MB 13.76 MB + +Comparison: +lua (eval) 13.76 MB +lua (chunk) 13.73 MB - 1.00x memory usage -0.03384 MB +luerl 13.76 MB - 1.00x memory usage -0.00677 MB diff --git a/bench_results/v0.4.0/string_ops.txt b/bench_results/v0.4.0/string_ops.txt new file mode 100644 index 0000000..249f816 --- /dev/null +++ b/bench_results/v0.4.0/string_ops.txt @@ -0,0 +1,106 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== String Concatenation via table.concat (n=100) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 25.94 K 38.55 μs ±12.21% 38.38 μs 58.63 μs +lua (eval) 25.35 K 39.45 μs ±6.04% 39.54 μs 45.04 μs +luerl 25.19 K 39.69 μs ±19.25% 39.42 μs 46.88 μs + +Comparison: +lua (chunk) 25.94 K +lua (eval) 25.35 K - 1.02x slower +0.89 μs +luerl 25.19 K - 1.03x slower +1.14 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 161.98 KB +lua (eval) 172.77 KB - 1.07x memory usage +10.79 KB +luerl 172.42 KB - 1.06x memory usage +10.44 KB + +**All measurements for memory usage were the same** + +=== String Formatting via string.format (n=100) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 9.75 K 102.58 μs ±11.83% 101.46 μs 124.58 μs +lua (eval) 9.70 K 103.12 μs ±7.60% 102.33 μs 124.83 μs +luerl 9.63 K 103.90 μs ±9.83% 102.83 μs 127.33 μs + +Comparison: +lua (chunk) 9.75 K +lua (eval) 9.70 K - 1.01x slower +0.54 μs +luerl 9.63 K - 1.01x slower +1.31 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 577.43 KB +lua (eval) 587.10 KB - 1.02x memory usage +9.67 KB +luerl 588.84 KB - 1.02x memory usage +11.41 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/summary.json b/bench_results/v0.4.0/summary.json new file mode 100644 index 0000000..d0fda5d --- /dev/null +++ b/bench_results/v0.4.0/summary.json @@ -0,0 +1,1257 @@ +{ + "fibonacci": { + "(single case)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "1.42", + "average": "703.34 ms", + "deviation": "±1.00%", + "median": "702.84 ms", + "p99": "715.45 ms", + "memory": "2.45 GB" + }, + { + "name": "lua (eval)", + "ips": "1.40", + "average": "712.63 ms", + "deviation": "±1.13%", + "median": "711.69 ms", + "p99": "732.64 ms", + "memory": "2.45 GB" + }, + { + "name": "luerl", + "ips": "1.38", + "average": "724.80 ms", + "deviation": "±3.25%", + "median": "720.40 ms", + "p99": "801.56 ms", + "memory": "2.45 GB" + } + ], + "comparison": [ + "lua (chunk) 1.42", + "lua (eval) 1.40 - 1.01x slower +9.29 ms", + "luerl 1.38 - 1.03x slower +21.46 ms" + ] + } + }, + "closures": { + "(single case)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "2.69 K", + "average": "372.35 μs", + "deviation": "±7.42%", + "median": "368.92 μs", + "p99": "507.15 μs", + "memory": "1.89 MB" + }, + { + "name": "lua (eval)", + "ips": "2.52 K", + "average": "397.20 μs", + "deviation": "±8.30%", + "median": "390.13 μs", + "p99": "515.81 μs", + "memory": "1.90 MB" + }, + { + "name": "luerl", + "ips": "2.50 K", + "average": "400.26 μs", + "deviation": "±9.75%", + "median": "391.42 μs", + "p99": "536.69 μs", + "memory": "1.90 MB" + } + ], + "comparison": [ + "lua (chunk) 2.69 K", + "lua (eval) 2.52 K - 1.07x slower +24.85 μs", + "luerl 2.50 K - 1.07x slower +27.91 μs" + ] + } + }, + "oop": { + "(single case)": { + "jobs": [ + { + "name": "luerl", + "ips": "9.11 K", + "average": "109.76 μs", + "deviation": "±13.66%", + "median": "106.50 μs", + "p99": "179.25 μs", + "memory": "381.45 KB" + }, + { + "name": "lua (eval)", + "ips": "8.85 K", + "average": "113.01 μs", + "deviation": "±18.72%", + "median": "108.58 μs", + "p99": "193.98 μs", + "memory": "382.52 KB" + }, + { + "name": "lua (chunk)", + "ips": "8.80 K", + "average": "113.70 μs", + "deviation": "±14.10%", + "median": "110.88 μs", + "p99": "195.87 μs", + "memory": "372.03 KB" + } + ], + "comparison": [ + "luerl 9.11 K", + "lua (eval) 8.85 K - 1.03x slower +3.25 μs", + "lua (chunk) 8.80 K - 1.04x slower +3.94 μs" + ] + } + }, + "string_ops": { + "String Concatenation via table.concat (n=100) (mode: full)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "25.94 K", + "average": "38.55 μs", + "deviation": "±12.21%", + "median": "38.38 μs", + "p99": "58.63 μs", + "memory": "161.98 KB" + }, + { + "name": "lua (eval)", + "ips": "25.35 K", + "average": "39.45 μs", + "deviation": "±6.04%", + "median": "39.54 μs", + "p99": "45.04 μs", + "memory": "172.77 KB" + }, + { + "name": "luerl", + "ips": "25.19 K", + "average": "39.69 μs", + "deviation": "±19.25%", + "median": "39.42 μs", + "p99": "46.88 μs", + "memory": "172.42 KB" + } + ], + "comparison": [ + "lua (chunk) 25.94 K", + "lua (eval) 25.35 K - 1.02x slower +0.89 μs", + "luerl 25.19 K - 1.03x slower +1.14 μs" + ] + }, + "String Formatting via string.format (n=100) (mode: full)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "9.75 K", + "average": "102.58 μs", + "deviation": "±11.83%", + "median": "101.46 μs", + "p99": "124.58 μs", + "memory": "577.43 KB" + }, + { + "name": "lua (eval)", + "ips": "9.70 K", + "average": "103.12 μs", + "deviation": "±7.60%", + "median": "102.33 μs", + "p99": "124.83 μs", + "memory": "587.10 KB" + }, + { + "name": "luerl", + "ips": "9.63 K", + "average": "103.90 μs", + "deviation": "±9.83%", + "median": "102.83 μs", + "p99": "127.33 μs", + "memory": "588.84 KB" + } + ], + "comparison": [ + "lua (chunk) 9.75 K", + "lua (eval) 9.70 K - 1.01x slower +0.54 μs", + "luerl 9.63 K - 1.01x slower +1.31 μs" + ] + } + }, + "string_format": { + "string.format: long literal-heavy format string (n=1000) (mode: full)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "249.24", + "average": "4.01 ms", + "deviation": "±6.43%", + "median": "3.91 ms", + "p99": "4.54 ms", + "memory": "20.36 MB" + }, + { + "name": "luerl", + "ips": "242.35", + "average": "4.13 ms", + "deviation": "±6.17%", + "median": "4.25 ms", + "p99": "4.54 ms", + "memory": "22.59 MB" + }, + { + "name": "lua (eval)", + "ips": "187.95", + "average": "5.32 ms", + "deviation": "±9.84%", + "median": "5.58 ms", + "p99": "6.77 ms", + "memory": "20.38 MB" + } + ], + "comparison": [ + "lua (chunk) 249.24", + "luerl 242.35 - 1.03x slower +0.114 ms", + "lua (eval) 187.95 - 1.33x slower +1.31 ms" + ] + }, + "string.format: width-flagged specifiers (n=1000) (mode: full)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "582.97", + "average": "1.72 ms", + "deviation": "±3.52%", + "median": "1.72 ms", + "p99": "1.86 ms", + "memory": "7.53 MB" + }, + { + "name": "lua (eval)", + "ips": "576.12", + "average": "1.74 ms", + "deviation": "±8.52%", + "median": "1.70 ms", + "p99": "2.41 ms", + "memory": "7.55 MB" + }, + { + "name": "luerl", + "ips": "572.77", + "average": "1.75 ms", + "deviation": "±8.30%", + "median": "1.73 ms", + "p99": "2.14 ms", + "memory": "7.55 MB" + } + ], + "comparison": [ + "lua (chunk) 582.97", + "lua (eval) 576.12 - 1.01x slower +0.0204 ms", + "luerl 572.77 - 1.02x slower +0.0306 ms" + ] + }, + "string.format: many specifiers (n=1000) (mode: full)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "370.09", + "average": "2.70 ms", + "deviation": "±4.67%", + "median": "2.78 ms", + "p99": "2.89 ms", + "memory": "13.76 MB" + }, + { + "name": "lua (chunk)", + "ips": "367.47", + "average": "2.72 ms", + "deviation": "±4.88%", + "median": "2.75 ms", + "p99": "2.99 ms", + "memory": "13.73 MB" + }, + { + "name": "luerl", + "ips": "362.27", + "average": "2.76 ms", + "deviation": "±3.93%", + "median": "2.80 ms", + "p99": "2.99 ms", + "memory": "13.76 MB" + } + ], + "comparison": [ + "lua (eval) 370.09", + "lua (chunk) 367.47 - 1.01x slower +0.0193 ms", + "luerl 362.27 - 1.02x slower +0.0584 ms" + ], + "memory_comparison": [ + "lua (eval) 13.76 MB", + "lua (chunk) 13.73 MB - 1.00x memory usage -0.03384 MB", + "luerl 13.76 MB - 1.00x memory usage -0.00677 MB" + ] + } + }, + "table_ops": { + "Table Build (mode: full)": { + "inputs": { + "With input large (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "6.50 K", + "average": "153.83 μs", + "deviation": "±11.45%", + "median": "151.96 μs", + "p99": "195.78 μs", + "memory": "993.93 KB" + }, + { + "name": "luerl", + "ips": "6.44 K", + "average": "155.20 μs", + "deviation": "±14.07%", + "median": "152.33 μs", + "p99": "259.75 μs", + "memory": "996.96 KB" + }, + { + "name": "lua (chunk)", + "ips": "6.35 K", + "average": "157.51 μs", + "deviation": "±18.23%", + "median": "149.92 μs", + "p99": "262.79 μs", + "memory": "983.38 KB" + } + ], + "comparison": [ + "lua (eval) 6.50 K", + "luerl 6.44 K - 1.01x slower +1.37 μs", + "lua (chunk) 6.35 K - 1.02x slower +3.68 μs" + ] + }, + "With input medium (n=100)": { + "jobs": [ + { + "name": "luerl", + "ips": "59.29 K", + "average": "16.87 μs", + "deviation": "±38.96%", + "median": "16.25 μs", + "p99": "26.79 μs", + "memory": "110.78 KB" + }, + { + "name": "lua (eval)", + "ips": "51.13 K", + "average": "19.56 μs", + "deviation": "±34.91%", + "median": "16.46 μs", + "p99": "41.71 μs", + "memory": "111.19 KB" + }, + { + "name": "lua (chunk)", + "ips": "43.75 K", + "average": "22.86 μs", + "deviation": "±38.05%", + "median": "25.96 μs", + "p99": "47.33 μs", + "memory": "100.13 KB" + } + ], + "comparison": [ + "luerl 59.29 K", + "lua (eval) 51.13 K - 1.16x slower +2.69 μs", + "lua (chunk) 43.75 K - 1.36x slower +5.99 μs" + ] + }, + "With input small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "405.22 K", + "average": "2.47 μs", + "deviation": "±243.17%", + "median": "2 μs", + "p99": "5.83 μs", + "memory": "12.45 KB" + }, + { + "name": "luerl", + "ips": "321.06 K", + "average": "3.11 μs", + "deviation": "±187.65%", + "median": "3 μs", + "p99": "4.54 μs", + "memory": "22.62 KB" + }, + { + "name": "lua (eval)", + "ips": "282.18 K", + "average": "3.54 μs", + "deviation": "±178.79%", + "median": "3.13 μs", + "p99": "9.29 μs", + "memory": "23.02 KB" + } + ], + "comparison": [ + "lua (chunk) 405.22 K", + "luerl 321.06 K - 1.26x slower +0.65 μs", + "lua (eval) 282.18 K - 1.44x slower +1.08 μs" + ] + } + } + }, + "Table Sort (mode: full)": { + "inputs": { + "With input large (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "5.61 K", + "average": "178.37 μs", + "deviation": "±12.59%", + "median": "174.29 μs", + "p99": "234.12 μs", + "memory": "1.17 MB" + }, + { + "name": "lua (eval)", + "ips": "5.45 K", + "average": "183.60 μs", + "deviation": "±106.15%", + "median": "177.67 μs", + "p99": "334.99 μs", + "memory": "1.18 MB" + }, + { + "name": "luerl", + "ips": "4.88 K", + "average": "205.11 μs", + "deviation": "±25.14%", + "median": "180.59 μs", + "p99": "336.78 μs", + "memory": "1.18 MB" + } + ], + "comparison": [ + "lua (chunk) 5.61 K", + "lua (eval) 5.45 K - 1.03x slower +5.23 μs", + "luerl 4.88 K - 1.15x slower +26.74 μs" + ] + }, + "With input medium (n=100)": { + "jobs": [ + { + "name": "luerl", + "ips": "51.03 K", + "average": "19.60 μs", + "deviation": "±20.90%", + "median": "19.08 μs", + "p99": "28.42 μs", + "memory": "133.34 KB" + }, + { + "name": "lua (chunk)", + "ips": "47.42 K", + "average": "21.09 μs", + "deviation": "±42.23%", + "median": "18.04 μs", + "p99": "51.33 μs", + "memory": "122.98 KB" + }, + { + "name": "lua (eval)", + "ips": "46.64 K", + "average": "21.44 μs", + "deviation": "±39.09%", + "median": "19.33 μs", + "p99": "45.92 μs", + "memory": "133.59 KB" + } + ], + "comparison": [ + "luerl 51.03 K", + "lua (chunk) 47.42 K - 1.08x slower +1.49 μs", + "lua (eval) 46.64 K - 1.09x slower +1.85 μs" + ] + }, + "With input small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "324.41 K", + "average": "3.08 μs", + "deviation": "±194.26%", + "median": "2.54 μs", + "p99": "7.08 μs", + "memory": "15.84 KB" + }, + { + "name": "luerl", + "ips": "271.87 K", + "average": "3.68 μs", + "deviation": "±124.45%", + "median": "3.54 μs", + "p99": "10 μs", + "memory": "25.98 KB" + }, + { + "name": "lua (eval)", + "ips": "245.19 K", + "average": "4.08 μs", + "deviation": "±136.48%", + "median": "3.67 μs", + "p99": "8.25 μs", + "memory": "26.25 KB" + } + ], + "comparison": [ + "lua (chunk) 324.41 K", + "luerl 271.87 K - 1.19x slower +0.60 μs", + "lua (eval) 245.19 K - 1.32x slower +1.00 μs" + ] + } + } + }, + "Table Iterate/Sum (mode: full)": { + "inputs": { + "With input large (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "3.81 K", + "average": "262.24 μs", + "deviation": "±19.86%", + "median": "243.42 μs", + "p99": "444.12 μs", + "memory": "1.35 MB" + }, + { + "name": "luerl", + "ips": "3.75 K", + "average": "266.52 μs", + "deviation": "±20.59%", + "median": "243.13 μs", + "p99": "416.21 μs", + "memory": "1.35 MB" + }, + { + "name": "lua (chunk)", + "ips": "3.60 K", + "average": "278.11 μs", + "deviation": "±24.12%", + "median": "242.42 μs", + "p99": "456.96 μs", + "memory": "1.34 MB" + } + ], + "comparison": [ + "lua (eval) 3.81 K", + "luerl 3.75 K - 1.02x slower +4.28 μs", + "lua (chunk) 3.60 K - 1.06x slower +15.87 μs" + ] + }, + "With input medium (n=100)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "37.84 K", + "average": "26.43 μs", + "deviation": "±9.28%", + "median": "26.08 μs", + "p99": "29.92 μs", + "memory": "149.42 KB" + }, + { + "name": "luerl", + "ips": "37.50 K", + "average": "26.66 μs", + "deviation": "±86.90%", + "median": "26.21 μs", + "p99": "36.50 μs", + "memory": "149.28 KB" + }, + { + "name": "lua (chunk)", + "ips": "35.12 K", + "average": "28.47 μs", + "deviation": "±26.41%", + "median": "25.17 μs", + "p99": "53.58 μs", + "memory": "139.25 KB" + } + ], + "comparison": [ + "lua (eval) 37.84 K", + "luerl 37.50 K - 1.01x slower +0.24 μs", + "lua (chunk) 35.12 K - 1.08x slower +2.05 μs" + ] + }, + "With input small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "245.60 K", + "average": "4.07 μs", + "deviation": "±313.25%", + "median": "3.29 μs", + "p99": "9.79 μs", + "memory": "16.70 KB" + }, + { + "name": "luerl", + "ips": "227.18 K", + "average": "4.40 μs", + "deviation": "±102.94%", + "median": "4.29 μs", + "p99": "10.88 μs", + "memory": "26.80 KB" + }, + { + "name": "lua (eval)", + "ips": "206.12 K", + "average": "4.85 μs", + "deviation": "±120.13%", + "median": "4.33 μs", + "p99": "9.83 μs", + "memory": "27.20 KB" + } + ], + "comparison": [ + "lua (chunk) 245.60 K", + "luerl 227.18 K - 1.08x slower +0.33 μs", + "lua (eval) 206.12 K - 1.19x slower +0.78 μs" + ] + } + } + }, + "Table Map + Reduce (mode: full)": { + "inputs": { + "With input large (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "2.21 K", + "average": "452.53 μs", + "deviation": "±7.79%", + "median": "446.25 μs", + "p99": "614.89 μs", + "memory": "2.45 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.19 K", + "average": "455.66 μs", + "deviation": "±9.09%", + "median": "444.34 μs", + "p99": "628.64 μs", + "memory": "2.43 MB" + }, + { + "name": "luerl", + "ips": "2.19 K", + "average": "456.58 μs", + "deviation": "±12.12%", + "median": "442.21 μs", + "p99": "659.87 μs", + "memory": "2.45 MB" + } + ], + "comparison": [ + "lua (eval) 2.21 K", + "lua (chunk) 2.19 K - 1.01x slower +3.14 μs", + "luerl 2.19 K - 1.01x slower +4.05 μs" + ] + }, + "With input medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "21.79 K", + "average": "45.90 μs", + "deviation": "±12.29%", + "median": "45.25 μs", + "p99": "53.13 μs", + "memory": "252.38 KB" + }, + { + "name": "luerl", + "ips": "21.47 K", + "average": "46.58 μs", + "deviation": "±5.77%", + "median": "46.42 μs", + "p99": "52.54 μs", + "memory": "262.48 KB" + }, + { + "name": "lua (eval)", + "ips": "21.44 K", + "average": "46.65 μs", + "deviation": "±6.16%", + "median": "46.50 μs", + "p99": "51.96 μs", + "memory": "263.02 KB" + } + ], + "comparison": [ + "lua (chunk) 21.79 K", + "luerl 21.47 K - 1.01x slower +0.68 μs", + "lua (eval) 21.44 K - 1.02x slower +0.75 μs" + ] + }, + "With input small (n=10)": { + "jobs": [ + { + "name": "luerl", + "ips": "148.83 K", + "average": "6.72 μs", + "deviation": "±69.12%", + "median": "6.58 μs", + "p99": "14.71 μs", + "memory": "39.39 KB" + }, + { + "name": "lua (chunk)", + "ips": "145.39 K", + "average": "6.88 μs", + "deviation": "±90.79%", + "median": "5.54 μs", + "p99": "14.96 μs", + "memory": "29.06 KB" + }, + { + "name": "lua (eval)", + "ips": "131.69 K", + "average": "7.59 μs", + "deviation": "±88.05%", + "median": "6.71 μs", + "p99": "16.42 μs", + "memory": "39.80 KB" + } + ], + "comparison": [ + "luerl 148.83 K", + "lua (chunk) 145.39 K - 1.02x slower +0.159 μs", + "lua (eval) 131.69 K - 1.13x slower +0.87 μs" + ] + } + } + }, + "Table Pairs (hash) (mode: full)": { + "inputs": { + "With input large (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "760.68", + "average": "1.31 ms", + "deviation": "±5.02%", + "median": "1.32 ms", + "p99": "1.45 ms", + "memory": "2.48 MB" + }, + { + "name": "lua (chunk)", + "ips": "759.91", + "average": "1.32 ms", + "deviation": "±5.40%", + "median": "1.32 ms", + "p99": "1.49 ms", + "memory": "2.47 MB" + }, + { + "name": "luerl", + "ips": "754.19", + "average": "1.33 ms", + "deviation": "±6.28%", + "median": "1.33 ms", + "p99": "1.45 ms", + "memory": "2.48 MB" + } + ], + "comparison": [ + "lua (eval) 760.68", + "lua (chunk) 759.91 - 1.00x slower +0.00133 ms", + "luerl 754.19 - 1.01x slower +0.0113 ms" + ] + }, + "With input medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "10.96 K", + "average": "91.23 μs", + "deviation": "±12.10%", + "median": "89.92 μs", + "p99": "135.19 μs", + "memory": "238.58 KB" + }, + { + "name": "lua (eval)", + "ips": "10.20 K", + "average": "97.99 μs", + "deviation": "±11.84%", + "median": "97.42 μs", + "p99": "138.25 μs", + "memory": "249.40 KB" + }, + { + "name": "luerl", + "ips": "9.96 K", + "average": "100.37 μs", + "deviation": "±20.92%", + "median": "99.96 μs", + "p99": "144.67 μs", + "memory": "248.99 KB" + } + ], + "comparison": [ + "lua (chunk) 10.96 K", + "lua (eval) 10.20 K - 1.07x slower +6.76 μs", + "luerl 9.96 K - 1.10x slower +9.14 μs" + ] + }, + "With input small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "126.80 K", + "average": "7.89 μs", + "deviation": "±150.76%", + "median": "6.50 μs", + "p99": "18.79 μs", + "memory": "25.83 KB" + }, + { + "name": "luerl", + "ips": "125.64 K", + "average": "7.96 μs", + "deviation": "±72.08%", + "median": "7.29 μs", + "p99": "24.29 μs", + "memory": "36.16 KB" + }, + { + "name": "lua (eval)", + "ips": "109.11 K", + "average": "9.17 μs", + "deviation": "±61.17%", + "median": "9.38 μs", + "p99": "28.33 μs", + "memory": "36.56 KB" + } + ], + "comparison": [ + "lua (chunk) 126.80 K", + "luerl 125.64 K - 1.01x slower +0.0728 μs", + "lua (eval) 109.11 K - 1.16x slower +1.28 μs" + ] + } + } + } + }, + "patterns": { + "patterns: find/match field extraction (n=200) (mode: full)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "947.52", + "average": "1.06 ms", + "deviation": "±3.75%", + "median": "1.06 ms", + "p99": "1.18 ms", + "memory": "6.70 MB" + }, + { + "name": "luerl", + "ips": "937.83", + "average": "1.07 ms", + "deviation": "±6.93%", + "median": "1.05 ms", + "p99": "1.22 ms", + "memory": "6.70 MB" + }, + { + "name": "lua (chunk)", + "ips": "919.82", + "average": "1.09 ms", + "deviation": "±4.39%", + "median": "1.07 ms", + "p99": "1.22 ms", + "memory": "6.69 MB" + } + ], + "comparison": [ + "lua (eval) 947.52", + "luerl 937.83 - 1.01x slower +0.0109 ms", + "lua (chunk) 919.82 - 1.03x slower +0.0318 ms" + ] + }, + "patterns: find-based tokenizer (n=200) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "731.44", + "average": "1.37 ms", + "deviation": "±3.23%", + "median": "1.36 ms", + "p99": "1.51 ms", + "memory": "6.65 MB" + }, + { + "name": "lua (chunk)", + "ips": "721.24", + "average": "1.39 ms", + "deviation": "±4.36%", + "median": "1.37 ms", + "p99": "1.57 ms", + "memory": "6.64 MB" + }, + { + "name": "lua (eval)", + "ips": "715.30", + "average": "1.40 ms", + "deviation": "±4.16%", + "median": "1.39 ms", + "p99": "1.49 ms", + "memory": "6.65 MB" + } + ], + "comparison": [ + "luerl 731.44", + "lua (chunk) 721.24 - 1.01x slower +0.0193 ms", + "lua (eval) 715.30 - 1.02x slower +0.0308 ms" + ] + }, + "patterns: gsub template substitution (n=200) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "452.66", + "average": "2.21 ms", + "deviation": "±4.59%", + "median": "2.21 ms", + "p99": "2.50 ms", + "memory": "11.75 MB" + }, + { + "name": "lua (eval)", + "ips": "446.40", + "average": "2.24 ms", + "deviation": "±8.19%", + "median": "2.19 ms", + "p99": "2.75 ms", + "memory": "11.75 MB" + }, + { + "name": "lua (chunk)", + "ips": "436.67", + "average": "2.29 ms", + "deviation": "±10.44%", + "median": "2.21 ms", + "p99": "3.18 ms", + "memory": "11.74 MB" + } + ], + "comparison": [ + "luerl 452.66", + "lua (eval) 446.40 - 1.01x slower +0.0310 ms", + "lua (chunk) 436.67 - 1.04x slower +0.0809 ms" + ], + "memory_comparison": [ + "luerl 11.75 MB", + "lua (eval) 11.75 MB - 1.00x memory usage +0.00021 MB", + "lua (chunk) 11.74 MB - 1.00x memory usage -0.00975 MB" + ] + } + }, + "metamethods": { + "metamethods: self-call method dispatch (n=200) (mode: full)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "2.04 K", + "average": "489.21 μs", + "deviation": "±10.11%", + "median": "476.58 μs", + "p99": "672.87 μs", + "memory": "1.42 MB" + }, + { + "name": "lua (eval)", + "ips": "1.99 K", + "average": "503.66 μs", + "deviation": "±9.40%", + "median": "488.92 μs", + "p99": "678.61 μs", + "memory": "1.43 MB" + }, + { + "name": "luerl", + "ips": "1.98 K", + "average": "505.65 μs", + "deviation": "±10.20%", + "median": "492.79 μs", + "p99": "680.89 μs", + "memory": "1.43 MB" + } + ], + "comparison": [ + "lua (chunk) 2.04 K", + "lua (eval) 1.99 K - 1.03x slower +14.45 μs", + "luerl 1.98 K - 1.03x slower +16.44 μs" + ] + }, + "metamethods: 3-level __index chain (n=200) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "4.58 K", + "average": "218.46 μs", + "deviation": "±4.01%", + "median": "217.92 μs", + "p99": "233.19 μs", + "memory": "668.59 KB" + }, + { + "name": "lua (chunk)", + "ips": "4.57 K", + "average": "219.03 μs", + "deviation": "±10.32%", + "median": "216.42 μs", + "p99": "254.20 μs", + "memory": "658.30 KB" + }, + { + "name": "lua (eval)", + "ips": "4.52 K", + "average": "221.04 μs", + "deviation": "±5.51%", + "median": "219.33 μs", + "p99": "245.33 μs", + "memory": "668.80 KB" + } + ], + "comparison": [ + "luerl 4.58 K", + "lua (chunk) 4.57 K - 1.00x slower +0.57 μs", + "lua (eval) 4.52 K - 1.01x slower +2.57 μs" + ] + }, + "metamethods: arithmetic/relational metamethods (n=200) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "519.76", + "average": "1.92 ms", + "deviation": "±9.38%", + "median": "1.88 ms", + "p99": "2.25 ms", + "memory": "6.24 MB" + }, + { + "name": "lua (chunk)", + "ips": "513.47", + "average": "1.95 ms", + "deviation": "±15.04%", + "median": "1.88 ms", + "p99": "2.30 ms", + "memory": "6.23 MB" + }, + { + "name": "lua (eval)", + "ips": "508.61", + "average": "1.97 ms", + "deviation": "±10.60%", + "median": "1.89 ms", + "p99": "2.93 ms", + "memory": "6.24 MB" + } + ], + "comparison": [ + "luerl 519.76", + "lua (chunk) 513.47 - 1.01x slower +0.0236 ms", + "lua (eval) 508.61 - 1.02x slower +0.0422 ms" + ] + } + }, + "pcall_varargs": { + "call protocol: pcall, success path (n=500) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "3.44 K", + "average": "291.08 μs", + "deviation": "±3.72%", + "median": "289.67 μs", + "p99": "313 μs", + "memory": "1.11 MB" + }, + { + "name": "lua (eval)", + "ips": "3.42 K", + "average": "292.45 μs", + "deviation": "±3.54%", + "median": "291.17 μs", + "p99": "315.44 μs", + "memory": "1.11 MB" + }, + { + "name": "lua (chunk)", + "ips": "3.40 K", + "average": "293.76 μs", + "deviation": "±7.51%", + "median": "290.50 μs", + "p99": "338.21 μs", + "memory": "1.10 MB" + } + ], + "comparison": [ + "luerl 3.44 K", + "lua (eval) 3.42 K - 1.00x slower +1.36 μs", + "lua (chunk) 3.40 K - 1.01x slower +2.68 μs" + ] + }, + "call protocol: pcall, raise + catch (n=500) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "2.13 K", + "average": "468.84 μs", + "deviation": "±5.39%", + "median": "466.38 μs", + "p99": "505.90 μs", + "memory": "1.55 MB" + }, + { + "name": "lua (eval)", + "ips": "2.13 K", + "average": "470.00 μs", + "deviation": "±3.58%", + "median": "467.50 μs", + "p99": "498.92 μs", + "memory": "1.55 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.11 K", + "average": "474.55 μs", + "deviation": "±9.26%", + "median": "469.29 μs", + "p99": "583.95 μs", + "memory": "1.54 MB" + } + ], + "comparison": [ + "luerl 2.13 K", + "lua (eval) 2.13 K - 1.00x slower +1.15 μs", + "lua (chunk) 2.11 K - 1.01x slower +5.70 μs" + ] + }, + "call protocol: varargs + multiple returns (n=500) (mode: full)": { + "jobs": [ + { + "name": "luerl", + "ips": "418.17", + "average": "2.39 ms", + "deviation": "±4.44%", + "median": "2.36 ms", + "p99": "2.70 ms", + "memory": "8.73 MB" + }, + { + "name": "lua (eval)", + "ips": "400.02", + "average": "2.50 ms", + "deviation": "±11.76%", + "median": "2.39 ms", + "p99": "3.48 ms", + "memory": "8.73 MB" + }, + { + "name": "lua (chunk)", + "ips": "391.27", + "average": "2.56 ms", + "deviation": "±14.33%", + "median": "2.40 ms", + "p99": "3.52 ms", + "memory": "8.72 MB" + } + ], + "comparison": [ + "luerl 418.17", + "lua (eval) 400.02 - 1.05x slower +0.109 ms", + "lua (chunk) 391.27 - 1.07x slower +0.164 ms" + ] + } + }, + "vm_new": { + "VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full)": { + "jobs": [ + { + "name": "lua (new, no sandbox)", + "ips": "66.32 K", + "average": "15.08 μs", + "deviation": "±21.55%", + "median": "14.88 μs", + "p99": "22.58 μs", + "memory": "51.77 KB" + }, + { + "name": "luerl (init)", + "ips": "64.85 K", + "average": "15.42 μs", + "deviation": "±46.27%", + "median": "15.04 μs", + "p99": "30.25 μs", + "memory": "51.64 KB" + }, + { + "name": "lua (new, custom exclude)", + "ips": "47.89 K", + "average": "20.88 μs", + "deviation": "±63.00%", + "median": "20.63 μs", + "p99": "31.67 μs", + "memory": "72.27 KB" + }, + { + "name": "lua (new)", + "ips": "46.57 K", + "average": "21.47 μs", + "deviation": "±15.57%", + "median": "21.33 μs", + "p99": "31.25 μs", + "memory": "73.78 KB" + } + ], + "comparison": [ + "lua (new, no sandbox) 66.32 K", + "luerl (init) 64.85 K - 1.02x slower +0.34 μs", + "lua (new, custom exclude) 47.89 K - 1.38x slower +5.80 μs", + "lua (new) 46.57 K - 1.42x slower +6.40 μs" + ] + }, + "_cold_start": { + "first_call": "10070.0 us", + "second_call": "48.0 us" + } + }, + "encode_decode": { + "raw": "lua 0.4.0 — encode!/decode! decomposition\n(decode+deep_cast column: enabled)\n==================================================================================\nop shape N total_us per_elem_ns\nencode int_list 8 0.27 33.3\ndecode int_list 8 0.09 11.2\ndec+cast int_list 8 0.11 13.7\nencode int_list 64 1.61 25.1\ndecode int_list 64 0.54 8.5\ndec+cast int_list 64 0.96 15.0\nencode int_list 512 26.38 51.5\ndecode int_list 512 5.11 10.0\ndec+cast int_list 512 8.60 16.8\nencode int_list 4096 226.97 55.4\ndecode int_list 4096 43.47 10.6\ndec+cast int_list 4096 88.67 21.6\n----------------------------------------------------------------------------------\nencode float_list 8 0.22 27.7\ndecode float_list 8 0.09 11.2\ndec+cast float_list 8 0.11 13.2\nencode float_list 64 1.64 25.7\ndecode float_list 64 0.55 8.5\ndec+cast float_list 64 0.94 14.7\nencode float_list 512 27.00 52.7\ndecode float_list 512 5.85 11.4\ndec+cast float_list 512 9.00 17.6\nencode float_list 4096 239.42 58.5\ndecode float_list 4096 64.74 15.8\ndec+cast float_list 4096 98.40 24.0\n----------------------------------------------------------------------------------\nencode bool_list 8 0.21 26.4\ndecode bool_list 8 0.08 10.0\ndec+cast bool_list 8 0.10 12.6\nencode bool_list 64 1.60 25.0\ndecode bool_list 64 0.56 8.7\ndec+cast bool_list 64 0.97 15.1\nencode bool_list 512 25.68 50.1\ndecode bool_list 512 4.80 9.4\ndec+cast bool_list 512 8.38 16.4\nencode bool_list 4096 224.95 54.9\ndecode bool_list 4096 38.99 9.5\ndec+cast bool_list 4096 84.05 20.5\n----------------------------------------------------------------------------------\nencode short_string_list 8 0.21 26.7\ndecode short_string_list 8 0.08 10.5\ndec+cast short_string_list 8 0.10 12.9\nencode short_string_list 64 1.62 25.3\ndecode short_string_list 64 0.55 8.6\ndec+cast short_string_list 64 0.98 15.4\nencode short_string_list 512 26.45 51.7\ndecode short_string_list 512 4.86 9.5\ndec+cast short_string_list 512 9.05 17.7\nencode short_string_list 4096 202.93 49.5\ndecode short_string_list 4096 43.46 10.6\ndec+cast short_string_list 4096 73.06 17.8\n----------------------------------------------------------------------------------\nencode long_string_list 8 0.22 27.2\ndecode long_string_list 8 0.08 10.4\ndec+cast long_string_list 8 0.10 12.7\nencode long_string_list 64 1.67 26.1\ndecode long_string_list 64 0.59 9.2\ndec+cast long_string_list 64 1.03 16.1\nencode long_string_list 512 28.42 55.5\ndecode long_string_list 512 5.16 10.1\ndec+cast long_string_list 512 9.09 17.7\nencode long_string_list 4096 465.96 113.8\ndecode long_string_list 4096 54.39 13.3\ndec+cast long_string_list 4096 108.40 26.5\n----------------------------------------------------------------------------------\nencode string_map 8 0.52 65.3\ndecode string_map 8 0.07 8.2\ndec+cast string_map 8 0.17 21.1\nencode string_map 64 6.56 102.5\ndecode string_map 64 0.40 6.2\ndec+cast string_map 64 3.21 50.2\nencode string_map 512 113.10 220.9\ndecode string_map 512 4.27 8.3\ndec+cast string_map 512 39.39 76.9\nencode string_map 4096 1295.09 316.2\ndecode string_map 4096 42.02 10.3\ndec+cast string_map 4096 367.76 89.8\n----------------------------------------------------------------------------------\nencode int_map 8 0.25 31.2\ndecode int_map 8 0.09 10.7\ndec+cast int_map 8 0.10 12.9\nencode int_map 64 3.56 55.6\ndecode int_map 64 0.57 8.9\ndec+cast int_map 64 0.97 15.1\nencode int_map 512 51.12 99.8\ndecode int_map 512 4.79 9.4\ndec+cast int_map 512 8.76 17.1\nencode int_map 4096 443.17 108.2\ndecode int_map 4096 40.30 9.8\ndec+cast int_map 4096 87.24 21.3\n----------------------------------------------------------------------------------\nencode record_list 8 2.07 258.7\ndecode record_list 8 0.40 49.9\ndec+cast record_list 8 0.80 100.1\nencode record_list 64 31.54 492.8\ndecode record_list 64 4.54 70.9\ndec+cast record_list 64 7.32 114.4\nencode record_list 512 287.96 562.4\ndecode record_list 512 52.67 102.9\ndec+cast record_list 512 77.64 151.6\nencode record_list 4096 2346.38 572.8\ndecode record_list 4096 406.88 99.3\ndec+cast record_list 4096 658.70 160.8\n----------------------------------------------------------------------------------\nnested chain (depth sweep) — isolates recursion/traversal from fan-out\nop shape N total_us per_elem_ns\nencode nested_chain 4 0.51 257.3\ndecode nested_chain 4 0.12 61.7\ndec+cast nested_chain 4 0.24 118.7\nencode nested_chain 16 2.06 1032.0\ndecode nested_chain 16 0.63 314.9\ndec+cast nested_chain 16 1.10 550.2\nencode nested_chain 64 8.53 4265.3\ndecode nested_chain 64 4.16 2079.7\ndec+cast nested_chain 64 6.63 3317.0\nencode nested_chain 256 34.34 17170.4\ndecode nested_chain 256 37.82 18909.7\ndec+cast nested_chain 256 47.43 23716.3\n==================================================================================\ncomposite anchor — the PR's `original_nested` (matches the 18us/108us figure)\nop shape N total_us per_elem_ns\nencode original_nested 75 3.48 870.0\ndecode original_nested 75 0.58 145.4\ndec+cast original_nested 75 1.32 329.3\n" + } +} \ No newline at end of file diff --git a/bench_results/v0.4.0/table_ops.txt b/bench_results/v0.4.0/table_ops.txt new file mode 100644 index 0000000..dfa3ff5 --- /dev/null +++ b/bench_results/v0.4.0/table_ops.txt @@ -0,0 +1,476 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== Table Build (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (eval) 6.50 K 153.83 μs ±11.45% 151.96 μs 195.78 μs +luerl 6.44 K 155.20 μs ±14.07% 152.33 μs 259.75 μs +lua (chunk) 6.35 K 157.51 μs ±18.23% 149.92 μs 262.79 μs + +Comparison: +lua (eval) 6.50 K +luerl 6.44 K - 1.01x slower +1.37 μs +lua (chunk) 6.35 K - 1.02x slower +3.68 μs + +Memory usage statistics: + +Name Memory usage +lua (eval) 993.93 KB +luerl 996.96 KB - 1.00x memory usage +3.03 KB +lua (chunk) 983.38 KB - 0.99x memory usage -10.54688 KB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +luerl 59.29 K 16.87 μs ±38.96% 16.25 μs 26.79 μs +lua (eval) 51.13 K 19.56 μs ±34.91% 16.46 μs 41.71 μs +lua (chunk) 43.75 K 22.86 μs ±38.05% 25.96 μs 47.33 μs + +Comparison: +luerl 59.29 K +lua (eval) 51.13 K - 1.16x slower +2.69 μs +lua (chunk) 43.75 K - 1.36x slower +5.99 μs + +Memory usage statistics: + +Name Memory usage +luerl 110.78 KB +lua (eval) 111.19 KB - 1.00x memory usage +0.41 KB +lua (chunk) 100.13 KB - 0.90x memory usage -10.64844 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 405.22 K 2.47 μs ±243.17% 2 μs 5.83 μs +luerl 321.06 K 3.11 μs ±187.65% 3 μs 4.54 μs +lua (eval) 282.18 K 3.54 μs ±178.79% 3.13 μs 9.29 μs + +Comparison: +lua (chunk) 405.22 K +luerl 321.06 K - 1.26x slower +0.65 μs +lua (eval) 282.18 K - 1.44x slower +1.08 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 12.45 KB +luerl 22.62 KB - 1.82x memory usage +10.16 KB +lua (eval) 23.02 KB - 1.85x memory usage +10.57 KB + +**All measurements for memory usage were the same** + +=== Table Sort (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (chunk) 5.61 K 178.37 μs ±12.59% 174.29 μs 234.12 μs +lua (eval) 5.45 K 183.60 μs ±106.15% 177.67 μs 334.99 μs +luerl 4.88 K 205.11 μs ±25.14% 180.59 μs 336.78 μs + +Comparison: +lua (chunk) 5.61 K +lua (eval) 5.45 K - 1.03x slower +5.23 μs +luerl 4.88 K - 1.15x slower +26.74 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1.17 MB +lua (eval) 1.18 MB - 1.01x memory usage +0.0107 MB +luerl 1.18 MB - 1.01x memory usage +0.00999 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +luerl 51.03 K 19.60 μs ±20.90% 19.08 μs 28.42 μs +lua (chunk) 47.42 K 21.09 μs ±42.23% 18.04 μs 51.33 μs +lua (eval) 46.64 K 21.44 μs ±39.09% 19.33 μs 45.92 μs + +Comparison: +luerl 51.03 K +lua (chunk) 47.42 K - 1.08x slower +1.49 μs +lua (eval) 46.64 K - 1.09x slower +1.85 μs + +Memory usage statistics: + +Name Memory usage +luerl 133.34 KB +lua (chunk) 122.98 KB - 0.92x memory usage -10.35156 KB +lua (eval) 133.59 KB - 1.00x memory usage +0.25 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 324.41 K 3.08 μs ±194.26% 2.54 μs 7.08 μs +luerl 271.87 K 3.68 μs ±124.45% 3.54 μs 10 μs +lua (eval) 245.19 K 4.08 μs ±136.48% 3.67 μs 8.25 μs + +Comparison: +lua (chunk) 324.41 K +luerl 271.87 K - 1.19x slower +0.60 μs +lua (eval) 245.19 K - 1.32x slower +1.00 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 15.84 KB +luerl 25.98 KB - 1.64x memory usage +10.13 KB +lua (eval) 26.25 KB - 1.66x memory usage +10.41 KB + +**All measurements for memory usage were the same** + +=== Table Iterate/Sum (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (eval) 3.81 K 262.24 μs ±19.86% 243.42 μs 444.12 μs +luerl 3.75 K 266.52 μs ±20.59% 243.13 μs 416.21 μs +lua (chunk) 3.60 K 278.11 μs ±24.12% 242.42 μs 456.96 μs + +Comparison: +lua (eval) 3.81 K +luerl 3.75 K - 1.02x slower +4.28 μs +lua (chunk) 3.60 K - 1.06x slower +15.87 μs + +Memory usage statistics: + +Name Memory usage +lua (eval) 1.35 MB +luerl 1.35 MB - 1.00x memory usage -0.00027 MB +lua (chunk) 1.34 MB - 0.99x memory usage -0.01018 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (eval) 37.84 K 26.43 μs ±9.28% 26.08 μs 29.92 μs +luerl 37.50 K 26.66 μs ±86.90% 26.21 μs 36.50 μs +lua (chunk) 35.12 K 28.47 μs ±26.41% 25.17 μs 53.58 μs + +Comparison: +lua (eval) 37.84 K +luerl 37.50 K - 1.01x slower +0.24 μs +lua (chunk) 35.12 K - 1.08x slower +2.05 μs + +Memory usage statistics: + +Name Memory usage +lua (eval) 149.42 KB +luerl 149.28 KB - 1.00x memory usage -0.14063 KB +lua (chunk) 139.25 KB - 0.93x memory usage -10.17188 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 245.60 K 4.07 μs ±313.25% 3.29 μs 9.79 μs +luerl 227.18 K 4.40 μs ±102.94% 4.29 μs 10.88 μs +lua (eval) 206.12 K 4.85 μs ±120.13% 4.33 μs 9.83 μs + +Comparison: +lua (chunk) 245.60 K +luerl 227.18 K - 1.08x slower +0.33 μs +lua (eval) 206.12 K - 1.19x slower +0.78 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 16.70 KB +luerl 26.80 KB - 1.61x memory usage +10.10 KB +lua (eval) 27.20 KB - 1.63x memory usage +10.51 KB + +**All measurements for memory usage were the same** + +=== Table Map + Reduce (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (eval) 2.21 K 452.53 μs ±7.79% 446.25 μs 614.89 μs +lua (chunk) 2.19 K 455.66 μs ±9.09% 444.34 μs 628.64 μs +luerl 2.19 K 456.58 μs ±12.12% 442.21 μs 659.87 μs + +Comparison: +lua (eval) 2.21 K +lua (chunk) 2.19 K - 1.01x slower +3.14 μs +luerl 2.19 K - 1.01x slower +4.05 μs + +Memory usage statistics: + +Name Memory usage +lua (eval) 2.45 MB +lua (chunk) 2.43 MB - 1.00x memory usage -0.01137 MB +luerl 2.45 MB - 1.00x memory usage -0.00098 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 21.79 K 45.90 μs ±12.29% 45.25 μs 53.13 μs +luerl 21.47 K 46.58 μs ±5.77% 46.42 μs 52.54 μs +lua (eval) 21.44 K 46.65 μs ±6.16% 46.50 μs 51.96 μs + +Comparison: +lua (chunk) 21.79 K +luerl 21.47 K - 1.01x slower +0.68 μs +lua (eval) 21.44 K - 1.02x slower +0.75 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 252.38 KB +luerl 262.48 KB - 1.04x memory usage +10.11 KB +lua (eval) 263.02 KB - 1.04x memory usage +10.65 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +luerl 148.83 K 6.72 μs ±69.12% 6.58 μs 14.71 μs +lua (chunk) 145.39 K 6.88 μs ±90.79% 5.54 μs 14.96 μs +lua (eval) 131.69 K 7.59 μs ±88.05% 6.71 μs 16.42 μs + +Comparison: +luerl 148.83 K +lua (chunk) 145.39 K - 1.02x slower +0.159 μs +lua (eval) 131.69 K - 1.13x slower +0.87 μs + +Memory usage statistics: + +Name Memory usage +luerl 39.39 KB +lua (chunk) 29.06 KB - 0.74x memory usage -10.32813 KB +lua (eval) 39.80 KB - 1.01x memory usage +0.41 KB + +**All measurements for memory usage were the same** + +=== Table Pairs (hash) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (eval) 760.68 1.31 ms ±5.02% 1.32 ms 1.45 ms +lua (chunk) 759.91 1.32 ms ±5.40% 1.32 ms 1.49 ms +luerl 754.19 1.33 ms ±6.28% 1.33 ms 1.45 ms + +Comparison: +lua (eval) 760.68 +lua (chunk) 759.91 - 1.00x slower +0.00133 ms +luerl 754.19 - 1.01x slower +0.0113 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 2.48 MB +lua (chunk) 2.47 MB - 1.00x memory usage -0.01072 MB +luerl 2.48 MB - 1.00x memory usage +0.00007 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 10.96 K 91.23 μs ±12.10% 89.92 μs 135.19 μs +lua (eval) 10.20 K 97.99 μs ±11.84% 97.42 μs 138.25 μs +luerl 9.96 K 100.37 μs ±20.92% 99.96 μs 144.67 μs + +Comparison: +lua (chunk) 10.96 K +lua (eval) 10.20 K - 1.07x slower +6.76 μs +luerl 9.96 K - 1.10x slower +9.14 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 238.58 KB +lua (eval) 249.40 KB - 1.05x memory usage +10.82 KB +luerl 248.99 KB - 1.04x memory usage +10.41 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 126.80 K 7.89 μs ±150.76% 6.50 μs 18.79 μs +luerl 125.64 K 7.96 μs ±72.08% 7.29 μs 24.29 μs +lua (eval) 109.11 K 9.17 μs ±61.17% 9.38 μs 28.33 μs + +Comparison: +lua (chunk) 126.80 K +luerl 125.64 K - 1.01x slower +0.0728 μs +lua (eval) 109.11 K - 1.16x slower +1.28 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 25.83 KB +luerl 36.16 KB - 1.40x memory usage +10.33 KB +lua (eval) 36.56 KB - 1.42x memory usage +10.73 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v0.4.0/timestamp.txt b/bench_results/v0.4.0/timestamp.txt new file mode 100644 index 0000000..0c188fe --- /dev/null +++ b/bench_results/v0.4.0/timestamp.txt @@ -0,0 +1 @@ +2026-07-28T13:59:36Z diff --git a/bench_results/v0.4.0/versions.txt b/bench_results/v0.4.0/versions.txt new file mode 100644 index 0000000..184e5f9 --- /dev/null +++ b/bench_results/v0.4.0/versions.txt @@ -0,0 +1,3 @@ +Erlang/OTP 29 [erts-17.0] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] + +Elixir 1.20.0 (compiled with Erlang/OTP 29) diff --git a/bench_results/v0.4.0/vm_new.txt b/bench_results/v0.4.0/vm_new.txt new file mode 100644 index 0000000..f09f0f4 --- /dev/null +++ b/bench_results/v0.4.0/vm_new.txt @@ -0,0 +1,72 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) + │ + 177 │ defp format_error(_, {:undefined, args}) do + │ ~ + │ + └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 + + +=== VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full) === + +Lua.new() one-time vs repeat cost (single samples, informational): + first call on this node : 10070.0 us + second call : 48.0 us + +The first figure includes any one-time template build and first-time module +loading. Benchee's steady-state numbers below are the per-request cost after +that point. + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 52 s +Excluding outliers: false + +Benchmarking lua (new) ... +Benchmarking lua (new, custom exclude) ... +Benchmarking lua (new, no sandbox) ... +Benchmarking luerl (init) ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (new, no sandbox) 66.32 K 15.08 μs ±21.55% 14.88 μs 22.58 μs +luerl (init) 64.85 K 15.42 μs ±46.27% 15.04 μs 30.25 μs +lua (new, custom exclude) 47.89 K 20.88 μs ±63.00% 20.63 μs 31.67 μs +lua (new) 46.57 K 21.47 μs ±15.57% 21.33 μs 31.25 μs + +Comparison: +lua (new, no sandbox) 66.32 K +luerl (init) 64.85 K - 1.02x slower +0.34 μs +lua (new, custom exclude) 47.89 K - 1.38x slower +5.80 μs +lua (new) 46.57 K - 1.42x slower +6.40 μs + +Memory usage statistics: + +Name Memory usage +lua (new, no sandbox) 51.77 KB +luerl (init) 51.64 KB - 1.00x memory usage -0.12500 KB +lua (new, custom exclude) 72.27 KB - 1.40x memory usage +20.51 KB +lua (new) 73.78 KB - 1.43x memory usage +22.02 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/closures.txt b/bench_results/v1.0.0/closures.txt new file mode 100644 index 0000000..c5e8d5b --- /dev/null +++ b/bench_results/v1.0.0/closures.txt @@ -0,0 +1,54 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 2.58 K 388.21 μs ±7.60% 382.50 μs 546.24 μs +lua (chunk) 2.09 K 478.08 μs ±6.68% 473.54 μs 613.46 μs +lua (eval) 2.01 K 498.43 μs ±16.91% 482.54 μs 1019.38 μs + +Comparison: +luerl 2.58 K +lua (chunk) 2.09 K - 1.23x slower +89.87 μs +lua (eval) 2.01 K - 1.28x slower +110.23 μs + +Memory usage statistics: + +Name average deviation median 99th % +luerl 1.90 MB ±0.00% 1.90 MB 1.90 MB +lua (chunk) 2.65 MB ±0.08% 2.65 MB 2.65 MB +lua (eval) 2.65 MB ±0.09% 2.65 MB 2.66 MB + +Comparison: +luerl 1.90 MB +lua (chunk) 2.65 MB - 1.39x memory usage +0.75 MB +lua (eval) 2.65 MB - 1.40x memory usage +0.76 MB diff --git a/bench_results/v1.0.0/cpu.txt b/bench_results/v1.0.0/cpu.txt new file mode 100644 index 0000000..de5c8ad --- /dev/null +++ b/bench_results/v1.0.0/cpu.txt @@ -0,0 +1 @@ +Apple M4 diff --git a/bench_results/v1.0.0/encode_decode.txt b/bench_results/v1.0.0/encode_decode.txt new file mode 100644 index 0000000..da298cb --- /dev/null +++ b/bench_results/v1.0.0/encode_decode.txt @@ -0,0 +1,136 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +lua 1.0.0 — encode!/decode! decomposition +(decode+deep_cast column: enabled) +================================================================================== +op shape N total_us per_elem_ns +encode int_list 8 0.90 113.0 +decode int_list 8 0.28 34.7 +dec+cast int_list 8 0.31 38.5 +encode int_list 64 13.52 211.2 +decode int_list 64 4.23 66.0 +dec+cast int_list 64 7.14 111.6 +encode int_list 512 151.37 295.6 +decode int_list 512 48.44 94.6 +dec+cast int_list 512 73.99 144.5 +encode int_list 4096 1601.11 390.9 +decode int_list 4096 583.70 142.5 +dec+cast int_list 4096 785.61 191.8 +---------------------------------------------------------------------------------- +encode float_list 8 0.78 97.1 +decode float_list 8 0.24 29.8 +dec+cast float_list 8 0.26 32.9 +encode float_list 64 11.47 179.2 +decode float_list 64 3.55 55.4 +dec+cast float_list 64 7.03 109.9 +encode float_list 512 156.43 305.5 +decode float_list 512 45.74 89.3 +dec+cast float_list 512 72.74 142.1 +encode float_list 4096 1611.38 393.4 +decode float_list 4096 551.08 134.5 +dec+cast float_list 4096 839.19 204.9 +---------------------------------------------------------------------------------- +encode bool_list 8 0.77 96.7 +decode bool_list 8 0.23 28.7 +dec+cast bool_list 8 0.25 31.3 +encode bool_list 64 11.21 175.2 +decode bool_list 64 3.46 54.1 +dec+cast bool_list 64 7.00 109.4 +encode bool_list 512 153.74 300.3 +decode bool_list 512 47.42 92.6 +dec+cast bool_list 512 74.01 144.6 +encode bool_list 4096 1569.25 383.1 +decode bool_list 4096 571.91 139.6 +dec+cast bool_list 4096 775.34 189.3 +---------------------------------------------------------------------------------- +encode short_string_list 8 0.78 96.9 +decode short_string_list 8 0.24 29.8 +dec+cast short_string_list 8 0.26 32.6 +encode short_string_list 64 11.40 178.1 +decode short_string_list 64 3.53 55.2 +dec+cast short_string_list 64 7.08 110.6 +encode short_string_list 512 155.05 302.8 +decode short_string_list 512 49.88 97.4 +dec+cast short_string_list 512 72.83 142.2 +encode short_string_list 4096 1715.16 418.7 +decode short_string_list 4096 718.28 175.4 +dec+cast short_string_list 4096 763.05 186.3 +---------------------------------------------------------------------------------- +encode long_string_list 8 0.78 96.9 +decode long_string_list 8 0.24 30.5 +dec+cast long_string_list 8 0.26 33.0 +encode long_string_list 64 11.20 174.9 +decode long_string_list 64 3.49 54.5 +dec+cast long_string_list 64 7.31 114.3 +encode long_string_list 512 157.18 307.0 +decode long_string_list 512 42.76 83.5 +dec+cast long_string_list 512 76.03 148.5 +encode long_string_list 4096 2488.38 607.5 +decode long_string_list 4096 1495.20 365.0 +dec+cast long_string_list 4096 1707.00 416.7 +---------------------------------------------------------------------------------- +encode string_map 8 1.60 199.5 +decode string_map 8 0.10 12.4 +dec+cast string_map 8 0.20 25.5 +encode string_map 64 30.47 476.1 +decode string_map 64 0.74 11.6 +dec+cast string_map 64 3.95 61.7 +encode string_map 512 240.62 470.0 +decode string_map 512 11.04 21.6 +dec+cast string_map 512 39.76 77.7 +encode string_map 4096 2194.00 535.6 +decode string_map 4096 71.08 17.4 +dec+cast string_map 4096 335.83 82.0 +---------------------------------------------------------------------------------- +encode int_map 8 0.83 103.4 +decode int_map 8 0.24 29.8 +dec+cast int_map 8 0.26 32.2 +encode int_map 64 10.97 171.5 +decode int_map 64 3.68 57.5 +dec+cast int_map 64 5.96 93.1 +encode int_map 512 109.35 213.6 +decode int_map 512 27.78 54.3 +dec+cast int_map 512 72.41 141.4 +encode int_map 4096 1307.14 319.1 +decode int_map 4096 576.06 140.6 +dec+cast int_map 4096 881.09 215.1 +---------------------------------------------------------------------------------- +encode record_list 8 5.07 633.8 +decode record_list 8 0.78 98.1 +dec+cast record_list 8 1.20 149.9 +encode record_list 64 47.83 747.4 +decode record_list 64 8.26 129.1 +dec+cast record_list 64 14.64 228.8 +encode record_list 512 565.88 1105.2 +decode record_list 512 118.02 230.5 +dec+cast record_list 512 196.29 383.4 +encode record_list 4096 4732.25 1155.3 +decode record_list 4096 1094.36 267.2 +dec+cast record_list 4096 1628.34 397.5 +---------------------------------------------------------------------------------- +nested chain (depth sweep) — isolates recursion/traversal from fan-out +op shape N total_us per_elem_ns +encode nested_chain 4 1.07 535.2 +decode nested_chain 4 0.21 103.7 +dec+cast nested_chain 4 0.32 161.9 +encode nested_chain 16 4.19 2094.1 +decode nested_chain 16 0.91 456.5 +dec+cast nested_chain 16 1.33 665.1 +encode nested_chain 64 16.94 8469.5 +decode nested_chain 64 3.92 1961.9 +dec+cast nested_chain 64 6.28 3140.2 +encode nested_chain 256 68.77 34386.2 +decode nested_chain 256 18.23 9114.9 +dec+cast nested_chain 256 32.31 16156.7 +================================================================================== +composite anchor — the PR's `original_nested` (matches the 18us/108us figure) +op shape N total_us per_elem_ns +encode original_nested 75 17.11 4276.4 +decode original_nested 75 3.21 802.7 +dec+cast original_nested 75 5.26 1314.8 diff --git a/bench_results/v1.0.0/environment.md b/bench_results/v1.0.0/environment.md new file mode 100644 index 0000000..bd4c848 --- /dev/null +++ b/bench_results/v1.0.0/environment.md @@ -0,0 +1,36 @@ +# Benchmark environment — v1.0.0 + +- **Ref**: `v1.0.0` (commit `69e13a6`) +- **Mode**: `full` (`LUA_BENCH_MODE=full`) +- **CPU**: Apple M4 (see `cpu.txt`) +- **Elixir / OTP**: Elixir 1.20.0, Erlang/OTP 29 [erts-17.0] [64-bit] [jit] (see `versions.txt`) +- **Worktree setup timestamp**: Tue Jul 28 10:48:16 EDT 2026 (see `timestamp.txt`) +- **Run date (this document)**: Tue Jul 28 11:20:58 EDT 2026 + +## Command form + +Each workload was run serially, one process per file, on an otherwise quiet +machine, from the worktree root: + +``` +LUA_BENCH_MODE=full MIX_ENV=benchmark mix run benchmarks/.exs +``` + +Workloads run: `fibonacci`, `closures`, `oop`, `string_ops`, `string_format`, +`table_ops`, `patterns`, `metamethods`, `pcall_varargs`, `vm_new`, +`encode_decode`. + +`encode_decode.exs` is not Benchee-based (it uses a `:timer.tc` harness +directly), but was invoked with the same command form for consistency. + +C Lua via `luaport` was not available in this environment (no local luaport +build) and was skipped by each script's own fallback path; all comparisons +below are lua (chunk)/lua (eval) vs. luerl only. + +## Artifacts + +- `cpu.txt`, `versions.txt`, `timestamp.txt` — raw environment probes captured + at worktree setup time. +- `.txt` — raw stdout of each `mix run` invocation (11 files). +- `summary.json` — parsed structured form of the above (workload -> case -> + jobs/comparison/memory). diff --git a/bench_results/v1.0.0/fibonacci.txt b/bench_results/v1.0.0/fibonacci.txt new file mode 100644 index 0000000..09c735a --- /dev/null +++ b/bench_results/v1.0.0/fibonacci.txt @@ -0,0 +1,51 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 1.37 730.63 ms ±0.64% 730.11 ms 743.77 ms +lua (chunk) 1.26 794.25 ms ±1.02% 792.42 ms 811.45 ms +lua (eval) 1.24 807.88 ms ±1.38% 805.75 ms 831.46 ms + +Comparison: +luerl 1.37 +lua (chunk) 1.26 - 1.09x slower +63.62 ms +lua (eval) 1.24 - 1.11x slower +77.25 ms + +Memory usage statistics: + +Name Memory usage +luerl 2.45 GB +lua (chunk) 2.90 GB - 1.18x memory usage +0.44 GB +lua (eval) 2.90 GB - 1.18x memory usage +0.44 GB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/metamethods.txt b/bench_results/v1.0.0/metamethods.txt new file mode 100644 index 0000000..911f422 --- /dev/null +++ b/bench_results/v1.0.0/metamethods.txt @@ -0,0 +1,144 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== metamethods: self-call method dispatch (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 2.06 K 484.65 μs ±9.07% 472.63 μs 658.27 μs +lua (eval) 1.77 K 565.06 μs ±8.07% 552.29 μs 755.54 μs +lua (chunk) 1.75 K 569.99 μs ±12.20% 553.75 μs 849.77 μs + +Comparison: +luerl 2.06 K +lua (eval) 1.77 K - 1.17x slower +80.41 μs +lua (chunk) 1.75 K - 1.18x slower +85.34 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.43 MB +lua (eval) 1.85 MB - 1.30x memory usage +0.43 MB +lua (chunk) 1.84 MB - 1.29x memory usage +0.42 MB + +**All measurements for memory usage were the same** + +=== metamethods: 3-level __index chain (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 4.19 K 238.51 μs ±3.78% 237.13 μs 257.04 μs +lua (eval) 4.01 K 249.27 μs ±5.91% 248.25 μs 273.67 μs +luerl 3.95 K 253.10 μs ±42.54% 221.29 μs 343.51 μs + +Comparison: +lua (chunk) 4.19 K +lua (eval) 4.01 K - 1.05x slower +10.76 μs +luerl 3.95 K - 1.06x slower +14.59 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 788.91 KB +lua (eval) 798.82 KB - 1.01x memory usage +9.91 KB +luerl 668.59 KB - 0.85x memory usage -120.32031 KB + +**All measurements for memory usage were the same** + +=== metamethods: arithmetic/relational metamethods (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 523.21 1.91 ms ±4.47% 1.88 ms 2.20 ms +lua (chunk) 455.21 2.20 ms ±4.68% 2.23 ms 2.54 ms +lua (eval) 453.68 2.20 ms ±6.62% 2.22 ms 2.63 ms + +Comparison: +luerl 523.21 +lua (chunk) 455.21 - 1.15x slower +0.29 ms +lua (eval) 453.68 - 1.15x slower +0.29 ms + +Memory usage statistics: + +Name Memory usage +luerl 6.24 MB +lua (chunk) 7.07 MB - 1.13x memory usage +0.83 MB +lua (eval) 7.08 MB - 1.13x memory usage +0.84 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/oop.txt b/bench_results/v1.0.0/oop.txt new file mode 100644 index 0000000..0b8ca06 --- /dev/null +++ b/bench_results/v1.0.0/oop.txt @@ -0,0 +1,51 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 8.38 K 119.27 μs ±16.72% 116.13 μs 217.71 μs +lua (chunk) 7.88 K 126.88 μs ±17.97% 123.50 μs 232.21 μs +lua (eval) 7.59 K 131.77 μs ±16.80% 126.29 μs 228.05 μs + +Comparison: +luerl 8.38 K +lua (chunk) 7.88 K - 1.06x slower +7.62 μs +lua (eval) 7.59 K - 1.10x slower +12.50 μs + +Memory usage statistics: + +Name Memory usage +luerl 381.45 KB +lua (chunk) 502.34 KB - 1.32x memory usage +120.89 KB +lua (eval) 511.52 KB - 1.34x memory usage +130.07 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/patterns.txt b/bench_results/v1.0.0/patterns.txt new file mode 100644 index 0000000..43d7a99 --- /dev/null +++ b/bench_results/v1.0.0/patterns.txt @@ -0,0 +1,147 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== patterns: find/match field extraction (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 972.71 1.03 ms ±9.96% 1.01 ms 1.24 ms +lua (eval) 970.99 1.03 ms ±3.16% 1.02 ms 1.13 ms +luerl 906.75 1.10 ms ±5.98% 1.09 ms 1.26 ms + +Comparison: +lua (chunk) 972.71 +lua (eval) 970.99 - 1.00x slower +0.00182 ms +luerl 906.75 - 1.07x slower +0.0748 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 3.72 MB +lua (eval) 3.73 MB - 1.00x memory usage +0.00805 MB +luerl 6.70 MB - 1.80x memory usage +2.98 MB + +**All measurements for memory usage were the same** + +=== patterns: find-based tokenizer (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 733.20 1.36 ms ±5.05% 1.35 ms 1.48 ms +lua (chunk) 626.48 1.60 ms ±4.81% 1.57 ms 1.73 ms +lua (eval) 611.43 1.64 ms ±8.70% 1.61 ms 1.99 ms + +Comparison: +luerl 733.20 +lua (chunk) 626.48 - 1.17x slower +0.23 ms +lua (eval) 611.43 - 1.20x slower +0.27 ms + +Memory usage statistics: + +Name average deviation median 99th % +luerl 6.65 MB ±0.00% 6.65 MB 6.65 MB +lua (chunk) 10.11 MB ±0.00% 10.11 MB 10.11 MB +lua (eval) 10.12 MB ±0.00% 10.12 MB 10.12 MB + +Comparison: +luerl 6.65 MB +lua (chunk) 10.11 MB - 1.52x memory usage +3.46 MB +lua (eval) 10.12 MB - 1.52x memory usage +3.48 MB + +=== patterns: gsub template substitution (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 491.94 2.03 ms ±2.07% 2.02 ms 2.14 ms +lua (eval) 488.05 2.05 ms ±4.20% 2.04 ms 2.22 ms +luerl 450.89 2.22 ms ±4.50% 2.17 ms 2.47 ms + +Comparison: +lua (chunk) 491.94 +lua (eval) 488.05 - 1.01x slower +0.0162 ms +luerl 450.89 - 1.09x slower +0.185 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 5.05 MB +lua (eval) 5.06 MB - 1.00x memory usage +0.00848 MB +luerl 11.75 MB - 2.33x memory usage +6.71 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/pcall_varargs.txt b/bench_results/v1.0.0/pcall_varargs.txt new file mode 100644 index 0000000..01b5211 --- /dev/null +++ b/bench_results/v1.0.0/pcall_varargs.txt @@ -0,0 +1,144 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== call protocol: pcall, success path (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 3.49 K 286.44 μs ±12.19% 282.13 μs 352.91 μs +luerl 3.44 K 290.86 μs ±3.49% 289.50 μs 309.63 μs +lua (eval) 3.34 K 299.49 μs ±5.85% 297.75 μs 329.07 μs + +Comparison: +lua (chunk) 3.49 K +luerl 3.44 K - 1.02x slower +4.42 μs +lua (eval) 3.34 K - 1.05x slower +13.05 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1.31 MB +luerl 1.11 MB - 0.85x memory usage -0.20367 MB +lua (eval) 1.32 MB - 1.01x memory usage +0.00981 MB + +**All measurements for memory usage were the same** + +=== call protocol: pcall, raise + catch (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 1.87 K 533.92 μs ±29.41% 485.38 μs 1138.98 μs +lua (chunk) 1.11 K 904.68 μs ±17.56% 877.33 μs 1891.77 μs +lua (eval) 1.00 K 1001.11 μs ±31.79% 899.42 μs 2144.79 μs + +Comparison: +luerl 1.87 K +lua (chunk) 1.11 K - 1.69x slower +370.76 μs +lua (eval) 1.00 K - 1.88x slower +467.19 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.55 MB +lua (chunk) 3.35 MB - 2.15x memory usage +1.79 MB +lua (eval) 3.36 MB - 2.16x memory usage +1.81 MB + +**All measurements for memory usage were the same** + +=== call protocol: varargs + multiple returns (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 410.14 2.44 ms ±4.02% 2.40 ms 2.73 ms +lua (eval) 253.78 3.94 ms ±6.97% 3.89 ms 4.79 ms +lua (chunk) 237.27 4.21 ms ±16.27% 4.01 ms 8.17 ms + +Comparison: +luerl 410.14 +lua (eval) 253.78 - 1.62x slower +1.50 ms +lua (chunk) 237.27 - 1.73x slower +1.78 ms + +Memory usage statistics: + +Name Memory usage +luerl 8.73 MB +lua (eval) 19.95 MB - 2.29x memory usage +11.22 MB +lua (chunk) 19.94 MB - 2.28x memory usage +11.21 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/string_format.txt b/bench_results/v1.0.0/string_format.txt new file mode 100644 index 0000000..7465780 --- /dev/null +++ b/bench_results/v1.0.0/string_format.txt @@ -0,0 +1,147 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== string.format: long literal-heavy format string (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 1063.59 0.94 ms ±10.28% 0.90 ms 1.17 ms +lua (chunk) 966.41 1.03 ms ±6.22% 1.03 ms 1.16 ms +luerl 241.01 4.15 ms ±10.25% 4.32 ms 4.71 ms + +Comparison: +lua (eval) 1063.59 +lua (chunk) 966.41 - 1.10x slower +0.0945 ms +luerl 241.01 - 4.41x slower +3.21 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 3.27 MB +lua (chunk) 3.26 MB - 1.00x memory usage -0.00974 MB +luerl 22.59 MB - 6.92x memory usage +19.32 MB + +**All measurements for memory usage were the same** + +=== string.format: width-flagged specifiers (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 668.09 1.50 ms ±2.73% 1.49 ms 1.62 ms +lua (chunk) 662.60 1.51 ms ±4.32% 1.50 ms 1.63 ms +luerl 549.59 1.82 ms ±5.86% 1.79 ms 2.09 ms + +Comparison: +lua (eval) 668.09 +lua (chunk) 662.60 - 1.01x slower +0.0124 ms +luerl 549.59 - 1.22x slower +0.32 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 5.22 MB +lua (chunk) 5.21 MB - 1.00x memory usage -0.01022 MB +luerl 7.55 MB - 1.45x memory usage +2.33 MB + +**All measurements for memory usage were the same** + +=== string.format: many specifiers (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 418.38 2.39 ms ±8.03% 2.39 ms 3.24 ms +lua (chunk) 413.90 2.42 ms ±8.21% 2.35 ms 3.05 ms +luerl 358.58 2.79 ms ±4.13% 2.82 ms 3.00 ms + +Comparison: +lua (eval) 418.38 +lua (chunk) 413.90 - 1.01x slower +0.0259 ms +luerl 358.58 - 1.17x slower +0.40 ms + +Memory usage statistics: + +Name average deviation median 99th % +lua (eval) 14.86 MB ±0.00% 14.86 MB 14.86 MB +lua (chunk) 14.86 MB ±0.00% 14.86 MB 14.86 MB +luerl 13.76 MB ±0.00% 13.76 MB 13.76 MB + +Comparison: +lua (eval) 14.86 MB +lua (chunk) 14.86 MB - 1.00x memory usage -0.00894 MB +luerl 13.76 MB - 0.93x memory usage -1.10735 MB diff --git a/bench_results/v1.0.0/string_ops.txt b/bench_results/v1.0.0/string_ops.txt new file mode 100644 index 0000000..adb575e --- /dev/null +++ b/bench_results/v1.0.0/string_ops.txt @@ -0,0 +1,99 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== String Concatenation via table.concat (n=100) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 27.97 K 35.76 μs ±9.62% 35.67 μs 41.97 μs +lua (eval) 25.79 K 38.77 μs ±11.58% 38.58 μs 45.21 μs +luerl 24.73 K 40.44 μs ±7.76% 40.17 μs 46.08 μs + +Comparison: +lua (chunk) 27.97 K +lua (eval) 25.79 K - 1.08x slower +3.01 μs +luerl 24.73 K - 1.13x slower +4.68 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 171.33 KB +lua (eval) 180.13 KB - 1.05x memory usage +8.80 KB +luerl 172.42 KB - 1.01x memory usage +1.09 KB + +**All measurements for memory usage were the same** + +=== String Formatting via string.format (n=100) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 12.44 K 80.35 μs ±7.41% 80.21 μs 93.29 μs +lua (eval) 11.53 K 86.74 μs ±27.72% 85.58 μs 111.49 μs +luerl 9.55 K 104.75 μs ±11.48% 103.38 μs 122.63 μs + +Comparison: +lua (chunk) 12.44 K +lua (eval) 11.53 K - 1.08x slower +6.39 μs +luerl 9.55 K - 1.30x slower +24.39 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 333.77 KB +lua (eval) 343.45 KB - 1.03x memory usage +9.68 KB +luerl 588.84 KB - 1.76x memory usage +255.07 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/summary.json b/bench_results/v1.0.0/summary.json new file mode 100644 index 0000000..09df202 --- /dev/null +++ b/bench_results/v1.0.0/summary.json @@ -0,0 +1,1290 @@ +{ + "fibonacci": { + "default": { + "jobs": [ + { + "name": "luerl", + "ips": "1.37", + "average": "730.63 ms", + "deviation": "±0.64%", + "median": "730.11 ms", + "p99": "743.77 ms", + "memory": "2.45 GB" + }, + { + "name": "lua (chunk)", + "ips": "1.26", + "average": "794.25 ms", + "deviation": "±1.02%", + "median": "792.42 ms", + "p99": "811.45 ms", + "memory": "2.90 GB" + }, + { + "name": "lua (eval)", + "ips": "1.24", + "average": "807.88 ms", + "deviation": "±1.38%", + "median": "805.75 ms", + "p99": "831.46 ms", + "memory": "2.90 GB" + } + ], + "comparison": [ + "luerl 1.37", + "lua (chunk) 1.26 - 1.09x slower +63.62 ms", + "lua (eval) 1.24 - 1.11x slower +77.25 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "closures": { + "default": { + "jobs": [ + { + "name": "luerl", + "ips": "2.58 K", + "average": "388.21 μs", + "deviation": "±7.60%", + "median": "382.50 μs", + "p99": "546.24 μs", + "memory": "1.90 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.09 K", + "average": "478.08 μs", + "deviation": "±6.68%", + "median": "473.54 μs", + "p99": "613.46 μs", + "memory": "2.65 MB" + }, + { + "name": "lua (eval)", + "ips": "2.01 K", + "average": "498.43 μs", + "deviation": "±16.91%", + "median": "482.54 μs", + "p99": "1019.38 μs", + "memory": "2.65 MB" + } + ], + "comparison": [ + "luerl 2.58 K", + "lua (chunk) 2.09 K - 1.23x slower +89.87 μs", + "lua (eval) 2.01 K - 1.28x slower +110.23 μs" + ], + "memory_comparison": [ + "luerl 1.90 MB", + "lua (chunk) 2.65 MB - 1.39x memory usage +0.75 MB", + "lua (eval) 2.65 MB - 1.40x memory usage +0.76 MB" + ] + } + }, + "oop": { + "default": { + "jobs": [ + { + "name": "luerl", + "ips": "8.38 K", + "average": "119.27 μs", + "deviation": "±16.72%", + "median": "116.13 μs", + "p99": "217.71 μs", + "memory": "381.45 KB" + }, + { + "name": "lua (chunk)", + "ips": "7.88 K", + "average": "126.88 μs", + "deviation": "±17.97%", + "median": "123.50 μs", + "p99": "232.21 μs", + "memory": "502.34 KB" + }, + { + "name": "lua (eval)", + "ips": "7.59 K", + "average": "131.77 μs", + "deviation": "±16.80%", + "median": "126.29 μs", + "p99": "228.05 μs", + "memory": "511.52 KB" + } + ], + "comparison": [ + "luerl 8.38 K", + "lua (chunk) 7.88 K - 1.06x slower +7.62 μs", + "lua (eval) 7.59 K - 1.10x slower +12.50 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "string_ops": { + "String Concatenation via table.concat (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "27.97 K", + "average": "35.76 μs", + "deviation": "±9.62%", + "median": "35.67 μs", + "p99": "41.97 μs", + "memory": "171.33 KB" + }, + { + "name": "lua (eval)", + "ips": "25.79 K", + "average": "38.77 μs", + "deviation": "±11.58%", + "median": "38.58 μs", + "p99": "45.21 μs", + "memory": "180.13 KB" + }, + { + "name": "luerl", + "ips": "24.73 K", + "average": "40.44 μs", + "deviation": "±7.76%", + "median": "40.17 μs", + "p99": "46.08 μs", + "memory": "172.42 KB" + } + ], + "comparison": [ + "lua (chunk) 27.97 K", + "lua (eval) 25.79 K - 1.08x slower +3.01 μs", + "luerl 24.73 K - 1.13x slower +4.68 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "String Formatting via string.format (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "12.44 K", + "average": "80.35 μs", + "deviation": "±7.41%", + "median": "80.21 μs", + "p99": "93.29 μs", + "memory": "333.77 KB" + }, + { + "name": "lua (eval)", + "ips": "11.53 K", + "average": "86.74 μs", + "deviation": "±27.72%", + "median": "85.58 μs", + "p99": "111.49 μs", + "memory": "343.45 KB" + }, + { + "name": "luerl", + "ips": "9.55 K", + "average": "104.75 μs", + "deviation": "±11.48%", + "median": "103.38 μs", + "p99": "122.63 μs", + "memory": "588.84 KB" + } + ], + "comparison": [ + "lua (chunk) 12.44 K", + "lua (eval) 11.53 K - 1.08x slower +6.39 μs", + "luerl 9.55 K - 1.30x slower +24.39 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "string_format": { + "string.format: long literal-heavy format string (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "1063.59", + "average": "0.94 ms", + "deviation": "±10.28%", + "median": "0.90 ms", + "p99": "1.17 ms", + "memory": "3.27 MB" + }, + { + "name": "lua (chunk)", + "ips": "966.41", + "average": "1.03 ms", + "deviation": "±6.22%", + "median": "1.03 ms", + "p99": "1.16 ms", + "memory": "3.26 MB" + }, + { + "name": "luerl", + "ips": "241.01", + "average": "4.15 ms", + "deviation": "±10.25%", + "median": "4.32 ms", + "p99": "4.71 ms", + "memory": "22.59 MB" + } + ], + "comparison": [ + "lua (eval) 1063.59", + "lua (chunk) 966.41 - 1.10x slower +0.0945 ms", + "luerl 241.01 - 4.41x slower +3.21 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "string.format: width-flagged specifiers (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "668.09", + "average": "1.50 ms", + "deviation": "±2.73%", + "median": "1.49 ms", + "p99": "1.62 ms", + "memory": "5.22 MB" + }, + { + "name": "lua (chunk)", + "ips": "662.60", + "average": "1.51 ms", + "deviation": "±4.32%", + "median": "1.50 ms", + "p99": "1.63 ms", + "memory": "5.21 MB" + }, + { + "name": "luerl", + "ips": "549.59", + "average": "1.82 ms", + "deviation": "±5.86%", + "median": "1.79 ms", + "p99": "2.09 ms", + "memory": "7.55 MB" + } + ], + "comparison": [ + "lua (eval) 668.09", + "lua (chunk) 662.60 - 1.01x slower +0.0124 ms", + "luerl 549.59 - 1.22x slower +0.32 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "string.format: many specifiers (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "418.38", + "average": "2.39 ms", + "deviation": "±8.03%", + "median": "2.39 ms", + "p99": "3.24 ms", + "memory": "14.86 MB" + }, + { + "name": "lua (chunk)", + "ips": "413.90", + "average": "2.42 ms", + "deviation": "±8.21%", + "median": "2.35 ms", + "p99": "3.05 ms", + "memory": "14.86 MB" + }, + { + "name": "luerl", + "ips": "358.58", + "average": "2.79 ms", + "deviation": "±4.13%", + "median": "2.82 ms", + "p99": "3.00 ms", + "memory": "13.76 MB" + } + ], + "comparison": [ + "lua (eval) 418.38", + "lua (chunk) 413.90 - 1.01x slower +0.0259 ms", + "luerl 358.58 - 1.17x slower +0.40 ms" + ], + "memory_comparison": [ + "lua (eval) 14.86 MB", + "lua (chunk) 14.86 MB - 1.00x memory usage -0.00894 MB", + "luerl 13.76 MB - 0.93x memory usage -1.10735 MB" + ] + } + }, + "table_ops": { + "Table Build": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "luerl", + "ips": "6.37 K", + "average": "157.04 μs", + "deviation": "±11.82%", + "median": "154.21 μs", + "p99": "218.16 μs", + "memory": "0.97 MB" + }, + { + "name": "lua (chunk)", + "ips": "6.23 K", + "average": "160.45 μs", + "deviation": "±8.91%", + "median": "157.88 μs", + "p99": "198.69 μs", + "memory": "1.00 MB" + }, + { + "name": "lua (eval)", + "ips": "5.90 K", + "average": "169.48 μs", + "deviation": "±14.14%", + "median": "164.58 μs", + "p99": "287.61 μs", + "memory": "1.01 MB" + } + ], + "comparison": [ + "luerl 6.37 K", + "lua (chunk) 6.23 K - 1.02x slower +3.41 μs", + "lua (eval) 5.90 K - 1.08x slower +12.44 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "60.98 K", + "average": "16.40 μs", + "deviation": "±32.33%", + "median": "16.17 μs", + "p99": "24.29 μs", + "memory": "105.41 KB" + }, + { + "name": "luerl", + "ips": "58.70 K", + "average": "17.04 μs", + "deviation": "±26.63%", + "median": "16.50 μs", + "p99": "26.67 μs", + "memory": "110.77 KB" + }, + { + "name": "lua (eval)", + "ips": "53.06 K", + "average": "18.85 μs", + "deviation": "±33.45%", + "median": "18.42 μs", + "p99": "27.04 μs", + "memory": "114.74 KB" + } + ], + "comparison": [ + "lua (chunk) 60.98 K", + "luerl 58.70 K - 1.04x slower +0.64 μs", + "lua (eval) 53.06 K - 1.15x slower +2.45 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "437.49 K", + "average": "2.29 μs", + "deviation": "±237.41%", + "median": "2.25 μs", + "p99": "3.42 μs", + "memory": "13.80 KB" + }, + { + "name": "luerl", + "ips": "317.31 K", + "average": "3.15 μs", + "deviation": "±243.81%", + "median": "3.04 μs", + "p99": "4.71 μs", + "memory": "22.62 KB" + }, + { + "name": "lua (eval)", + "ips": "205.32 K", + "average": "4.87 μs", + "deviation": "±231.75%", + "median": "4.46 μs", + "p99": "10.42 μs", + "memory": "23.45 KB" + } + ], + "comparison": [ + "lua (chunk) 437.49 K", + "luerl 317.31 K - 1.38x slower +0.87 μs", + "lua (eval) 205.32 K - 2.13x slower +2.58 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Sort": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "luerl", + "ips": "5.52 K", + "average": "181.29 μs", + "deviation": "±10.01%", + "median": "178.00 μs", + "p99": "220.42 μs", + "memory": "1.18 MB" + }, + { + "name": "lua (chunk)", + "ips": "4.27 K", + "average": "234.42 μs", + "deviation": "±13.69%", + "median": "227.63 μs", + "p99": "411.33 μs", + "memory": "1.33 MB" + }, + { + "name": "lua (eval)", + "ips": "4.19 K", + "average": "238.72 μs", + "deviation": "±9.85%", + "median": "234.88 μs", + "p99": "335.73 μs", + "memory": "1.34 MB" + } + ], + "comparison": [ + "luerl 5.52 K", + "lua (chunk) 4.27 K - 1.29x slower +53.13 μs", + "lua (eval) 4.19 K - 1.32x slower +57.42 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "luerl", + "ips": "50.83 K", + "average": "19.67 μs", + "deviation": "±20.44%", + "median": "19.13 μs", + "p99": "28.17 μs", + "memory": "133.33 KB" + }, + { + "name": "lua (chunk)", + "ips": "42.73 K", + "average": "23.40 μs", + "deviation": "±35.27%", + "median": "23.21 μs", + "p99": "26.75 μs", + "memory": "141.38 KB" + }, + { + "name": "lua (eval)", + "ips": "38.35 K", + "average": "26.07 μs", + "deviation": "±37.69%", + "median": "25.58 μs", + "p99": "35.33 μs", + "memory": "151.13 KB" + } + ], + "comparison": [ + "luerl 50.83 K", + "lua (chunk) 42.73 K - 1.19x slower +3.73 μs", + "lua (eval) 38.35 K - 1.33x slower +6.40 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "311.25 K", + "average": "3.21 μs", + "deviation": "±183.71%", + "median": "3.17 μs", + "p99": "4.50 μs", + "memory": "19.06 KB" + }, + { + "name": "luerl", + "ips": "266.96 K", + "average": "3.75 μs", + "deviation": "±163.56%", + "median": "3.58 μs", + "p99": "9.50 μs", + "memory": "25.98 KB" + }, + { + "name": "lua (eval)", + "ips": "169.47 K", + "average": "5.90 μs", + "deviation": "±170.93%", + "median": "5.50 μs", + "p99": "15.75 μs", + "memory": "28.78 KB" + } + ], + "comparison": [ + "lua (chunk) 311.25 K", + "luerl 266.96 K - 1.17x slower +0.53 μs", + "lua (eval) 169.47 K - 1.84x slower +2.69 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Iterate/Sum": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "4.10 K", + "average": "243.68 μs", + "deviation": "±11.81%", + "median": "235.09 μs", + "p99": "342.27 μs", + "memory": "1.47 MB" + }, + { + "name": "lua (eval)", + "ips": "4.07 K", + "average": "245.42 μs", + "deviation": "±12.09%", + "median": "241.67 μs", + "p99": "362.47 μs", + "memory": "1.48 MB" + }, + { + "name": "luerl", + "ips": "4.06 K", + "average": "246.09 μs", + "deviation": "±7.76%", + "median": "243.06 μs", + "p99": "342.83 μs", + "memory": "1.35 MB" + } + ], + "comparison": [ + "lua (chunk) 4.10 K", + "lua (eval) 4.07 K - 1.01x slower +1.75 μs", + "luerl 4.06 K - 1.01x slower +2.41 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "40.08 K", + "average": "24.95 μs", + "deviation": "±25.49%", + "median": "24.25 μs", + "p99": "52.88 μs", + "memory": "154.27 KB" + }, + { + "name": "lua (eval)", + "ips": "36.72 K", + "average": "27.23 μs", + "deviation": "±17.19%", + "median": "26.67 μs", + "p99": "38.21 μs", + "memory": "164.09 KB" + }, + { + "name": "luerl", + "ips": "36.18 K", + "average": "27.64 μs", + "deviation": "±35.39%", + "median": "27.00 μs", + "p99": "57.79 μs", + "memory": "149.14 KB" + } + ], + "comparison": [ + "lua (chunk) 40.08 K", + "lua (eval) 36.72 K - 1.09x slower +2.28 μs", + "luerl 36.18 K - 1.11x slower +2.69 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "306.46 K", + "average": "3.26 μs", + "deviation": "±172.56%", + "median": "3.21 μs", + "p99": "4.38 μs", + "memory": "19.87 KB" + }, + { + "name": "luerl", + "ips": "231.29 K", + "average": "4.32 μs", + "deviation": "±138.54%", + "median": "4.25 μs", + "p99": "6.33 μs", + "memory": "26.80 KB" + }, + { + "name": "lua (eval)", + "ips": "177.92 K", + "average": "5.62 μs", + "deviation": "±101.11%", + "median": "5.42 μs", + "p99": "9.25 μs", + "memory": "29.51 KB" + } + ], + "comparison": [ + "lua (chunk) 306.46 K", + "luerl 231.29 K - 1.33x slower +1.06 μs", + "lua (eval) 177.92 K - 1.72x slower +2.36 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Map + Reduce": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "luerl", + "ips": "2.15 K", + "average": "464.93 μs", + "deviation": "±7.90%", + "median": "449.96 μs", + "p99": "600.72 μs", + "memory": "2.44 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.05 K", + "average": "487.99 μs", + "deviation": "±6.39%", + "median": "480.24 μs", + "p99": "622.53 μs", + "memory": "2.92 MB" + }, + { + "name": "lua (eval)", + "ips": "2.01 K", + "average": "496.87 μs", + "deviation": "±13.21%", + "median": "487.17 μs", + "p99": "772.35 μs", + "memory": "2.93 MB" + } + ], + "comparison": [ + "luerl 2.15 K", + "lua (chunk) 2.05 K - 1.05x slower +23.06 μs", + "lua (eval) 2.01 K - 1.07x slower +31.94 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "luerl", + "ips": "21.41 K", + "average": "46.71 μs", + "deviation": "±6.09%", + "median": "46.50 μs", + "p99": "52.29 μs", + "memory": "262.35 KB" + }, + { + "name": "lua (chunk)", + "ips": "20.87 K", + "average": "47.91 μs", + "deviation": "±8.43%", + "median": "47.54 μs", + "p99": "53.79 μs", + "memory": "303.87 KB" + }, + { + "name": "lua (eval)", + "ips": "19.56 K", + "average": "51.12 μs", + "deviation": "±6.66%", + "median": "50.58 μs", + "p99": "60.46 μs", + "memory": "314.32 KB" + } + ], + "comparison": [ + "luerl 21.41 K", + "lua (chunk) 20.87 K - 1.03x slower +1.20 μs", + "lua (eval) 19.56 K - 1.09x slower +4.40 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "168.91 K", + "average": "5.92 μs", + "deviation": "±76.95%", + "median": "5.83 μs", + "p99": "13.38 μs", + "memory": "37.09 KB" + }, + { + "name": "luerl", + "ips": "145.75 K", + "average": "6.86 μs", + "deviation": "±94.69%", + "median": "6.59 μs", + "p99": "15.96 μs", + "memory": "39.39 KB" + }, + { + "name": "lua (eval)", + "ips": "102.03 K", + "average": "9.80 μs", + "deviation": "±71.35%", + "median": "8.21 μs", + "p99": "22.96 μs", + "memory": "46.72 KB" + } + ], + "comparison": [ + "lua (chunk) 168.91 K", + "luerl 145.75 K - 1.16x slower +0.94 μs", + "lua (eval) 102.03 K - 1.66x slower +3.88 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Pairs (hash)": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "1.02 K", + "average": "984.62 μs", + "deviation": "±7.98%", + "median": "995.70 μs", + "p99": "1143.86 μs", + "memory": "2.06 MB" + }, + { + "name": "lua (eval)", + "ips": "1.01 K", + "average": "990.11 μs", + "deviation": "±8.33%", + "median": "1002.18 μs", + "p99": "1157.35 μs", + "memory": "2.05 MB" + }, + { + "name": "luerl", + "ips": "0.75 K", + "average": "1332.28 μs", + "deviation": "±5.93%", + "median": "1334.35 μs", + "p99": "1489.09 μs", + "memory": "2.48 MB" + } + ], + "comparison": [ + "lua (chunk) 1.02 K", + "lua (eval) 1.01 K - 1.01x slower +5.49 μs", + "luerl 0.75 K - 1.35x slower +347.66 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "13.46 K", + "average": "74.27 μs", + "deviation": "±10.12%", + "median": "72.92 μs", + "p99": "99.63 μs", + "memory": "211.49 KB" + }, + { + "name": "lua (eval)", + "ips": "12.33 K", + "average": "81.10 μs", + "deviation": "±25.10%", + "median": "78.13 μs", + "p99": "167.55 μs", + "memory": "221.46 KB" + }, + { + "name": "luerl", + "ips": "10.92 K", + "average": "91.56 μs", + "deviation": "±15.90%", + "median": "86.83 μs", + "p99": "142.16 μs", + "memory": "248.84 KB" + } + ], + "comparison": [ + "lua (chunk) 13.46 K", + "lua (eval) 12.33 K - 1.09x slower +6.83 μs", + "luerl 10.92 K - 1.23x slower +17.29 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "130.58 K", + "average": "7.66 μs", + "deviation": "±70.51%", + "median": "7.33 μs", + "p99": "24.42 μs", + "memory": "25.10 KB" + }, + { + "name": "luerl", + "ips": "130.19 K", + "average": "7.68 μs", + "deviation": "±68.45%", + "median": "7.25 μs", + "p99": "23.38 μs", + "memory": "36.16 KB" + }, + { + "name": "lua (eval)", + "ips": "94.66 K", + "average": "10.56 μs", + "deviation": "±56.52%", + "median": "9.92 μs", + "p99": "29.25 μs", + "memory": "34.91 KB" + } + ], + "comparison": [ + "lua (chunk) 130.58 K", + "luerl 130.19 K - 1.00x slower +0.0229 μs", + "lua (eval) 94.66 K - 1.38x slower +2.91 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + } + }, + "patterns": { + "patterns: find/match field extraction (n=200)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "972.71", + "average": "1.03 ms", + "deviation": "±9.96%", + "median": "1.01 ms", + "p99": "1.24 ms", + "memory": "3.72 MB" + }, + { + "name": "lua (eval)", + "ips": "970.99", + "average": "1.03 ms", + "deviation": "±3.16%", + "median": "1.02 ms", + "p99": "1.13 ms", + "memory": "3.73 MB" + }, + { + "name": "luerl", + "ips": "906.75", + "average": "1.10 ms", + "deviation": "±5.98%", + "median": "1.09 ms", + "p99": "1.26 ms", + "memory": "6.70 MB" + } + ], + "comparison": [ + "lua (chunk) 972.71", + "lua (eval) 970.99 - 1.00x slower +0.00182 ms", + "luerl 906.75 - 1.07x slower +0.0748 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "patterns: find-based tokenizer (n=200)": { + "jobs": [ + { + "name": "luerl", + "ips": "733.20", + "average": "1.36 ms", + "deviation": "±5.05%", + "median": "1.35 ms", + "p99": "1.48 ms", + "memory": "6.65 MB" + }, + { + "name": "lua (chunk)", + "ips": "626.48", + "average": "1.60 ms", + "deviation": "±4.81%", + "median": "1.57 ms", + "p99": "1.73 ms", + "memory": "10.11 MB" + }, + { + "name": "lua (eval)", + "ips": "611.43", + "average": "1.64 ms", + "deviation": "±8.70%", + "median": "1.61 ms", + "p99": "1.99 ms", + "memory": "10.12 MB" + } + ], + "comparison": [ + "luerl 733.20", + "lua (chunk) 626.48 - 1.17x slower +0.23 ms", + "lua (eval) 611.43 - 1.20x slower +0.27 ms" + ], + "memory_comparison": [ + "luerl 6.65 MB", + "lua (chunk) 10.11 MB - 1.52x memory usage +3.46 MB", + "lua (eval) 10.12 MB - 1.52x memory usage +3.48 MB" + ] + }, + "patterns: gsub template substitution (n=200)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "491.94", + "average": "2.03 ms", + "deviation": "±2.07%", + "median": "2.02 ms", + "p99": "2.14 ms", + "memory": "5.05 MB" + }, + { + "name": "lua (eval)", + "ips": "488.05", + "average": "2.05 ms", + "deviation": "±4.20%", + "median": "2.04 ms", + "p99": "2.22 ms", + "memory": "5.06 MB" + }, + { + "name": "luerl", + "ips": "450.89", + "average": "2.22 ms", + "deviation": "±4.50%", + "median": "2.17 ms", + "p99": "2.47 ms", + "memory": "11.75 MB" + } + ], + "comparison": [ + "lua (chunk) 491.94", + "lua (eval) 488.05 - 1.01x slower +0.0162 ms", + "luerl 450.89 - 1.09x slower +0.185 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "metamethods": { + "metamethods: self-call method dispatch (n=200)": { + "jobs": [ + { + "name": "luerl", + "ips": "2.06 K", + "average": "484.65 μs", + "deviation": "±9.07%", + "median": "472.63 μs", + "p99": "658.27 μs", + "memory": "1.43 MB" + }, + { + "name": "lua (eval)", + "ips": "1.77 K", + "average": "565.06 μs", + "deviation": "±8.07%", + "median": "552.29 μs", + "p99": "755.54 μs", + "memory": "1.85 MB" + }, + { + "name": "lua (chunk)", + "ips": "1.75 K", + "average": "569.99 μs", + "deviation": "±12.20%", + "median": "553.75 μs", + "p99": "849.77 μs", + "memory": "1.84 MB" + } + ], + "comparison": [ + "luerl 2.06 K", + "lua (eval) 1.77 K - 1.17x slower +80.41 μs", + "lua (chunk) 1.75 K - 1.18x slower +85.34 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "metamethods: 3-level __index chain (n=200)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "4.19 K", + "average": "238.51 μs", + "deviation": "±3.78%", + "median": "237.13 μs", + "p99": "257.04 μs", + "memory": "788.91 KB" + }, + { + "name": "lua (eval)", + "ips": "4.01 K", + "average": "249.27 μs", + "deviation": "±5.91%", + "median": "248.25 μs", + "p99": "273.67 μs", + "memory": "798.82 KB" + }, + { + "name": "luerl", + "ips": "3.95 K", + "average": "253.10 μs", + "deviation": "±42.54%", + "median": "221.29 μs", + "p99": "343.51 μs", + "memory": "668.59 KB" + } + ], + "comparison": [ + "lua (chunk) 4.19 K", + "lua (eval) 4.01 K - 1.05x slower +10.76 μs", + "luerl 3.95 K - 1.06x slower +14.59 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "metamethods: arithmetic/relational metamethods (n=200)": { + "jobs": [ + { + "name": "luerl", + "ips": "523.21", + "average": "1.91 ms", + "deviation": "±4.47%", + "median": "1.88 ms", + "p99": "2.20 ms", + "memory": "6.24 MB" + }, + { + "name": "lua (chunk)", + "ips": "455.21", + "average": "2.20 ms", + "deviation": "±4.68%", + "median": "2.23 ms", + "p99": "2.54 ms", + "memory": "7.07 MB" + }, + { + "name": "lua (eval)", + "ips": "453.68", + "average": "2.20 ms", + "deviation": "±6.62%", + "median": "2.22 ms", + "p99": "2.63 ms", + "memory": "7.08 MB" + } + ], + "comparison": [ + "luerl 523.21", + "lua (chunk) 455.21 - 1.15x slower +0.29 ms", + "lua (eval) 453.68 - 1.15x slower +0.29 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "pcall_varargs": { + "call protocol: pcall, success path (n=500)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "3.49 K", + "average": "286.44 μs", + "deviation": "±12.19%", + "median": "282.13 μs", + "p99": "352.91 μs", + "memory": "1.31 MB" + }, + { + "name": "luerl", + "ips": "3.44 K", + "average": "290.86 μs", + "deviation": "±3.49%", + "median": "289.50 μs", + "p99": "309.63 μs", + "memory": "1.11 MB" + }, + { + "name": "lua (eval)", + "ips": "3.34 K", + "average": "299.49 μs", + "deviation": "±5.85%", + "median": "297.75 μs", + "p99": "329.07 μs", + "memory": "1.32 MB" + } + ], + "comparison": [ + "lua (chunk) 3.49 K", + "luerl 3.44 K - 1.02x slower +4.42 μs", + "lua (eval) 3.34 K - 1.05x slower +13.05 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "call protocol: pcall, raise + catch (n=500)": { + "jobs": [ + { + "name": "luerl", + "ips": "1.87 K", + "average": "533.92 μs", + "deviation": "±29.41%", + "median": "485.38 μs", + "p99": "1138.98 μs", + "memory": "1.55 MB" + }, + { + "name": "lua (chunk)", + "ips": "1.11 K", + "average": "904.68 μs", + "deviation": "±17.56%", + "median": "877.33 μs", + "p99": "1891.77 μs", + "memory": "3.35 MB" + }, + { + "name": "lua (eval)", + "ips": "1.00 K", + "average": "1001.11 μs", + "deviation": "±31.79%", + "median": "899.42 μs", + "p99": "2144.79 μs", + "memory": "3.36 MB" + } + ], + "comparison": [ + "luerl 1.87 K", + "lua (chunk) 1.11 K - 1.69x slower +370.76 μs", + "lua (eval) 1.00 K - 1.88x slower +467.19 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "call protocol: varargs + multiple returns (n=500)": { + "jobs": [ + { + "name": "luerl", + "ips": "410.14", + "average": "2.44 ms", + "deviation": "±4.02%", + "median": "2.40 ms", + "p99": "2.73 ms", + "memory": "8.73 MB" + }, + { + "name": "lua (eval)", + "ips": "253.78", + "average": "3.94 ms", + "deviation": "±6.97%", + "median": "3.89 ms", + "p99": "4.79 ms", + "memory": "19.95 MB" + }, + { + "name": "lua (chunk)", + "ips": "237.27", + "average": "4.21 ms", + "deviation": "±16.27%", + "median": "4.01 ms", + "p99": "8.17 ms", + "memory": "19.94 MB" + } + ], + "comparison": [ + "luerl 410.14", + "lua (eval) 253.78 - 1.62x slower +1.50 ms", + "lua (chunk) 237.27 - 1.73x slower +1.78 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "vm_new": { + "VM instantiation: Lua.new/1 vs :luerl.init/0": { + "jobs": [ + { + "name": "luerl (init)", + "ips": "64.86 K", + "average": "15.42 μs", + "deviation": "±23.43%", + "median": "15.21 μs", + "p99": "24.25 μs", + "memory": "51.64 KB" + }, + { + "name": "lua (new, no sandbox)", + "ips": "33.30 K", + "average": "30.03 μs", + "deviation": "±11.23%", + "median": "29.71 μs", + "p99": "38.46 μs", + "memory": "68.76 KB" + }, + { + "name": "lua (new, custom exclude)", + "ips": "27.37 K", + "average": "36.54 μs", + "deviation": "±8.96%", + "median": "35.96 μs", + "p99": "42.63 μs", + "memory": "90.97 KB" + }, + { + "name": "lua (new)", + "ips": "26.58 K", + "average": "37.63 μs", + "deviation": "±62.22%", + "median": "36.67 μs", + "p99": "71.96 μs", + "memory": "91.88 KB" + } + ], + "comparison": [ + "luerl (init) 64.86 K", + "lua (new, no sandbox) 33.30 K - 1.95x slower +14.62 μs", + "lua (new, custom exclude) 27.37 K - 2.37x slower +21.12 μs", + "lua (new) 26.58 K - 2.44x slower +22.21 μs" + ], + "memory_note": "**All measurements for memory usage were the same**", + "cold_call": "6654.0 us", + "second_call": "93.0 us" + } + }, + "encode_decode": { + "raw": "lua 1.0.0 — encode!/decode! decomposition\n(decode+deep_cast column: enabled)\n==================================================================================\nop shape N total_us per_elem_ns\nencode int_list 8 0.90 113.0\ndecode int_list 8 0.28 34.7\ndec+cast int_list 8 0.31 38.5\nencode int_list 64 13.52 211.2\ndecode int_list 64 4.23 66.0\ndec+cast int_list 64 7.14 111.6\nencode int_list 512 151.37 295.6\ndecode int_list 512 48.44 94.6\ndec+cast int_list 512 73.99 144.5\nencode int_list 4096 1601.11 390.9\ndecode int_list 4096 583.70 142.5\ndec+cast int_list 4096 785.61 191.8\n----------------------------------------------------------------------------------\nencode float_list 8 0.78 97.1\ndecode float_list 8 0.24 29.8\ndec+cast float_list 8 0.26 32.9\nencode float_list 64 11.47 179.2\ndecode float_list 64 3.55 55.4\ndec+cast float_list 64 7.03 109.9\nencode float_list 512 156.43 305.5\ndecode float_list 512 45.74 89.3\ndec+cast float_list 512 72.74 142.1\nencode float_list 4096 1611.38 393.4\ndecode float_list 4096 551.08 134.5\ndec+cast float_list 4096 839.19 204.9\n----------------------------------------------------------------------------------\nencode bool_list 8 0.77 96.7\ndecode bool_list 8 0.23 28.7\ndec+cast bool_list 8 0.25 31.3\nencode bool_list 64 11.21 175.2\ndecode bool_list 64 3.46 54.1\ndec+cast bool_list 64 7.00 109.4\nencode bool_list 512 153.74 300.3\ndecode bool_list 512 47.42 92.6\ndec+cast bool_list 512 74.01 144.6\nencode bool_list 4096 1569.25 383.1\ndecode bool_list 4096 571.91 139.6\ndec+cast bool_list 4096 775.34 189.3\n----------------------------------------------------------------------------------\nencode short_string_list 8 0.78 96.9\ndecode short_string_list 8 0.24 29.8\ndec+cast short_string_list 8 0.26 32.6\nencode short_string_list 64 11.40 178.1\ndecode short_string_list 64 3.53 55.2\ndec+cast short_string_list 64 7.08 110.6\nencode short_string_list 512 155.05 302.8\ndecode short_string_list 512 49.88 97.4\ndec+cast short_string_list 512 72.83 142.2\nencode short_string_list 4096 1715.16 418.7\ndecode short_string_list 4096 718.28 175.4\ndec+cast short_string_list 4096 763.05 186.3\n----------------------------------------------------------------------------------\nencode long_string_list 8 0.78 96.9\ndecode long_string_list 8 0.24 30.5\ndec+cast long_string_list 8 0.26 33.0\nencode long_string_list 64 11.20 174.9\ndecode long_string_list 64 3.49 54.5\ndec+cast long_string_list 64 7.31 114.3\nencode long_string_list 512 157.18 307.0\ndecode long_string_list 512 42.76 83.5\ndec+cast long_string_list 512 76.03 148.5\nencode long_string_list 4096 2488.38 607.5\ndecode long_string_list 4096 1495.20 365.0\ndec+cast long_string_list 4096 1707.00 416.7\n----------------------------------------------------------------------------------\nencode string_map 8 1.60 199.5\ndecode string_map 8 0.10 12.4\ndec+cast string_map 8 0.20 25.5\nencode string_map 64 30.47 476.1\ndecode string_map 64 0.74 11.6\ndec+cast string_map 64 3.95 61.7\nencode string_map 512 240.62 470.0\ndecode string_map 512 11.04 21.6\ndec+cast string_map 512 39.76 77.7\nencode string_map 4096 2194.00 535.6\ndecode string_map 4096 71.08 17.4\ndec+cast string_map 4096 335.83 82.0\n----------------------------------------------------------------------------------\nencode int_map 8 0.83 103.4\ndecode int_map 8 0.24 29.8\ndec+cast int_map 8 0.26 32.2\nencode int_map 64 10.97 171.5\ndecode int_map 64 3.68 57.5\ndec+cast int_map 64 5.96 93.1\nencode int_map 512 109.35 213.6\ndecode int_map 512 27.78 54.3\ndec+cast int_map 512 72.41 141.4\nencode int_map 4096 1307.14 319.1\ndecode int_map 4096 576.06 140.6\ndec+cast int_map 4096 881.09 215.1\n----------------------------------------------------------------------------------\nencode record_list 8 5.07 633.8\ndecode record_list 8 0.78 98.1\ndec+cast record_list 8 1.20 149.9\nencode record_list 64 47.83 747.4\ndecode record_list 64 8.26 129.1\ndec+cast record_list 64 14.64 228.8\nencode record_list 512 565.88 1105.2\ndecode record_list 512 118.02 230.5\ndec+cast record_list 512 196.29 383.4\nencode record_list 4096 4732.25 1155.3\ndecode record_list 4096 1094.36 267.2\ndec+cast record_list 4096 1628.34 397.5\n----------------------------------------------------------------------------------\nnested chain (depth sweep) — isolates recursion/traversal from fan-out\nop shape N total_us per_elem_ns\nencode nested_chain 4 1.07 535.2\ndecode nested_chain 4 0.21 103.7\ndec+cast nested_chain 4 0.32 161.9\nencode nested_chain 16 4.19 2094.1\ndecode nested_chain 16 0.91 456.5\ndec+cast nested_chain 16 1.33 665.1\nencode nested_chain 64 16.94 8469.5\ndecode nested_chain 64 3.92 1961.9\ndec+cast nested_chain 64 6.28 3140.2\nencode nested_chain 256 68.77 34386.2\ndecode nested_chain 256 18.23 9114.9\ndec+cast nested_chain 256 32.31 16156.7\n==================================================================================\ncomposite anchor — the PR's `original_nested` (matches the 18us/108us figure)\nop shape N total_us per_elem_ns\nencode original_nested 75 17.11 4276.4\ndecode original_nested 75 3.21 802.7\ndec+cast original_nested 75 5.26 1314.8\n" + } +} diff --git a/bench_results/v1.0.0/table_ops.txt b/bench_results/v1.0.0/table_ops.txt new file mode 100644 index 0000000..91cc3a0 --- /dev/null +++ b/bench_results/v1.0.0/table_ops.txt @@ -0,0 +1,469 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== Table Build (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +luerl 6.37 K 157.04 μs ±11.82% 154.21 μs 218.16 μs +lua (chunk) 6.23 K 160.45 μs ±8.91% 157.88 μs 198.69 μs +lua (eval) 5.90 K 169.48 μs ±14.14% 164.58 μs 287.61 μs + +Comparison: +luerl 6.37 K +lua (chunk) 6.23 K - 1.02x slower +3.41 μs +lua (eval) 5.90 K - 1.08x slower +12.44 μs + +Memory usage statistics: + +Name Memory usage +luerl 0.97 MB +lua (chunk) 1.00 MB - 1.03x memory usage +0.0267 MB +lua (eval) 1.01 MB - 1.04x memory usage +0.0361 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 60.98 K 16.40 μs ±32.33% 16.17 μs 24.29 μs +luerl 58.70 K 17.04 μs ±26.63% 16.50 μs 26.67 μs +lua (eval) 53.06 K 18.85 μs ±33.45% 18.42 μs 27.04 μs + +Comparison: +lua (chunk) 60.98 K +luerl 58.70 K - 1.04x slower +0.64 μs +lua (eval) 53.06 K - 1.15x slower +2.45 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 105.41 KB +luerl 110.77 KB - 1.05x memory usage +5.35 KB +lua (eval) 114.74 KB - 1.09x memory usage +9.33 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 437.49 K 2.29 μs ±237.41% 2.25 μs 3.42 μs +luerl 317.31 K 3.15 μs ±243.81% 3.04 μs 4.71 μs +lua (eval) 205.32 K 4.87 μs ±231.75% 4.46 μs 10.42 μs + +Comparison: +lua (chunk) 437.49 K +luerl 317.31 K - 1.38x slower +0.87 μs +lua (eval) 205.32 K - 2.13x slower +2.58 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 13.80 KB +luerl 22.62 KB - 1.64x memory usage +8.82 KB +lua (eval) 23.45 KB - 1.70x memory usage +9.66 KB + +**All measurements for memory usage were the same** + +=== Table Sort (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +luerl 5.52 K 181.29 μs ±10.01% 178.00 μs 220.42 μs +lua (chunk) 4.27 K 234.42 μs ±13.69% 227.63 μs 411.33 μs +lua (eval) 4.19 K 238.72 μs ±9.85% 234.88 μs 335.73 μs + +Comparison: +luerl 5.52 K +lua (chunk) 4.27 K - 1.29x slower +53.13 μs +lua (eval) 4.19 K - 1.32x slower +57.42 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.18 MB +lua (chunk) 1.33 MB - 1.13x memory usage +0.156 MB +lua (eval) 1.34 MB - 1.14x memory usage +0.164 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +luerl 50.83 K 19.67 μs ±20.44% 19.13 μs 28.17 μs +lua (chunk) 42.73 K 23.40 μs ±35.27% 23.21 μs 26.75 μs +lua (eval) 38.35 K 26.07 μs ±37.69% 25.58 μs 35.33 μs + +Comparison: +luerl 50.83 K +lua (chunk) 42.73 K - 1.19x slower +3.73 μs +lua (eval) 38.35 K - 1.33x slower +6.40 μs + +Memory usage statistics: + +Name Memory usage +luerl 133.33 KB +lua (chunk) 141.38 KB - 1.06x memory usage +8.05 KB +lua (eval) 151.13 KB - 1.13x memory usage +17.80 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 311.25 K 3.21 μs ±183.71% 3.17 μs 4.50 μs +luerl 266.96 K 3.75 μs ±163.56% 3.58 μs 9.50 μs +lua (eval) 169.47 K 5.90 μs ±170.93% 5.50 μs 15.75 μs + +Comparison: +lua (chunk) 311.25 K +luerl 266.96 K - 1.17x slower +0.53 μs +lua (eval) 169.47 K - 1.84x slower +2.69 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 19.06 KB +luerl 25.98 KB - 1.36x memory usage +6.91 KB +lua (eval) 28.78 KB - 1.51x memory usage +9.72 KB + +**All measurements for memory usage were the same** + +=== Table Iterate/Sum (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (chunk) 4.10 K 243.68 μs ±11.81% 235.09 μs 342.27 μs +lua (eval) 4.07 K 245.42 μs ±12.09% 241.67 μs 362.47 μs +luerl 4.06 K 246.09 μs ±7.76% 243.06 μs 342.83 μs + +Comparison: +lua (chunk) 4.10 K +lua (eval) 4.07 K - 1.01x slower +1.75 μs +luerl 4.06 K - 1.01x slower +2.41 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1.47 MB +lua (eval) 1.48 MB - 1.01x memory usage +0.0110 MB +luerl 1.35 MB - 0.92x memory usage -0.12469 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 40.08 K 24.95 μs ±25.49% 24.25 μs 52.88 μs +lua (eval) 36.72 K 27.23 μs ±17.19% 26.67 μs 38.21 μs +luerl 36.18 K 27.64 μs ±35.39% 27.00 μs 57.79 μs + +Comparison: +lua (chunk) 40.08 K +lua (eval) 36.72 K - 1.09x slower +2.28 μs +luerl 36.18 K - 1.11x slower +2.69 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 154.27 KB +lua (eval) 164.09 KB - 1.06x memory usage +9.81 KB +luerl 149.14 KB - 0.97x memory usage -5.13281 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 306.46 K 3.26 μs ±172.56% 3.21 μs 4.38 μs +luerl 231.29 K 4.32 μs ±138.54% 4.25 μs 6.33 μs +lua (eval) 177.92 K 5.62 μs ±101.11% 5.42 μs 9.25 μs + +Comparison: +lua (chunk) 306.46 K +luerl 231.29 K - 1.33x slower +1.06 μs +lua (eval) 177.92 K - 1.72x slower +2.36 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 19.87 KB +luerl 26.80 KB - 1.35x memory usage +6.93 KB +lua (eval) 29.51 KB - 1.49x memory usage +9.64 KB + +**All measurements for memory usage were the same** + +=== Table Map + Reduce (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +luerl 2.15 K 464.93 μs ±7.90% 449.96 μs 600.72 μs +lua (chunk) 2.05 K 487.99 μs ±6.39% 480.24 μs 622.53 μs +lua (eval) 2.01 K 496.87 μs ±13.21% 487.17 μs 772.35 μs + +Comparison: +luerl 2.15 K +lua (chunk) 2.05 K - 1.05x slower +23.06 μs +lua (eval) 2.01 K - 1.07x slower +31.94 μs + +Memory usage statistics: + +Name Memory usage +luerl 2.44 MB +lua (chunk) 2.92 MB - 1.19x memory usage +0.47 MB +lua (eval) 2.93 MB - 1.20x memory usage +0.48 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +luerl 21.41 K 46.71 μs ±6.09% 46.50 μs 52.29 μs +lua (chunk) 20.87 K 47.91 μs ±8.43% 47.54 μs 53.79 μs +lua (eval) 19.56 K 51.12 μs ±6.66% 50.58 μs 60.46 μs + +Comparison: +luerl 21.41 K +lua (chunk) 20.87 K - 1.03x slower +1.20 μs +lua (eval) 19.56 K - 1.09x slower +4.40 μs + +Memory usage statistics: + +Name Memory usage +luerl 262.35 KB +lua (chunk) 303.87 KB - 1.16x memory usage +41.52 KB +lua (eval) 314.32 KB - 1.20x memory usage +51.97 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 168.91 K 5.92 μs ±76.95% 5.83 μs 13.38 μs +luerl 145.75 K 6.86 μs ±94.69% 6.59 μs 15.96 μs +lua (eval) 102.03 K 9.80 μs ±71.35% 8.21 μs 22.96 μs + +Comparison: +lua (chunk) 168.91 K +luerl 145.75 K - 1.16x slower +0.94 μs +lua (eval) 102.03 K - 1.66x slower +3.88 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 37.09 KB +luerl 39.39 KB - 1.06x memory usage +2.30 KB +lua (eval) 46.72 KB - 1.26x memory usage +9.63 KB + +**All measurements for memory usage were the same** + +=== Table Pairs (hash) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (chunk) 1.02 K 984.62 μs ±7.98% 995.70 μs 1143.86 μs +lua (eval) 1.01 K 990.11 μs ±8.33% 1002.18 μs 1157.35 μs +luerl 0.75 K 1332.28 μs ±5.93% 1334.35 μs 1489.09 μs + +Comparison: +lua (chunk) 1.02 K +lua (eval) 1.01 K - 1.01x slower +5.49 μs +luerl 0.75 K - 1.35x slower +347.66 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 2.06 MB +lua (eval) 2.05 MB - 0.99x memory usage -0.01064 MB +luerl 2.48 MB - 1.20x memory usage +0.42 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 13.46 K 74.27 μs ±10.12% 72.92 μs 99.63 μs +lua (eval) 12.33 K 81.10 μs ±25.10% 78.13 μs 167.55 μs +luerl 10.92 K 91.56 μs ±15.90% 86.83 μs 142.16 μs + +Comparison: +lua (chunk) 13.46 K +lua (eval) 12.33 K - 1.09x slower +6.83 μs +luerl 10.92 K - 1.23x slower +17.29 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 211.49 KB +lua (eval) 221.46 KB - 1.05x memory usage +9.97 KB +luerl 248.84 KB - 1.18x memory usage +37.34 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 130.58 K 7.66 μs ±70.51% 7.33 μs 24.42 μs +luerl 130.19 K 7.68 μs ±68.45% 7.25 μs 23.38 μs +lua (eval) 94.66 K 10.56 μs ±56.52% 9.92 μs 29.25 μs + +Comparison: +lua (chunk) 130.58 K +luerl 130.19 K - 1.00x slower +0.0229 μs +lua (eval) 94.66 K - 1.38x slower +2.91 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 25.10 KB +luerl 36.16 KB - 1.44x memory usage +11.05 KB +lua (eval) 34.91 KB - 1.39x memory usage +9.81 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.0/timestamp.txt b/bench_results/v1.0.0/timestamp.txt new file mode 100644 index 0000000..ff5029f --- /dev/null +++ b/bench_results/v1.0.0/timestamp.txt @@ -0,0 +1 @@ +Tue Jul 28 10:48:16 EDT 2026 diff --git a/bench_results/v1.0.0/versions.txt b/bench_results/v1.0.0/versions.txt new file mode 100644 index 0000000..184e5f9 --- /dev/null +++ b/bench_results/v1.0.0/versions.txt @@ -0,0 +1,3 @@ +Erlang/OTP 29 [erts-17.0] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] + +Elixir 1.20.0 (compiled with Erlang/OTP 29) diff --git a/bench_results/v1.0.0/vm_new.txt b/bench_results/v1.0.0/vm_new.txt new file mode 100644 index 0000000..26ae1a4 --- /dev/null +++ b/bench_results/v1.0.0/vm_new.txt @@ -0,0 +1,65 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + +=== VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full) === + +Lua.new() one-time vs repeat cost (single samples, informational): + first call on this node : 6654.0 us + second call : 93.0 us + +The first figure includes any one-time template build and first-time module +loading. Benchee's steady-state numbers below are the per-request cost after +that point. + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 52 s +Excluding outliers: false + +Benchmarking lua (new) ... +Benchmarking lua (new, custom exclude) ... +Benchmarking lua (new, no sandbox) ... +Benchmarking luerl (init) ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl (init) 64.86 K 15.42 μs ±23.43% 15.21 μs 24.25 μs +lua (new, no sandbox) 33.30 K 30.03 μs ±11.23% 29.71 μs 38.46 μs +lua (new, custom exclude) 27.37 K 36.54 μs ±8.96% 35.96 μs 42.63 μs +lua (new) 26.58 K 37.63 μs ±62.22% 36.67 μs 71.96 μs + +Comparison: +luerl (init) 64.86 K +lua (new, no sandbox) 33.30 K - 1.95x slower +14.62 μs +lua (new, custom exclude) 27.37 K - 2.37x slower +21.12 μs +lua (new) 26.58 K - 2.44x slower +22.21 μs + +Memory usage statistics: + +Name Memory usage +luerl (init) 51.64 KB +lua (new, no sandbox) 68.76 KB - 1.33x memory usage +17.12 KB +lua (new, custom exclude) 90.97 KB - 1.76x memory usage +39.33 KB +lua (new) 91.88 KB - 1.78x memory usage +40.23 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/closures.txt b/bench_results/v1.0.2/closures.txt new file mode 100644 index 0000000..cf110a9 --- /dev/null +++ b/bench_results/v1.0.2/closures.txt @@ -0,0 +1,54 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 2.68 K 372.50 μs ±10.92% 365.04 μs 532.92 μs +lua (chunk) 2.63 K 379.93 μs ±8.39% 373.71 μs 512.83 μs +luerl 2.53 K 395.50 μs ±7.19% 389.21 μs 548.26 μs + +Comparison: +lua (eval) 2.68 K +lua (chunk) 2.63 K - 1.02x slower +7.43 μs +luerl 2.53 K - 1.06x slower +23.01 μs + +Memory usage statistics: + +Name average deviation median 99th % +lua (eval) 2.12 MB ±0.11% 2.12 MB 2.13 MB +lua (chunk) 2.11 MB ±0.11% 2.11 MB 2.12 MB +luerl 1.90 MB ±0.00% 1.90 MB 1.90 MB + +Comparison: +lua (eval) 2.12 MB +lua (chunk) 2.11 MB - 1.00x memory usage -0.01023 MB +luerl 1.90 MB - 0.89x memory usage -0.22699 MB diff --git a/bench_results/v1.0.2/commit.txt b/bench_results/v1.0.2/commit.txt new file mode 100644 index 0000000..27bf7c3 --- /dev/null +++ b/bench_results/v1.0.2/commit.txt @@ -0,0 +1 @@ +3a0d3923249cbb05380a3f17d3920a0b3ed55cb0 diff --git a/bench_results/v1.0.2/cpu.txt b/bench_results/v1.0.2/cpu.txt new file mode 100644 index 0000000..de5c8ad --- /dev/null +++ b/bench_results/v1.0.2/cpu.txt @@ -0,0 +1 @@ +Apple M4 diff --git a/bench_results/v1.0.2/encode_decode.txt b/bench_results/v1.0.2/encode_decode.txt new file mode 100644 index 0000000..8302981 --- /dev/null +++ b/bench_results/v1.0.2/encode_decode.txt @@ -0,0 +1,136 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +lua 1.0.2 — encode!/decode! decomposition +(decode+deep_cast column: enabled) +================================================================================== +op shape N total_us per_elem_ns +encode int_list 8 0.81 101.5 +decode int_list 8 0.29 35.9 +dec+cast int_list 8 0.33 40.9 +encode int_list 64 14.59 228.0 +decode int_list 64 4.94 77.1 +dec+cast int_list 64 7.11 111.0 +encode int_list 512 158.65 309.9 +decode int_list 512 47.04 91.9 +dec+cast int_list 512 72.78 142.1 +encode int_list 4096 1568.66 383.0 +decode int_list 4096 565.55 138.1 +dec+cast int_list 4096 892.14 217.8 +---------------------------------------------------------------------------------- +encode float_list 8 0.78 98.0 +decode float_list 8 0.25 31.8 +dec+cast float_list 8 0.27 33.6 +encode float_list 64 11.77 183.9 +decode float_list 64 3.76 58.7 +dec+cast float_list 64 6.99 109.2 +encode float_list 512 162.64 317.6 +decode float_list 512 47.30 92.4 +dec+cast float_list 512 75.86 148.2 +encode float_list 4096 1614.23 394.1 +decode float_list 4096 547.77 133.7 +dec+cast float_list 4096 805.73 196.7 +---------------------------------------------------------------------------------- +encode bool_list 8 0.79 99.1 +decode bool_list 8 0.24 29.9 +dec+cast bool_list 8 0.27 33.9 +encode bool_list 64 14.65 228.8 +decode bool_list 64 4.69 73.3 +dec+cast bool_list 64 6.88 107.4 +encode bool_list 512 158.50 309.6 +decode bool_list 512 45.36 88.6 +dec+cast bool_list 512 71.69 140.0 +encode bool_list 4096 1572.86 384.0 +decode bool_list 4096 581.14 141.9 +dec+cast bool_list 4096 792.78 193.6 +---------------------------------------------------------------------------------- +encode short_string_list 8 0.78 97.8 +decode short_string_list 8 0.24 30.0 +dec+cast short_string_list 8 0.27 34.2 +encode short_string_list 64 11.49 179.6 +decode short_string_list 64 3.51 54.9 +dec+cast short_string_list 64 6.93 108.3 +encode short_string_list 512 162.32 317.0 +decode short_string_list 512 48.13 94.0 +dec+cast short_string_list 512 74.26 145.0 +encode short_string_list 4096 1664.81 406.4 +decode short_string_list 4096 550.34 134.4 +dec+cast short_string_list 4096 810.06 197.8 +---------------------------------------------------------------------------------- +encode long_string_list 8 0.79 98.9 +decode long_string_list 8 0.25 31.1 +dec+cast long_string_list 8 0.26 33.0 +encode long_string_list 64 11.67 182.4 +decode long_string_list 64 3.66 57.3 +dec+cast long_string_list 64 6.75 105.5 +encode long_string_list 512 169.63 331.3 +decode long_string_list 512 47.95 93.7 +dec+cast long_string_list 512 75.67 147.8 +encode long_string_list 4096 2667.06 651.1 +decode long_string_list 4096 1339.42 327.0 +dec+cast long_string_list 4096 1705.52 416.4 +---------------------------------------------------------------------------------- +encode string_map 8 0.72 89.9 +decode string_map 8 0.11 13.2 +dec+cast string_map 8 0.21 26.1 +encode string_map 64 12.18 190.3 +decode string_map 64 0.74 11.6 +dec+cast string_map 64 3.58 56.0 +encode string_map 512 103.35 201.9 +decode string_map 512 9.32 18.2 +dec+cast string_map 512 39.57 77.3 +encode string_map 4096 1046.22 255.4 +decode string_map 4096 79.30 19.4 +dec+cast string_map 4096 343.00 83.7 +---------------------------------------------------------------------------------- +encode int_map 8 0.81 100.9 +decode int_map 8 0.25 31.6 +dec+cast int_map 8 0.27 33.9 +encode int_map 64 11.36 177.6 +decode int_map 64 3.45 54.0 +dec+cast int_map 64 5.64 88.1 +encode int_map 512 111.00 216.8 +decode int_map 512 32.55 63.6 +dec+cast int_map 512 72.18 141.0 +encode int_map 4096 1503.89 367.2 +decode int_map 4096 599.78 146.4 +dec+cast int_map 4096 850.84 207.7 +---------------------------------------------------------------------------------- +encode record_list 8 3.10 387.3 +decode record_list 8 0.89 111.3 +dec+cast record_list 8 1.60 199.4 +encode record_list 64 45.68 713.8 +decode record_list 64 12.94 202.3 +dec+cast record_list 64 18.97 296.4 +encode record_list 512 422.71 825.6 +decode record_list 512 115.71 226.0 +dec+cast record_list 512 154.32 301.4 +encode record_list 4096 3814.88 931.4 +decode record_list 4096 1139.67 278.2 +dec+cast record_list 4096 1656.06 404.3 +---------------------------------------------------------------------------------- +nested chain (depth sweep) — isolates recursion/traversal from fan-out +op shape N total_us per_elem_ns +encode nested_chain 4 0.68 338.0 +decode nested_chain 4 0.24 122.2 +dec+cast nested_chain 4 0.36 179.7 +encode nested_chain 16 2.77 1387.2 +decode nested_chain 16 1.11 552.9 +dec+cast nested_chain 16 1.54 768.1 +encode nested_chain 64 10.94 5468.5 +decode nested_chain 64 6.00 3002.0 +dec+cast nested_chain 64 8.37 4183.6 +encode nested_chain 256 44.55 22273.9 +decode nested_chain 256 28.00 13998.0 +dec+cast nested_chain 256 37.24 18618.2 +================================================================================== +composite anchor — the PR's `original_nested` (matches the 18us/108us figure) +op shape N total_us per_elem_ns +encode original_nested 75 12.26 3065.6 +decode original_nested 75 3.29 821.3 +dec+cast original_nested 75 5.66 1413.8 diff --git a/bench_results/v1.0.2/environment.md b/bench_results/v1.0.2/environment.md new file mode 100644 index 0000000..453867b --- /dev/null +++ b/bench_results/v1.0.2/environment.md @@ -0,0 +1,36 @@ +# Benchmark environment — main (1.0.2) + +- **Ref**: `main` — the main checkout itself, commit `3a0d3923249cbb05380a3f17d3920a0b3ed55cb0` (the 1.0.2 release). No worktree was created or removed for this run; the main checkout was read but not modified. +- **Mode**: `full` (`LUA_BENCH_MODE=full`) +- **CPU**: Apple M4 (see `cpu.txt`) +- **Elixir / OTP**: Elixir 1.20.0, Erlang/OTP 29 [erts-17.0] [64-bit] [jit] (see `versions.txt`) +- **Run timestamp**: Tue Jul 28 11:14:31 EDT 2026 (see `timestamp.txt`) +- **This document written**: Tue Jul 28 11:37:22 EDT 2026 + +## Command form + +Each workload was run serially, one process per file, on an otherwise quiet +machine, from the main checkout root: + +``` +LUA_BENCH_MODE=full MIX_ENV=benchmark mix run benchmarks/.exs +``` + +Workloads run: `fibonacci`, `closures`, `oop`, `string_ops`, `string_format`, +`table_ops`, `patterns`, `metamethods`, `pcall_varargs`, `vm_new`, +`encode_decode`. + +`encode_decode.exs` is not Benchee-based (it uses a `:timer.tc` harness +directly), but was invoked with the same command form for consistency. + +C Lua via `luaport` was not available in this environment (no local luaport +build) and was skipped by each script's own fallback path; all comparisons +below are lua (chunk)/lua (eval) vs. luerl only. + +## Artifacts + +- `cpu.txt`, `versions.txt`, `timestamp.txt`, `commit.txt` — raw environment + probes. +- `.txt` — raw stdout of each `mix run` invocation (11 files). +- `summary.json` — parsed structured form of the above (workload -> case -> + jobs/comparison/memory), same schema as `results/v1.0.0/summary.json`. diff --git a/bench_results/v1.0.2/fibonacci.txt b/bench_results/v1.0.2/fibonacci.txt new file mode 100644 index 0000000..6695d13 --- /dev/null +++ b/bench_results/v1.0.2/fibonacci.txt @@ -0,0 +1,51 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 2.32 431.14 ms ±1.78% 429.30 ms 454.01 ms +lua (chunk) 2.30 435.33 ms ±2.05% 434.12 ms 469.44 ms +luerl 1.35 742.61 ms ±1.33% 741.95 ms 764.71 ms + +Comparison: +lua (eval) 2.32 +lua (chunk) 2.30 - 1.01x slower +4.18 ms +luerl 1.35 - 1.72x slower +311.46 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 1016.62 MB +lua (chunk) 1016.63 MB - 1.00x memory usage +0.00925 MB +luerl 2513.67 MB - 2.47x memory usage +1497.05 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/metamethods.txt b/bench_results/v1.0.2/metamethods.txt new file mode 100644 index 0000000..601736c --- /dev/null +++ b/bench_results/v1.0.2/metamethods.txt @@ -0,0 +1,144 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== metamethods: self-call method dispatch (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 2.10 K 476.79 μs ±13.40% 462.04 μs 730.43 μs +luerl 2.05 K 487.93 μs ±8.87% 474.63 μs 657.89 μs +lua (chunk) 2.04 K 490.34 μs ±7.16% 485.83 μs 642.74 μs + +Comparison: +lua (eval) 2.10 K +luerl 2.05 K - 1.02x slower +11.14 μs +lua (chunk) 2.04 K - 1.03x slower +13.55 μs + +Memory usage statistics: + +Name Memory usage +lua (eval) 1.37 MB +luerl 1.43 MB - 1.04x memory usage +0.0544 MB +lua (chunk) 1.36 MB - 0.99x memory usage -0.01070 MB + +**All measurements for memory usage were the same** + +=== metamethods: 3-level __index chain (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 5.24 K 190.75 μs ±4.32% 190.08 μs 206.47 μs +lua (eval) 5.09 K 196.41 μs ±6.91% 194.58 μs 221.08 μs +luerl 4.60 K 217.23 μs ±4.39% 216 μs 233.25 μs + +Comparison: +lua (chunk) 5.24 K +lua (eval) 5.09 K - 1.03x slower +5.65 μs +luerl 4.60 K - 1.14x slower +26.48 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 576.76 KB +lua (eval) 587.70 KB - 1.02x memory usage +10.95 KB +luerl 668.59 KB - 1.16x memory usage +91.83 KB + +**All measurements for memory usage were the same** + +=== metamethods: arithmetic/relational metamethods (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 533.57 1.87 ms ±5.87% 1.92 ms 2.18 ms +luerl 528.86 1.89 ms ±5.87% 1.85 ms 2.20 ms +lua (chunk) 521.21 1.92 ms ±12.56% 1.93 ms 3.10 ms + +Comparison: +lua (eval) 533.57 +luerl 528.86 - 1.01x slower +0.0167 ms +lua (chunk) 521.21 - 1.02x slower +0.0444 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 5.71 MB +luerl 6.24 MB - 1.09x memory usage +0.53 MB +lua (chunk) 5.70 MB - 1.00x memory usage -0.01050 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/oop.txt b/bench_results/v1.0.2/oop.txt new file mode 100644 index 0000000..b918533 --- /dev/null +++ b/bench_results/v1.0.2/oop.txt @@ -0,0 +1,51 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 10.47 K 95.55 μs ±13.61% 93.21 μs 154.21 μs +lua (eval) 10.27 K 97.33 μs ±13.81% 94.67 μs 153.75 μs +luerl 7.14 K 140.03 μs ±21.43% 133.50 μs 231.96 μs + +Comparison: +lua (chunk) 10.47 K +lua (eval) 10.27 K - 1.02x slower +1.78 μs +luerl 7.14 K - 1.47x slower +44.48 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 368.40 KB +lua (eval) 378.94 KB - 1.03x memory usage +10.54 KB +luerl 381.45 KB - 1.04x memory usage +13.05 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/patterns.txt b/bench_results/v1.0.2/patterns.txt new file mode 100644 index 0000000..06134f0 --- /dev/null +++ b/bench_results/v1.0.2/patterns.txt @@ -0,0 +1,144 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== patterns: find/match field extraction (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 998.95 1.00 ms ±3.02% 1.00 ms 1.06 ms +lua (eval) 986.14 1.01 ms ±10.37% 1.00 ms 1.36 ms +luerl 923.97 1.08 ms ±3.61% 1.08 ms 1.18 ms + +Comparison: +lua (chunk) 998.95 +lua (eval) 986.14 - 1.01x slower +0.0130 ms +luerl 923.97 - 1.08x slower +0.0812 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 3.21 MB +lua (eval) 3.22 MB - 1.00x memory usage +0.0110 MB +luerl 6.70 MB - 2.09x memory usage +3.49 MB + +**All measurements for memory usage were the same** + +=== patterns: find-based tokenizer (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 729.19 1.37 ms ±3.06% 1.36 ms 1.48 ms +lua (chunk) 673.77 1.48 ms ±3.57% 1.46 ms 1.61 ms +lua (eval) 661.88 1.51 ms ±4.44% 1.49 ms 1.76 ms + +Comparison: +luerl 729.19 +lua (chunk) 673.77 - 1.08x slower +0.113 ms +lua (eval) 661.88 - 1.10x slower +0.139 ms + +Memory usage statistics: + +Name Memory usage +luerl 6.65 MB +lua (chunk) 9.10 MB - 1.37x memory usage +2.45 MB +lua (eval) 9.11 MB - 1.37x memory usage +2.46 MB + +**All measurements for memory usage were the same** + +=== patterns: gsub template substitution (n=200) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 513.54 1.95 ms ±2.83% 1.95 ms 2.06 ms +lua (chunk) 480.12 2.08 ms ±16.12% 1.98 ms 3.12 ms +luerl 418.97 2.39 ms ±17.21% 2.20 ms 3.51 ms + +Comparison: +lua (eval) 513.54 +lua (chunk) 480.12 - 1.07x slower +0.136 ms +luerl 418.97 - 1.23x slower +0.44 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 4.50 MB +lua (chunk) 4.49 MB - 1.00x memory usage -0.01124 MB +luerl 11.75 MB - 2.61x memory usage +7.25 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/pcall_varargs.txt b/bench_results/v1.0.2/pcall_varargs.txt new file mode 100644 index 0000000..80d1aa5 --- /dev/null +++ b/bench_results/v1.0.2/pcall_varargs.txt @@ -0,0 +1,144 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== call protocol: pcall, success path (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 4.47 K 223.70 μs ±4.04% 222.88 μs 241.97 μs +lua (eval) 4.44 K 225.34 μs ±6.09% 224.42 μs 243.03 μs +luerl 3.40 K 294.17 μs ±6.42% 291.54 μs 317.97 μs + +Comparison: +lua (chunk) 4.47 K +lua (eval) 4.44 K - 1.01x slower +1.63 μs +luerl 3.40 K - 1.31x slower +70.46 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 895.93 KB +lua (eval) 911.78 KB - 1.02x memory usage +15.85 KB +luerl 1137.10 KB - 1.27x memory usage +241.17 KB + +**All measurements for memory usage were the same** + +=== call protocol: pcall, raise + catch (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 2.12 K 471.09 μs ±3.42% 468.42 μs 503.21 μs +lua (chunk) 1.36 K 733.85 μs ±2.60% 727.13 μs 781.15 μs +lua (eval) 1.35 K 742.90 μs ±5.35% 734.58 μs 824.44 μs + +Comparison: +luerl 2.12 K +lua (chunk) 1.36 K - 1.56x slower +262.76 μs +lua (eval) 1.35 K - 1.58x slower +271.81 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.55 MB +lua (chunk) 2.67 MB - 1.72x memory usage +1.12 MB +lua (eval) 2.68 MB - 1.72x memory usage +1.12 MB + +**All measurements for memory usage were the same** + +=== call protocol: varargs + multiple returns (n=500) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +luerl 409.82 2.44 ms ±4.21% 2.41 ms 2.74 ms +lua (eval) 306.73 3.26 ms ±1.81% 3.27 ms 3.44 ms +lua (chunk) 300.00 3.33 ms ±10.97% 3.28 ms 5.01 ms + +Comparison: +luerl 409.82 +lua (eval) 306.73 - 1.34x slower +0.82 ms +lua (chunk) 300.00 - 1.37x slower +0.89 ms + +Memory usage statistics: + +Name Memory usage +luerl 8.73 MB +lua (eval) 16.71 MB - 1.91x memory usage +7.98 MB +lua (chunk) 16.70 MB - 1.91x memory usage +7.97 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/string_format.txt b/bench_results/v1.0.2/string_format.txt new file mode 100644 index 0000000..779a9a4 --- /dev/null +++ b/bench_results/v1.0.2/string_format.txt @@ -0,0 +1,144 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== string.format: long literal-heavy format string (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 1.38 K 722.93 μs ±4.47% 716.04 μs 787.74 μs +lua (eval) 1.23 K 815.14 μs ±12.46% 759.94 μs 989.38 μs +luerl 0.25 K 3990.98 μs ±7.83% 3921.50 μs 5009.73 μs + +Comparison: +lua (chunk) 1.38 K +lua (eval) 1.23 K - 1.13x slower +92.22 μs +luerl 0.25 K - 5.52x slower +3268.06 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 2.03 MB +lua (eval) 2.04 MB - 1.01x memory usage +0.0106 MB +luerl 22.59 MB - 11.11x memory usage +20.56 MB + +**All measurements for memory usage were the same** + +=== string.format: width-flagged specifiers (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 812.70 1.23 ms ±5.82% 1.22 ms 1.33 ms +lua (eval) 804.55 1.24 ms ±2.99% 1.24 ms 1.32 ms +luerl 571.68 1.75 ms ±3.48% 1.74 ms 1.86 ms + +Comparison: +lua (chunk) 812.70 +lua (eval) 804.55 - 1.01x slower +0.0125 ms +luerl 571.68 - 1.42x slower +0.52 ms + +Memory usage statistics: + +Name Memory usage +lua (chunk) 3.52 MB +lua (eval) 3.53 MB - 1.00x memory usage +0.0132 MB +luerl 7.55 MB - 2.14x memory usage +4.03 MB + +**All measurements for memory usage were the same** + +=== string.format: many specifiers (n=1000) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (eval) 577.23 1.73 ms ±12.08% 1.70 ms 2.30 ms +lua (chunk) 566.54 1.77 ms ±3.99% 1.77 ms 1.94 ms +luerl 359.44 2.78 ms ±7.17% 2.78 ms 3.20 ms + +Comparison: +lua (eval) 577.23 +lua (chunk) 566.54 - 1.02x slower +0.0327 ms +luerl 359.44 - 1.61x slower +1.05 ms + +Memory usage statistics: + +Name Memory usage +lua (eval) 6.64 MB +lua (chunk) 6.63 MB - 1.00x memory usage -0.01004 MB +luerl 13.76 MB - 2.07x memory usage +7.11 MB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/string_ops.txt b/bench_results/v1.0.2/string_ops.txt new file mode 100644 index 0000000..a3d9c4b --- /dev/null +++ b/bench_results/v1.0.2/string_ops.txt @@ -0,0 +1,99 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== String Concatenation via table.concat (n=100) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 31.46 K 31.79 μs ±14.86% 31.21 μs 41.21 μs +lua (eval) 29.39 K 34.03 μs ±10.70% 33.63 μs 42.46 μs +luerl 24.71 K 40.47 μs ±7.41% 40.17 μs 45.88 μs + +Comparison: +lua (chunk) 31.46 K +lua (eval) 29.39 K - 1.07x slower +2.24 μs +luerl 24.71 K - 1.27x slower +8.68 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 147.68 KB +lua (eval) 161.46 KB - 1.09x memory usage +13.78 KB +luerl 172.42 KB - 1.17x memory usage +24.74 KB + +**All measurements for memory usage were the same** + +=== String Formatting via string.format (n=100) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 39 s +Excluding outliers: false + +Benchmarking lua (chunk) ... +Benchmarking lua (eval) ... +Benchmarking luerl ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (chunk) 14.08 K 71.03 μs ±8.11% 72.21 μs 80.46 μs +lua (eval) 13.20 K 75.76 μs ±16.78% 76.17 μs 92.94 μs +luerl 9.54 K 104.79 μs ±7.40% 103.04 μs 123.13 μs + +Comparison: +lua (chunk) 14.08 K +lua (eval) 13.20 K - 1.07x slower +4.73 μs +luerl 9.54 K - 1.48x slower +33.75 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 241.46 KB +lua (eval) 252.21 KB - 1.04x memory usage +10.75 KB +luerl 588.84 KB - 2.44x memory usage +347.38 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/summary.json b/bench_results/v1.0.2/summary.json new file mode 100644 index 0000000..c146681 --- /dev/null +++ b/bench_results/v1.0.2/summary.json @@ -0,0 +1,1282 @@ +{ + "fibonacci": { + "default": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "2.32", + "average": "431.14 ms", + "deviation": "±1.78%", + "median": "429.30 ms", + "p99": "454.01 ms", + "memory": "1016.62 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.30", + "average": "435.33 ms", + "deviation": "±2.05%", + "median": "434.12 ms", + "p99": "469.44 ms", + "memory": "1016.63 MB" + }, + { + "name": "luerl", + "ips": "1.35", + "average": "742.61 ms", + "deviation": "±1.33%", + "median": "741.95 ms", + "p99": "764.71 ms", + "memory": "2513.67 MB" + } + ], + "comparison": [ + "lua (eval) 2.32", + "lua (chunk) 2.30 - 1.01x slower +4.18 ms", + "luerl 1.35 - 1.72x slower +311.46 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "closures": { + "default": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "2.68 K", + "average": "372.50 μs", + "deviation": "±10.92%", + "median": "365.04 μs", + "p99": "532.92 μs", + "memory": "2.12 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.63 K", + "average": "379.93 μs", + "deviation": "±8.39%", + "median": "373.71 μs", + "p99": "512.83 μs", + "memory": "2.11 MB" + }, + { + "name": "luerl", + "ips": "2.53 K", + "average": "395.50 μs", + "deviation": "±7.19%", + "median": "389.21 μs", + "p99": "548.26 μs", + "memory": "1.90 MB" + } + ], + "comparison": [ + "lua (eval) 2.68 K", + "lua (chunk) 2.63 K - 1.02x slower +7.43 μs", + "luerl 2.53 K - 1.06x slower +23.01 μs" + ], + "memory_comparison": [ + "lua (eval) 2.12 MB", + "lua (chunk) 2.11 MB - 1.00x memory usage -0.01023 MB", + "luerl 1.90 MB - 0.89x memory usage -0.22699 MB" + ] + } + }, + "oop": { + "default": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "10.47 K", + "average": "95.55 μs", + "deviation": "±13.61%", + "median": "93.21 μs", + "p99": "154.21 μs", + "memory": "368.40 KB" + }, + { + "name": "lua (eval)", + "ips": "10.27 K", + "average": "97.33 μs", + "deviation": "±13.81%", + "median": "94.67 μs", + "p99": "153.75 μs", + "memory": "378.94 KB" + }, + { + "name": "luerl", + "ips": "7.14 K", + "average": "140.03 μs", + "deviation": "±21.43%", + "median": "133.50 μs", + "p99": "231.96 μs", + "memory": "381.45 KB" + } + ], + "comparison": [ + "lua (chunk) 10.47 K", + "lua (eval) 10.27 K - 1.02x slower +1.78 μs", + "luerl 7.14 K - 1.47x slower +44.48 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "string_ops": { + "String Concatenation via table.concat (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "31.46 K", + "average": "31.79 μs", + "deviation": "±14.86%", + "median": "31.21 μs", + "p99": "41.21 μs", + "memory": "147.68 KB" + }, + { + "name": "lua (eval)", + "ips": "29.39 K", + "average": "34.03 μs", + "deviation": "±10.70%", + "median": "33.63 μs", + "p99": "42.46 μs", + "memory": "161.46 KB" + }, + { + "name": "luerl", + "ips": "24.71 K", + "average": "40.47 μs", + "deviation": "±7.41%", + "median": "40.17 μs", + "p99": "45.88 μs", + "memory": "172.42 KB" + } + ], + "comparison": [ + "lua (chunk) 31.46 K", + "lua (eval) 29.39 K - 1.07x slower +2.24 μs", + "luerl 24.71 K - 1.27x slower +8.68 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "String Formatting via string.format (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "14.08 K", + "average": "71.03 μs", + "deviation": "±8.11%", + "median": "72.21 μs", + "p99": "80.46 μs", + "memory": "241.46 KB" + }, + { + "name": "lua (eval)", + "ips": "13.20 K", + "average": "75.76 μs", + "deviation": "±16.78%", + "median": "76.17 μs", + "p99": "92.94 μs", + "memory": "252.21 KB" + }, + { + "name": "luerl", + "ips": "9.54 K", + "average": "104.79 μs", + "deviation": "±7.40%", + "median": "103.04 μs", + "p99": "123.13 μs", + "memory": "588.84 KB" + } + ], + "comparison": [ + "lua (chunk) 14.08 K", + "lua (eval) 13.20 K - 1.07x slower +4.73 μs", + "luerl 9.54 K - 1.48x slower +33.75 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "string_format": { + "string.format: long literal-heavy format string (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "1.38 K", + "average": "722.93 μs", + "deviation": "±4.47%", + "median": "716.04 μs", + "p99": "787.74 μs", + "memory": "2.03 MB" + }, + { + "name": "lua (eval)", + "ips": "1.23 K", + "average": "815.14 μs", + "deviation": "±12.46%", + "median": "759.94 μs", + "p99": "989.38 μs", + "memory": "2.04 MB" + }, + { + "name": "luerl", + "ips": "0.25 K", + "average": "3990.98 μs", + "deviation": "±7.83%", + "median": "3921.50 μs", + "p99": "5009.73 μs", + "memory": "22.59 MB" + } + ], + "comparison": [ + "lua (chunk) 1.38 K", + "lua (eval) 1.23 K - 1.13x slower +92.22 μs", + "luerl 0.25 K - 5.52x slower +3268.06 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "string.format: width-flagged specifiers (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "812.70", + "average": "1.23 ms", + "deviation": "±5.82%", + "median": "1.22 ms", + "p99": "1.33 ms", + "memory": "3.52 MB" + }, + { + "name": "lua (eval)", + "ips": "804.55", + "average": "1.24 ms", + "deviation": "±2.99%", + "median": "1.24 ms", + "p99": "1.32 ms", + "memory": "3.53 MB" + }, + { + "name": "luerl", + "ips": "571.68", + "average": "1.75 ms", + "deviation": "±3.48%", + "median": "1.74 ms", + "p99": "1.86 ms", + "memory": "7.55 MB" + } + ], + "comparison": [ + "lua (chunk) 812.70", + "lua (eval) 804.55 - 1.01x slower +0.0125 ms", + "luerl 571.68 - 1.42x slower +0.52 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "string.format: many specifiers (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "577.23", + "average": "1.73 ms", + "deviation": "±12.08%", + "median": "1.70 ms", + "p99": "2.30 ms", + "memory": "6.64 MB" + }, + { + "name": "lua (chunk)", + "ips": "566.54", + "average": "1.77 ms", + "deviation": "±3.99%", + "median": "1.77 ms", + "p99": "1.94 ms", + "memory": "6.63 MB" + }, + { + "name": "luerl", + "ips": "359.44", + "average": "2.78 ms", + "deviation": "±7.17%", + "median": "2.78 ms", + "p99": "3.20 ms", + "memory": "13.76 MB" + } + ], + "comparison": [ + "lua (eval) 577.23", + "lua (chunk) 566.54 - 1.02x slower +0.0327 ms", + "luerl 359.44 - 1.61x slower +1.05 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "table_ops": { + "Table Build": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "6.74 K", + "average": "148.28 μs", + "deviation": "±10.00%", + "median": "145.92 μs", + "p99": "180.79 μs", + "memory": "1023.05 KB" + }, + { + "name": "lua (eval)", + "ips": "6.55 K", + "average": "152.58 μs", + "deviation": "±18.06%", + "median": "148.88 μs", + "p99": "259.21 μs", + "memory": "1033.39 KB" + }, + { + "name": "luerl", + "ips": "6.48 K", + "average": "154.42 μs", + "deviation": "±9.20%", + "median": "152.71 μs", + "p99": "189.18 μs", + "memory": "996.96 KB" + } + ], + "comparison": [ + "lua (chunk) 6.74 K", + "lua (eval) 6.55 K - 1.03x slower +4.30 μs", + "luerl 6.48 K - 1.04x slower +6.15 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "65.36 K", + "average": "15.30 μs", + "deviation": "±46.45%", + "median": "15 μs", + "p99": "24.63 μs", + "memory": "104.51 KB" + }, + { + "name": "luerl", + "ips": "58.37 K", + "average": "17.13 μs", + "deviation": "±38.71%", + "median": "16.63 μs", + "p99": "27.96 μs", + "memory": "110.78 KB" + }, + { + "name": "lua (eval)", + "ips": "56.97 K", + "average": "17.55 μs", + "deviation": "±21.90%", + "median": "17.21 μs", + "p99": "26.13 μs", + "memory": "115.14 KB" + } + ], + "comparison": [ + "lua (chunk) 65.36 K", + "luerl 58.37 K - 1.12x slower +1.83 μs", + "lua (eval) 56.97 K - 1.15x slower +2.25 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "473.88 K", + "average": "2.11 μs", + "deviation": "±259.11%", + "median": "2.04 μs", + "p99": "3.21 μs", + "memory": "12.89 KB" + }, + { + "name": "luerl", + "ips": "319.03 K", + "average": "3.13 μs", + "deviation": "±185.41%", + "median": "3.04 μs", + "p99": "4.54 μs", + "memory": "22.62 KB" + }, + { + "name": "lua (eval)", + "ips": "234.54 K", + "average": "4.26 μs", + "deviation": "±124.39%", + "median": "4.13 μs", + "p99": "11.13 μs", + "memory": "23.84 KB" + } + ], + "comparison": [ + "lua (chunk) 473.88 K", + "luerl 319.03 K - 1.49x slower +1.02 μs", + "lua (eval) 234.54 K - 2.02x slower +2.15 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Sort": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "luerl", + "ips": "5.41 K", + "average": "184.80 μs", + "deviation": "±18.74%", + "median": "179.09 μs", + "p99": "413.57 μs", + "memory": "1.17 MB" + }, + { + "name": "lua (chunk)", + "ips": "4.88 K", + "average": "205.09 μs", + "deviation": "±13.18%", + "median": "199.34 μs", + "p99": "322.32 μs", + "memory": "1.22 MB" + }, + { + "name": "lua (eval)", + "ips": "4.82 K", + "average": "207.48 μs", + "deviation": "±16.82%", + "median": "202.59 μs", + "p99": "278.13 μs", + "memory": "1.24 MB" + } + ], + "comparison": [ + "luerl 5.41 K", + "lua (chunk) 4.88 K - 1.11x slower +20.29 μs", + "lua (eval) 4.82 K - 1.12x slower +22.68 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "luerl", + "ips": "51.19 K", + "average": "19.54 μs", + "deviation": "±19.26%", + "median": "19.04 μs", + "p99": "28.13 μs", + "memory": "133.16 KB" + }, + { + "name": "lua (chunk)", + "ips": "47.57 K", + "average": "21.02 μs", + "deviation": "±75.79%", + "median": "20.38 μs", + "p99": "44.00 μs", + "memory": "129.16 KB" + }, + { + "name": "lua (eval)", + "ips": "43.10 K", + "average": "23.20 μs", + "deviation": "±19.83%", + "median": "22.79 μs", + "p99": "28.67 μs", + "memory": "140.11 KB" + } + ], + "comparison": [ + "luerl 51.19 K", + "lua (chunk) 47.57 K - 1.08x slower +1.49 μs", + "lua (eval) 43.10 K - 1.19x slower +3.67 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "343.00 K", + "average": "2.92 μs", + "deviation": "±179.73%", + "median": "2.88 μs", + "p99": "4.17 μs", + "memory": "16.81 KB" + }, + { + "name": "luerl", + "ips": "267.77 K", + "average": "3.73 μs", + "deviation": "±158.30%", + "median": "3.63 μs", + "p99": "5.67 μs", + "memory": "25.98 KB" + }, + { + "name": "lua (eval)", + "ips": "193.98 K", + "average": "5.16 μs", + "deviation": "±94.77%", + "median": "5.04 μs", + "p99": "8.29 μs", + "memory": "27.75 KB" + } + ], + "comparison": [ + "lua (chunk) 343.00 K", + "luerl 267.77 K - 1.28x slower +0.82 μs", + "lua (eval) 193.98 K - 1.77x slower +2.24 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Iterate/Sum": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "4.61 K", + "average": "216.80 μs", + "deviation": "±10.43%", + "median": "212.71 μs", + "p99": "315.74 μs", + "memory": "1.47 MB" + }, + { + "name": "lua (eval)", + "ips": "4.45 K", + "average": "224.56 μs", + "deviation": "±17.51%", + "median": "216.17 μs", + "p99": "421.98 μs", + "memory": "1.48 MB" + }, + { + "name": "luerl", + "ips": "4.01 K", + "average": "249.67 μs", + "deviation": "±8.67%", + "median": "245.63 μs", + "p99": "340.05 μs", + "memory": "1.35 MB" + } + ], + "comparison": [ + "lua (chunk) 4.61 K", + "lua (eval) 4.45 K - 1.04x slower +7.76 μs", + "luerl 4.01 K - 1.15x slower +32.87 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "45.22 K", + "average": "22.12 μs", + "deviation": "±15.19%", + "median": "21.75 μs", + "p99": "31.46 μs", + "memory": "152.91 KB" + }, + { + "name": "lua (eval)", + "ips": "40.41 K", + "average": "24.74 μs", + "deviation": "±35.77%", + "median": "24.29 μs", + "p99": "34.58 μs", + "memory": "164.10 KB" + }, + { + "name": "luerl", + "ips": "37.45 K", + "average": "26.70 μs", + "deviation": "±12.27%", + "median": "26.50 μs", + "p99": "36.50 μs", + "memory": "149.13 KB" + } + ], + "comparison": [ + "lua (chunk) 45.22 K", + "lua (eval) 40.41 K - 1.12x slower +2.63 μs", + "luerl 37.45 K - 1.21x slower +4.58 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "335.95 K", + "average": "2.98 μs", + "deviation": "±201.50%", + "median": "2.88 μs", + "p99": "7 μs", + "memory": "18.55 KB" + }, + { + "name": "luerl", + "ips": "220.48 K", + "average": "4.54 μs", + "deviation": "±148.78%", + "median": "4.29 μs", + "p99": "11.21 μs", + "memory": "26.80 KB" + }, + { + "name": "lua (eval)", + "ips": "197.51 K", + "average": "5.06 μs", + "deviation": "±103.13%", + "median": "4.96 μs", + "p99": "12.13 μs", + "memory": "29.64 KB" + } + ], + "comparison": [ + "lua (chunk) 335.95 K", + "luerl 220.48 K - 1.52x slower +1.56 μs", + "lua (eval) 197.51 K - 1.70x slower +2.09 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Map + Reduce": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "2.22 K", + "average": "450.65 μs", + "deviation": "±6.37%", + "median": "443.17 μs", + "p99": "596.71 μs", + "memory": "2.91 MB" + }, + { + "name": "luerl", + "ips": "2.20 K", + "average": "454.46 μs", + "deviation": "±6.67%", + "median": "446.63 μs", + "p99": "600.22 μs", + "memory": "2.44 MB" + }, + { + "name": "lua (eval)", + "ips": "2.20 K", + "average": "455.45 μs", + "deviation": "±6.47%", + "median": "447.96 μs", + "p99": "597.34 μs", + "memory": "2.92 MB" + } + ], + "comparison": [ + "lua (chunk) 2.22 K", + "luerl 2.20 K - 1.01x slower +3.81 μs", + "lua (eval) 2.20 K - 1.01x slower +4.80 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "22.40 K", + "average": "44.64 μs", + "deviation": "±9.68%", + "median": "44.21 μs", + "p99": "55.09 μs", + "memory": "302.05 KB" + }, + { + "name": "luerl", + "ips": "21.26 K", + "average": "47.05 μs", + "deviation": "±8.41%", + "median": "46.63 μs", + "p99": "56.42 μs", + "memory": "262.49 KB" + }, + { + "name": "lua (eval)", + "ips": "20.96 K", + "average": "47.70 μs", + "deviation": "±17.39%", + "median": "46.71 μs", + "p99": "67.33 μs", + "memory": "313.15 KB" + } + ], + "comparison": [ + "lua (chunk) 22.40 K", + "luerl 21.26 K - 1.05x slower +2.41 μs", + "lua (eval) 20.96 K - 1.07x slower +3.07 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "186.34 K", + "average": "5.37 μs", + "deviation": "±89.27%", + "median": "5.21 μs", + "p99": "12.79 μs", + "memory": "35.40 KB" + }, + { + "name": "luerl", + "ips": "149.86 K", + "average": "6.67 μs", + "deviation": "±89.30%", + "median": "6.58 μs", + "p99": "13.71 μs", + "memory": "39.39 KB" + }, + { + "name": "lua (eval)", + "ips": "132.59 K", + "average": "7.54 μs", + "deviation": "±81.79%", + "median": "7.42 μs", + "p99": "15 μs", + "memory": "46.17 KB" + } + ], + "comparison": [ + "lua (chunk) 186.34 K", + "luerl 149.86 K - 1.24x slower +1.31 μs", + "lua (eval) 132.59 K - 1.41x slower +2.18 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + }, + "Table Pairs (hash)": { + "by_input": { + "large (n=1000)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "1.02 K", + "average": "982.50 μs", + "deviation": "±9.43%", + "median": "999.49 μs", + "p99": "1154.53 μs", + "memory": "2.06 MB" + }, + { + "name": "lua (chunk)", + "ips": "1.00 K", + "average": "998.42 μs", + "deviation": "±13.14%", + "median": "1001.12 μs", + "p99": "1477.94 μs", + "memory": "2.05 MB" + }, + { + "name": "luerl", + "ips": "0.74 K", + "average": "1347.55 μs", + "deviation": "±4.60%", + "median": "1349.77 μs", + "p99": "1465.70 μs", + "memory": "2.48 MB" + } + ], + "comparison": [ + "lua (eval) 1.02 K", + "lua (chunk) 1.00 K - 1.02x slower +15.92 μs", + "luerl 0.74 K - 1.37x slower +365.05 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "medium (n=100)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "13.32 K", + "average": "75.07 μs", + "deviation": "±9.83%", + "median": "74.46 μs", + "p99": "96.75 μs", + "memory": "209.77 KB" + }, + { + "name": "lua (eval)", + "ips": "12.97 K", + "average": "77.10 μs", + "deviation": "±11.95%", + "median": "75.33 μs", + "p99": "106.04 μs", + "memory": "221.48 KB" + }, + { + "name": "luerl", + "ips": "11.37 K", + "average": "87.93 μs", + "deviation": "±10.51%", + "median": "86.75 μs", + "p99": "123.60 μs", + "memory": "248.85 KB" + } + ], + "comparison": [ + "lua (chunk) 13.32 K", + "lua (eval) 12.97 K - 1.03x slower +2.04 μs", + "luerl 11.37 K - 1.17x slower +12.86 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "small (n=10)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "134.58 K", + "average": "7.43 μs", + "deviation": "±85.96%", + "median": "7.13 μs", + "p99": "18.33 μs", + "memory": "24.09 KB" + }, + { + "name": "luerl", + "ips": "128.74 K", + "average": "7.77 μs", + "deviation": "±71.32%", + "median": "7.38 μs", + "p99": "23.96 μs", + "memory": "36.16 KB" + }, + { + "name": "lua (eval)", + "ips": "98.68 K", + "average": "10.13 μs", + "deviation": "±86.13%", + "median": "9.54 μs", + "p99": "24.92 μs", + "memory": "34.87 KB" + } + ], + "comparison": [ + "lua (chunk) 134.58 K", + "luerl 128.74 K - 1.05x slower +0.34 μs", + "lua (eval) 98.68 K - 1.36x slower +2.70 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + } + } + }, + "patterns": { + "patterns: find/match field extraction (n=200)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "998.95", + "average": "1.00 ms", + "deviation": "±3.02%", + "median": "1.00 ms", + "p99": "1.06 ms", + "memory": "3.21 MB" + }, + { + "name": "lua (eval)", + "ips": "986.14", + "average": "1.01 ms", + "deviation": "±10.37%", + "median": "1.00 ms", + "p99": "1.36 ms", + "memory": "3.22 MB" + }, + { + "name": "luerl", + "ips": "923.97", + "average": "1.08 ms", + "deviation": "±3.61%", + "median": "1.08 ms", + "p99": "1.18 ms", + "memory": "6.70 MB" + } + ], + "comparison": [ + "lua (chunk) 998.95", + "lua (eval) 986.14 - 1.01x slower +0.0130 ms", + "luerl 923.97 - 1.08x slower +0.0812 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "patterns: find-based tokenizer (n=200)": { + "jobs": [ + { + "name": "luerl", + "ips": "729.19", + "average": "1.37 ms", + "deviation": "±3.06%", + "median": "1.36 ms", + "p99": "1.48 ms", + "memory": "6.65 MB" + }, + { + "name": "lua (chunk)", + "ips": "673.77", + "average": "1.48 ms", + "deviation": "±3.57%", + "median": "1.46 ms", + "p99": "1.61 ms", + "memory": "9.10 MB" + }, + { + "name": "lua (eval)", + "ips": "661.88", + "average": "1.51 ms", + "deviation": "±4.44%", + "median": "1.49 ms", + "p99": "1.76 ms", + "memory": "9.11 MB" + } + ], + "comparison": [ + "luerl 729.19", + "lua (chunk) 673.77 - 1.08x slower +0.113 ms", + "lua (eval) 661.88 - 1.10x slower +0.139 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "patterns: gsub template substitution (n=200)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "513.54", + "average": "1.95 ms", + "deviation": "±2.83%", + "median": "1.95 ms", + "p99": "2.06 ms", + "memory": "4.50 MB" + }, + { + "name": "lua (chunk)", + "ips": "480.12", + "average": "2.08 ms", + "deviation": "±16.12%", + "median": "1.98 ms", + "p99": "3.12 ms", + "memory": "4.49 MB" + }, + { + "name": "luerl", + "ips": "418.97", + "average": "2.39 ms", + "deviation": "±17.21%", + "median": "2.20 ms", + "p99": "3.51 ms", + "memory": "11.75 MB" + } + ], + "comparison": [ + "lua (eval) 513.54", + "lua (chunk) 480.12 - 1.07x slower +0.136 ms", + "luerl 418.97 - 1.23x slower +0.44 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "metamethods": { + "metamethods: self-call method dispatch (n=200)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "2.10 K", + "average": "476.79 μs", + "deviation": "±13.40%", + "median": "462.04 μs", + "p99": "730.43 μs", + "memory": "1.37 MB" + }, + { + "name": "luerl", + "ips": "2.05 K", + "average": "487.93 μs", + "deviation": "±8.87%", + "median": "474.63 μs", + "p99": "657.89 μs", + "memory": "1.43 MB" + }, + { + "name": "lua (chunk)", + "ips": "2.04 K", + "average": "490.34 μs", + "deviation": "±7.16%", + "median": "485.83 μs", + "p99": "642.74 μs", + "memory": "1.36 MB" + } + ], + "comparison": [ + "lua (eval) 2.10 K", + "luerl 2.05 K - 1.02x slower +11.14 μs", + "lua (chunk) 2.04 K - 1.03x slower +13.55 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "metamethods: 3-level __index chain (n=200)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "5.24 K", + "average": "190.75 μs", + "deviation": "±4.32%", + "median": "190.08 μs", + "p99": "206.47 μs", + "memory": "576.76 KB" + }, + { + "name": "lua (eval)", + "ips": "5.09 K", + "average": "196.41 μs", + "deviation": "±6.91%", + "median": "194.58 μs", + "p99": "221.08 μs", + "memory": "587.70 KB" + }, + { + "name": "luerl", + "ips": "4.60 K", + "average": "217.23 μs", + "deviation": "±4.39%", + "median": "216 μs", + "p99": "233.25 μs", + "memory": "668.59 KB" + } + ], + "comparison": [ + "lua (chunk) 5.24 K", + "lua (eval) 5.09 K - 1.03x slower +5.65 μs", + "luerl 4.60 K - 1.14x slower +26.48 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "metamethods: arithmetic/relational metamethods (n=200)": { + "jobs": [ + { + "name": "lua (eval)", + "ips": "533.57", + "average": "1.87 ms", + "deviation": "±5.87%", + "median": "1.92 ms", + "p99": "2.18 ms", + "memory": "5.71 MB" + }, + { + "name": "luerl", + "ips": "528.86", + "average": "1.89 ms", + "deviation": "±5.87%", + "median": "1.85 ms", + "p99": "2.20 ms", + "memory": "6.24 MB" + }, + { + "name": "lua (chunk)", + "ips": "521.21", + "average": "1.92 ms", + "deviation": "±12.56%", + "median": "1.93 ms", + "p99": "3.10 ms", + "memory": "5.70 MB" + } + ], + "comparison": [ + "lua (eval) 533.57", + "luerl 528.86 - 1.01x slower +0.0167 ms", + "lua (chunk) 521.21 - 1.02x slower +0.0444 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "pcall_varargs": { + "call protocol: pcall, success path (n=500)": { + "jobs": [ + { + "name": "lua (chunk)", + "ips": "4.47 K", + "average": "223.70 μs", + "deviation": "±4.04%", + "median": "222.88 μs", + "p99": "241.97 μs", + "memory": "895.93 KB" + }, + { + "name": "lua (eval)", + "ips": "4.44 K", + "average": "225.34 μs", + "deviation": "±6.09%", + "median": "224.42 μs", + "p99": "243.03 μs", + "memory": "911.78 KB" + }, + { + "name": "luerl", + "ips": "3.40 K", + "average": "294.17 μs", + "deviation": "±6.42%", + "median": "291.54 μs", + "p99": "317.97 μs", + "memory": "1137.10 KB" + } + ], + "comparison": [ + "lua (chunk) 4.47 K", + "lua (eval) 4.44 K - 1.01x slower +1.63 μs", + "luerl 3.40 K - 1.31x slower +70.46 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "call protocol: pcall, raise + catch (n=500)": { + "jobs": [ + { + "name": "luerl", + "ips": "2.12 K", + "average": "471.09 μs", + "deviation": "±3.42%", + "median": "468.42 μs", + "p99": "503.21 μs", + "memory": "1.55 MB" + }, + { + "name": "lua (chunk)", + "ips": "1.36 K", + "average": "733.85 μs", + "deviation": "±2.60%", + "median": "727.13 μs", + "p99": "781.15 μs", + "memory": "2.67 MB" + }, + { + "name": "lua (eval)", + "ips": "1.35 K", + "average": "742.90 μs", + "deviation": "±5.35%", + "median": "734.58 μs", + "p99": "824.44 μs", + "memory": "2.68 MB" + } + ], + "comparison": [ + "luerl 2.12 K", + "lua (chunk) 1.36 K - 1.56x slower +262.76 μs", + "lua (eval) 1.35 K - 1.58x slower +271.81 μs" + ], + "memory_note": "**All measurements for memory usage were the same**" + }, + "call protocol: varargs + multiple returns (n=500)": { + "jobs": [ + { + "name": "luerl", + "ips": "409.82", + "average": "2.44 ms", + "deviation": "±4.21%", + "median": "2.41 ms", + "p99": "2.74 ms", + "memory": "8.73 MB" + }, + { + "name": "lua (eval)", + "ips": "306.73", + "average": "3.26 ms", + "deviation": "±1.81%", + "median": "3.27 ms", + "p99": "3.44 ms", + "memory": "16.71 MB" + }, + { + "name": "lua (chunk)", + "ips": "300.00", + "average": "3.33 ms", + "deviation": "±10.97%", + "median": "3.28 ms", + "p99": "5.01 ms", + "memory": "16.70 MB" + } + ], + "comparison": [ + "luerl 409.82", + "lua (eval) 306.73 - 1.34x slower +0.82 ms", + "lua (chunk) 300.00 - 1.37x slower +0.89 ms" + ], + "memory_note": "**All measurements for memory usage were the same**" + } + }, + "vm_new": { + "VM instantiation: Lua.new/1 vs :luerl.init/0": { + "jobs": [ + { + "name": "lua (new, no sandbox)", + "ips": "1.90 M", + "average": "0.53 μs", + "deviation": "±805.41%", + "median": "0.50 μs", + "p99": "0.63 μs", + "memory": "0.95 KB" + }, + { + "name": "lua (new)", + "ips": "1.59 M", + "average": "0.63 μs", + "deviation": "±1209.11%", + "median": "0.58 μs", + "p99": "0.75 μs", + "memory": "0.88 KB" + }, + { + "name": "lua (new, custom exclude)", + "ips": "0.151 M", + "average": "6.63 μs", + "deviation": "±88.13%", + "median": "6.54 μs", + "p99": "8.79 μs", + "memory": "22.57 KB" + }, + { + "name": "luerl (init)", + "ips": "0.0656 M", + "average": "15.24 μs", + "deviation": "±36.68%", + "median": "15.13 μs", + "p99": "22.42 μs", + "memory": "51.64 KB" + } + ], + "comparison": [ + "lua (new, no sandbox) 1.90 M", + "lua (new) 1.59 M - 1.20x slower +0.104 μs", + "lua (new, custom exclude) 0.151 M - 12.61x slower +6.10 μs", + "luerl (init) 0.0656 M - 29.00x slower +14.71 μs" + ], + "memory_note": "**All measurements for memory usage were the same**", + "cold_call": "6892.0 us", + "second_call": "1.0 us" + } + }, + "encode_decode": { + "raw": "lua 1.0.2 — encode!/decode! decomposition\n(decode+deep_cast column: enabled)\n==================================================================================\nop shape N total_us per_elem_ns\nencode int_list 8 0.81 101.5\ndecode int_list 8 0.29 35.9\ndec+cast int_list 8 0.33 40.9\nencode int_list 64 14.59 228.0\ndecode int_list 64 4.94 77.1\ndec+cast int_list 64 7.11 111.0\nencode int_list 512 158.65 309.9\ndecode int_list 512 47.04 91.9\ndec+cast int_list 512 72.78 142.1\nencode int_list 4096 1568.66 383.0\ndecode int_list 4096 565.55 138.1\ndec+cast int_list 4096 892.14 217.8\n----------------------------------------------------------------------------------\nencode float_list 8 0.78 98.0\ndecode float_list 8 0.25 31.8\ndec+cast float_list 8 0.27 33.6\nencode float_list 64 11.77 183.9\ndecode float_list 64 3.76 58.7\ndec+cast float_list 64 6.99 109.2\nencode float_list 512 162.64 317.6\ndecode float_list 512 47.30 92.4\ndec+cast float_list 512 75.86 148.2\nencode float_list 4096 1614.23 394.1\ndecode float_list 4096 547.77 133.7\ndec+cast float_list 4096 805.73 196.7\n----------------------------------------------------------------------------------\nencode bool_list 8 0.79 99.1\ndecode bool_list 8 0.24 29.9\ndec+cast bool_list 8 0.27 33.9\nencode bool_list 64 14.65 228.8\ndecode bool_list 64 4.69 73.3\ndec+cast bool_list 64 6.88 107.4\nencode bool_list 512 158.50 309.6\ndecode bool_list 512 45.36 88.6\ndec+cast bool_list 512 71.69 140.0\nencode bool_list 4096 1572.86 384.0\ndecode bool_list 4096 581.14 141.9\ndec+cast bool_list 4096 792.78 193.6\n----------------------------------------------------------------------------------\nencode short_string_list 8 0.78 97.8\ndecode short_string_list 8 0.24 30.0\ndec+cast short_string_list 8 0.27 34.2\nencode short_string_list 64 11.49 179.6\ndecode short_string_list 64 3.51 54.9\ndec+cast short_string_list 64 6.93 108.3\nencode short_string_list 512 162.32 317.0\ndecode short_string_list 512 48.13 94.0\ndec+cast short_string_list 512 74.26 145.0\nencode short_string_list 4096 1664.81 406.4\ndecode short_string_list 4096 550.34 134.4\ndec+cast short_string_list 4096 810.06 197.8\n----------------------------------------------------------------------------------\nencode long_string_list 8 0.79 98.9\ndecode long_string_list 8 0.25 31.1\ndec+cast long_string_list 8 0.26 33.0\nencode long_string_list 64 11.67 182.4\ndecode long_string_list 64 3.66 57.3\ndec+cast long_string_list 64 6.75 105.5\nencode long_string_list 512 169.63 331.3\ndecode long_string_list 512 47.95 93.7\ndec+cast long_string_list 512 75.67 147.8\nencode long_string_list 4096 2667.06 651.1\ndecode long_string_list 4096 1339.42 327.0\ndec+cast long_string_list 4096 1705.52 416.4\n----------------------------------------------------------------------------------\nencode string_map 8 0.72 89.9\ndecode string_map 8 0.11 13.2\ndec+cast string_map 8 0.21 26.1\nencode string_map 64 12.18 190.3\ndecode string_map 64 0.74 11.6\ndec+cast string_map 64 3.58 56.0\nencode string_map 512 103.35 201.9\ndecode string_map 512 9.32 18.2\ndec+cast string_map 512 39.57 77.3\nencode string_map 4096 1046.22 255.4\ndecode string_map 4096 79.30 19.4\ndec+cast string_map 4096 343.00 83.7\n----------------------------------------------------------------------------------\nencode int_map 8 0.81 100.9\ndecode int_map 8 0.25 31.6\ndec+cast int_map 8 0.27 33.9\nencode int_map 64 11.36 177.6\ndecode int_map 64 3.45 54.0\ndec+cast int_map 64 5.64 88.1\nencode int_map 512 111.00 216.8\ndecode int_map 512 32.55 63.6\ndec+cast int_map 512 72.18 141.0\nencode int_map 4096 1503.89 367.2\ndecode int_map 4096 599.78 146.4\ndec+cast int_map 4096 850.84 207.7\n----------------------------------------------------------------------------------\nencode record_list 8 3.10 387.3\ndecode record_list 8 0.89 111.3\ndec+cast record_list 8 1.60 199.4\nencode record_list 64 45.68 713.8\ndecode record_list 64 12.94 202.3\ndec+cast record_list 64 18.97 296.4\nencode record_list 512 422.71 825.6\ndecode record_list 512 115.71 226.0\ndec+cast record_list 512 154.32 301.4\nencode record_list 4096 3814.88 931.4\ndecode record_list 4096 1139.67 278.2\ndec+cast record_list 4096 1656.06 404.3\n----------------------------------------------------------------------------------\nnested chain (depth sweep) — isolates recursion/traversal from fan-out\nop shape N total_us per_elem_ns\nencode nested_chain 4 0.68 338.0\ndecode nested_chain 4 0.24 122.2\ndec+cast nested_chain 4 0.36 179.7\nencode nested_chain 16 2.77 1387.2\ndecode nested_chain 16 1.11 552.9\ndec+cast nested_chain 16 1.54 768.1\nencode nested_chain 64 10.94 5468.5\ndecode nested_chain 64 6.00 3002.0\ndec+cast nested_chain 64 8.37 4183.6\nencode nested_chain 256 44.55 22273.9\ndecode nested_chain 256 28.00 13998.0\ndec+cast nested_chain 256 37.24 18618.2\n==================================================================================\ncomposite anchor — the PR's `original_nested` (matches the 18us/108us figure)\nop shape N total_us per_elem_ns\nencode original_nested 75 12.26 3065.6\ndecode original_nested 75 3.29 821.3\ndec+cast original_nested 75 5.66 1413.8\n" + } +} diff --git a/bench_results/v1.0.2/table_ops.txt b/bench_results/v1.0.2/table_ops.txt new file mode 100644 index 0000000..6d49993 --- /dev/null +++ b/bench_results/v1.0.2/table_ops.txt @@ -0,0 +1,469 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + +luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks + +=== Table Build (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (chunk) 6.74 K 148.28 μs ±10.00% 145.92 μs 180.79 μs +lua (eval) 6.55 K 152.58 μs ±18.06% 148.88 μs 259.21 μs +luerl 6.48 K 154.42 μs ±9.20% 152.71 μs 189.18 μs + +Comparison: +lua (chunk) 6.74 K +lua (eval) 6.55 K - 1.03x slower +4.30 μs +luerl 6.48 K - 1.04x slower +6.15 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1023.05 KB +lua (eval) 1033.39 KB - 1.01x memory usage +10.34 KB +luerl 996.96 KB - 0.97x memory usage -26.09375 KB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 65.36 K 15.30 μs ±46.45% 15 μs 24.63 μs +luerl 58.37 K 17.13 μs ±38.71% 16.63 μs 27.96 μs +lua (eval) 56.97 K 17.55 μs ±21.90% 17.21 μs 26.13 μs + +Comparison: +lua (chunk) 65.36 K +luerl 58.37 K - 1.12x slower +1.83 μs +lua (eval) 56.97 K - 1.15x slower +2.25 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 104.51 KB +luerl 110.78 KB - 1.06x memory usage +6.27 KB +lua (eval) 115.14 KB - 1.10x memory usage +10.63 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 473.88 K 2.11 μs ±259.11% 2.04 μs 3.21 μs +luerl 319.03 K 3.13 μs ±185.41% 3.04 μs 4.54 μs +lua (eval) 234.54 K 4.26 μs ±124.39% 4.13 μs 11.13 μs + +Comparison: +lua (chunk) 473.88 K +luerl 319.03 K - 1.49x slower +1.02 μs +lua (eval) 234.54 K - 2.02x slower +2.15 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 12.89 KB +luerl 22.62 KB - 1.75x memory usage +9.73 KB +lua (eval) 23.84 KB - 1.85x memory usage +10.95 KB + +**All measurements for memory usage were the same** + +=== Table Sort (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +luerl 5.41 K 184.80 μs ±18.74% 179.09 μs 413.57 μs +lua (chunk) 4.88 K 205.09 μs ±13.18% 199.34 μs 322.32 μs +lua (eval) 4.82 K 207.48 μs ±16.82% 202.59 μs 278.13 μs + +Comparison: +luerl 5.41 K +lua (chunk) 4.88 K - 1.11x slower +20.29 μs +lua (eval) 4.82 K - 1.12x slower +22.68 μs + +Memory usage statistics: + +Name Memory usage +luerl 1.17 MB +lua (chunk) 1.22 MB - 1.04x memory usage +0.0500 MB +lua (eval) 1.24 MB - 1.05x memory usage +0.0602 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +luerl 51.19 K 19.54 μs ±19.26% 19.04 μs 28.13 μs +lua (chunk) 47.57 K 21.02 μs ±75.79% 20.38 μs 44.00 μs +lua (eval) 43.10 K 23.20 μs ±19.83% 22.79 μs 28.67 μs + +Comparison: +luerl 51.19 K +lua (chunk) 47.57 K - 1.08x slower +1.49 μs +lua (eval) 43.10 K - 1.19x slower +3.67 μs + +Memory usage statistics: + +Name Memory usage +luerl 133.16 KB +lua (chunk) 129.16 KB - 0.97x memory usage -3.99219 KB +lua (eval) 140.11 KB - 1.05x memory usage +6.95 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 343.00 K 2.92 μs ±179.73% 2.88 μs 4.17 μs +luerl 267.77 K 3.73 μs ±158.30% 3.63 μs 5.67 μs +lua (eval) 193.98 K 5.16 μs ±94.77% 5.04 μs 8.29 μs + +Comparison: +lua (chunk) 343.00 K +luerl 267.77 K - 1.28x slower +0.82 μs +lua (eval) 193.98 K - 1.77x slower +2.24 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 16.81 KB +luerl 25.98 KB - 1.55x memory usage +9.16 KB +lua (eval) 27.75 KB - 1.65x memory usage +10.94 KB + +**All measurements for memory usage were the same** + +=== Table Iterate/Sum (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (chunk) 4.61 K 216.80 μs ±10.43% 212.71 μs 315.74 μs +lua (eval) 4.45 K 224.56 μs ±17.51% 216.17 μs 421.98 μs +luerl 4.01 K 249.67 μs ±8.67% 245.63 μs 340.05 μs + +Comparison: +lua (chunk) 4.61 K +lua (eval) 4.45 K - 1.04x slower +7.76 μs +luerl 4.01 K - 1.15x slower +32.87 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 1.47 MB +lua (eval) 1.48 MB - 1.01x memory usage +0.0119 MB +luerl 1.35 MB - 0.91x memory usage -0.12543 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 45.22 K 22.12 μs ±15.19% 21.75 μs 31.46 μs +lua (eval) 40.41 K 24.74 μs ±35.77% 24.29 μs 34.58 μs +luerl 37.45 K 26.70 μs ±12.27% 26.50 μs 36.50 μs + +Comparison: +lua (chunk) 45.22 K +lua (eval) 40.41 K - 1.12x slower +2.63 μs +luerl 37.45 K - 1.21x slower +4.58 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 152.91 KB +lua (eval) 164.10 KB - 1.07x memory usage +11.20 KB +luerl 149.13 KB - 0.98x memory usage -3.78125 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 335.95 K 2.98 μs ±201.50% 2.88 μs 7 μs +luerl 220.48 K 4.54 μs ±148.78% 4.29 μs 11.21 μs +lua (eval) 197.51 K 5.06 μs ±103.13% 4.96 μs 12.13 μs + +Comparison: +lua (chunk) 335.95 K +luerl 220.48 K - 1.52x slower +1.56 μs +lua (eval) 197.51 K - 1.70x slower +2.09 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 18.55 KB +luerl 26.80 KB - 1.44x memory usage +8.25 KB +lua (eval) 29.64 KB - 1.60x memory usage +11.09 KB + +**All measurements for memory usage were the same** + +=== Table Map + Reduce (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (chunk) 2.22 K 450.65 μs ±6.37% 443.17 μs 596.71 μs +luerl 2.20 K 454.46 μs ±6.67% 446.63 μs 600.22 μs +lua (eval) 2.20 K 455.45 μs ±6.47% 447.96 μs 597.34 μs + +Comparison: +lua (chunk) 2.22 K +luerl 2.20 K - 1.01x slower +3.81 μs +lua (eval) 2.20 K - 1.01x slower +4.80 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 2.91 MB +luerl 2.44 MB - 0.84x memory usage -0.46843 MB +lua (eval) 2.92 MB - 1.00x memory usage +0.0121 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 22.40 K 44.64 μs ±9.68% 44.21 μs 55.09 μs +luerl 21.26 K 47.05 μs ±8.41% 46.63 μs 56.42 μs +lua (eval) 20.96 K 47.70 μs ±17.39% 46.71 μs 67.33 μs + +Comparison: +lua (chunk) 22.40 K +luerl 21.26 K - 1.05x slower +2.41 μs +lua (eval) 20.96 K - 1.07x slower +3.07 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 302.05 KB +luerl 262.49 KB - 0.87x memory usage -39.55469 KB +lua (eval) 313.15 KB - 1.04x memory usage +11.10 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 186.34 K 5.37 μs ±89.27% 5.21 μs 12.79 μs +luerl 149.86 K 6.67 μs ±89.30% 6.58 μs 13.71 μs +lua (eval) 132.59 K 7.54 μs ±81.79% 7.42 μs 15 μs + +Comparison: +lua (chunk) 186.34 K +luerl 149.86 K - 1.24x slower +1.31 μs +lua (eval) 132.59 K - 1.41x slower +2.18 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 35.40 KB +luerl 39.39 KB - 1.11x memory usage +3.99 KB +lua (eval) 46.17 KB - 1.30x memory usage +10.77 KB + +**All measurements for memory usage were the same** + +=== Table Pairs (hash) (mode: full) === + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: large (n=1000), medium (n=100), small (n=10) +Estimated total run time: 1 min 57 s +Excluding outliers: false + +Benchmarking lua (chunk) with input large (n=1000) ... +Benchmarking lua (chunk) with input medium (n=100) ... +Benchmarking lua (chunk) with input small (n=10) ... +Benchmarking lua (eval) with input large (n=1000) ... +Benchmarking lua (eval) with input medium (n=100) ... +Benchmarking lua (eval) with input small (n=10) ... +Benchmarking luerl with input large (n=1000) ... +Benchmarking luerl with input medium (n=100) ... +Benchmarking luerl with input small (n=10) ... +Calculating statistics... +Formatting results... + +##### With input large (n=1000) ##### +Name ips average deviation median 99th % +lua (eval) 1.02 K 982.50 μs ±9.43% 999.49 μs 1154.53 μs +lua (chunk) 1.00 K 998.42 μs ±13.14% 1001.12 μs 1477.94 μs +luerl 0.74 K 1347.55 μs ±4.60% 1349.77 μs 1465.70 μs + +Comparison: +lua (eval) 1.02 K +lua (chunk) 1.00 K - 1.02x slower +15.92 μs +luerl 0.74 K - 1.37x slower +365.05 μs + +Memory usage statistics: + +Name Memory usage +lua (eval) 2.06 MB +lua (chunk) 2.05 MB - 0.99x memory usage -0.01048 MB +luerl 2.48 MB - 1.20x memory usage +0.42 MB + +**All measurements for memory usage were the same** + +##### With input medium (n=100) ##### +Name ips average deviation median 99th % +lua (chunk) 13.32 K 75.07 μs ±9.83% 74.46 μs 96.75 μs +lua (eval) 12.97 K 77.10 μs ±11.95% 75.33 μs 106.04 μs +luerl 11.37 K 87.93 μs ±10.51% 86.75 μs 123.60 μs + +Comparison: +lua (chunk) 13.32 K +lua (eval) 12.97 K - 1.03x slower +2.04 μs +luerl 11.37 K - 1.17x slower +12.86 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 209.77 KB +lua (eval) 221.48 KB - 1.06x memory usage +11.72 KB +luerl 248.85 KB - 1.19x memory usage +39.09 KB + +**All measurements for memory usage were the same** + +##### With input small (n=10) ##### +Name ips average deviation median 99th % +lua (chunk) 134.58 K 7.43 μs ±85.96% 7.13 μs 18.33 μs +luerl 128.74 K 7.77 μs ±71.32% 7.38 μs 23.96 μs +lua (eval) 98.68 K 10.13 μs ±86.13% 9.54 μs 24.92 μs + +Comparison: +lua (chunk) 134.58 K +luerl 128.74 K - 1.05x slower +0.34 μs +lua (eval) 98.68 K - 1.36x slower +2.70 μs + +Memory usage statistics: + +Name Memory usage +lua (chunk) 24.09 KB +luerl 36.16 KB - 1.50x memory usage +12.06 KB +lua (eval) 34.87 KB - 1.45x memory usage +10.77 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/v1.0.2/timestamp.txt b/bench_results/v1.0.2/timestamp.txt new file mode 100644 index 0000000..2a01292 --- /dev/null +++ b/bench_results/v1.0.2/timestamp.txt @@ -0,0 +1 @@ +Tue Jul 28 11:14:31 EDT 2026 diff --git a/bench_results/v1.0.2/versions.txt b/bench_results/v1.0.2/versions.txt new file mode 100644 index 0000000..184e5f9 --- /dev/null +++ b/bench_results/v1.0.2/versions.txt @@ -0,0 +1,3 @@ +Erlang/OTP 29 [erts-17.0] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] + +Elixir 1.20.0 (compiled with Erlang/OTP 29) diff --git a/bench_results/v1.0.2/vm_new.txt b/bench_results/v1.0.2/vm_new.txt new file mode 100644 index 0000000..7ece932 --- /dev/null +++ b/bench_results/v1.0.2/vm_new.txt @@ -0,0 +1,65 @@ +zoxide: detected a possible configuration issue. +Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). + +If the issue persists, consider filing an issue at: +https://github.com/ajeetdsouza/zoxide/issues + +Disable this message by setting _ZO_DOCTOR=0. + + +=== VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full) === + +Lua.new() one-time vs repeat cost (single samples, informational): + first call on this node : 6892.0 us + second call : 1.0 us + +The first figure includes any one-time template build and first-time module +loading. Benchee's steady-state numbers below are the per-request cost after +that point. + +Operating System: macOS +CPU Information: Apple M4 +Number of Available Cores: 10 +Available memory: 32 GB +Elixir 1.20.0 +Erlang 29.0 +JIT enabled: true + +Benchmark suite executing with the following configuration: +warmup: 2 s +time: 10 s +memory time: 1 s +reduction time: 0 ns +parallel: 1 +inputs: none specified +Estimated total run time: 52 s +Excluding outliers: false + +Benchmarking lua (new) ... +Benchmarking lua (new, custom exclude) ... +Benchmarking lua (new, no sandbox) ... +Benchmarking luerl (init) ... +Calculating statistics... +Formatting results... + +Name ips average deviation median 99th % +lua (new, no sandbox) 1.90 M 0.53 μs ±805.41% 0.50 μs 0.63 μs +lua (new) 1.59 M 0.63 μs ±1209.11% 0.58 μs 0.75 μs +lua (new, custom exclude) 0.151 M 6.63 μs ±88.13% 6.54 μs 8.79 μs +luerl (init) 0.0656 M 15.24 μs ±36.68% 15.13 μs 22.42 μs + +Comparison: +lua (new, no sandbox) 1.90 M +lua (new) 1.59 M - 1.20x slower +0.104 μs +lua (new, custom exclude) 0.151 M - 12.61x slower +6.10 μs +luerl (init) 0.0656 M - 29.00x slower +14.71 μs + +Memory usage statistics: + +Name Memory usage +lua (new, no sandbox) 0.95 KB +lua (new) 0.88 KB - 0.92x memory usage -0.07813 KB +lua (new, custom exclude) 22.57 KB - 23.68x memory usage +21.62 KB +luerl (init) 51.64 KB - 54.18x memory usage +50.69 KB + +**All measurements for memory usage were the same** diff --git a/bench_results/versions-2026-07-28.md b/bench_results/versions-2026-07-28.md new file mode 100644 index 0000000..e04a1b6 --- /dev/null +++ b/bench_results/versions-2026-07-28.md @@ -0,0 +1,574 @@ +# Cross-version benchmarks — v0.4.0 vs v1.0.0 vs 1.0.2 + +Recorded 2026-07-28 for the comparative perf run tracked in +[#267](https://github.com/tv-labs/lua/issues/267). + +This report **supersedes the 1.0.0-era figures in +[`benchmarks/BASELINE.md`](../benchmarks/BASELINE.md)**. That file remains in +the tree as the historical record of the 1.0.0 perf gate; every number in it +predates the call-convention, peephole and bootstrap work that landed after +1.0.0, and several of its conclusions no longer hold (notably its "every +workload is within 25% of Luerl" gate statement — see +[Where we are still behind](#where-we-are-still-behind)). Use this document, +not `BASELINE.md`, for current numbers. + +## What is being compared + +Three points in this library's history, each measured in its own process, on +the same machine, in the same sitting: + +| Column | Ref | What it is | +|---|---|---| +| **v0.4.0** | tag `v0.4.0` | The last release before the native VM. `Lua` was a thin Elixir wrapper over [Luerl](https://github.com/rvirding/luerl) 1.5.1 — parsing, evaluation and the whole standard library were Luerl's. | +| **v1.0.0** | tag `v1.0.0`, commit `69e13a6` | The first release running this project's own Lua 5.3 VM: own lexer, parser, compiler, bytecode dispatcher and standard library. No Luerl on the execution path. | +| **1.0.2** | `main`, commit `3a0d392` | Current release. Adds the peephole/fused-opcode pass ([#403](https://github.com/tv-labs/lua/pull/403)), the one-allocation register file and static-arity call convention ([#405](https://github.com/tv-labs/lua/pull/405)), and the memoized boot template ([#398](https://github.com/tv-labs/lua/pull/398)). | + +### Why Luerl appears in every table + +Every benchmark script runs a Luerl 1.5.1 job **in the same process, in the +same Benchee run, against the same Lua source** as the `lua` jobs. Luerl is +therefore not a competitor here so much as a **control**: it is the one thing +held constant across all three refs, so it absorbs machine drift, thermal +state, and OTP-level variation. When a ratio moves between columns, the Luerl +control tells you whether the library moved or the machine did. + +The control held very steady. Across the three independent runs, Luerl's +`:luerl.init()` median was 15.04 µs / 15.21 µs / 15.13 µs, and its +`string.format` many-specifier median was 2.80 ms / 2.82 ms / 2.78 ms — under +1% spread. Where the control *did* drift, +it is called out inline. + +Ratios below are always **`lua (chunk)` median ÷ same-run `luerl` median**. +Lower is better; **below 1.00 means faster than Luerl in that same run**. + +### Environment + +- Apple M4 (arm64), Elixir 1.20.0, Erlang/OTP 29 [erts-17.0] [64-bit] [jit] +- `LUA_BENCH_MODE=full` (2 s warmup, 10 s measurement, memory measurement on, + and the multi-size sweep for the table workloads) +- Comparison control: Luerl `~> 1.5` (1.5.1) +- C Lua via `:luaport` was **not** available in this environment; each script's + own fallback path skipped it. There are no C Lua rows anywhere in this report. + +### Measurement discipline + +Every workload was run **serially, one `mix run` process at a time, on an +otherwise quiet machine**. This is not optional. Running these concurrently +with tests or other agents inflates deviation to the point where the table and +OOP cases swing wildly and orderings flip. All figures below are the +quiet-machine read. + +**Medians are quoted as primary throughout**, not averages. On workloads with +allocation-driven GC pauses the mean is pulled around by a small number of +long iterations; the median is the number an embedding host actually +experiences per call. Averages, p99s, deviations and memory figures are all in +the `summary.json` files if you want them. + +--- + +## Numeric code: recursion and call overhead + +`fibonacci` computes `fib(30)` recursively through a global function — roughly +2.7 million calls per iteration, each one a small integer compare, two +recursive calls and an add. It is deliberately the worst case for call +overhead: almost nothing happens between calls, so the measurement is +dominated by frame setup, argument passing and return. This is the workload +the static-arity call convention and one-allocation register file +([#405](https://github.com/tv-labs/lua/pull/405)) were built for. + +| Ref | lua (chunk) median | ips | luerl median (same run) | ratio | +|---|---|---|---|---| +| v0.4.0 | 702.84 ms | 1.42 | 720.40 ms | 0.98 | +| v1.0.0 | 792.42 ms | 1.26 | 730.11 ms | 1.09 | +| **1.0.2** | **434.12 ms** | **2.30** | 741.95 ms | **0.59** | + +1.0.0 shipped this workload 9% *slower* than the Luerl control. 1.0.2 runs it +**1.7× faster than Luerl** and 1.83× faster than 1.0.0. Allocation moved with +it: 2.90 GB per iteration on 1.0.0 → 1016.63 MB on 1.0.2, against Luerl's +2513.67 MB in the same run. + +`lua (eval)` is within 1% of `lua (chunk)` here (429.30 ms vs 434.12 ms) — at +this workload size the one-time parse is invisible next to 400 ms of +execution. + +## Closures and upvalues + +`closures` builds 100 counter closures through a factory function, each +capturing and mutating its own upvalue, then calls each one ten times. It +exercises closure allocation, upvalue capture, and mutation of a captured +local through a closure boundary — the machinery `BASELINE.md` flagged as the +weakest area at 1.0.0. + +| Ref | lua (chunk) median | ips | luerl median (same run) | ratio | +|---|---|---|---|---| +| v0.4.0 | 368.92 µs | 2.69 K | 391.42 µs | 0.94 | +| v1.0.0 | 473.54 µs | 2.09 K | 382.50 µs | 1.24 | +| **1.0.2** | **373.71 µs** | **2.63 K** | 389.21 µs | **0.96** | + +This is the clearest single vindication of the post-1.0 work: the 1.25× gap +`BASELINE.md` recorded as "at the bar" reproduces exactly (1.24× here) and is +now closed — 1.0.2 is marginally faster than the control and 1.27× faster than +1.0.0. + +## OOP via metatables + +Two workloads cover this. `oop` is the shallow, construction-dominated end: 50 +instances per iteration, each two field writes plus a `setmetatable`, then one +field-style method call whose body concatenates two fields. `metamethods` +covers dispatch: `:` self-calls (including a chained call on a freshly +constructed receiver), a three-level `__index` prototype chain resolving at +depths 1/2/3, and arithmetic/relational metamethods (`__add`, `__sub`, `__lt`, +`__eq`, `__tostring`) which route through the arithmetic opcodes' metamethod +fallback rather than through table indexing. + +| Case | Ref | lua (chunk) median | ips | luerl median | ratio | +|---|---|---|---|---|---| +| oop (construction) | v0.4.0 | 110.88 µs | 8.80 K | 106.50 µs | 1.04 | +| | v1.0.0 | 123.50 µs | 7.88 K | 116.13 µs | 1.06 | +| | **1.0.2** | **93.21 µs** | **10.47 K** | 133.50 µs | **0.70** | +| self-call dispatch | v0.4.0 | 476.58 µs | 2.04 K | 492.79 µs | 0.97 | +| | v1.0.0 | 553.75 µs | 1.75 K | 472.63 µs | 1.17 | +| | **1.0.2** | **485.83 µs** | **2.04 K** | 474.63 µs | **1.02** | +| 3-level `__index` chain | v0.4.0 | 216.42 µs | 4.57 K | 217.92 µs | 0.99 | +| | v1.0.0 | 237.13 µs | 4.19 K | 221.29 µs | 1.07 | +| | **1.0.2** | **190.08 µs** | **5.24 K** | 216 µs | **0.88** | +| arithmetic metamethods | v0.4.0 | 1.88 ms | 513.47 | 1.88 ms | 1.00 | +| | v1.0.0 | 2.23 ms | 455.21 | 1.88 ms | 1.19 | +| | **1.0.2** | **1.93 ms** | **521.21** | 1.85 ms | **1.04** | + +Prototype-chain walking is now 12% faster than the control and self-call +dispatch has essentially reached parity (1.02×) from 1.17× at 1.0.0 — the +`call_self` fusion in [#405](https://github.com/tv-labs/lua/pull/405) doing its +job. Arithmetic metamethods improved from 1.19× to 1.04× but have not quite +closed. + +**Control drift warning:** the `oop` Luerl median moved 106.50 → 116.13 → +133.50 µs across the three runs (deviations ±13.7% to ±21.4%), a 25% spread on +the one job that should be identical. `oop`'s ratio column is the least +trustworthy in this report; the raw `lua (chunk)` medians (110.88 → 123.50 → +93.21 µs) tell the story more reliably than the ratios do. + +## Strings: building and formatting + +`string_ops` covers string building: `table.concat` over 100 parts, and +`string.format("item_%d=%f")` in a 100-iteration loop. `string_format` pushes +the formatter on three separate axes at n=1000: a long literal-heavy format +string (~430 characters of literal text around three specifiers, so the cost +is copying literal bytes), many width-flagged specifiers (`%-20s`, `%8d`, +`%12.4f`, `%6x` — the padding path on every conversion), and a dozen +specifiers interleaved with short literals (the conversion-heavy counterpart). + +| Case | Ref | lua (chunk) median | ips | luerl median | ratio | +|---|---|---|---|---|---| +| `table.concat` (n=100) | v0.4.0 | 38.38 µs | 25.94 K | 39.42 µs | 0.97 | +| | v1.0.0 | 35.67 µs | 27.97 K | 40.17 µs | 0.89 | +| | **1.0.2** | **31.21 µs** | **31.46 K** | 40.17 µs | **0.78** | +| `string.format` loop (n=100) | v0.4.0 | 101.46 µs | 9.75 K | 102.83 µs | 0.99 | +| | v1.0.0 | 80.21 µs | 12.44 K | 103.38 µs | 0.78 | +| | **1.0.2** | **72.21 µs** | **14.08 K** | 103.04 µs | **0.70** | +| format: long literal-heavy | v0.4.0 | 3.91 ms | 249.24 | 4.25 ms | 0.92 | +| | v1.0.0 | 1.03 ms | 966.41 | 4.32 ms | 0.24 | +| | **1.0.2** | **716.04 µs** | **1.38 K** | 3921.50 µs | **0.18** | +| format: width-flagged | v0.4.0 | 1.72 ms | 582.97 | 1.73 ms | 0.99 | +| | v1.0.0 | 1.50 ms | 662.60 | 1.79 ms | 0.84 | +| | **1.0.2** | **1.22 ms** | **812.70** | 1.74 ms | **0.70** | +| format: many specifiers | v0.4.0 | 2.75 ms | 367.47 | 2.80 ms | 0.98 | +| | v1.0.0 | 2.35 ms | 413.90 | 2.82 ms | 0.83 | +| | **1.0.2** | **1.77 ms** | **566.54** | 2.78 ms | **0.64** | + +`string.format` is this library's strongest area. The literal-heavy case runs +**5.5× faster than Luerl** (716.04 µs vs 3921.50 µs) and allocates 2.03 MB +against Luerl's 22.59 MB — an 11× allocation reduction — thanks to the +bare-specifier fast path, the exact bignum fixed-precision float formatter, and +the parsed-template cache. Every string case improved monotonically across all +three versions. + +## Tables: array and hash parts + +`table_ops` covers five operations, swept over n=10/100/1000 in full mode: +building an array of squares, sorting a reverse-ordered array (worst case), +summing by index, a two-pass map-then-reduce, and a full `pairs` walk over a +**string-keyed** table, which forces every entry into the hash part and drives +the memoized hash-iteration path. + +The **n=1000** column is quoted below. The n=10 and n=100 cells are in the +JSON but carry deviations of ±20–46% at 2–25 µs, which is not enough signal to +publish a ratio from; one such cell (v0.4.0 build at n=100) reads 1.60× purely +from noise, on a ref where the `lua` and `luerl` rows are the same engine. + +| Operation (n=1000) | Ref | lua (chunk) median | ips | luerl median | ratio | +|---|---|---|---|---|---| +| build | v0.4.0 | 149.92 µs | 6.35 K | 152.33 µs | 0.98 | +| | v1.0.0 | 157.88 µs | 6.23 K | 154.21 µs | 1.02 | +| | **1.0.2** | **145.92 µs** | **6.74 K** | 152.71 µs | **0.96** | +| sort | v0.4.0 | 174.29 µs | 5.61 K | 180.59 µs | 0.97 | +| | v1.0.0 | 227.63 µs | 4.27 K | 178.00 µs | 1.28 | +| | **1.0.2** | **199.34 µs** | **4.88 K** | 179.09 µs | **1.11** | +| iterate / sum | v0.4.0 | 242.42 µs | 3.60 K | 243.13 µs | 1.00 | +| | v1.0.0 | 235.09 µs | 4.10 K | 243.06 µs | 0.97 | +| | **1.0.2** | **212.71 µs** | **4.61 K** | 245.63 µs | **0.87** | +| map + reduce | v0.4.0 | 444.34 µs | 2.19 K | 442.21 µs | 1.00 | +| | v1.0.0 | 480.24 µs | 2.05 K | 449.96 µs | 1.07 | +| | **1.0.2** | **443.17 µs** | **2.22 K** | 446.63 µs | **0.99** | +| `pairs` (hash part) | v0.4.0 | 1.32 ms | 759.91 | 1.33 ms | 0.99 | +| | v1.0.0 | 995.70 µs | 1.02 K | 1334.35 µs | 0.75 | +| | **1.0.2** | **1001.12 µs** | **1.00 K** | 1349.77 µs | **0.74** | + +Hash-part iteration is the standout: **1.35× faster than Luerl** at n=1000, +where the memoized order-index makes each `next` step O(1) instead of rescanning +an order list. `table.sort` is the one table operation still behind (1.11×, +improved from 1.28× at 1.0.0) — the array part is rebuilt with a single +`:array.from_list/2` write-back, but the comparison-driven sort itself remains +more expensive than Luerl's. + +## String patterns + +`patterns` exercises the Lua pattern engine, which had no coverage at all +before this report. Three cases: field extraction from a log line with +`string.find` and `string.match` (character classes, quantifiers, escaped magic +characters, captures); a tokenizer that re-enters `string.find(s, "[^,]+", pos)` +at an advancing init offset; and template substitution using all three `gsub` +replacement kinds — a table replacement driven by a capture, a string +replacement over `%s+`, and a function replacement invoked per word. + +The tokenizer deliberately avoids `string.gmatch`. Luerl 1.5.1 raises +`{:badarg, :gmatch, ...}` for any `gmatch` call, so a `gmatch`-based tokenizer +could not be measured against the control on identical Lua source, and keeping +the source identical across engines is this suite's fairness contract. + +| Case | Ref | lua (chunk) median | ips | luerl median | ratio | +|---|---|---|---|---|---| +| find/match extraction | v0.4.0 | 1.07 ms | 919.82 | 1.05 ms | 1.02 | +| | v1.0.0 | 1.01 ms | 972.71 | 1.09 ms | 0.93 | +| | **1.0.2** | **1.00 ms** | **998.95** | 1.08 ms | **0.93** | +| find-based tokenizer | v0.4.0 | 1.37 ms | 721.24 | 1.36 ms | 1.01 | +| | v1.0.0 | 1.57 ms | 626.48 | 1.35 ms | 1.16 | +| | **1.0.2** | **1.46 ms** | **673.77** | 1.36 ms | **1.07** | +| gsub template substitution | v0.4.0 | 2.21 ms | 436.67 | 2.21 ms | 1.00 | +| | v1.0.0 | 2.02 ms | 491.94 | 2.17 ms | 0.93 | +| | **1.0.2** | **1.98 ms** | **480.12** | 2.20 ms | **0.90** | + +Extraction and `gsub` are 7–10% faster than the control. The tokenizer is 7% +behind: it restarts the matcher once per token with a fresh start position, and +that per-restart setup is where the remaining gap sits. Allocation is mixed — +`gsub` allocates 4.49 MB against Luerl's 11.75 MB, but the tokenizer allocates +9.10 MB against Luerl's 6.65 MB. + +## Call protocol: protected calls, varargs, multiple returns + +`pcall_varargs` covers the parts of the call protocol that every host +embedding depends on and that had no coverage before this report. Three cases, +all n=500: protected calls that all succeed (the production-common case, which +isolates the cost of entering and leaving a protected frame); protected calls +that all raise a string error and are caught (error construction, unwinding, +returning `false, err`); and variadic handling — `select("#", ...)`, +positional `select(i, ...)`, tail-position `f(...)` forwarding, +`table.pack`/`table.unpack` round-tripping, and multiple-return destructuring. + +| Case | Ref | lua (chunk) median | ips | luerl median | ratio | +|---|---|---|---|---|---| +| pcall, success path | v0.4.0 | 290.50 µs | 3.40 K | 289.67 µs | 1.00 | +| | v1.0.0 | 282.13 µs | 3.49 K | 289.50 µs | 0.97 | +| | **1.0.2** | **222.88 µs** | **4.47 K** | 291.54 µs | **0.76** | +| pcall, raise + catch | v0.4.0 | 469.29 µs | 2.11 K | 466.38 µs | 1.01 | +| | v1.0.0 | 877.33 µs | 1.11 K | 485.38 µs | 1.81 | +| | **1.0.2** | **727.13 µs** | **1.36 K** | 468.42 µs | **1.55** | +| varargs + multiple returns | v0.4.0 | 2.40 ms | 391.27 | 2.36 ms | 1.02 | +| | v1.0.0 | 4.01 ms | 237.27 | 2.40 ms | 1.67 | +| | **1.0.2** | **3.28 ms** | **300.00** | 2.41 ms | **1.36** | + +The success path — the one that actually runs on every request in a host that +wraps script entry points in `pcall` — is **1.31× faster than Luerl** and 1.27× +faster than 1.0.0. The other two are the subject of +[Where we are still behind](#where-we-are-still-behind). + +## `Lua.new/1`: VM instantiation + +`vm_new` measures the cost of standing up a VM before any Lua runs. A host +that builds a fresh sandbox per request — the recommended isolation model — +pays this on every request. Three instantiation shapes are measured, chosen +because `:sandboxed` and `:exclude` are the only `new/1` options that exist +across the whole version history: + +- **`Lua.new()`** — the default deny-list sandbox. What most embedders call. +- **`Lua.new(sandboxed: [])`** — standard library installed, zero sandbox + passes. The closest like-for-like analogue of a bare `:luerl.init()`, which + also performs no sandboxing. **Compare the Luerl row against this row.** +- **`Lua.new(exclude: [[:require]])`** — the default deny-list minus one entry. + "I want the sandbox but need one thing back." + +| Shape | v0.4.0 median | v1.0.0 median | 1.0.2 median | +|---|---|---|---| +| `Lua.new()` | 21.33 µs | 36.67 µs | **0.58 µs** | +| `Lua.new(sandboxed: [])` | 14.88 µs | 29.71 µs | **0.50 µs** | +| `Lua.new(exclude: [[:require]])` | 20.63 µs | 35.96 µs | **6.54 µs** | +| `:luerl.init()` (control) | 15.04 µs | 15.21 µs | 15.13 µs | + +Allocation for `Lua.new()` fell from 91.88 KB (1.0.0) to **0.88 KB** (1.0.2) — +104× less — which is the expected signature of returning a shared memoized +template rather than rebuilding one. + +### Validating the "~100×" claim + +The 1.0.2 changelog and [#419](https://github.com/tv-labs/lua/pull/419) claim +"`Lua.new/1` ~100× faster via a memoized boot-time VM template +([#398](https://github.com/tv-labs/lua/pull/398))". The claim holds in +substance, with four qualifications that anyone quoting it should carry: + +1. **Measured 63× here, not 100×, and that is expected.** 36.67 µs → 0.58 µs + by median is **63×**; by average (37.63 µs → 0.63 µs) it is **60×**. These + runs are under `mix run`, which puts the VM in `:interactive` code-loading + mode. In that mode a cache hit re-verifies the module-reload fingerprint + that #398 describes as costing ~0.2 µs of a ~0.42 µs hit. Under `:embedded` + mode — releases — that check is skipped entirely and the ~100× figure is + credible. **Quote the mode alongside the multiplier.** +2. **It is shape-specific.** Only the exactly-default `Lua.new()` hits the + fully-sandboxed template. Pass any custom `:sandboxed` or `:exclude` and you + get the shared pre-sandbox install but still pay your own sandbox pass: + 35.96 µs → 6.54 µs, i.e. **5.5×, not 63×**. Passing `sandboxed:` is common, + so an unqualified "100× faster `Lua.new`" overstates it for a real fraction + of users. +3. **The history is non-monotonic.** v1.0.0 *regressed* instantiation relative + to v0.4.0 (36.67 µs vs 21.33 µs — 1.72× slower), because the native VM's + standard-library install cost more than Luerl's `init`. 1.0.2 does not + merely recover that; it beats v0.4.0 by **36.8×**. A two-point + "0.4.0 → 1.0.2" comparison would hide a real regression that existed in + between. +4. **There is a one-time cost, and it is not zero.** The template is written + once per node. The first `Lua.new()` on a node measured 6892 µs on 1.0.2, + the second 1.0 µs. That first figure is an **upper bound, not the template + build cost** — under `mix run` it also absorbs first-time loading of the + standard-library modules, which a release has already done at boot. (For + scale, the same cold measurement is 6654 µs on v1.0.0 and 10070 µs on + v0.4.0, i.e. the *larger* cold number belongs to the *slower* library.) Do + not publish "the first call costs 6.9 ms" as a property of the memoization. + +**Unequal work across refs.** `Lua.new()` does not sandbox the same number of +paths on every ref: v0.4.0's default deny-list has 14 entries, v1.0.0 and 1.0.2 +have 27 (the single `[:io]` entry was split into 14 per-function paths). The +`Lua.new()` row is therefore "what the public API costs on that release", not +identical work. The `sandboxed: []` row is the identical-work comparison, and +it is the one the Luerl control should be read against. + +**Deviation on the sub-microsecond rows.** 1.0.2's `Lua.new()` reports +±1209% deviation. This is Benchee measuring an operation near timer +resolution and batching to compensate, not instability — the median (0.58 µs) +and p99 (0.75 µs) are tight. Use medians for these rows and do not reproduce +the deviation column without this note. + +## Host boundary: `encode!` / `decode!` + +`encode_decode` measures the Elixir↔Lua data boundary — `Lua.encode!/2`, +`Lua.decode!/2`, and `Lua.Table.deep_cast/1` — across container shapes (integer +/ float / boolean / short-string / long-string lists, string-keyed maps, +integer-keyed maps, record-shaped maps) at N = 8/64/512/4096, plus a +nested-chain depth sweep and a composite anchor. It uses its own `:timer.tc` +harness rather than Benchee, reports per-element nanoseconds so a super-linear +curve is visible, and **has no Luerl comparator** — it measures this library's +own boundary code, which on v0.4.0 happens to be Luerl's term conversion. + +This is the one area where the native VM is **materially behind the +Luerl-backed v0.4.0**, and it should be published as such. + +| Operation | v0.4.0 | v1.0.0 | 1.0.2 | 1.0.2 vs v0.4.0 | +|---|---|---|---|---| +| `encode` int_list, N=4096 | 226.97 µs | 1601.11 µs | 1568.66 µs | **6.91× slower** | +| `decode` int_list, N=4096 | 43.47 µs | 583.70 µs | 565.55 µs | **13.01× slower** | +| `decode` long_string_list, N=4096 | 54.39 µs | 1495.20 µs | 1339.42 µs | **24.63× slower** | +| `encode` record_list, N=4096 | 2346.38 µs | 4732.25 µs | 3814.88 µs | 1.63× slower | +| `encode` string_map, N=4096 | 1295.09 µs | 2194.00 µs | 1046.22 µs | **0.81× (faster)** | +| `dec+cast` string_map, N=4096 | 367.76 µs | 335.83 µs | 343.00 µs | 0.93× (faster) | +| `encode` original_nested (composite) | 3.48 µs | 17.11 µs | 12.26 µs | 3.52× slower | +| `decode` original_nested (composite) | 0.58 µs | 3.21 µs | 3.29 µs | 5.67× slower | + +Two things are true at once. **List-shaped payloads regressed badly** at the +boundary and have barely improved since 1.0.0 — integer lists ~7× slower to +encode and ~13× slower to decode than v0.4.0, long-string lists ~25× slower to +decode. **String-keyed maps went the other way**: 1.0.2 encodes them 1.24× +faster than v0.4.0 and 2.1× faster than 1.0.0 (2194.00 → 1046.22 µs), and +`decode + deep_cast` on them is at parity or better across all three refs. + +If your host passes large lists across the boundary per request, v0.4.0 was +faster at that specific thing and this is a known gap, not a measurement +artifact. It is the clearest optimisation target this report surfaces. + +## Where we are still behind + +Three cases where 1.0.2 is slower than the same-run Luerl control. All three +improved relative to 1.0.0; none has closed. + +| Case | v1.0.0 ratio | 1.0.2 ratio | 1.0.2 median vs control | +|---|---|---|---| +| pcall, raise + catch | 1.81× | **1.55×** | 727.13 µs vs 468.42 µs | +| varargs + multiple returns | 1.67× | **1.36×** | 3.28 ms vs 2.41 ms | +| `table.sort` (n=1000) | 1.28× | **1.11×** | 199.34 µs vs 179.09 µs | +| patterns: find-based tokenizer | 1.16× | **1.07×** | 1.46 ms vs 1.36 ms | + +**The raise path is not an apples-to-apples work comparison.** On 1.0.2, +`error("negative")` produces `":1: negative"` — position-prefixed, which +is what PUC-Lua does at error level 1. Luerl (and therefore v0.4.0) produces a +bare `"negative"` with no position information. 1.0.2 is doing strictly more +work per raise, and the extra work is the *conformant* behaviour. The 1.55× +should be read as "we pay 1.55× for a more correct error value", not as pure +overhead. It is still worth optimising — position capture need not cost this +much — but it is not a like-for-like loss. + +**Varargs is a genuine loss.** 1.36× slower and allocating 16.70 MB against +the control's 8.73 MB (1.91×) on the same source. Variadic collection and +`table.pack`/`unpack` round-tripping are doing measurably more allocation than +they need to. This is the most actionable pure-performance gap in the suite. + +For the record, `BASELINE.md`'s 1.0.0 gate statement — "every workload is +within 25% of Luerl on the chunk path" — was true of the workloads that +existed when it was written. It is not a property of the current, wider suite: +the pcall-raise and varargs cases were added for this report and both exceed +that band, on 1.0.0 as well as on 1.0.2. + +## Caveats + +1. **v0.4.0 *is* Luerl.** The v0.4.0 column is a thin Elixir wrapper over + `luerl 1.5.1`, so its `lua` rows and its own `luerl` control row measure + substantially the same engine. The data shows this directly: + `Lua.new(sandboxed: [])` at 14.88 µs vs `:luerl.init()` at 15.04 µs (1.1% + apart); `table.concat` at 38.38 µs vs 39.42 µs; `gsub` substitution at + 2.21 ms vs 2.21 ms; arithmetic metamethods at 1.88 ms vs 1.88 ms. **v0.4.0 + is not an independent data point.** Read it as "what an embedder got before + the native VM", and read the v0.4.0 ratio column as a noise floor — it + should sit at 1.00, and where it strays (0.94 on closures, 1.04 on oop) that + is the measurement error budget for this whole report. + +2. **Coroutines are absent by design and are not benchmarked.** The + `coroutine` library is not implemented on v1.0.0 or 1.0.2 — an explicit 1.0 + capability exclusion, recorded as such in the Lua 5.3 suite skip list. Any + coroutine workload would run only on v0.4.0. This is a real gap in + "representative Lua" coverage and is disclosed rather than benchmarked + around. If your embedding needs coroutines, this library does not currently + provide them. + +3. **Four of these workloads are new as of this report.** `patterns`, + `metamethods`, `pcall_varargs` and `vm_new` were added specifically for this + comparison, to cover the pattern engine, self-call/metamethod dispatch, the + protected-call and variadic protocol, and instantiation — none of which the + pre-existing suite touched. They are run against all three refs from the + same source files, so the cross-version numbers are valid, but they have no + history before 2026-07-28 and no `BASELINE.md` counterpart. + +4. **Single machine, single OTP version.** Everything here is one Apple M4 on + Elixir 1.20.0 / OTP 29. Ratios against the same-run Luerl control should + travel reasonably well; absolute microsecond figures will not. Nothing here + has been reproduced on x86_64, on another OTP release, or under a different + scheduler configuration. + +5. **Sub-microsecond rows need medians.** The `Lua.new()` rows on 1.0.2 report + deviations in the hundreds to over a thousand percent because the operation + is near timer resolution and Benchee batches to compensate. Medians and p99s + are tight and meaningful; the deviation column is not, and should not be + republished for those rows without explanation. + +6. **`table_ops` small and medium sizes are too noisy to publish ratios from.** + At n=10 and n=100 the deviations run ±20–46% on operations taking 2–25 µs. + One such cell (v0.4.0 build at n=100) reads 1.60× purely from noise on a ref + where both rows are the same engine. Only the n=1000 column is quoted above. + +7. **No C Lua reference.** `:luaport` was unavailable in this environment, so + there is no PUC-Lua/C row anywhere here. Nothing in this report should be + read as a comparison against reference Lua; it is a comparison of this + library against itself over time, with Luerl as the control. + +8. **`lua (eval)` vs `lua (chunk)`.** The `chunk` path — compile once, run many + — is quoted throughout because it is the production embedding path. The + `eval` path (parse on every call) is within a few percent on most workloads; + the gap is largest where the script is small relative to its parse, and it + inverts on a few cases where measurement noise exceeds the difference. Full + `eval` figures are in the JSON. + +## Reproduction + +All raw stdout, parsed JSON and environment probes for this run are under +[`v0.4.0/`](./v0.4.0/), [`v1.0.0/`](./v1.0.0/) and [`v1.0.2/`](./v1.0.2/), +one directory per released version. Every figure in this +report is traceable to the `median` field of a job in the corresponding +`summary.json`. To regenerate: + +### 1.0.2 / main + +Benchee and Luerl are gated to the `:benchmark` env, so `MIX_ENV=benchmark` is +mandatory. + +```sh +MIX_ENV=benchmark mix deps.get +for w in fibonacci closures oop string_ops string_format table_ops \ + patterns metamethods pcall_varargs vm_new encode_decode; do + LUA_BENCH_MODE=full MIX_ENV=benchmark mix run "benchmarks/$w.exs" +done +``` + +Run these **serially**, as written — not in parallel, and not alongside a test +suite. `mix lua.bench --workload ` is a convenience wrapper that sets the +env for you, but note that invoking it with no `--workload` also picks up +`array_vs_map_probe` (a data-structure probe that runs no Lua) and +`dispatcher_vs_interpreter` (an internal A/B that reaches into +`Lua.Compiler.Prototype` and `Lua.VM.State`). Neither belongs in a +cross-version comparison, and the latter cannot run on v0.4.0 at all. + +### v1.0.0 + +`benchmarks/` at tag `v1.0.0` is byte-identical to the 1.0.2 tree for the +pre-existing workloads, and `mix.exs` deps are the same. Copy in the four +workloads added for this report, then run as above: + +```sh +git worktree add --detach /tmp/lua-v1.0.0 v1.0.0 +cp benchmarks/{patterns,metamethods,pcall_varargs,vm_new}.exs \ + /tmp/lua-v1.0.0/benchmarks/ +cd /tmp/lua-v1.0.0 && MIX_ENV=benchmark mix deps.get +# then the same loop as above +``` + +### v0.4.0 + +Three adaptations are required: + +1. **There is no `benchmarks/` directory at that tag.** Copy the whole + directory in from the current tree. +2. **There is no `benchee` dependency.** Add + `{:benchee, "~> 1.3", only: :benchmark}` to `deps/0` in the worktree's + `mix.exs`, then `MIX_ENV=benchmark mix deps.get`. (`luerl` is an + unconditional runtime dependency at v0.4.0, so the control rows need no + adaptation on any ref.) +3. **Nothing else.** In particular the `load_chunk!/2` state threading that + v0.4.0 requires is **already committed** in every benchmark file. At v0.4.0 + a `%Lua.Chunk{}` holds a `:ref` into the state it was loaded against, so + discarding that state invalidates the chunk and the `lua (chunk)` job dies + with `key N not found`. Every file now writes + `{chunk, lua} = Lua.load_chunk!(lua, ...)`, which is correct and free on all + three refs. + +```sh +git worktree add --detach /tmp/lua-v0.4.0 v0.4.0 +cp -R benchmarks /tmp/lua-v0.4.0/benchmarks +# add {:benchee, "~> 1.3", only: :benchmark} to deps/0 in /tmp/lua-v0.4.0/mix.exs +cd /tmp/lua-v0.4.0 && MIX_ENV=benchmark mix deps.get +# then the same loop as above +``` + +### Language constraints when editing workloads + +Empirically verified against all three refs. A workload that violates any of +these cannot be measured across the whole history: + +- **Use global `function f(...)`, not `local function f(...)`, for anything + self-recursive.** `local function` self-recursion fails on v0.4.0 + (`undefined function nil`). A forward-declared `local f; f = function ...` + also works. +- **No `string.gmatch`.** Raises `{:badarg, :gmatch, ...}` in Luerl 1.5.1, so + it is broken on v0.4.0 and unusable for the control row on any ref. It works + correctly on v1.0.0 and 1.0.2. +- **No `goto`/labels** (parse error on v0.4.0), **no `xpcall`** (broken on + v0.4.0), **no `coroutine.*`** (absent on v1.0.0 and 1.0.2). +- **No non-string `error()` values.** v0.4.0 stringifies them; 1.0.2 preserves + the table. +- **Beware `select("#", ...)`** — returns a float on v0.4.0/Luerl and an + integer on v1.0.0/1.0.2. Harmless in `pcall_varargs` (same iteration count), + but it makes a dedicated integer-arithmetic benchmark measure float math on + one column and integer math on another. From d0fcc6ffd521ee93f64f6ea3a8bb0d3c00c74dca Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 14:30:02 -0400 Subject: [PATCH 4/8] website: host the version benchmark page at /benchmarks.html Self-contained static page (no external assets, light/dark aware) charting the v0.4.0 -> v1.0.0 -> v1.0.2 medians against the same-run Luerl control, with the honest-gaps and methodology notes from bench_results/versions-2026-07-28.md. --- website/lib/website_web.ex | 2 +- website/priv/static/benchmarks.html | 388 ++++++++++++++++++++++++++++ 2 files changed, 389 insertions(+), 1 deletion(-) create mode 100644 website/priv/static/benchmarks.html diff --git a/website/lib/website_web.ex b/website/lib/website_web.ex index 1526230..f600f08 100644 --- a/website/lib/website_web.ex +++ b/website/lib/website_web.ex @@ -17,7 +17,7 @@ defmodule DemoWeb do those modules here. """ - def static_paths, do: ~w(assets fonts images favicon.ico robots.txt) + def static_paths, do: ~w(assets fonts images favicon.ico robots.txt benchmarks.html) def router do quote do diff --git a/website/priv/static/benchmarks.html b/website/priv/static/benchmarks.html new file mode 100644 index 0000000..520d55d --- /dev/null +++ b/website/priv/static/benchmarks.html @@ -0,0 +1,388 @@ + + + + + +Lua on the BEAM — version benchmarks (0.4.0 → 1.0.0 → 1.0.2) + + + +
+
+

tv-labs/lua · full-mode benchee · 2026-07-28

+

Lua on the BEAM: 0.4.0 → 1.0.0 → 1.0.2

+

Three releases of the lua Elixir library, measured on the same machine in the same + sitting, with Luerl 1.5 run inside every benchmark as a same-run control. v0.4.0 was a thin + wrapper over Luerl; 1.0.0 introduced the native VM; 1.0.2 is the release cut today.

+

The headline: 1.0.2 is the first release faster than Luerl on most workloads — + and Lua.new() is now effectively free.

+
+ +
+
+
Lua.new() median
+
0.58 µs
+
63× faster than 1.0.0 (36.67 µs)
+
Steady-state, default options, under mix run. ~100× credible in an + :embedded release; ~5× when passing custom sandbox options (PR #419 / #398).
+
+
+
Lua.new() allocation
+
0.88 KB
+
104× less than 1.0.0 (91.88 KB)
+
Memoized boot-time VM template; one-time ~7 ms cold build per node.
+
+
+
fibonacci fib(30) median
+
434 ms
+
1.7× faster than same-run Luerl
+
Was 1.09× slower on 1.0.0 (792 ms). Allocation: 1.0 GB vs Luerl's 2.5 GB.
+
+
+ +
+

Runtime vs Luerl, per workload

+

Each dot is a release's compiled-chunk median divided by the Luerl median from the + same run — left of the parity line is faster than Luerl. Hover a dot for the underlying medians. + Log scale.

+
+
+ v0.4.0 (Luerl wrapper) + v1.0.0 + 1.0.2 + │ line = Luerl parity (1.0) +
+
+
← faster than Luerlslower than Luerl →
+
+
+ +
+
+
+
+ +
+

Medians across releases

+

Compiled-chunk path (the production embedding path: compile once, run many). + Ratio column is 1.0.2 ÷ same-run Luerl; green means faster than Luerl.

+
+ + + + + +
Workloadv0.4.0v1.0.01.0.21.0.2 vs Luerl
+
+
+ +
+
+

Where 1.0.2 is still behind Luerl

+

pcall raise + catch (1.55× slower). Partly apples-to-oranges: 1.0.2 builds + position-prefixed error messages ("<eval>:1: negative", the PUC-Lua-conformant + behavior) where Luerl returns the bare value — it does strictly more work per raise. Still, + it regressed at 1.0.0 (1.81×) and has only partially recovered.

+

varargs + multiple returns (1.36× slower, ~1.9× the allocation). A genuine + gap with no conformance excuse — the clearest optimization target for 1.1.x.

+

Host-boundary decode of large lists. Not in the chart (its own harness): + decoding a 4,096-element integer list is ~13× slower than the v0.4.0/Luerl era; long-string + lists ~25×. String-keyed maps moved the other way (1.24× faster). Worth a look before 1.1.

+
+
+ +
+

How to read these numbers

+
    +
  • v0.4.0 is not an independent series. It wraps luerl 1.5.1; its own + no-sandbox instantiation lands within ~1% of raw :luerl.init(). Its dots hugging the + parity line is by construction, and it's why the Luerl control column is meaningful across all + three runs.
    • +
  • Discipline: Apple M4 · Elixir 1.20.0 / OTP 29 · LUA_BENCH_MODE=full + (10 s measure, 2 s warmup, 1 s memory) · one mix run at a time on a quiet machine · + medians quoted, not averages. C Lua (luaport) was unavailable; Luerl is the reference.
    • +
  • The suite got wider for this report. patterns, metamethods, pcall/varargs, and + Lua.new workloads are new — chosen to make the suite representative of real Lua + (pattern engine, : method dispatch, protected calls), not just of what was optimized.
    • +
  • Coroutines are not benchmarked — they are an intentional 1.0 capability + exclusion in the library, not an omission from the suite.
    • +
  • Sub-microsecond rows (Lua.new() on 1.0.2) show large Benchee + deviation percentages due to batching at that timescale; medians are stable across runs.
    • +
+
+ + +
+ +
+ + +

+† 1.0.2 builds PUC-Lua-conformant position-prefixed error messages; Luerl returns the bare error value, +doing less work per raise.

+ + From a4d9c5b05661d8afdfaed5f0cbad66d44604cb23 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 15:15:12 -0400 Subject: [PATCH 5/8] release: use an absolute URL for the benchmark report link in the changelog The changelog ships as a hexdocs extra, where repo-relative file links don't resolve and fail mix docs --warnings-as-errors. --- CHANGELOG.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ba7df75..5104040 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,7 +56,8 @@ is in the [`1.0.0-rc.0`](#100-rc0---2026-05-26) entry below. 0.6µs median, with per-call allocation down from ~92KB to under 1KB — and ~5.5x faster when a custom sandbox is passed (36.0µs down to 6.5µs), as measured by `benchmarks/vm_new.exs` under `mix run` - ([full figures](bench_results/versions-2026-07-28.md)). Installing the + ([full figures](https://github.com/tv-labs/lua/blob/main/bench_results/versions-2026-07-28.md)). + Installing the standard library is pure and deterministic, so the boot-time VM template is now built once per node and memoized in `:persistent_term`; every later `Lua.new/1` starts from the shared template copy-on-write. In `:interactive` mode (dev, IEx, tests) the From 971d18c0c1301963cba2d1b5b7b25bd541ce8828 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 16:04:20 -0400 Subject: [PATCH 6/8] bench: strip local-environment preamble from recorded captures MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every committed capture opened with a zoxide shell-configuration notice, and the v0.4.0 ones additionally carried the unused-clause warning that tag's own `Lua.Util.format_error/2` emits when compiled. Neither is benchmark output: the notice is local shell state, and the warning belongs to a released tag we measure but do not change (`main` has only `format_error/1`). Strips leading noise only, up to each file's first real line — 344 deletions, no insertions, nothing inside a capture touched. --- bench_results/v0.4.0/closures.txt | 15 --------------- bench_results/v0.4.0/encode_decode.txt | 15 --------------- bench_results/v0.4.0/fibonacci.txt | 15 --------------- bench_results/v0.4.0/metamethods.txt | 15 --------------- bench_results/v0.4.0/oop.txt | 15 --------------- bench_results/v0.4.0/patterns.txt | 15 --------------- bench_results/v0.4.0/pcall_varargs.txt | 15 --------------- bench_results/v0.4.0/string_format.txt | 15 --------------- bench_results/v0.4.0/string_ops.txt | 15 --------------- bench_results/v0.4.0/table_ops.txt | 15 --------------- bench_results/v0.4.0/vm_new.txt | 16 ---------------- bench_results/v1.0.0/closures.txt | 8 -------- bench_results/v1.0.0/encode_decode.txt | 8 -------- bench_results/v1.0.0/fibonacci.txt | 8 -------- bench_results/v1.0.0/metamethods.txt | 8 -------- bench_results/v1.0.0/oop.txt | 8 -------- bench_results/v1.0.0/patterns.txt | 8 -------- bench_results/v1.0.0/pcall_varargs.txt | 8 -------- bench_results/v1.0.0/string_format.txt | 8 -------- bench_results/v1.0.0/string_ops.txt | 8 -------- bench_results/v1.0.0/table_ops.txt | 8 -------- bench_results/v1.0.0/vm_new.txt | 9 --------- bench_results/v1.0.2/closures.txt | 8 -------- bench_results/v1.0.2/encode_decode.txt | 8 -------- bench_results/v1.0.2/fibonacci.txt | 8 -------- bench_results/v1.0.2/metamethods.txt | 8 -------- bench_results/v1.0.2/oop.txt | 8 -------- bench_results/v1.0.2/patterns.txt | 8 -------- bench_results/v1.0.2/pcall_varargs.txt | 8 -------- bench_results/v1.0.2/string_format.txt | 8 -------- bench_results/v1.0.2/string_ops.txt | 8 -------- bench_results/v1.0.2/table_ops.txt | 8 -------- bench_results/v1.0.2/vm_new.txt | 9 --------- 33 files changed, 344 deletions(-) diff --git a/bench_results/v0.4.0/closures.txt b/bench_results/v0.4.0/closures.txt index dac5991..8e16e2e 100644 --- a/bench_results/v0.4.0/closures.txt +++ b/bench_results/v0.4.0/closures.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v0.4.0/encode_decode.txt b/bench_results/v0.4.0/encode_decode.txt index 32d5f5a..0f54da6 100644 --- a/bench_results/v0.4.0/encode_decode.txt +++ b/bench_results/v0.4.0/encode_decode.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - lua 0.4.0 — encode!/decode! decomposition (decode+deep_cast column: enabled) ================================================================================== diff --git a/bench_results/v0.4.0/fibonacci.txt b/bench_results/v0.4.0/fibonacci.txt index 1927342..62e549c 100644 --- a/bench_results/v0.4.0/fibonacci.txt +++ b/bench_results/v0.4.0/fibonacci.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v0.4.0/metamethods.txt b/bench_results/v0.4.0/metamethods.txt index 38564c2..a60332a 100644 --- a/bench_results/v0.4.0/metamethods.txt +++ b/bench_results/v0.4.0/metamethods.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === metamethods: self-call method dispatch (n=200) (mode: full) === diff --git a/bench_results/v0.4.0/oop.txt b/bench_results/v0.4.0/oop.txt index 580705e..b167197 100644 --- a/bench_results/v0.4.0/oop.txt +++ b/bench_results/v0.4.0/oop.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v0.4.0/patterns.txt b/bench_results/v0.4.0/patterns.txt index bc1ea94..cc66c55 100644 --- a/bench_results/v0.4.0/patterns.txt +++ b/bench_results/v0.4.0/patterns.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === patterns: find/match field extraction (n=200) (mode: full) === diff --git a/bench_results/v0.4.0/pcall_varargs.txt b/bench_results/v0.4.0/pcall_varargs.txt index c6e6759..efb881e 100644 --- a/bench_results/v0.4.0/pcall_varargs.txt +++ b/bench_results/v0.4.0/pcall_varargs.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === call protocol: pcall, success path (n=500) (mode: full) === diff --git a/bench_results/v0.4.0/string_format.txt b/bench_results/v0.4.0/string_format.txt index 79888e7..b992107 100644 --- a/bench_results/v0.4.0/string_format.txt +++ b/bench_results/v0.4.0/string_format.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === string.format: long literal-heavy format string (n=1000) (mode: full) === diff --git a/bench_results/v0.4.0/string_ops.txt b/bench_results/v0.4.0/string_ops.txt index 249f816..7823a7a 100644 --- a/bench_results/v0.4.0/string_ops.txt +++ b/bench_results/v0.4.0/string_ops.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === String Concatenation via table.concat (n=100) (mode: full) === diff --git a/bench_results/v0.4.0/table_ops.txt b/bench_results/v0.4.0/table_ops.txt index dfa3ff5..d0e203b 100644 --- a/bench_results/v0.4.0/table_ops.txt +++ b/bench_results/v0.4.0/table_ops.txt @@ -1,18 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === Table Build (mode: full) === diff --git a/bench_results/v0.4.0/vm_new.txt b/bench_results/v0.4.0/vm_new.txt index f09f0f4..7d4449c 100644 --- a/bench_results/v0.4.0/vm_new.txt +++ b/bench_results/v0.4.0/vm_new.txt @@ -1,19 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - warning: this clause of defp format_error/2 is never used (or it will always fail/warn when invoked) - │ - 177 │ defp format_error(_, {:undefined, args}) do - │ ~ - │ - └─ lib/lua/util.ex:177:8: Lua.Util.format_error/2 - - === VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full) === Lua.new() one-time vs repeat cost (single samples, informational): diff --git a/bench_results/v1.0.0/closures.txt b/bench_results/v1.0.0/closures.txt index c5e8d5b..244e466 100644 --- a/bench_results/v1.0.0/closures.txt +++ b/bench_results/v1.0.0/closures.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v1.0.0/encode_decode.txt b/bench_results/v1.0.0/encode_decode.txt index da298cb..f1b2d16 100644 --- a/bench_results/v1.0.0/encode_decode.txt +++ b/bench_results/v1.0.0/encode_decode.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - lua 1.0.0 — encode!/decode! decomposition (decode+deep_cast column: enabled) ================================================================================== diff --git a/bench_results/v1.0.0/fibonacci.txt b/bench_results/v1.0.0/fibonacci.txt index 09c735a..61c6c97 100644 --- a/bench_results/v1.0.0/fibonacci.txt +++ b/bench_results/v1.0.0/fibonacci.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v1.0.0/metamethods.txt b/bench_results/v1.0.0/metamethods.txt index 911f422..07e5541 100644 --- a/bench_results/v1.0.0/metamethods.txt +++ b/bench_results/v1.0.0/metamethods.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === metamethods: self-call method dispatch (n=200) (mode: full) === diff --git a/bench_results/v1.0.0/oop.txt b/bench_results/v1.0.0/oop.txt index 0b8ca06..e5607b9 100644 --- a/bench_results/v1.0.0/oop.txt +++ b/bench_results/v1.0.0/oop.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v1.0.0/patterns.txt b/bench_results/v1.0.0/patterns.txt index 43d7a99..442f790 100644 --- a/bench_results/v1.0.0/patterns.txt +++ b/bench_results/v1.0.0/patterns.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === patterns: find/match field extraction (n=200) (mode: full) === diff --git a/bench_results/v1.0.0/pcall_varargs.txt b/bench_results/v1.0.0/pcall_varargs.txt index 01b5211..9048cf7 100644 --- a/bench_results/v1.0.0/pcall_varargs.txt +++ b/bench_results/v1.0.0/pcall_varargs.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === call protocol: pcall, success path (n=500) (mode: full) === diff --git a/bench_results/v1.0.0/string_format.txt b/bench_results/v1.0.0/string_format.txt index 7465780..39bfaf5 100644 --- a/bench_results/v1.0.0/string_format.txt +++ b/bench_results/v1.0.0/string_format.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === string.format: long literal-heavy format string (n=1000) (mode: full) === diff --git a/bench_results/v1.0.0/string_ops.txt b/bench_results/v1.0.0/string_ops.txt index adb575e..bf8b3b9 100644 --- a/bench_results/v1.0.0/string_ops.txt +++ b/bench_results/v1.0.0/string_ops.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === String Concatenation via table.concat (n=100) (mode: full) === diff --git a/bench_results/v1.0.0/table_ops.txt b/bench_results/v1.0.0/table_ops.txt index 91cc3a0..5b463f7 100644 --- a/bench_results/v1.0.0/table_ops.txt +++ b/bench_results/v1.0.0/table_ops.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === Table Build (mode: full) === diff --git a/bench_results/v1.0.0/vm_new.txt b/bench_results/v1.0.0/vm_new.txt index 26ae1a4..9ade4d5 100644 --- a/bench_results/v1.0.0/vm_new.txt +++ b/bench_results/v1.0.0/vm_new.txt @@ -1,12 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - === VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full) === Lua.new() one-time vs repeat cost (single samples, informational): diff --git a/bench_results/v1.0.2/closures.txt b/bench_results/v1.0.2/closures.txt index cf110a9..c2a370a 100644 --- a/bench_results/v1.0.2/closures.txt +++ b/bench_results/v1.0.2/closures.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v1.0.2/encode_decode.txt b/bench_results/v1.0.2/encode_decode.txt index 8302981..204913b 100644 --- a/bench_results/v1.0.2/encode_decode.txt +++ b/bench_results/v1.0.2/encode_decode.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - lua 1.0.2 — encode!/decode! decomposition (decode+deep_cast column: enabled) ================================================================================== diff --git a/bench_results/v1.0.2/fibonacci.txt b/bench_results/v1.0.2/fibonacci.txt index 6695d13..153dfe1 100644 --- a/bench_results/v1.0.2/fibonacci.txt +++ b/bench_results/v1.0.2/fibonacci.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v1.0.2/metamethods.txt b/bench_results/v1.0.2/metamethods.txt index 601736c..5079db1 100644 --- a/bench_results/v1.0.2/metamethods.txt +++ b/bench_results/v1.0.2/metamethods.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === metamethods: self-call method dispatch (n=200) (mode: full) === diff --git a/bench_results/v1.0.2/oop.txt b/bench_results/v1.0.2/oop.txt index b918533..1e6a3ae 100644 --- a/bench_results/v1.0.2/oop.txt +++ b/bench_results/v1.0.2/oop.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks Operating System: macOS CPU Information: Apple M4 diff --git a/bench_results/v1.0.2/patterns.txt b/bench_results/v1.0.2/patterns.txt index 06134f0..9bd65ef 100644 --- a/bench_results/v1.0.2/patterns.txt +++ b/bench_results/v1.0.2/patterns.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === patterns: find/match field extraction (n=200) (mode: full) === diff --git a/bench_results/v1.0.2/pcall_varargs.txt b/bench_results/v1.0.2/pcall_varargs.txt index 80d1aa5..ebeacdd 100644 --- a/bench_results/v1.0.2/pcall_varargs.txt +++ b/bench_results/v1.0.2/pcall_varargs.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === call protocol: pcall, success path (n=500) (mode: full) === diff --git a/bench_results/v1.0.2/string_format.txt b/bench_results/v1.0.2/string_format.txt index 779a9a4..97284a3 100644 --- a/bench_results/v1.0.2/string_format.txt +++ b/bench_results/v1.0.2/string_format.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === string.format: long literal-heavy format string (n=1000) (mode: full) === diff --git a/bench_results/v1.0.2/string_ops.txt b/bench_results/v1.0.2/string_ops.txt index a3d9c4b..9ac629f 100644 --- a/bench_results/v1.0.2/string_ops.txt +++ b/bench_results/v1.0.2/string_ops.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === String Concatenation via table.concat (n=100) (mode: full) === diff --git a/bench_results/v1.0.2/table_ops.txt b/bench_results/v1.0.2/table_ops.txt index 6d49993..d6f1180 100644 --- a/bench_results/v1.0.2/table_ops.txt +++ b/bench_results/v1.0.2/table_ops.txt @@ -1,11 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - luaport not available ({:luaport, {~c"no such file or directory", ~c"luaport.app"}}) — skipping C Lua benchmarks === Table Build (mode: full) === diff --git a/bench_results/v1.0.2/vm_new.txt b/bench_results/v1.0.2/vm_new.txt index 7ece932..fb12a49 100644 --- a/bench_results/v1.0.2/vm_new.txt +++ b/bench_results/v1.0.2/vm_new.txt @@ -1,12 +1,3 @@ -zoxide: detected a possible configuration issue. -Please ensure that zoxide is initialized right at the end of your shell configuration file (usually ~/.zshrc). - -If the issue persists, consider filing an issue at: -https://github.com/ajeetdsouza/zoxide/issues - -Disable this message by setting _ZO_DOCTOR=0. - - === VM instantiation: Lua.new/1 vs :luerl.init/0 (mode: full) === Lua.new() one-time vs repeat cost (single samples, informational): From c9dce4fe36e25cd632953aacb080b97b13e286e3 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 16:04:25 -0400 Subject: [PATCH 7/8] bench: use placeholders for machine paths in v0.4.0 environment notes The worktree setup and cleanup commands were recorded with this machine's absolute checkout and scratchpad paths, which are noise to anyone reproducing the run. Uses `` and `` instead, matching how the file already referred to the worktree further down and how the v1.0.0 notes were written. --- bench_results/v0.4.0/environment.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/bench_results/v0.4.0/environment.md b/bench_results/v0.4.0/environment.md index 2d7f888..897695d 100644 --- a/bench_results/v0.4.0/environment.md +++ b/bench_results/v0.4.0/environment.md @@ -12,10 +12,9 @@ ## Worktree setup ``` -git -C /Users/dave/code/tvlabs/lua worktree add --detach \ - /private/tmp/claude-501/-Users-dave-code-tvlabs-lua/baeced00-0f95-4dc6-8d8e-c16607ffb143/scratchpad/bench-v0.4.0 v0.4.0 +git -C worktree add --detach v0.4.0 -cp -R /Users/dave/code/tvlabs/lua/benchmarks /benchmarks +cp -R /benchmarks /benchmarks # mix.exs deps edit (worktree only): added # {:benchee, "~> 1.3", only: :benchmark} @@ -45,8 +44,7 @@ Excluded: `dispatcher_vs_interpreter.exs` (crashes on v0.4.0), `array_vs_map_pro ## Cleanup ``` -git -C /Users/dave/code/tvlabs/lua worktree remove --force \ - /private/tmp/claude-501/-Users-dave-code-tvlabs-lua/baeced00-0f95-4dc6-8d8e-c16607ffb143/scratchpad/bench-v0.4.0 +git -C worktree remove --force ``` ## Results From d50cb5647e1c110f28fe763678c77efa0b3c1fa5 Mon Sep 17 00:00:00 2001 From: Dave Lucia Date: Tue, 28 Jul 2026 16:04:36 -0400 Subject: [PATCH 8/8] website: render the version benchmarks from the recorded results MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The benchmark page shipped as a hand-written static file under priv/static: its numbers were transcribed into JS literals with no link to the data they came from, it rendered outside the site layout, and nothing linked to it. Replaces it with a dead view at `/benchmarks`, linked from the nav and footer. `Website.Benchmarks` reads the committed `bench_results//summary.json` files at compile time, so the page has one source of truth for every figure — the tables, the ratio plot and the headline tiles are all derived, and the plot is rendered server-side rather than assembled by client JS. Recording a release is the whole update: version directories are found by glob and ordered with `Version.compare/2`, so a new `summary.json` becomes a new column and the newest `versions-.md` becomes the linked report. Only a new *workload* needs a row spec, and only the prose is written by hand. The loader normalises the case-key differences in the v0.4.0 run (`" (mode: full)"` suffixes, `inputs` vs `by_input`) so one row spec resolves against every ref. Every derived figure was checked against the numbers the static page carried; all 36 cells and 27 ratios match. Verified in a release container, since the results directory has to be copied into the image for the compile-time read to work. --- Dockerfile | 4 + bench_results/README.md | 28 +- website/lib/website/benchmarks.ex | 460 ++++++++++++++++++ website/lib/website_web.ex | 2 +- website/lib/website_web/components/layouts.ex | 14 + .../controllers/page_controller.ex | 15 + .../lib/website_web/controllers/page_html.ex | 52 ++ .../page_html/benchmarks.html.heex | 280 +++++++++++ website/lib/website_web/router.ex | 1 + website/priv/static/benchmarks.html | 388 --------------- .../controllers/page_controller_test.exs | 58 +++ 11 files changed, 911 insertions(+), 391 deletions(-) create mode 100644 website/lib/website/benchmarks.ex create mode 100644 website/lib/website_web/controllers/page_html/benchmarks.html.heex delete mode 100644 website/priv/static/benchmarks.html diff --git a/Dockerfile b/Dockerfile index 7e74e94..9f5f0e3 100644 --- a/Dockerfile +++ b/Dockerfile @@ -56,6 +56,10 @@ COPY website/priv priv COPY website/lib lib COPY website/assets assets +# Recorded benchmark results, read at compile time by lib/website/benchmarks.ex +# (@external_resource) to render /benchmarks. Compile fails loudly without it. +COPY bench_results /app/bench_results + # mix compile must run BEFORE assets.deploy because Phoenix's LiveView # colocated-hooks compiler generates files under _build/ that esbuild # resolves via NODE_PATH (`phoenix-colocated/website`). diff --git a/bench_results/README.md b/bench_results/README.md index 7afbdc3..07cbc16 100644 --- a/bench_results/README.md +++ b/bench_results/README.md @@ -107,7 +107,31 @@ series grows one column per release. 4. Write or extend a report named `-.md` quoting **medians**, not averages — several workloads have allocation-driven GC pauses that pull the mean around. -5. Add a row to the Contents table above, and update the hosted page at - `website/priv/static/benchmarks.html` if the headline story changed. +5. Add a row to the Contents table above. 6. Do not edit `benchmarks/BASELINE.md`. It is the historical 1.0.0 gate record; a newer report supersedes it by saying so. + +### What updates itself + +The hosted page at [`/benchmarks`](../website/lib/website_web/controllers/page_html/benchmarks.html.heex) +reads these directories directly, so steps 1–5 are the whole job: + +- **A new `/summary.json` becomes a new column.** Version directories + are found by glob and ordered with `Version.compare/2` — no list to extend. +- **A new `versions-.md` becomes the linked report**, and its date becomes + the page's dated eyebrow. The newest report filename wins. +- **Headline tiles and the "still behind Luerl" figures re-derive** from the new + column, including the "N× faster than " deltas. + +Two things still need a human: + +- **A new workload needs a row spec** in `Website.Benchmarks` (`@rows`) before it + appears — which cases are worth showing, and under what name, is editorial. + A version that lacks a workload another version has renders as `—`. +- **The prose** — the headline claim and the analysis paragraphs — is written, + not generated. Re-read it when the story changes. + +`Website.Benchmarks` registers each `summary.json` as an `@external_resource`, +so editing recorded results recompiles the page in dev. The container build +copies this directory in (see `Dockerfile`); compilation fails loudly rather +than shipping an empty page if it is missing. diff --git a/website/lib/website/benchmarks.ex b/website/lib/website/benchmarks.ex new file mode 100644 index 0000000..88ee2f7 --- /dev/null +++ b/website/lib/website/benchmarks.ex @@ -0,0 +1,460 @@ +defmodule Website.Benchmarks do + @moduledoc """ + The recorded cross-version benchmark results, read from `bench_results/`. + + Every released version has a directory of committed benchmark output at + `bench_results//`, and this module turns those `summary.json` files + into the rows rendered by `/benchmarks`. + + Version directories are **discovered by glob** and ordered by + `Version.compare/2`, so recording a new release is the whole update: drop + `bench_results//summary.json` in place and the page grows a column. + The same goes for the report link, which tracks the newest + `bench_results/versions-.md`. + + What is *not* automatic is `@rows` below — the editorial choice of which + workload cases are worth showing, and under what name. A new workload only + appears once it has a row spec. A version that lacks a workload another + version has renders as `—` in that cell rather than failing. + + All file reads happen at compile time and are registered as + `@external_resource`, so editing recorded results recompiles the page. + """ + + @bench_results Path.expand("../../../bench_results", __DIR__) + + # label: what the row is called on the page + # sub: second line, chart only + # at: {workload, case, input | nil} — case/input keys as normalised below + # job: the Benchee job whose number we quote + # vs: the same-run control job the ratio is taken against + # metric: :median (time) or :memory (allocation) + # chart?: whether the row also gets a dot in the ratio plot + # warn?: the newest release is behind the control here — flagged in both views + # dagger?: the comparison is not like-for-like; the page footnotes why + @rows [ + %{ + label: "Lua.new() — steady state", + sub: "default options", + at: {"vm_new", "VM instantiation: Lua.new/1 vs :luerl.init/0", nil}, + job: "lua (new)", + vs: "luerl (init)", + metric: :median, + chart?: false, + warn?: false, + dagger?: false + }, + %{ + label: "Lua.new() — allocation", + sub: "default options", + at: {"vm_new", "VM instantiation: Lua.new/1 vs :luerl.init/0", nil}, + job: "lua (new)", + vs: "luerl (init)", + metric: :memory, + chart?: false, + warn?: false, + dagger?: false + }, + %{ + label: "fibonacci fib(30)", + sub: "recursive calls", + at: {"fibonacci", "default", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "fibonacci — allocation", + sub: "recursive calls", + at: {"fibonacci", "default", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :memory, + chart?: false, + warn?: false, + dagger?: false + }, + %{ + label: "string.format (literal-heavy)", + sub: "literal-heavy template", + at: {"string_format", "string.format: long literal-heavy format string (n=1000)", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "pcall success path (n=500)", + sub: "protected call, no raise", + at: {"pcall_varargs", "call protocol: pcall, success path (n=500)", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "pairs over hash part (n=1000)", + sub: "hash-part iteration", + at: {"table_ops", "Table Pairs (hash)", "large (n=1000)"}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "__index 3-level chain", + sub: "prototype lookup", + at: {"metamethods", "metamethods: 3-level __index chain (n=200)", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "closures", + sub: "factory + upvalue mutation", + at: {"closures", "default", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "table.sort (n=1000)", + sub: "reverse-ordered input", + at: {"table_ops", "Table Sort", "large (n=1000)"}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: false, + dagger?: false + }, + %{ + label: "varargs + multi-return (n=500)", + sub: "call protocol", + at: {"pcall_varargs", "call protocol: varargs + multiple returns (n=500)", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: true, + dagger?: false + }, + %{ + label: "pcall raise + catch (n=500)", + sub: "does strictly more work", + at: {"pcall_varargs", "call protocol: pcall, raise + catch (n=500)", nil}, + job: "lua (chunk)", + vs: "luerl", + metric: :median, + chart?: true, + warn?: true, + dagger?: true + } + ] + + summaries = Path.wildcard(Path.join(@bench_results, "*/summary.json")) + + if summaries == [] do + raise """ + no benchmark summaries found under #{@bench_results} + + /benchmarks renders the committed results in bench_results//. If \ + this is a container build, the image needs the directory: + + COPY bench_results /app/bench_results + """ + end + + for path <- summaries do + @external_resource path + end + + reports = Path.wildcard(Path.join(@bench_results, "versions-*.md")) + + # --- compile-time loading ------------------------------------------------ + + # v0.4.0's run labelled cases differently: banners carry a " (mode: full)" + # suffix, the single-case workloads say "(single case)" where later runs say + # "default", per-input results sit under "inputs" rather than "by_input", and + # input labels are prefixed "With input ". Normalise so one row spec resolves + # against every ref. + normalise_case = fn name -> + case String.replace_suffix(name, " (mode: full)", "") do + "(single case)" -> "default" + other -> other + end + end + + normalise_input = fn name -> String.replace_prefix(name, "With input ", "") end + + parse = fn + nil -> + nil + + value -> + case Regex.run(~r/^([\d.]+)\s*(\S+)$/, String.replace(value, "μ", "µ")) do + [_, number, unit] -> + scale = + case unit do + "ns" -> 1 + "µs" -> 1_000 + "ms" -> 1_000_000 + "s" -> 1_000_000_000 + "B" -> 1 + "KB" -> 1_024 + "MB" -> 1_024 * 1_024 + "GB" -> 1_024 * 1_024 * 1_024 + _ -> nil + end + + # Benchee drops the decimal point on some values ("216 µs"), so + # Float.parse rather than String.to_float. + with true <- is_integer(scale), {number, ""} <- Float.parse(number) do + number * scale + else + _ -> nil + end + + _ -> + nil + end + end + + data = + for path <- summaries, into: %{} do + version = path |> Path.dirname() |> Path.basename() |> String.trim_leading("v") + + cases = + path + |> File.read!() + |> Jason.decode!() + |> Map.new(fn {workload, body} -> + normalised = + case body do + %{"raw" => _} -> + %{} + + cases -> + Map.new(cases, fn {name, one} -> + by_input = Map.get(one, "by_input") || Map.get(one, "inputs") + + value = + if by_input, + do: Map.new(by_input, fn {k, v} -> {normalise_input.(k), v} end), + else: one + + {normalise_case.(name), value} + end) + end + + {workload, normalised} + end) + + {version, cases} + end + + @versions data |> Map.keys() |> Enum.sort(&(Version.compare(&1, &2) != :gt)) + + table_rows = + for spec <- @rows do + {workload, case_name, input} = spec.at + + values = + for version <- @versions, into: %{} do + jobs = + with %{^workload => workloads} <- data[version], + %{^case_name => one} <- workloads do + case input do + nil -> Map.get(one, "jobs") + key -> one |> Map.get(key, %{}) |> Map.get("jobs") + end + else + _ -> nil + end + + find = fn name -> + jobs && Enum.find(jobs, &(&1["name"] == name)) + end + + field = if spec.metric == :memory, do: "memory", else: "median" + mine = find.(spec.job) + control = find.(spec.vs) + + shown = mine && mine[field] + against = control && control[field] + + mine_number = parse.(shown) + against_number = parse.(against) + + ratio = + if is_number(mine_number) and is_number(against_number) and against_number > 0 do + Float.round(mine_number / against_number, 2) + end + + {version, %{value: shown, control: against, ratio: ratio, numeric: mine_number}} + end + + Map.put(spec, :values, values) + end + + @table_rows table_rows + + @report reports |> Enum.map(&Path.basename/1) |> Enum.max(fn -> nil end) + + @report_date (case @report && Regex.run(~r/(\d{4}-\d{2}-\d{2})/, @report) do + [_, date] -> Date.from_iso8601!(date) + _ -> nil + end) + + # --- public API ---------------------------------------------------------- + + @doc """ + Recorded versions, oldest first. + """ + def versions, do: @versions + + @doc """ + The newest recorded version — the one the page's headline numbers describe. + """ + def latest, do: List.last(@versions) + + @doc """ + Table rows: one per row spec, each carrying a value per version. + """ + def rows, do: @table_rows + + @doc """ + The subset of rows plotted as ratio-vs-control dots. + """ + def chart_rows, do: Enum.filter(@table_rows, & &1.chart?) + + @doc """ + Look up one row by label, for the headline tiles. + """ + def row(label), do: Enum.find(@table_rows, &(&1.label == label)) + + @doc """ + The value a row recorded for a version, or `nil` if that version lacks it. + """ + def value(row, version), do: get_in(row.values, [version, :value]) + + @doc """ + Whether a version holds the best (lowest) number in its row. Both metrics — + duration and allocation — are better when smaller. + """ + def best?(row, version) do + numbers = for {_, %{numeric: n}} <- row.values, is_number(n), do: n + mine = get_in(row.values, [version, :numeric]) + + is_number(mine) and numbers != [] and mine == Enum.min(numbers) + end + + @doc """ + The version before the newest — what the headline improvements are measured against. + """ + def previous, do: Enum.at(@versions, -2) + + @doc """ + The figures quoted in the page's headline tiles and callout, derived from the + rows so that recording a new release moves them without an edit here. + """ + def headline do + new = row("Lua.new() — steady state") + new_memory = row("Lua.new() — allocation") + fib = row("fibonacci fib(30)") + fib_memory = row("fibonacci — allocation") + raise_catch = row("pcall raise + catch (n=500)") + varargs = row("varargs + multi-return (n=500)") + + %{ + previous: previous(), + new_median: value(new, latest()), + new_median_prev: value(new, previous()), + new_speedup: improvement(new, previous(), latest()), + new_memory: value(new_memory, latest()), + new_memory_prev: value(new_memory, previous()), + new_memory_factor: improvement(new_memory, previous(), latest()), + fib_median: value(fib, latest()), + fib_median_prev: value(fib, previous()), + fib_speedup: inverse(ratio(fib, latest())), + fib_prev_ratio: ratio(fib, previous()), + fib_memory: value(fib_memory, latest()), + fib_memory_control: get_in(fib_memory.values, [latest(), :control]), + raise_ratio: ratio(raise_catch, latest()), + raise_prev_ratio: ratio(raise_catch, previous()), + varargs_ratio: ratio(varargs, latest()) + } + end + + @doc """ + A row's ratio against its same-run control for one version. + """ + def ratio(row, version), do: get_in(row.values, [version, :ratio]) + + @doc """ + How many times better `to` is than `from` in a row, as display text. + """ + def improvement(row, from, to) do + with a when is_number(a) <- get_in(row.values, [from, :numeric]), + b when is_number(b) <- get_in(row.values, [to, :numeric]), + true <- b > 0 do + format_factor(a / b) + else + _ -> nil + end + end + + @doc """ + Opacity for a version's colour, ramping oldest (faintest) to newest (solid). + """ + def version_weight(_index, count) when count < 2, do: 1.0 + def version_weight(index, count), do: Float.round(0.35 + 0.65 * (index / (count - 1)), 2) + + @doc """ + Filename of the newest campaign report, e.g. `versions-2026-07-28.md`. + """ + def report, do: @report + + @doc """ + Date of the newest campaign report, parsed from its filename. + """ + def report_date, do: @report_date + + @doc """ + Log-scale x position, as a 0..100 percentage, for a ratio in the dot plot. + """ + def plot_x(ratio) when is_number(ratio) do + :math.log(ratio / xmin()) / :math.log(xmax() / xmin()) * 100 + end + + def xmin, do: 0.15 + def xmax, do: 2.1 + + @doc """ + Gridline/tick positions for the plot axis. + """ + def ticks, do: [{0.25, "4× faster"}, {0.5, "2× faster"}, {1.0, "parity"}, {2.0, "2× slower"}] + + # A ratio below parity, restated as the speedup it represents. + defp inverse(ratio) when is_number(ratio) and ratio > 0, do: format_factor(1 / ratio) + defp inverse(_ratio), do: nil + + # Large factors read better whole ("63×"); small ones need the decimal ("1.7×"). + defp format_factor(factor) when factor >= 10, do: factor |> round() |> Integer.to_string() + defp format_factor(factor), do: :erlang.float_to_binary(factor, decimals: 1) +end diff --git a/website/lib/website_web.ex b/website/lib/website_web.ex index f600f08..1526230 100644 --- a/website/lib/website_web.ex +++ b/website/lib/website_web.ex @@ -17,7 +17,7 @@ defmodule DemoWeb do those modules here. """ - def static_paths, do: ~w(assets fonts images favicon.ico robots.txt benchmarks.html) + def static_paths, do: ~w(assets fonts images favicon.ico robots.txt) def router do quote do diff --git a/website/lib/website_web/components/layouts.ex b/website/lib/website_web/components/layouts.ex index f711bfa..664cbab 100644 --- a/website/lib/website_web/components/layouts.ex +++ b/website/lib/website_web/components/layouts.ex @@ -63,6 +63,7 @@ defmodule DemoWeb.Layouts do <.nav_link href="/playground" active={@active == :playground}>Playground <.nav_link href="/tour" active={@active == :tour}>Tour <.nav_link href="/reference/opcodes" active={@active == :opcodes}>Opcodes + <.nav_link href="/benchmarks" active={@active == :benchmarks}>Benchmarks <.nav_link href="/about" active={@active == :about}>About +
  • + <.link + navigate="/benchmarks" + class={@active == :benchmarks && "active text-primary bg-primary/10"} + > + Benchmarks + +
  • <.link navigate="/about" @@ -221,6 +230,11 @@ defmodule DemoWeb.Layouts do Opcode reference
    • +
  • + <.link navigate={~p"/benchmarks"} class="text-base-content/70 hover:text-primary"> + Benchmarks + +
  • <.link navigate={~p"/about"} class="text-base-content/70 hover:text-primary"> About diff --git a/website/lib/website_web/controllers/page_controller.ex b/website/lib/website_web/controllers/page_controller.ex index 8906060..44e9c61 100644 --- a/website/lib/website_web/controllers/page_controller.ex +++ b/website/lib/website_web/controllers/page_controller.ex @@ -1,6 +1,8 @@ defmodule DemoWeb.PageController do use DemoWeb, :controller + alias Website.Benchmarks + def home(conn, _params) do %{source: fib_source} = hd(Website.LuaSandbox.home_snippets()) @@ -15,6 +17,19 @@ defmodule DemoWeb.PageController do render(conn, :about, page_title: "About") end + def benchmarks(conn, _params) do + render(conn, :benchmarks, + page_title: "Benchmarks", + versions: Benchmarks.versions(), + latest: Benchmarks.latest(), + rows: Benchmarks.rows(), + chart_rows: Benchmarks.chart_rows(), + headline: Benchmarks.headline(), + report: Benchmarks.report(), + report_date: Benchmarks.report_date() + ) + end + def health(conn, _params) do send_resp(conn, 200, "ok") end diff --git a/website/lib/website_web/controllers/page_html.ex b/website/lib/website_web/controllers/page_html.ex index 0d04475..1ff4104 100644 --- a/website/lib/website_web/controllers/page_html.ex +++ b/website/lib/website_web/controllers/page_html.ex @@ -6,8 +6,56 @@ defmodule DemoWeb.PageHTML do """ use DemoWeb, :html + alias Website.Benchmarks + embed_templates "page_html/*" + attr :label, :string, required: true + attr :value, :string, required: true + attr :delta, :string, default: nil + slot :inner_block, required: true + + def stat_tile(assigns) do + ~H""" +
    +
    + {@label} +
    +
    {@value}
    +
    {@delta}
    +
    + {render_slot(@inner_block)} +
    +
    + """ + end + + attr :value, :float, default: nil + + def ratio(assigns) do + ~H""" + — + + {:erlang.float_to_binary(@value, decimals: 2)}× + + """ + end + + @doc """ + Hover text for a dot in the ratio plot. + """ + def dot_title(row, version) do + values = row.values[version] + + """ + #{row.label} — #{version} + chunk median: #{values.value} + same-run Luerl: #{values.control} + ratio: #{values.ratio}× #{ratio_word(values.ratio)} + """ + |> String.trim() + end + attr :icon, :string, required: true attr :title, :string, required: true attr :accent, :string, default: "primary" @@ -30,6 +78,10 @@ defmodule DemoWeb.PageHTML do """ end + defp ratio_word(ratio) when ratio < 1, do: "(faster)" + defp ratio_word(ratio) when ratio > 1, do: "(slower)" + defp ratio_word(_ratio), do: "" + defp accent_bg("primary"), do: "bg-primary/15" defp accent_bg("secondary"), do: "bg-secondary/15" defp accent_bg("accent"), do: "bg-accent/15" diff --git a/website/lib/website_web/controllers/page_html/benchmarks.html.heex b/website/lib/website_web/controllers/page_html/benchmarks.html.heex new file mode 100644 index 0000000..e1b7579 --- /dev/null +++ b/website/lib/website_web/controllers/page_html/benchmarks.html.heex @@ -0,0 +1,280 @@ + + <%!-- ============== HERO ============== --%> +
    + + +
    +

    + tv-labs/lua · full-mode benchee · {@report_date} +

    +

    + Lua on the BEAM: {Enum.join(@versions, " → ")} +

    +

    + {length(@versions)} releases of the lua + Elixir library, measured on the same machine in the same sitting, with Luerl run inside + every benchmark as a same-run control. {List.first(@versions)} was + a thin wrapper over Luerl; {@headline.previous} introduced the native VM; {@latest} is the + current release. +

    +

    + The headline: + {@latest} is the first release faster than Luerl on most workloads — and + Lua.new() + is now effectively free. +

    +
    +
    + + <%!-- ============== HEADLINE TILES ============== --%> +
    +
    + <.stat_tile + label="Lua.new() median" + value={@headline.new_median} + delta={"#{@headline.new_speedup}× faster than #{@headline.previous} (#{@headline.new_median_prev})"} + > + Steady-state, default options, under mix run. ~100× credible in an + :embedded + release; ~5× when passing custom sandbox options. + + + <.stat_tile + label="Lua.new() allocation" + value={@headline.new_memory} + delta={"#{@headline.new_memory_factor}× less than #{@headline.previous} (#{@headline.new_memory_prev})"} + > + Memoized boot-time VM template; one-time ~7 ms cold build per node. + + + <.stat_tile + label="fibonacci fib(30) median" + value={@headline.fib_median} + delta={"#{@headline.fib_speedup}× faster than same-run Luerl"} + > + Was {@headline.fib_prev_ratio}× slower + on {@headline.previous} ({@headline.fib_median_prev}). Allocation: {@headline.fib_memory} vs Luerl's {@headline.fib_memory_control}. + +
    +
    + + <%!-- ============== RATIO PLOT ============== --%> +
    +

    Runtime vs Luerl, per workload

    +

    + Each dot is a release's compiled-chunk median divided by the Luerl median from the same + run — left of the parity line is faster than Luerl. Hover a dot for the underlying + medians. Log scale. +

    + +
    +
    + + + + {version} +  (Luerl wrapper) + + + │ line = Luerl parity (1.0) +
    + +
    +
    +
    + ← faster than Luerl + slower than Luerl → +
    + +
    +
    + + {row.label} + + {row.sub} +
    + +
    +
    +
    +
    +
    + + + +
    +
    + +
    + +
    + + {label} + +
    +
    +
    +
    +
    +
    + + <%!-- ============== MEDIANS TABLE ============== --%> +
    +

    Medians across releases

    +

    + Compiled-chunk path (the production embedding path: compile once, run many). Ratio column + is {@latest} ÷ same-run Luerl; green means faster than Luerl. +

    + +
    + + + + + + + + + + + + + + + +
    Workload{version}{@latest} vs Luerl
    + + {row.label} + + + + {Benchmarks.value(row, version) || "—"} + <.ratio value={Benchmarks.ratio(row, @latest)} />
    +
    + +

    + † {@latest} builds PUC-Lua-conformant position-prefixed error messages; Luerl returns the + bare error value, doing less work per raise. +

    +
    + + <%!-- ============== WHERE WE'RE BEHIND ============== --%> +
    +
    +

    Where {@latest} is still behind Luerl

    +
    +

    + + pcall raise + catch ({@headline.raise_ratio}× slower). + + Partly apples-to-oranges: {@latest} builds position-prefixed error messages + ("<eval>:1: negative", the PUC-Lua-conformant behavior) where Luerl + returns the bare value — it does strictly more work per raise. Still, it regressed at {@headline.previous} ({@headline.raise_prev_ratio}×) and has only partially recovered. +

    +

    + + varargs + multiple returns ({@headline.varargs_ratio}× slower, ~1.9× the allocation). + + A genuine gap with no conformance excuse — the clearest optimization target for 1.1.x. +

    +

    + Host-boundary decode of large lists. + Not in the chart (its own harness): decoding a 4,096-element integer list is ~13× slower + than the {List.first(@versions)}/Luerl era; long-string lists ~25×. String-keyed maps + moved the other way (1.24× faster). Worth a look before 1.1. +

    +
    +
    +
    + + <%!-- ============== HOW TO READ ============== --%> +
    +

    How to read these numbers

    +
      +
    • + + {List.first(@versions)} is not an independent series. + + It wraps luerl 1.5.1; its own no-sandbox instantiation lands within ~1% of raw :luerl.init(). Its dots hugging the parity line is by construction, and it's + why the Luerl control column is meaningful across all {length(@versions)} runs. +
      • +
    • + Discipline: + Apple M4 · Elixir 1.20.0 / OTP 29 · LUA_BENCH_MODE=full + (10 s measure, 2 s warmup, 1 s memory) · one mix run + at a time on a quiet machine · medians quoted, not averages. C Lua (luaport) was + unavailable; Luerl is the reference. +
      • +
    • + The suite got wider for this report. + patterns, metamethods, pcall/varargs, and Lua.new + workloads are new — chosen to make the suite representative of real Lua (pattern engine, + : + method dispatch, protected calls), not just of what was optimized. +
      • +
    • + Coroutines are not benchmarked + — they are an intentional 1.0 capability exclusion in the library, not an omission from + the suite. +
      • +
    • + Sub-microsecond rows + (Lua.new() on {@latest}) show large Benchee deviation percentages due to + batching at that timescale; medians are stable across runs. +
      • +
    + +

    + Full report with per-workload analysis and reproduction instructions: + + bench_results/{@report} + + in tv-labs/lua. Raw Benchee outputs and parsed JSON are committed alongside, one directory + per released version. +

    +
    + diff --git a/website/lib/website_web/router.ex b/website/lib/website_web/router.ex index 4569c20..dadfdbd 100644 --- a/website/lib/website_web/router.ex +++ b/website/lib/website_web/router.ex @@ -23,6 +23,7 @@ defmodule DemoWeb.Router do get "/", PageController, :home get "/about", PageController, :about + get "/benchmarks", PageController, :benchmarks live "/playground", PlaygroundLive, :index live "/playground/:example", PlaygroundLive, :example live "/tour", TourLive, :index diff --git a/website/priv/static/benchmarks.html b/website/priv/static/benchmarks.html deleted file mode 100644 index 520d55d..0000000 --- a/website/priv/static/benchmarks.html +++ /dev/null @@ -1,388 +0,0 @@ - - - - - -Lua on the BEAM — version benchmarks (0.4.0 → 1.0.0 → 1.0.2) - - - -
    -
    -

    tv-labs/lua · full-mode benchee · 2026-07-28

    -

    Lua on the BEAM: 0.4.0 → 1.0.0 → 1.0.2

    -

    Three releases of the lua Elixir library, measured on the same machine in the same - sitting, with Luerl 1.5 run inside every benchmark as a same-run control. v0.4.0 was a thin - wrapper over Luerl; 1.0.0 introduced the native VM; 1.0.2 is the release cut today.

    -

    The headline: 1.0.2 is the first release faster than Luerl on most workloads — - and Lua.new() is now effectively free.

    -
    - -
    -
    -
    Lua.new() median
    -
    0.58 µs
    -
    63× faster than 1.0.0 (36.67 µs)
    -
    Steady-state, default options, under mix run. ~100× credible in an - :embedded release; ~5× when passing custom sandbox options (PR #419 / #398).
    -
    -
    -
    Lua.new() allocation
    -
    0.88 KB
    -
    104× less than 1.0.0 (91.88 KB)
    -
    Memoized boot-time VM template; one-time ~7 ms cold build per node.
    -
    -
    -
    fibonacci fib(30) median
    -
    434 ms
    -
    1.7× faster than same-run Luerl
    -
    Was 1.09× slower on 1.0.0 (792 ms). Allocation: 1.0 GB vs Luerl's 2.5 GB.
    -
    -
    - -
    -

    Runtime vs Luerl, per workload

    -

    Each dot is a release's compiled-chunk median divided by the Luerl median from the - same run — left of the parity line is faster than Luerl. Hover a dot for the underlying medians. - Log scale.

    -
    -
    - v0.4.0 (Luerl wrapper) - v1.0.0 - 1.0.2 - │ line = Luerl parity (1.0) -
    -
    -
    ← faster than Luerlslower than Luerl →
    -
    -
    - -
    -
    -
    -
    - -
    -

    Medians across releases

    -

    Compiled-chunk path (the production embedding path: compile once, run many). - Ratio column is 1.0.2 ÷ same-run Luerl; green means faster than Luerl.

    -
    - - - - - -
    Workloadv0.4.0v1.0.01.0.21.0.2 vs Luerl
    -
    -
    - -
    -
    -

    Where 1.0.2 is still behind Luerl

    -

    pcall raise + catch (1.55× slower). Partly apples-to-oranges: 1.0.2 builds - position-prefixed error messages ("<eval>:1: negative", the PUC-Lua-conformant - behavior) where Luerl returns the bare value — it does strictly more work per raise. Still, - it regressed at 1.0.0 (1.81×) and has only partially recovered.

    -

    varargs + multiple returns (1.36× slower, ~1.9× the allocation). A genuine - gap with no conformance excuse — the clearest optimization target for 1.1.x.

    -

    Host-boundary decode of large lists. Not in the chart (its own harness): - decoding a 4,096-element integer list is ~13× slower than the v0.4.0/Luerl era; long-string - lists ~25×. String-keyed maps moved the other way (1.24× faster). Worth a look before 1.1.

    -
    -
    - -
    -

    How to read these numbers

    -
      -
    • v0.4.0 is not an independent series. It wraps luerl 1.5.1; its own - no-sandbox instantiation lands within ~1% of raw :luerl.init(). Its dots hugging the - parity line is by construction, and it's why the Luerl control column is meaningful across all - three runs.
      • -
    • Discipline: Apple M4 · Elixir 1.20.0 / OTP 29 · LUA_BENCH_MODE=full - (10 s measure, 2 s warmup, 1 s memory) · one mix run at a time on a quiet machine · - medians quoted, not averages. C Lua (luaport) was unavailable; Luerl is the reference.
      • -
    • The suite got wider for this report. patterns, metamethods, pcall/varargs, and - Lua.new workloads are new — chosen to make the suite representative of real Lua - (pattern engine, : method dispatch, protected calls), not just of what was optimized.
      • -
    • Coroutines are not benchmarked — they are an intentional 1.0 capability - exclusion in the library, not an omission from the suite.
      • -
    • Sub-microsecond rows (Lua.new() on 1.0.2) show large Benchee - deviation percentages due to batching at that timescale; medians are stable across runs.
      • -
    -
    - - -
    - -
    - - -

    -† 1.0.2 builds PUC-Lua-conformant position-prefixed error messages; Luerl returns the bare error value, -doing less work per raise.

    - - diff --git a/website/test/website_web/controllers/page_controller_test.exs b/website/test/website_web/controllers/page_controller_test.exs index 4a52cb4..d2600a3 100644 --- a/website/test/website_web/controllers/page_controller_test.exs +++ b/website/test/website_web/controllers/page_controller_test.exs @@ -1,10 +1,68 @@ defmodule DemoWeb.PageControllerTest do use DemoWeb.ConnCase + alias Website.Benchmarks + test "GET / renders the Lua showcase landing page", %{conn: conn} do body = conn |> get(~p"/") |> html_response(200) assert body =~ "Lua, on the" assert body =~ "Playground" assert body =~ "Tour" end + + describe "GET /benchmarks" do + test "renders a column per recorded version", %{conn: conn} do + body = conn |> get(~p"/benchmarks") |> html_response(200) + + for version <- Benchmarks.versions() do + assert body =~ version + end + end + + test "renders every row's recorded value rather than a placeholder", %{conn: conn} do + body = conn |> get(~p"/benchmarks") |> html_response(200) + + for row <- Benchmarks.rows(), version <- Benchmarks.versions() do + value = Benchmarks.value(row, version) + + assert value, "#{row.label} has no value for #{version}" + assert body =~ value, "#{row.label}/#{version} (#{value}) missing from the page" + end + end + + test "links the campaign report it was written from", %{conn: conn} do + body = conn |> get(~p"/benchmarks") |> html_response(200) + + assert body =~ "bench_results/#{Benchmarks.report()}" + end + + test "is reachable from the site navigation", %{conn: conn} do + body = conn |> get(~p"/") |> html_response(200) + + assert body =~ ~p"/benchmarks" + end + end + + describe "recorded results" do + test "every row resolves against every recorded version" do + for row <- Benchmarks.rows(), version <- Benchmarks.versions() do + assert is_binary(Benchmarks.value(row, version)), + "#{row.label} did not resolve for #{version} — check the case-name normalisation" + end + end + + test "ratios are taken against the same-run control" do + for row <- Benchmarks.rows(), version <- Benchmarks.versions() do + assert is_float(Benchmarks.ratio(row, version)), + "#{row.label}/#{version} has no ratio — control job #{row.vs} missing?" + end + end + + test "versions are ordered oldest to newest" do + assert Benchmarks.versions() == + Enum.sort(Benchmarks.versions(), &(Version.compare(&1, &2) != :gt)) + + assert Benchmarks.latest() == List.last(Benchmarks.versions()) + end + end end