Repository navigation
bench(baremetal): Sandstorm against Kata, microsandbox, CubeSandbox, gVisor and containers on one KVM machine - #384
Merged
Merged
Conversation
compare.py measures start to first answer and memory per instance kept running, for the same two programs, on: a plain process, bubblewrap, Docker (runc), Podman, Docker + gVisor, Kata 4.2, microsandbox 0.7.8, CubeSandbox 0.7.3, and Sandstorm on gVisor and on Firecracker, cold and from a warm template. The runtimes from bench/fleet/baselines.py are reused as they are. fork.py, fork_msb.py and fork_cube.py measure snapshotting a running sandbox and cloning it, each the way its own API does it. setup.sh brings up the two Sandstorm nodes, with their own port ranges, and microsandbox's images. Results on 2x Xeon E5-2637 v2 in results/2026-10-30, with the raw rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d the pool fix Warm start 129/199 -> 62/74 ms (web/app64), ahead of CubeSandbox's 72/78; snapshot 52 -> 27 ms; clone 20 ~1,562 -> ~980 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cold 308/624 -> 234/545 ms, warm 168/186 -> 107/123 ms (web/app64); clone 20 ~2,760 -> ~2,280 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, Kata and Docker, with two charts bench/baremetal/charts.py draws start-and-memory and clone charts from the results directory with the fleet charts' palette and primitives. CubeSandbox's clone row is added to fork.jsonl as fork_cube.py printed it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… setup and fleet chart for the 2c2g comparison Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every runtime was run on the same bare-metal machine with KVM (2× Xeon E5-2637 v2, 31 GB), with the same two programs and the same harness. Results, method and raw rows are in
bench/baremetal/results/2026-10-30.For snapshot and clone, Sandstorm is fastest at snapshotting a running sandbox (52 ms, against 252 ms for CubeSandbox) and at cloning one (~187 ms, against 542 ms). CubeSandbox is faster at warm starts on this older CPU and at cloning 20 at once; both are open items in the write-up.
compare.py: start-to-first-answer and per-instance memory. Reusesbench/fleet/baselines.pyfor plain process, bubblewrap, runc, runsc and Podman, and adds Kata, microsandbox, CubeSandbox and Sandstorm.fork.py,fork_msb.py,fork_cube.py: snapshot and clone, each through that runtime's own API.setup.sh: brings up two Sandstorm nodes (gVisor and Firecracker) with separate port ranges, and loads microsandbox's images.The Sandstorm numbers depend on the fixes in #380, which came out of this comparison.
🤖 Generated with Claude Code