From 22a5f5df9f68ae7648f3206b3d620bd12afd48a4 Mon Sep 17 00:00:00 2001 From: akshathayadav378-boop <287247196+akshathayadav378-boop@users.noreply.github.com> Date: Sat, 14 Nov 2026 10:38:58 +0800 Subject: [PATCH] docs(strata): say the bench subcommand is not in this build --- edge0-strata/docs/BENCHMARK_PROTOCOL.md | 4 ++++ edge0-strata/docs/COMMUNITY_BENCHMARKS.md | 3 +++ 2 files changed, 7 insertions(+) diff --git a/edge0-strata/docs/BENCHMARK_PROTOCOL.md b/edge0-strata/docs/BENCHMARK_PROTOCOL.md index 949f9d8f..9a708344 100644 --- a/edge0-strata/docs/BENCHMARK_PROTOCOL.md +++ b/edge0-strata/docs/BENCHMARK_PROTOCOL.md @@ -3,6 +3,10 @@ **Status: draft, implemented in `strata bench run`; pilot validation pending.** Protocol ID: `strata-qwen38-thinking-v1-draft.1`. Result format: `strata-standard-benchmark`, schema version `1`. +> **Availability in this repository:** the `edge0-strata` build here has no `bench` subcommand, so +> `strata bench run` cannot be used yet. Use the community scripts in +> [COMMUNITY_BENCHMARKS.md](COMMUNITY_BENCHMARKS.md) until the runner is ported. + This document specifies a repeatable community benchmark and the data a future website can consume. The required suite took **2 to 4 minutes after model loading** on the two machines it has been run on (RTX 3060 and 2x RTX A4000, IQ3_XXS, Strata 0.2.1: 3 min 22 s and 2 min 22 s; the optional concurrency stage added 1.5 to 2 minutes). diff --git a/edge0-strata/docs/COMMUNITY_BENCHMARKS.md b/edge0-strata/docs/COMMUNITY_BENCHMARKS.md index 1c518d6b..b75e64b2 100644 --- a/edge0-strata/docs/COMMUNITY_BENCHMARKS.md +++ b/edge0-strata/docs/COMMUNITY_BENCHMARKS.md @@ -9,6 +9,9 @@ sampling, twelve sanity checks, about 2 to 4 minutes after the model has loaded the hardware, versions and every request. For a quick look at your own machine, [`BENCH.bat` / `./bench.sh`](#run-the-benchmark) takes 3 to 10 minutes and writes a report in the format below. +> The `edge0-strata` build in this repository has no `bench` subcommand yet, so the standard benchmark +> cannot be run here; the community scripts above are the way to measure for now. + The protocol is still a draft until a pilot has run on two kinds of hardware. The reports listed below were made before it existed and are not standard results; they stay valid under this guide.