MODEL-MM-QWEN4-EXP W8: e2e through examples/server, the llama.cpp concurrency ladder, and the four document surfaces

[localai-bot]

W8 of #1978 — the tail that turns a
loadable model into the thing the developer actually asked for: landed on main,
benchmarked, running e2e through examples/server, documented.

Filed now, before it can start, because the pieces it should REUSE are easy to
rediscover expensively, and because two of its requirements constrain earlier waves.

Depends on

W5 (assembled forward) and W6a/W6b (#1989),
because the only artifact that fits a host we own is the GGUF. Not startable before both.

1. e2e through examples/server

examples/server/main.cpp is the target and it already exists. Per AGENTS.md "Shared
seams", it is a thin client of include/vllm.h and must stay one — the model reaches it
through ModelRegistry::Forward and the registered server path, not through a bespoke
entry point. Per "Nothing lands dead", the reachability proof for this row is a request
served by that binary on its default configuration, not a unit test.

Multimodal is in scope: the checkpoint declares image_token_id, video_token_id and a
vision_config, and the vision tower is an unmodified Qwen3.5-Moe ViT we already have.

2. Benchmarks — MOST OF THE HARNESS ALREADY EXISTS. Do not rebuild it.

tools/bench/run_serve_low.py, with serve_low_common.py, summarize_serve_low.py and
make_serve_low_corpus.py. What it already gives you, and why each part matters here:

• The timed client is an unmodified upstream module (sglang.bench_serving, pinned at
  28b095c); the local wrapper owns only command construction and validation. That is the
  difference between a benchmark and a self-report, and it is why this row must not grow
  its own stopwatch.
• Concurrency and repetitions are first-class, with artifacts written per
  c{concurrency}-r{repetition}.
• Fail-closed artifact validation, plus untimed serving preflights.

What W8 adds: a llama.cpp server arm. llama-server is OpenAI-compatible, so the same
pinned client can drive both engines against the identical artifact — which is exactly the
quant-matched condition the llama-cpp oracle's recorded scope requires.

The gate is G4 in the spec, and its shape is not negotiable
(#1990): a concurrency ladder of at
least c = 1, 4, 8, 16, 32 is the headline; prefill and decode reported separately because
input length splits them; memory as an axis, not a footnote; llama.cpp in its production
configuration; identical artifact, prompts, token counts and sampling; idle host;
reproduced with a same-binary A/B.

Do not benchmark on this box while other waves are building. Host loadavg reached 157
during W1-W6a and that alone reds test_cpu_x86_llamacpp_floor
(#618) by denying it a quiet window. A
contended measurement is not a measurement.

3. Documentation, and each surface has exactly one trigger

• docs/USAGE.md — the weights row, and it is owed in the same change that makes
  any arm reachable, not later. File name, size, the exact HuggingFace repo AND revision,
  grouped by arm, with a sha256 for each quantized artifact and the refused arms named
  beside them. A repo id alone is not a pin, because checkpoints get re-quantized in place
  under an unchanged name. Weights are the one part of a port a reader cannot infer from
  the code.
• docs/FEATURES.md — the row already exists from W1 and currently reads "REGISTERED,
  NOT LOADABLE"; it changes when that stops being true.
• docs/BENCHMARKS.md — one index row plus one docs/benchmarks/<benchmark-id>.md
  detail file, when a public benchmark ID is added.
• README.md — only if the headline or quick start changes.

Known artifact, for the weights row

unsloth/Qwen3.8-Flash-Next-GGUF, UD-IQ1_S/Qwen3.8-Flash-Next-UD-IQ1_S-0000{1,2,3}-of-00003.gguf,
67.56 GiB of weights (10.9 MB + 50.0 GB + 22.5 GB), general.architecture = qwen4exp,
1224 tensors. Pin the revision and record a sha256 per shard; do not cite the repo id
alone. Note it carries no MTP weights — that is #1993's
problem, and docs/USAGE.md should say so beside the arm rather than leave a reader to
discover it.

Done when

A request served by examples/server on its default configuration returns correct output
on the real checkpoint; the G4 ladder is recorded with values, ratios and the exact build
and run recipe on an idle host; and the four document surfaces say what is true. No axis
may be reported as satisfied on a contended host, and no ceiling may be declared if a
first measurement disappoints.