diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 7cd159742..d1c848f80 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -761,3 +761,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#2040](https://github.com/mudler/vllm.cpp/issues/2040) | `ENG-HYBRID-PLACEMENT` | W3c: replace W3b's hand-written per-model round trip with ONE shared seam every architecture routes through, exploiting that every MoE block already has the same `(Dev, weights, params, [T,H] dh, T)` shape — and correct `docs/ENVIRONMENT.md` and `docs/FEATURES.md`, which W3b made false by leaving them saying the knobs place nothing | bug | | [#2046](https://github.com/mudler/vllm.cpp/issues/2046) | `ENG-HYBRID-PLACEMENT` | `qwen3_5.cpp` kept private `Dev`/`DBuf`/`MakeTensor`/`Reshape` copies instead of the shared `dense_device_glue.h` set — the off-framework divergence its own `ResidentWeight` comment records, where a repair reached 25 model files and not this one. The private types also had INTERNAL LINKAGE, which is what forced the MoE placement seam to carry a glue-templated second spelling; migrating collapses it back to one | bug | | [#2050](https://github.com/mudler/vllm.cpp/issues/2050) | `ENG-HYBRID-PLACEMENT` | Laguna's FFN is host-orchestrated token-at-a-time — per-token host rows, the router on the host through `MatmulNK`, and a host scalar combine loop — so a device-shaped MoE entry wrapping those loops would put it in the placement seam's wired list while moving nothing and adding a round trip: supported to read, a regression to measure. The real repair is a device-resident batched FFN, which is a model rework with a performance gate | gap | +| [#1756](https://github.com/mudler/vllm.cpp/issues/1756) | `ENG-PUBLIC-DOC-PROJECTIONS` | Three user-facing benchmark references retained the old public-document shape after #1714: README sent two claims to the internal benchmark record, the speculative-decoding guide described the compact index as the old record, and the llama.cpp detail page named deleted `docs/STATUS.md`. Fixed in flow by linking the public benchmark detail pages directly. No result, support claim, or README news headline changed | bug | diff --git a/.agents/specs/public-doc-projections.md b/.agents/specs/public-doc-projections.md index 0c260ab78..d2224cae8 100644 --- a/.agents/specs/public-doc-projections.md +++ b/.agents/specs/public-doc-projections.md @@ -318,3 +318,12 @@ The current-main reconciliation preserved the `docs/benchmarks/at-a-glance.md`. The operator gate passed every change-related check. The unrelated shared-host contention means this result does not claim fully green integration. + +### Follow-up link repair + +Issue [#1756](https://github.com/mudler/vllm.cpp/issues/1756) repaired three +references that retained the old public-document shape after the split. README +benchmark claims now lead to public detail pages. The speculative-decoding +guide links its benchmark detail directly. The llama.cpp detail page no longer +names the deleted `docs/STATUS.md`. No result, support claim, or news headline +changed. diff --git a/README.md b/README.md index 67265cecb..0207d9133 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ treat c2 through c32 as ties. The tokens are identical either way, and the insta 9.1 GiB. Cold start to first `/health`: **36.5 s vs vLLM's 221.5 s (6.1x)**, provisional -([detail](.agents/benchmark-record.md)). +([measurement details](docs/benchmarks/vllm-online-serving.md)). Peak host memory is a clean win at **24.88 GiB vs vLLM's 28.18 GiB**, with no Python stack behind it: @@ -156,7 +156,8 @@ shape-gated to prefill (95.9% on the default build). MTP is **token-identical to vLLM's MTP and about 4% faster** at c1 on Qwen3.6-27B-NVFP4, on both gate models end to end. Block-diffusion DFlash runs about 2x over spec-off but stays below vLLM's -throughput. That gap is an open bf16 acceptance floor, tracked in the benchmark record. +throughput. The [speculative-decoding measurements](docs/benchmarks/speculative-decoding.md) track +that open bf16 acceptance floor. Full per-axis grids, memory tables, the nine residual axes, and exact reproduction recipes: [docs/BENCHMARKS.md](docs/BENCHMARKS.md). The two figures above are rendered from these measured diff --git a/docs/SPECULATIVE-DECODING.md b/docs/SPECULATIVE-DECODING.md index d6395ff18..91597ae07 100644 --- a/docs/SPECULATIVE-DECODING.md +++ b/docs/SPECULATIVE-DECODING.md @@ -362,11 +362,12 @@ with a SINGLE COLD oracle invocation per paired run, so the denominator paid compile-JIT the numerator did not; with the oracle warm and generation length matched, the paired measurement is **0.834x**. Those two cannot be differenced directly, because the gate host was reimaged in between and is no longer the same -machine. The deciding experiment, a single cold oracle invocation on the CURRENT -box, is specified in the benchmark record and has not yet run. Until it does, no -speed claim in either direction is supportable. The acceptance-rate band and the -other target families remain owed. A GGUF target, and a target architecture with no aux -multi-tap, are both refused by name. +machine. The deciding experiment is a single cold oracle invocation on the +current box. The [speculative-decoding benchmark page](benchmarks/speculative-decoding.md) +records the pending run. Until it runs, no speed claim in either direction is +supportable. The acceptance-rate band and the other target families remain +owed. The loader refuses a GGUF target or a target architecture without an +auxiliary multi-tap by name. ```bash main --model /models/Qwen3-4B \ @@ -457,8 +458,8 @@ Speculation helps both engines about 1.5 to 1.6x at this operating point, and ou engine is already about 4% faster than vLLM with speculation off, so the lead is preserved with it on. The extra state speculation needs (a doubled recurrent-state slot at k=1, plus the draft cache and head) costs about 3.6 GB, well inside the -box's unified memory. The full A/B, including the higher-concurrency numbers, is -in [the benchmark record](BENCHMARKS.md). +box's unified memory. The [speculative-decoding benchmark page](benchmarks/speculative-decoding.md) +contains the full A/B, including the higher-concurrency numbers. ## Concurrency above 1 diff --git a/docs/benchmarks/llama-cpp-cpu.md b/docs/benchmarks/llama-cpp-cpu.md index 0365b4309..d2dd17b7e 100644 --- a/docs/benchmarks/llama-cpp-cpu.md +++ b/docs/benchmarks/llama-cpp-cpu.md @@ -34,7 +34,7 @@ set is swept over every tracked file, not listed: the query is in the | Flips first | Verdict | Exposure | |---|---|---| | 1 | Vulkan `BENCH-VK-LLAMA` decode 4.36 vs 4.35, `MET` | 0.23% margin inside a 0.69% 7-leg spread. Its own source calls it "a narrow pass, not a comfortable one", so any move can flip it. The README states it as "matches" | -| 2 | Muse Glimmer in128 prefill `1.023x` (also `STATUS.md`) | 2.3% margin inside our own arm's 4.5% leg spread, n=4. Its denominator is already stock `7044859`, 84 commits from `b10451`, so the noise floor is what exposes it | +| 2 | Muse Glimmer in128 prefill `1.023x` (also in [At a glance](at-a-glance.md)) | 2.3% margin inside our own arm's 4.5% leg spread, n=4. Its denominator is already stock `7044859`, 84 commits from `b10451`, so the noise floor is what exposes it | | 3 | This table's peak memory `1.01x` PARITY and decode `0.97x` tie | ties by declaration, not wins. A denominator that moves at all in llama.cpp's favour turns both into recorded gaps | | 4 | keep-f16's `1.01x` RSS and "prefill `1.18x` AHEAD", quoted in product code for `VT_GGUF_KEEP_F16` default ON | buys 1.05 GiB of peak RSS (3.885 to 2.832) for ~9% prefill (224 to 204 tok/s) and ~1.4% decode, tokens identical. **Decided 2026-08-17: the default stays ON on our own arms, so it needs no re-take.** Both ratios do | | 5 | `KERNEL-GEMM-CPU-TILED` NEON vs stock ggml sgemm, "at parity, ahead on 4 of 6 shapes" | off this page, in the kernel matrix. Bands overlap, 216-242 vs 208-215 GFLOP/s, and one shape is already behind |