feat(KV-FP8): W3 -- half-sized KV blocks the runner actually serves, and a default scale that knows it was declared (#1593)

.agents/quantization-matrix.md

@@ -157,7 +157,7 @@ Pinned vLLM source: `vllm/config/cache.py:19-36`.
 
 | ID | Item | Upstream | Our code | Tests/evidence | Spike/spec | State | Owner |
 |---|---|---|---|---|---|---|---|
-| `QUANT-KV-FP8` | fp8, fp8_e4m3, fp8_e5m2 | `vllm/config/cache.py:19-25`; `vllm/model_executor/layers/quantization/kv_cache.py:42-191`; store `cache_kernels.cu:241-252`; scale convention `quant_utils.cuh:296-308` | **W1 CPU fp8-e4m3 store+read LANDED**: [codec](../include/vt/fp8_kv.h#L39), [store kernel](../src/vt/cpu/cpu_cache.cpp#L143), [read dequant](../src/vt/cpu/cpu_paged_attn.cpp#L82), [config parse](../include/vllm/v1/kv_cache_dtype.h#L37). **W2 CUDA fp8-e4m3 store+read LANDED** ([#1593](https://github.com/mudler/vllm.cpp/issues/1593)): [store kernel](../src/vt/cuda/cuda_cache.cu), [read dequant](../src/vt/cuda/cuda_paged_attn.cu) -- gate [test_cuda_fp8_kv_cache](../tests/vt/test_cuda_fp8_kv_cache.cpp), whose DEVICE cases are UNEXECUTED; the CUDA TUs COMPILE (CI `cuda-fat-build`, ten architectures, run 32495320287 on `4d71e776e`) but that job builds with tests OFF, so none has been executed (spec `## Owed`). e5m2 compute, per-head scales, the Metal/ROCm arms and the runner integration are named later bricks (see spec) | [test_ops_fp8_kv_cache](../tests/vt/test_ops_fp8_kv_cache.cpp#L1) — 8 cases / 511 assertions, round-trip within the e4m3 band + fp8-vs-bf16 NMSE<1% + paged-attention e2e; RED-first (wrong store direction fails 3/480) | [fp8-kv-cache](specs/fp8-kv-cache.md) | `PARTIAL` | - |
+| `QUANT-KV-FP8` | fp8, fp8_e4m3, fp8_e5m2 | `vllm/config/cache.py:19-25`; `vllm/model_executor/layers/quantization/kv_cache.py:42-191`; store `cache_kernels.cu:241-252`; scale convention `quant_utils.cuh:296-308` | **W1 CPU fp8-e4m3 store+read LANDED**: [codec](../include/vt/fp8_kv.h#L39), [store kernel](../src/vt/cpu/cpu_cache.cpp#L143), [read dequant](../src/vt/cpu/cpu_paged_attn.cpp#L82), [config parse](../include/vllm/v1/kv_cache_dtype.h#L37). **W2 CUDA fp8-e4m3 store+read LANDED** ([#1593](https://github.com/mudler/vllm.cpp/issues/1593)): [store kernel](../src/vt/cuda/cuda_cache.cu), [read dequant](../src/vt/cuda/cuda_paged_attn.cu) -- gate [test_cuda_fp8_kv_cache](../tests/vt/test_cuda_fp8_kv_cache.cpp), whose DEVICE cases are UNEXECUTED; the CUDA TUs COMPILE (CI `cuda-fat-build`, ten architectures, run 32495320287 on `4d71e776e`) but that job builds with tests OFF, so none has been executed (spec `## Owed`). **W3 runner integration LANDED** ([#1593](https://github.com/mudler/vllm.cpp/issues/1593)): [`--kv-cache-dtype`](../src/vllm/entrypoints/openai/server_main.cpp) reaches `EngineParams`, the checkpoint's `kv_cache_quant_algo` is honoured when no flag is typed, [`ApplyCacheDType`](../src/vllm/v1/kv_cache_interface.cpp) retypes every attention spec (group and heterogeneous per-layer alike) so one byte budget buys exactly 2x the blocks, and [`kv_cache_route.h`](../include/vllm/model_executor/models/kv_cache_route.h) is the ONE place the store and the read decide float versus fp8. An fp8 cache DISABLES FA-2 prefill and decode, the WMMA ladder and the vectorized decode kernels, which are bf16-native; the resulting throughput cost is unmeasured and recorded as such. e5m2 compute, per-head scales, the Metal/ROCm arms, the C-ABI field, the 16 unrouted architectures (15 of which refuse with a message that names neither fp8 nor the flag) and the `k_scale`/`v_scale` weight-loader read are named later bricks (see spec) | [test_ops_fp8_kv_cache](../tests/vt/test_ops_fp8_kv_cache.cpp#L1) — 8 cases / 511 assertions, round-trip within the e4m3 band + fp8-vs-bf16 NMSE<1% + paged-attention e2e; RED-first (wrong store direction fails 3/480); W3 [test_kv_cache_fp8_wiring](../tests/vllm/entrypoints/test_kv_cache_fp8_wiring.cpp) — 31 cases, G1-G12, entering through `LoadedEngine` rather than by building a spec by hand, and bounding the fp8 pages against a bf16 run inside e4m3's own round-trip envelope, and [test_serve_kv_cache_dtype](../tests/vllm/entrypoints/openai/test_serve_kv_cache_dtype.cpp) — the flag through the REAL `VllmServerMain` | [fp8-kv-cache](specs/fp8-kv-cache.md) | `PARTIAL` | - |
 | `QUANT-KV-FP8-VENDOR` | fp8_inc, fp8_ds_mla | `vllm/config/cache.py:24-25`; vendor KV implementations selected by attention backend | - | no quantized KV cache | `planned: specs/vendor-fp8-kv-cache.md` | `INVENTORIED` | - |
 | `QUANT-KV-TURBO` | k8v4, 4bit_nc, k3v4_nc, 3bit_nc | `vllm/config/cache.py:28-33`; TurboQuant dependency path | - | no quantized KV cache | `planned: specs/turboquant-kv-cache.md` | `INVENTORIED` | - |
 | `QUANT-KV-PER-HEAD` | int4/int8/fp8 per-token-head | `vllm/config/cache.py:34`; quantized cache kernels selected by backend | - | no quantized KV cache | `planned: specs/per-head-kv-cache.md` | `INVENTORIED` | - |