fix(GATE-ISSUE-INDEX-OWNER-KEY): key the issue index on the issue AND its owning row, so a hand-off between two lanes is a record - #1744
Open
localai-bot wants to merge 8 commits into
Open
Conversation
…D its owning row (#1731) `check-agent-record.py` is red on `main` at `038ff61e5` because `.agents/issue-index.md` lists #1649 twice: once under `ENG-HF-MODEL-DOWNLOAD`, the lane that found and filed the bug, and once under `GATE-WINDOWS-PORTABILITY-TARGET-SCOPE`, the lane that fixed it. Neither row is wrong and neither branch could see the other. The spec argues that the ban on a second row is a leftover. It arrived in `8dd6508da`, when the surface was a keyed, editable intake table inside `roadmap_v1.md`. `51e0cb5b1` moved it out, made it append-only and gave it `merge=union`, and uniqueness by number survived that move unexamined. In an append-only log an update IS an append, so one row per issue means one statement per issue forever. The tree already pays for that twice in prose: `bpe-quadratic-merge.md` and `serve-request-length-guard.md` each record a fact that belongs in the index, because appending it would red the gate. The design keys uniqueness on `(issue number, owning row ID)` and reports both line numbers. #1619 measured the one corruption this refusal has ever caught, where the union driver duplicated a row BYTE-IDENTICALLY at two lines; a copied line carries the same owner, so the pair key still collides and that catch is unchanged. What is given up is a mistyped owner on a second row, which no reading of the file can separate from a legitimate hand-off. Spec only, so `check-agent-record.py` and `test_agent_record` stay red at this commit with the inherited #1649 duplicate. The implementation commit clears both. `test_cpu_x86_llamacpp_floor` also reds here at loadavg 45.97 with `waiting for quiet ... exit 4`, which is #618 and is not this change. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
… its owning row, so filing and fixing on two branches stops redding main (#1731) `scripts/check-agent-record.py` was red on `main` at `038ff61e5` with `.agents/issue-index.md: issue #1649 listed twice`, and every branch that merged `main` inherited it. The two rows are both correct: `ENG-HF-MODEL-DOWNLOAD` recorded the bug and the attribution to `a50c57d69`, and `GATE-WINDOWS-PORTABILITY-TARGET-SCOPE` recorded the fix and a second red the first row did not carry. Neither branch could see the other. THE BAN WAS A LEFTOVER FROM A DIFFERENT RECORD SURFACE. It arrived in `8dd6508da` (2026-08-09), when the intake table was a KEYED, editable table inside `.agents/roadmap_v1.md` and one row per issue was the whole design. `51e0cb5b1` moved it into `.agents/issue-index.md`, made it append-only and gave it `merge=union`; uniqueness by issue NUMBER survived that move unexamined. In an append-only log an update IS an append, so one row per issue permits one statement per issue for the life of the repository. The tree already pays for that twice in prose: `.agents/specs/bpe-quadratic-merge.md` records that `#1365`'s row under-describes its own re-scoped issue and that "Both available edits are gate failures, so the reconciliation is PROSE", and `.agents/specs/serve-request-length-guard.md` records the same refusal for `#1541`. The remedy the ban forces is the operation the file forbids, and `scripts/check-issue-index-append-only.py` refuses that too. `check_issue_index` now keys `seen` on `(issue number, owning row ID)` and holds the line number of the first row for each pair, so the refusal names the repeating line, the first line and the shared owner instead of only the number. The line numbers are derived at read time and stored nowhere; a line number written INTO an append-only file goes stale on the next append. THE ONE CORRUPTION THIS REFUSAL HAS CAUGHT IN THE FIELD IS UNAFFECTED, which is why the ban was narrowed rather than dropped. `#1619` measured the `merge=union` driver duplicating a row BYTE-IDENTICALLY at two lines, 538 where the correct union is 537; `git merge-tree --write-tree` called it clean and `check-issue-index-append-only.py` passed, because a duplicate is an ADDITION and that checker collects removals only. `check-agent-record.py` was the single gate that refused it. A duplicated LINE carries its owner with it, so the pair key still collides, and `test_a_byte_identical_duplicate_row_is_rejected` reproduces that shape. GIVEN UP DELIBERATELY: a mistyped owner on a second row, which no reading of this file can separate from a legitimate hand-off. REJECTED DELIBERATELY: a cap on rows per issue, which is AGENTS.md #Records' "limit an entry, not a shared file" anti-pattern at the scale of one issue and would force the third lane to delete an earlier row; and an owner-ID existence check, which would red `main` on day one because `GATE-WINDOWS-PORTABILITY-TARGET-SCOPE` and this row are unplaced gate rows with no matrix entry and both are legitimate owners. `UNOWNED_HIGH_WATER` stays at 33 and the unowned count stays PER ROW, because AGENTS.md states that obligation per row and this change owes one semantic move, not two. RED FIRST, measured on the unmodified checker: `IssueIntakeTable` ran 11 tests with 4 failures -- `test_a_second_row_under_a_different_owner_is_a_record`, `test_a_dashed_row_and_an_owned_row_are_not_a_duplicate`, `test_a_duplicate_under_one_owner_names_both_line_numbers` and `test_the_tracked_index_is_valid`. After: 113 tests OK across the module, up from 109 with 1 failure, and `scripts/check-agent-record.py` on the real tree returns rc 0. MUTATION, both directions, tree restored byte-for-byte and verified by sha256. Forcing the key back to the number alone (`key = (number, None)`, sha `710224b8`) reds exactly the three admission cases including the real tree, 3 of 113. Forcing the pair to the permissive value (`first = None`, sha `8e7cfe09`) reds exactly the three refusal cases, 3 of 113. The clean file is `7abe4aa4b3a8b0d776364207396be146e58dd9d34067b9f258c106a96e12d593` before and after. `test_a_duplicated_issue_is_rejected` is rescoped, not deleted: it appended its duplicate under a DASH while the first `#201` row is owned by `BACKEND-ROCM`, so it asserted the old key by accident. The owner now matches, which is the guarantee its name claims, and the different-owner half becomes a case of its own with the opposite expectation. The preamble gains a paragraph telling an appending agent which second row is legal, mirrored into `INDEX_PREAMBLE`, because an agent reads the preamble BEFORE appending and reads a checker message only after redding the gate. It inserts whole new lines and edits none, so `scripts/check-issue-index-append-only.py` reports `OK` and `git diff origin/main --numstat -- .agents/issue-index.md` is `8 0`. This change has to append its own second `#1731` row to land, which is its first legitimate case: the existing `#1731` row names `ENG-RECORD-CONFLICT-SURFACES`, whose spec says "No product source, kernel or gate semantic moves", so the fixing row is this one and AGENTS.md requires the index, the spec and the pull request body to agree. Two record anchors in `.agents/engine-matrix.md` moved because the test insert shifted them, and they are repaired in the same change: `tests/scripts/test_agent_record.py:1397` to `:1476` and `:1465` to `:1544`. The anchor-rot bucket returns to its baseline of 31 stale. `scripts/agent-preflight.sh --staged` is green except `test_cpu_x86_llamacpp_floor`, which is `#618`: its contended leg exits 4 `NO_QUIET_WINDOW` instead of 2 at loadavg 88.06. It failed identically on the spec-only tree at loadavg 45.97, so it is the box and not this change. `git merge-base --is-ancestor origin/main HEAD` is rc 0 and RANGE_COUNT is 1, so the trailer gates ran instead of taking the `TRAILER_BEHIND` skip. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
`origin/main` advanced to `2d2a66715` (#1739, CAMPPlus) while the preflight for this branch was running, so the branch stopped being a descendant of its base. The merge touches no file this branch changes: main added no `.agents/issue-index.md` row, and the row count is 620 on `origin/main`, 621 on this branch before the merge and 621 after, so the union driver dropped nothing. Gates rerun after the merge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
Pinned `origin/main` at 1a1d17e, which adds one commit (the llama.cpp oracle gateability record) touching no file this branch touches, so `.agents/issue-index.md` is not in the merge at all. The index row count is 621 on both sides of the merge: the union driver has dropped a tail row on a clean merge before (#1619), so the count is taken rather than assumed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…tree each number came from, and replace two unreproducible mutation hashes (#1731) The section named ONE "before" tree and quoted two numbers taken on two different ones. `IssueIntakeTable` has SEVEN cases on `origin/main`, exactly one of which fails; the recorded "11 tests, 4 failures" needs this branch's suite to exist at all, so it described the mixed tree under `origin/main`'s name. The table now carries three named columns -- `main`, the mixed red-first tree, and this branch -- and every number in it was taken at `66f055248`. The two mutation hashes could not be reproduced from the mutation text printed beside them. Re-measured in bare, single-substitution form so a reader can reproduce each: `first = None` is `cf8eea17...` and reds 3 of 113, `key = (number, None)` is `a14213fc...` and reds a DISJOINT 3 of 113, and a third mutation dropping the line numbers from the message is `fcddaa08...` and reds the one case that asserts them. Disjoint sets are the property worth having: one mutation proves the refusal still refuses, the other proves the narrowing is what admits the hand-off, and neither alone shows both. Two instrument failures are recorded because each would have read as a pass. The import check reported `AttributeError: 'NoneType' object has no attribute '__dict__'` on the PRISTINE file -- `@dataclass` resolves `__module__` through `sys.modules` and the module was not registered there -- so every mutant would have been reported as "does not build". It is now validated in both directions before use. Separately the harness was killed by a timeout part-way through the third mutation and left the tree at `fcddaa08...`; the restoration is proved by hash, not by the harness having finished. `UNOWNED_HIGH_WATER` needs no adjustment: 33 before and after, with 33 unowned rows on the branch, because the appended row names an owner. Also records the concurrent `row/FIX-ISSUE-INDEX-1649-DUP` lane and credits its two findings, both re-measured here first. #1733 duplicates #1731 by 17 minutes and its bold claim that the repair "is NOT blocked" is false, because `check-issue-index-append-only.py:50-51` diffs a COMMIT range: the same deleted bytes give rc 0 uncommitted and rc 1 committed. If this row lands, that lane's row deletion becomes unnecessary. No checker, test or index byte changes here. The spec is the only file touched. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
…g main's index wholesale Pinned `origin/main` at `6354755ba`, which lands the OTHER repair for #1731 (PR #1742): it deletes the `ENG-HF-MODEL-DOWNLOAD` `#1649` row as an argued append-only exception, appends a `#1733` row, and closes #1731. The immediate red is therefore already gone from `main`, and this row's remaining value is preventive rather than curative -- the narrowed key removes the CLASS, so the next filing-and-fixing hand-off does not have to spend a row deletion. `.agents/issue-index.md` is a keyed record, so the automatic three-way result was not accepted. The file was rebuilt as main's complete version plus this branch's two scoped edits re-applied: the preamble paragraph before the table header, and this row's `#1731` row appended at the END. The union driver's own result differed -- it placed this branch's row BEFORE main's newly appended `#1733` row rather than after it -- and the rebuilt ordering is the one committed. Row counts taken rather than assumed, because the union driver has silently dropped a tail row on a clean merge before (#1619): main 620, this branch 621, merged 621, with the last row this branch's and exactly one `#1649` row surviving, which is main's deletion respected rather than reverted. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
… so this row is preventive and its own index is the first live case `6354755ba` (PR #1742) landed the other repair while this row was being gated: it deleted the `ENG-HF-MODEL-DOWNLOAD` `#1649` row and closed #1731. `main` is green on `check-agent-record.py` by its own deletion, verified by running main's checker against main's index. This row therefore no longer cures a red, and the spec now says so instead of claiming a green it did not produce. What it still carries is the class. The deletion is a per-instance cost that recurs every time two lanes touch one issue; the narrowed key makes the next hand-off an append. The argument is no longer hypothetical, because after the merge this branch's own index carries #1731 TWICE -- at `:644` under `ENG-RECORD-CONFLICT-SURFACES` which filed it, and at `:646` under `GATE-ISSUE-INDEX-OWNER-KEY` which fixed it -- with zero duplicate `(number, owner)` pairs. Main's checker refuses this branch's index with `issue #1731 listed twice`; this branch's checker passes it. Re-measured after the merge rather than dated, because main's change disarmed part of the proof. `test_the_tracked_index_is_valid` red under the `key = (number, None)` mutation because the real index held two `#1649` rows; that reason is gone and it reds for a new one. Both predicate mutations still red three cases each at the same hashes. A mutation table left unrechecked here would have kept a green a later commit on main had stopped earning. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
This was referenced Aug 23, 2026
…re, and the ratchet it named as a guard cannot see an owed issue Three findings from the fresh review of #1744. The review PASSED the narrowed key itself, so nothing here re-litigates the design. The row was in breach of the rule it is about. `AGENTS.md` requires the issue in three places that agree, and this row had it in none: the spec named #1731, which #1742 CLOSED; the index carried no row for the open issue; and the pull request body never mentioned it. #1745 is the open issue, filed against this branch by name, and it now stands in all three. #1731 stays named through the spec as the first instance and as the red that no longer stands. A guard was asserted in prose that does not exist for most of the index. §4 of the spec and the docstring of test_a_dashed_row_and_an_owned_row_are_not_a_duplicate both said the opposite order "is already gated: an added dashed row raises the unowned count and reds the UNOWNED_HIGH_WATER ratchet". It does not, whenever any spec lists the issue under `## Owed`, because check_issue_index counts a dashed row only `if row_id is None and number not in owed`. Measured on this tree: 328 owed numbers, and 226 of 621 rows carry both an owner and an owed issue. Probing the real index with a dashed #168 row leaves this branch's checker silent at rc 0 with the unowned count still 33, while the contrast row #618, an issue no spec owes, reds it with `34 rows name no owner`. The index was restored byte-for-byte. The exemption is `AGENTS.md` working as written, so the sentences are corrected and the missing ORDERING rule is #1749, owed in the spec rather than built here: it is a second checker semantic in a change that owes one. Nothing proves the CLI runs check_issue_index. Its sole production call site is unguarded inside main(); replacing it with `pass` leaves the suite at 113 tests OK and makes the command accept a byte-identical duplicate row, the exact #1619 corruption the pristine tree refuses at rc 1. Inherited rather than introduced, since `git log -S` on that call returns one commit and this change touches only the body of the function, so it is filed as #1750 and owed, not repaired here. The docstring repair moved two symbols other records cite, and the record-anchor ratchet caught it in the same run: `stale: 33 > baseline 31`. The two ENG-RECORD-ANCHOR-RATCHET citations advance to where the symbols now are, and the baseline is untouched. The three appended index rows carry no `path:line` citation at all, because the index cannot be edited and a stale anchor written into it can never be repaired; they name symbols, and the line numbers live in the spec. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
This was referenced Aug 23, 2026
localai-bot
added a commit
that referenced
this pull request
Aug 23, 2026
…and a default scale that knows it was declared (#1593) (#1641) W1 landed the CPU fp8-e4m3 KV store and read, W2 the CUDA arm, and both landed with nothing reaching either from a production entry point. W3 is the wiring: `--kv-cache-dtype` on the server flag, the checkpoint's own `kv_cache_quant_algo` honoured when no flag is typed, KV blocks sized at one byte per element, and the `k_scale`/`v_scale` path. **Now based on `main`.** W2 merged as squash `53f5b74be`, so this branch's previous base `row/KV-FP8-W2` is dead and GitHub was computing mergeability against it. `origin/main` is merged in and the conflict set was re-derived here, not inherited. Issue: #1593 (the issue that carries W2 and W3). Spec: [`.agents/specs/fp8-kv-cache.md`](.agents/specs/fp8-kv-cache.md) `## W3 — the runner integration`. ## What the FIRST fresh review found, and what changed The review of `68d2e0d5a` returned **FAIL — not safe to merge**, with one functional bug. Every finding is repaired here, and the second review re-proved every one of them by mutation. ### The bug: `--kv-cache-dtype fp8` threw at the first forward on every real checkpoint `FullAttnBlockPaged` cast K and V to the cache dtype only under `if (kv.dtype == DType::kBF16)`, and on the fp8 route `kv.dtype` is `kI8`. So K still arrived as f32 — `attn_dt` is f32 for every fp8 cache, because `kv.dtype == DType::kBF16` is a term of both FA2 eligibility tests — while V arrived as whatever the `v_proj` GEMM emitted. That is **bf16** on the block-wise fp8 arm (`MatmulFp8BlockScaledD`), **bf16** on the NVFP4 arm under the default `VT_BF16_GEMM_OUT`, and **bf16** on ordinary torch safetensors (`MatmulBf16D`). `vt::ReshapeAndCacheFp8` quantizes from ONE source dtype, so the flag threw `reshape_and_cache_fp8: k/v must share one float dtype` at the first forward on every weight arm a user has — **including the #1574 campaign checkpoint this wave exists to serve**, so W3 as written did not deliver the matched comparison it is on the critical path for. The gate could not see it. `MakeDenseWeights` builds projection weights with no `nk` flag, `ProjectFullAttnQkv` serves those through `MatmulF32D`, and f32 pairs with f32. Only the per-tensor fp8 arm and that synthetic arm ever paired. The fp8 route now normalises both to bf16, exactly as the bf16-cache route already did, reusing the two `DBuf`s that were already allocated. bf16 rather than f32 because bf16 is the dtype upstream quantizes from: vLLM's model IS bf16 where `reshape_and_cache_flash` takes key/value (`cache_kernels.cu:314-401`). **G9** is the new case, and it enters through `LoadedEngine` with the raw torch Linear layout (`[N=out, K=in]`, `nk`) a real safetensors checkpoint ships. ### The merge conflict in `model_loader.cpp` was SEMANTIC, and both sides survive `main`'s SPEC-DRAFTER-CHAIN W1 refusal (#1522) and this row's resolution stanza both insert as the FIRST statement of `FromModelDir`, and both are load-bearing: that row's G5 requires the chain refusal *before any weight I/O*, and `ReadQuantConfigJson` opens a file inside `model_dir`. Taking either side drops a guarantee. Both survive, chain refusal first, and the order is **gated**. `tests/vllm/entrypoints/test_drafter_chain_reach.cpp` cannot catch an inversion: it points at a NONEXISTENT directory, and `ReadQuantConfigJson` answers `""` for one of those without opening anything, so the chain refusal arrives either way. G10's ordering case points at a directory that EXISTS and declares fp8, so an inverted order announces the declaration first — and that line is the evidence. Mutation **M19b** MOVES the refusal below the stanza (rather than deleting it) and reddens exactly that assertion. The keyed records — `.agents/engine-matrix.md`, `.agents/quantization-matrix.md`, `.agents/issue-index.md` — take the target-branch version with the scoped edit re-applied by hand. The union merge had **duplicated two `issue-index` rows** (#1593 and #1636 appear once on `main` and twice after the automatic merge); that is the known union-driver trap and it is undone here. ### Three surfaces that no gate reached | Surface | The review's mutation | Now gated by | |---|---|---| | `FromModelDir`'s resolution stanza + `ReadQuantConfigJson`, whose only caller it is | deleting the whole stanza left **19/19, 89/89 SUCCESS** | **G10**, through `LoadedEngine::FromModelDir` on a real directory | | the `--kv-cache-dtype` parse arm and `engine_params.kv_cache_dtype = args.kv_cache_dtype` | deleting both left the same green, and no test in the tree named the flag | new `tests/vllm/entrypoints/openai/test_serve_kv_cache_dtype.cpp`, driving the REAL `VllmServerMain` | | `ApplyCacheDType`'s `per_layer_attn_specs` loop | deleting it left the same green | **G11**, on the arithmetic it claims: every non-null spec retyped, and `KVBytesPerBlock` exactly halved | The serve gate works on a polarity rather than on a printed value: the loader announces the checkpoint's declaration only when nothing explicit was typed, so CASE 1 (no flag) requires the line and CASE 2 (`--kv-cache-dtype bfloat16`) requires its absence. Break either half of the flag's chain and the default `auto` arrives, the checkpoint wins, and the line comes back. ### The file precedence was inverted against upstream `ReadQuantConfigJson` read `hf_quant_config.json` first and fell back to `config.json:quantization_config`. `vllm/transformers_utils/config.py:751-761` does the reverse, under upstream's own comments: ModelOpt writes the inline document from 0.31.0 on, and the standalone file is what 0.29.0 and before wrote, consulted only when the inline one is `None`. Mirrored, and gated at both the resolver and the loader by G10 — a checkpoint re-quantized in place, inline document added and stale legacy file left beside it, otherwise resolves to the OLD declaration and quantizes a KV cache nobody asked to quantize, at half the page, silently. ### What turning it on costs, recorded rather than claimed An fp8 KV cache takes the model **off every fast attention kernel this engine has**. `qwen3_5.cpp` makes `kv.dtype == DType::kBF16` a term of both `fa2_prefill` and `fa2_decode`; `src/vt/cuda/cuda_paged_attn.cu:2941-2960` routes an fp8 read only through tiled prefill and block decode, because the WMMA ladder, the vendored FA-2 launchers and the vectorized decode-opt/GQA kernels are bf16-native by construction. W3's rationale is that fp8 KV makes campaign #1574 a matched comparison, where we currently lead **11.06 vs 9.71 tok/s** on the bf16 path. Turning it on plausibly LOSES that lead, and this session had no GPU and measured nothing. So the spec's `## W3`, `docs/USAGE.md`, `docs/FEATURES.md` and both matrix rows now say the exchange rate is unmeasured, and the fast-kernel routing is named under `## Owed`. No direction is claimed. ## What the SECOND fresh review found, and what changed The review of `947c5b65f` returned **FAIL**, with one blocking finding. Every finding is repaired here, and the first round's repairs are untouched. ### BLOCKING: the shared seam's fp8 routing was DEAD CODE `dense_attn::AttnBlock` is named in AGENTS.md `## Shared seams` as the decode seam, and W3 listed it as routed. It was not. The routing was written — `fp8_kv`, `WriteKvCache`, `ApplyKvCacheQuant` — underneath a preamble guard that still admitted only `kBF16` and `kF32`, and `IsFp8KvCache` is true only for `kI8`. So `fp8_kv` was provably false at every call and neither fp8 arm could execute. Not an inference: a contradiction between two lines of the same function. **The gate was blind to it.** The reviewer reverted the ENTIRE routing from the seam and got 26 cases / 120 assertions SUCCESS, because every case in the file entered through `Qwen3_5DenseModel::Forward` and none entered here. The seam is the production forward for Qwen3 dense (`qwen3.cpp:185`), Qwen3-MoE (`qwen3_moe.cpp:84`), Voxtral (`voxtral.cpp:102`) and the Llama, Mistral and InternLM2 registries that share `Qwen3DenseModel`. `--kv-cache-dtype fp8` on any of them sized the pool at half and then threw a message naming neither fp8, nor the flag, nor the row. The guard is widened exactly as `qwen3_5.cpp:5313` already was, and **G12** is the new gate. It enters through `Qwen3DenseModel::Forward`, which is what `ForwardQwen3ForCausalLM` calls under `ModelRegistry::Forward` (`qwen3_dense.cpp:113`), and it asserts three things a "it ran" case would not: the half-width pages carry bytes after the forward, two fresh stacks agree bit for bit, and the fp8 logits DIFFER from the same forward over a bf16 cache (320/320, max |delta| 3.39e-4) — which is what says the read really dequantized rather than the cache behaving like a float one. Mutation **MSEAM** reverts the seam's routing the way the reviewer did and reddens both cases. ### The #1574 checkpoint does NOT take the declared-fp8 path, and five places said it did Verified from the live artifact @ `36f717a2`: `config.json:quantization_config` carries `quant_method: "modelopt"`, `quant_algo: "MIXED_PRECISION"` and **no `kv_cache_*` key at all**; only `hf_quant_config.json` declares `kv_cache_quant_algo: "FP8"`. The first round's own file-precedence repair makes the inline document win, so the declaration is never read for this checkpoint — on this engine and on vLLM alike. Nothing is functionally wrong; the mirror is correct on both halves. What was wrong is what the spec, `kv_cache.h`, `cache.h`, `model_loader.cpp` and this body said about it. The campaign consequence, stated plainly: **for #1574 the flag has to be typed explicitly on both sides.** The competitors' own `serve.sh` already passes `--kv-cache-dtype fp8`, so this is consistent rather than a blocker. A new G10 case writes BOTH real documents into one directory and pins the answer at `auto`. ### The modelopt marker: NOT a divergence, and the reason took running upstream The review reported `src/vllm/config/cache.cpp` accepting `producer.name` as a divergence, on the grounds that `get_kv_cache_quant_algo_string` (`torch_utils.py:319`) gates on a top-level `quant_method` and nothing else. That transcription is of one function; the answer needs two. `ModelArchConfigConvertorBase._normalize_quantization_config` (`transformers_utils/model_arch_config_convertor.py:208-247`) runs FIRST, at `ModelConfig.__post_init__` (`config/model.py:577`), and INJECTS `quant_cfg["quant_method"] = "modelopt"` when `producer["name"] == "modelopt"`, mutating the very dict `hf_config.quantization_config` names. MEASURED by extracting both functions with `ast` from the files at `555967922` and running them over the live documents rather than retyping them: `nvidia/Llama-3.3-70B-Instruct-FP8`'s producer-only `hf_quant_config.json` answers `None` before normalization and `'fp8_e4m3'` after it. So accepting the producer name is the MIRROR. The comment that claimed it on the wrong grounds ("`modelopt_mixed_precision.h` already reads both shapes") is replaced by the chain and the measurement. Reading the injector did surface three real differences, and the marker set is now exactly upstream's: - `quantization.quant_method` is no longer accepted. Upstream writes and reads the marker at the TOP level and never looks inside, so a document whose only marker is nested resolved fp8 KV here and nowhere else. - A nested `modelopt_quant_config` key is now accepted (`:218-220`), which upstream treats as the legacy modelopt marker and we ignored. - The producer name is compared raw against the literal (`:222` is `==`), not prefix-matched and case-folded. `quant_method` keeps both, because `:238-246` lower-cases it before `:319` prefix-tests it. Same file, two tests, mirrored separately. G1's new marker case pins all six arms; restoring the nested-marker widening reddens two of them. **One difference remains and is recorded rather than copied.** The injector RAISES `ValueError: Unknown ModelOpt quant algo: <algo>` (`:235`) when the producer is modelopt and the nested `quant_algo` is neither FP8-family nor NVFP4 — which is what the #1574 legacy file (`MIXED_PRECISION`) gets, and we answer `fp8_e4m3` for it instead. That refusal is a WEIGHT-half validation living in a config convertor this port does not have, and moving it into the KV resolver would refuse a checkpoint whose weights `modelopt_mixed_precision.h` loads. It is unreachable for that checkpoint anyway, because its inline document wins. Spec `## Owed` names it. ### The refusal mechanism was wrong for 13 of the 16 unrouted architectures "Every other architecture is refused BY NAME at `vt::ReshapeAndCache`" holds for three. `granite:95`, `minicpm:96`, `phi3:78`, `gemma3:121`, `opt:125`, `stablelm:86`, `glm4:93`, `commandr:93`, `gemma:53`, `gemma2:135`, `phi:98`, `muse_glimmer:144` and `olmo2:94` each refuse at their own `"<arch>: KV cache must be bf16 or f32"` guard first, which names neither fp8 nor the flag. Only `gemma4`, `qwen3_vl` and `nemotron_h_device` reach the store guard. The safety property holds — nothing writes floats into a half-sized page — but the described mechanism did not, and the count was 16 architectures at 17 call sites rather than 17 architectures. **Round 3 traced each of the three and found only `qwen3_vl` reaches the store guard; the split is 14 / 1 / 1. See below.** Corrected in the spec, in `docs/USAGE.md`, in `docs/FEATURES.md` and in the engine-matrix row, and the 13-guard gap is now its own `## Owed` bullet. ### `docs/FEATURES.md` promised a doubled pool on a path that halves it `ResolveNumBlocks` (`model_loader.cpp:1446-1451`) returns the fixed 256-block fallback when neither `--num-blocks` nor `--kv-cache-memory` is given, so on the default path fp8 KV halves the pool BYTES rather than doubling the pool. The one-liner said "halves the block, doubles the pool" unqualified; `docs/USAGE.md`'s example passes `--kv-cache-memory` and was correct, and now says so explicitly. ### Preflight was reporting nothing about two gates `origin/main` had moved to `08c81a89` and was no longer an ancestor, so the diff-scoped `commit-trailers` and `commit-style` checks SKIPPED. `origin/main` is merged up and both execute. ## What the THIRD fresh review found, and what changed The review of `d0bcf80dd` returned **FAIL**, with one blocking finding. Every finding is repaired here, and rounds 1 and 2 are untouched. The round-2 `producer.name` MEDIUM was FALSIFIED by that review, which `ast`-extracted both pinned upstream functions and ran them: `_normalize_quantization_config` mutates `hf_config.quantization_config` in place during `ModelConfig.__post_init__`, before `resolve_kv_cache_dtype_string` reads it, so accepting `producer.name` is the mirror. It stays. ### BLOCKING: the gate could not distinguish a working fp8 KV cache from a corrupted one G12's counter-case asserted `differing > 0` and `max_abs < 1.0` on a delta its own `MESSAGE` reported as `0.000339303`. The bound sat about **3000x above the signal**, so it measured this toy model's insensitivity to its KV cache and not the cache. Two mutations of `kv_cache_route.h:63` walked through the whole 30/30 suite: | Mutation | Delta on the old axis | Old suite | |---|---|---| | **N1_KVSWAP** — pass `(q, v, k, k_cache, v_cache, ...)`, so the fp8 arm stores V into `k_cache` and K into `v_cache` | `0.0247495` (73x) | 30/30 SUCCESS | | **N4_STORE_SCALE8** — store with `kv.k_scale * 8.0F` / `kv.v_scale * 8.0F` while the read uses the unscaled values | `0.0063844` (19x) | 30/30 SUCCESS | The second one is exactly the defect the `k_scale`/`v_scale` plumbing exists to prevent. Nothing above G12 closed the hole either: G5 and G9, the two production-entry cases, assert token counts and determinism across two fresh stacks, and neither compares an fp8 run against a bf16-cache run. **The repair compares the CACHE BYTES, inside an envelope read off the FORMAT.** A new G12 case runs the same prompt twice — once with a bf16 page, once with an fp8 page — and checks every layer-0 element of both K and V: ``` |dequant(store(ref)) - ref| <= 2^-4 * |ref| + 2^-10 * scale ``` e4m3fn carries three explicit mantissa bits and rounds to nearest even (`include/vt/fp8_kv.h`), so a normal magnitude is within a half ulp — at most `2^-4` of its own value — and anything below the smallest normal `2^-6` sits on a uniform `2^-9` grid whose half step is `2^-10`, in absolute terms scaled by the per-tensor scale. Not one constant is fitted to a measurement, which is the point: a threshold sized to today's `3.4e-4` is the same hole one decimal place tighter. **Layer 0 is the whole population, and that is the design rather than a limitation.** Its K and V are functions of the embedding and the input layernorm alone, so the bf16 run and the fp8 run hand the store bit-identical floats and the float run's page IS the reference the fp8 page has to round. From layer 1 on the fp8 run's inputs already carry the previous layer's dequantization and no per-element envelope holds. The scales are `0.125` for K and `0.25` for V — non-unit and unequal, so a dropped, swapped or one-sided scale IN THE STORE leaves the envelope rather than staying inside it. Both are below one because the store divides by the scale and this model's layer-0 K and V land around `1e-2`; a scale above one pushes most elements under `2^-6`, where only the weaker absolute arm applies. Two anti-vacuity `REQUIRE`s hold the population to that: every one of the ten pages (five tokens, K and V) must carry at least one NORMAL, and no reference may reach e4m3's finite maximum, so the bound is the pure rounding envelope and never a clamp. MEASURED on the repaired tree, and both mutations are RED against it: | Tree | Elements outside the envelope | Worst ratio to the bound | |---|---|---| | repaired | **0 / 320** (281/320 normals over 10/10 pages) | 0 | | N1_KVSWAP | **316 / 320** | **416.9** | | N4_STORE_SCALE8 | **319 / 320** | **13.9** | A bf16-versus-fp8 comparison in G5/G9 through `LoadedEngine` was considered and is NOT what closes this. Both STORE mutations left every other case green, G5 and G9 included, which is the direct measurement that an engine-level token or determinism comparison cannot see a dequant defect this model absorbs. The old two assertions are kept beside the new case, not replaced. ### Round 4: that envelope gated the STORE, and the READ was still open The fourth review found the hole the third one's repair left. The envelope case decodes the cache bytes with its own `vt::LoadKvFp8E4M3` and never enters the production dequant (`cpu_paged_attn.cpp:167`); the one value in the case that IS downstream of that dequant carried no assertion at all. Every other case in the file that asserts a number downstream of the read runs at `k_scale == v_scale == 1`, where a k/v scale SWAP on the read is arithmetically inert. Mutating the production read to `const float v_scale = args.k_scale;` therefore left `test_kv_cache_fp8_wiring` at 31/31 and `test_ops_fp8_kv_cache` at 8/8, both SUCCESS, while every V the softmax saw was halved. **The read's SCALE is closed by INVARIANCE, and EXACTLY rather than by a tolerance** — its ROUTING is not, and that half is under `## Owed`. e4m3fn's normal grid is relative — for `|y|` in `[2^e, 2^(e+1))` the representable points are `m * 2^(e-3)` — and dividing by a power of two is exact in binary floating point, shifting `e` without touching the mantissa. So for any two power-of-two scales that both leave a value normal and unsaturated, ``` s * Dequant(Quantize(x / s)) == s' * Dequant(Quantize(x / s')) ``` bit for bit: the two runs' cache BYTES differ in every element's exponent field, and the floats the attention kernel is handed do not. The case runs the seam twice more, at `(2^-7, 2^-13)` and `(2^-11, 2^-9)`, and requires `memcmp`-level agreement on the logits. There is no constant to fit and none to widen later, which is the same discipline the envelope above is written to. Both pairs move BOTH sides on purpose: a pair that moved only one would let a read-side defect that depends on the other scale reproduce itself identically in both runs and cancel out. **That exactness is ASSERTED for LAYER 0, and the logit equality is an EMPIRICAL result for this fixture rather than a theorem.** The `scale_exact` `REQUIRE` decodes `bf16.buf[0]` and holds `320/320` LAYER-0 elements normal and unsaturated at all four scales — the measured magnitudes are `1.76e-4` to `1.32e-1` for K and `5.41e-5` to `4.22e-2` for V, against all-normal windows of `(2.94e-4, 1.13e-2]` and `(9.43e-5, 3.46e-3]` — because one subnormal would round on the ABSOLUTE `2^-9` grid, which is not scale invariant. `MakeSeamConfig` sets `num_hidden_layers = 2`, so that is 320 of the 640 elements each run stores, while the logits the case compares are a function of BOTH layers' caches. Layer 1 does NOT satisfy the precondition: decoding each run's own `buf[1]` at its own scales, `3/320` of its elements disagree by up to `7.62939e-06`, all K-side and an order of magnitude below layer 0's `1.76e-4` minimum, so at `kInvAKScale = 2^-7` they land in e4m3's SUBNORMAL region and that layer carries 2 SATURATED elements besides. `CHECK(inv_differing == 0)` therefore holds by the format property for layer 0 and by ABSORPTION for layer 1 — a `7.6e-6` cache perturbation vanishing in f32 accumulation before it reaches a logit. Extending `scale_exact` over every layer was the preferred repair and this fixture cannot satisfy it, so the claim is narrowed here and the fragility is recorded under `## Owed`. The second anti-vacuity `REQUIRE` is that the two caches really do hold different bytes, so the comparison is a statement about the read rather than about two identical buffers. MEASURED on the repaired tree: `0/320` logits differ, max `|delta|` exactly `0`. Three read-side mutations of `cpu_paged_attn.cpp:167` are RED, and the store envelope above reads `0/320` under all three: | Read mutation | Logits differing | max delta | |---|---|---| | `v_scale = args.k_scale` | **320 / 320** | `0.0673` | | `k_scale = args.v_scale` | **256 / 320** | `1.08e-4` | | `v_scale = 1.0F` | **320 / 320** | `2.47e-3` | The K-side one is why this is stated as exact equality rather than a bound. The three signals span **621x**, so a bound sized against the largest keeps nothing for the smallest: even a tight 12.4x margin against `0.0673` puts the constant at `5.4e-3`, which admits `1.08e-4` by **50x**. An absolute `1e-4` would in fact have CAUGHT the K-side one, by 8.4%, and a relative `1e-4` against the fixture's largest logit (`0.0627671`) is `6.3e-6` absolute and catches it by 17x — the argument is the span, and the refusal to fit a constant to whichever defect happened to be measured first, never any one number. **What this comparison still cannot see** is a read-side defect that is a function of BYTES and INDICES rather than of scales, because both fp8 runs commit it identically and it cancels. Measured on production code, both of these PASS the whole file at `31/31`, `487/487`, with `0/320 logits differ`: serving V out of the K page with K's scale (`cpu_paged_attn.cpp:174` `v_base = k_cache.data` with `:167` `v_scale = args.k_scale`), the read-side twin of the `N1_KVSWAP` store mutation the envelope DOES catch; and dropping the in-page token offset from the V read (`:270`, `off & 0`), which is pure indexing and scale-free. Closing that class needs a comparison against a REFERENCE — the bf16 run's page, or the case's own decode — and never a second fp8 run. It is under `## Owed`. The same review found `scripts/agent-integration.py --base origin/main` exiting 1 with `commit-trailers` and `commit-style` both SKIPPED: `origin/main` had moved to `c98ffd4d0` while this branch's last merge took `ff8f72807`, so neither gate reported anything about this tree. `main` is merged up and both execute again. ### Round 5: the code was correct, and two sentences about it were not The fifth review passed the change and rejected two claims written around it. It did not ask for a production edit and none was made; the two paragraphs above are where the repair lands, and both numbers in them were re-measured here rather than carried over. `origin/main` had moved again, to `ea9b7e30e`, so it is merged up once more and `commit-trailers` and `commit-style` execute against this tree rather than SKIP on a base that has moved. The first was the tolerance sentence. "A tolerance sized at `1e-4` would have let it through" is false as stated and false on either reading: `1.08408e-4` is 8.4% ABOVE an absolute `1e-4`, and a relative `1e-4` against the fixture's largest logit `0.0627671` is `6.3e-6` and catches it by 17x. The argument that survives is the 621x SPAN between the three read signals, not that constant. The second was the exactness scope. The comment said the invariance gate is exact "as a property of the format rather than a lucky measurement" and that "the `scale_exact` REQUIRE below holds the population to it". `scale_exact` decodes `bf16.buf[0]`, which is 320 of the 640 elements each run stores, and the logits it compares depend on both layers. Extending the assertion over every layer was the preferred repair and the fixture cannot carry it: layer 1 measures `3/320` elements disagreeing by up to `7.62939e-06`, subnormal at `2^-7`. So the prose is narrowed to what is asserted, the empirical part is named as empirical, and the fragility and the unclosed routing half are both under `## Owed`. ### The `## Owed` refusal accounting was wrong for 2 of the 3 architectures it named Round 2 said `gemma4`, `qwen3_vl` and `nemotron_h_device` reach the store guard. Traced with a `kI8` page against a bf16 model dtype, only `qwen3_vl` does. The true split is **14 / 1 / 1**: - `gemma4.cpp:306-315` takes `kv.dtype != adt`, allocates `DBuf kcast(d, kv.dtype /* kI8 */, ...)` and calls `vt::CastF32`, which refuses at `src/vt/ops.cpp:4087` with `"cast_f32: out must be f32"`. It never reaches the store, and that message names neither fp8, nor the flag, nor `gemma4` — so it is the WORST of the sixteen, not one of the three best. - `nemotron_h_device.cpp:1589-1593` carries an explicit `else { VT_CHECK(false, "NemotronH paged forward: ... The fp8 KV scheme ... is not selected here"); }` on the same cast, which fires first. It is the ONE refusal in the sixteen that names fp8 to the operator. - `qwen3_vl.cpp:198-200` carries no guard and no cast, so `vt::ReshapeAndCache` is what refuses it. Safety is unaffected — nothing writes floats into a half-sized page on any of the sixteen — and what moves is message quality: the store guard's better message is reached by 1 of 16, not 3. Corrected in the spec's narrative and in its `## Owed` bullet, which now also records that **G7 hand-builds its K/V tensors** and calls `vt::ReshapeAndCache` directly, so it gates the store guard's MESSAGE and never the claim that any particular architecture reaches it. ### The marker mirror accepted three shapes upstream answers `None` for `_normalize_quantization_config` injects `quant_method` only `if quant_algo is not None` (`:224`), and it reads that key out of `quant_cfg.get("quantization", {})` — an EMPTY-object fallback, unlike the reader's `quant_cfg.get("quantization", quant_cfg)` at `torch_utils.py:321`. `cache.cpp` accepted the producer name and the legacy key unconditionally, and read the legacy key out of the reader's `inner`. Three documents therefore resolved to `fp8_e4m3` here while upstream resolved `None`: | Document | Upstream | Before | Now | |---|---|---|---| | `producer.name == "modelopt"`, `quantization` with no `quant_algo` | `None` | `fp8_e4m3` | `auto` | | `quantization.modelopt_quant_config` with no `quant_algo` | `None` | `fp8_e4m3` | `auto` | | top-level `modelopt_quant_config`, no `quantization` key at all | `None` | `fp8_e4m3` | `auto` | Each of the three would halve the KV page on a checkpoint vLLM runs at the model dtype. Arm **(g)** of G1's marker case pins all three, alongside the same documents made acceptable by adding the `quant_algo` upstream requires, so what refuses them is the guard and not the shape. **No shipped fixture moved**: every real document in the suite carries a `quant_algo`. The **MIXED_PRECISION raise stays under `## Owed`**, and that is why the mirrored test is exactly `quant_algo is not None` rather than the FP8/NVFP4 family set: the only two upstream outcomes for a `quant_algo` that IS present are "inject" and "raise", and taking the raise's arm collapses them into one. Importing the raise would refuse a checkpoint whose weights this tree loads. ### This PR falsified four statements elsewhere and now carries the repair Before this branch nothing in production read `hf_quant_config.json`. `src/vllm/config/cache.cpp:207` now does, from `LoadedEngine::FromModelDir` (`model_loader.cpp:1988`). Four statements written against the old world read false at this head, and a record edit rides in the pull request whose change made it stale: | Site | Was | Now | |---|---|---| | `modelopt_mixed_precision.h:1006` | "a file no production path in this tree reads at all" | reads it, but only as the LEGACY FALLBACK behind `config.json`'s inline document, and this checkpoint ships that document | | `.agents/specs/qwen38-27b-quant-arms.md:1629` | "the declaration is invisible to the loader rather than ignored by it" | the same conclusion, reached through the precedence rather than through absence | | `.agents/specs/qwen38-27b-quant-arms.md:1841` | "No production path in this tree reads `hf_quant_config.json`" | names the reader, the fallback order, and that BOTH artifacts carry the inline document | | `test_qwen38_27b_modelopt_mtp_arm.cpp:482` (case NAME) and `:487-488` | "where NO production path reads it" / "the ONLY document any production path in this tree reads" | "in the file this artifact's `config.json` outranks", with the precedence spelled out | **No assertion changed.** They stay valid because `config.json`'s inline `quantization_config` wins on the one path that reads either. ### The 14 / 1 / 1 split had to reach four more records The recount above landed in the spec and stopped there. `docs/USAGE.md`, `docs/FEATURES.md`, the `KV-FP8` engine-matrix row and the `QUANT-KV-FP8` quantization-matrix row — all four written by THIS pull request — still told a reader that Gemma-4, Qwen3-VL and Nemotron-H are the three that name the flag. None of them names the flag. All four now describe what an operator actually sees, and the two matrix rows take the new gate count (31 cases, G1-G12) with one line on what G12's third case bounds; the quantization-matrix row was still saying 26 cases, G1-G11, and 17 unrouted architectures. ### A GCC 13 -O2 build of this branch did not compile `KvAlgoFromObject`'s `const auto num_bits = cond ? std::optional<int>(...) : std::nullopt;` fails `-Werror=maybe-uninitialized` at `-O2` on GCC 13.3, which is every `Release` and `RelWithDebInfo` build of this tree. CI configures with no `CMAKE_BUILD_TYPE` and never saw it; `build-newest-gcc` uses `Release` on a newer compiler that does not emit it. Written as a statement instead, and the translation unit now compiles clean under `-O2 -g -DNDEBUG -Wall -Wextra -Werror`. ## Gates `tests/vllm/entrypoints/test_kv_cache_fp8_wiring.cpp` — **31 cases / 481 assertions GREEN**, and `tests/vllm/entrypoints/openai/test_serve_kv_cache_dtype.cpp` — **3 cases / 26 assertions GREEN**, plus `tests/vllm/models/test_qwen38_27b_modelopt_mtp_arm.cpp` — **22 cases / 1687 assertions GREEN**, whose case name this round narrowed. CPU-only, `-Wall -Wextra -Werror`, zero warnings over a full build. The `src/vllm/config/cache.cpp` translation unit is compiled a second time at `-O2 -g -DNDEBUG` for the GCC 13 `-Werror=maybe-uninitialized` repair above, because CI configures with no `CMAKE_BUILD_TYPE` and cannot see it. `ctest` **584/584 PASSED, 0 failed** on the pushed head, on an idle box, over the whole tree built from scratch with `-Wall -Wextra -Werror` and zero warnings. `test_runner` PASSES here in 36.9 s, so the CI red ([#1602](#1602)) is not reproducible on this host; it is still counted as the pre-existing red rather than as fixed, because nothing in this branch touched it. Four suites are Skipped for absent checkpoints or an absent config: `test_modelopt_mixed_precision_checkpoint`, `test_voxtral_e2e`, `test_cli_offload_config`, `test_qwen35_paged_engine`. The count is 584 rather than CI's 593 because this configure sets `-DVLLM_CPP_BUILD_EXAMPLES=OFF`. The round-2 measurement, kept for the record: `ctest` **591/593 after the second round**, on a CONTENDED box (1-minute load average 62 to 100 through the run, other sessions building in their own worktrees). `test_runner` is the pre-existing red ([#1602](#1602)) and fails on an attention-backend selection message under a non-multiple-of-16 block size, which this change does not touch; it fails the same way on a serial re-run (20 cases, 543/544 assertions), so it is a code red that predates this branch and not a load artefact. `test_engine_core_proc` failed under `ctest -j 4` and **PASSES on a serial re-run** (14 cases / 118 assertions, `Status: SUCCESS!`), so it is counted as green rather than as a regression. Three suites are Skipped for absent checkpoints: `test_modelopt_mixed_precision_checkpoint`, `test_voxtral_e2e`, `test_qwen35_paged_engine`. `scripts/agent-preflight.sh --staged` is green except `test_cpu_x86_llamacpp_floor` ([#618](#618)), which fails on one run and passes on the next depending on the box's load: its harness leg is DISCARDED and RETRIED under contention and the case asserts `returncode == 2` against the 4 the retry returns. Nothing here was repaired. **CI READ on this head** (`b7921ceb0`, run [32627552373](https://github.com/mudler/vllm.cpp/actions/runs/32627552373), CONCLUDED): **17 pass, 6 skipping, 3 fail**, and all three are pre-existing and filed. `build-test-cpu` PASSES here, 593/593. `test_runner` ([#1602](#1602)) is green in that lane and in both sanitize lanes on this head, and green locally in 36.9 s. On the previous head it was the ONLY failing test in all three of `build-test-cpu`, `sanitize-cpu (address,undefined)` and `sanitize-cpu (thread)` — one flake reported as three job failures, not three defects. `cuda-fat-build`, `build-newest-gcc`, `build-test-cpu-arm64`, `build-test-vulkan`, both `verify` lanes, `commit-protocol-tag`, `documentation-checkpoint`, `pr-size`, `device-leakage`, `last-gated-commit`, `cuda-arch-features`, `vulkan-spirv-freshness`, `plan` and `drift-check` all pass. The three reds: | Job | What it reports | Whose | |---|---|---| | `windows-msvc-cpu` | `test_openai_api_server.exe exited with status -1073740791` (`0xC0000409`, `STATUS_STACK_BUFFER_OVERRUN`) | [#584](#584), open, reproduced unchanged on PR #1744 | | `windows-msvc-vulkan` | the same executable, the same status | [#584](#584) | | `agent-record` | `FileNotFoundError: [Errno 2] No such file or directory: 'hugo'` at `tests/scripts/test_check_site.py:130` | [#1722](#1722), [#1754](#1754), [#1764](#1764); PR #1726 is the fix in flight | **The `windows-msvc-*` signature MOVED, and that settles what the catch-up merge bought.** Both jobs were failing at step 3, *"Prove PowerShell, static CRT, and unsupported-tier contracts"* — [#1649](#1649), whose fix `2f2a70925` was NOT reachable from this branch's previous base. On this head step 3 PASSES in both, and what fails is step 4, on the pre-existing #584 crash. #1649 is closed for this branch; #584 is not this branch's and is not repaired here. `agent-record`'s case was added by #1714 (`1db7e59cf`, on `main`) and shells out to a binary the job never installs (`grep hugo .github/workflows/ci.yml` finds nothing). It passes here 7/7, because this host has `hugo`. Red on `main` and on every pull request; not repaired here. **`commit-trailers` and `commit-style` now RUN.** `origin/main` had moved to `08c81a89` and was no longer an ancestor, so both were SKIPPED and reported nothing at all about this tree. After the catch-up merge preflight prints `Commit trailers vs origin/main 08c81a8:` with both green, alongside `now-current range` and `issue-index append-only`. **Twelve mutations from the first round remain valid, eight more were run for the second, two for the third, and three more for this round**, each applied with its sha256 and applied diff printed, built with its exit status printed, run red, and restored byte-identically: | Mutation | Gate that went red | |---|---| | M13 the fp8 route stops normalising K/V to one dtype | G9 (the production arm) | | M14 the `--kv-cache-dtype` parse arm is dropped | serve CASE 2 and CASE 3 | | M15 the parsed flag never reaches `EngineParams` | serve CASE 2 and CASE 3 | | M16 the loader never reads the checkpoint's declaration | G10 | | M17 the legacy file outranks the current one again | G10 (precedence, both halves) | | M18 the per-layer specs are never retyped | G11 (9 assertions) | | M19 the chain refusal is deleted | G10 (ordering) | | M19b the chain refusal MOVES below the stanza | G10 (ordering) — the mutation M19 cannot make | | MSEAM the shared seam's fp8 routing is reverted whole (`cast_dt` back to `kv.dtype`, `WriteKvCache` -> `vt::ReshapeAndCache`, `ApplyKvCacheQuant` deleted) | G12, both cases — the mutation that stayed GREEN before this round | | MMARKER the nested `quantization.quant_method` widening is restored | G1's marker case, arms (c) and (d) | | N1_KVSWAP the fp8 store is handed `(q, v, k, ...)`, so V lands in `k_cache` and K in `v_cache` | G12's envelope case: 316/320 elements outside, worst ratio 416.9 — GREEN 30/30 before this round | | N4_STORE_SCALE8 the fp8 store uses `k_scale * 8` / `v_scale * 8` against an unscaled read | G12's envelope case: 319/320 outside, worst ratio 13.9 — GREEN 30/30 before this round | | N5_READ_VSWAP the production read dequantizes V with K's scale (`cpu_paged_attn.cpp:167`) | G12's envelope case, the new invariance half: 320/320 logits differ at max delta 0.0673 — GREEN 31/31 AND `test_ops_fp8_kv_cache` 8/8 before this round | | N6_READ_KSWAP the production read dequantizes K with V's scale | same: 256/320 logits differ at max delta 1.08e-4 | | N7_READ_VDROP the production read drops the V scale (`v_scale = 1.0F`) | same: 320/320 logits differ at max delta 2.47e-3 | | N_LOW1 the marker predicate is restored to its pre-repair form (unguarded producer/legacy key, legacy key read out of `inner`) | G1's marker case, arm (g): 7 assertions red across all three shapes | The driver prints the applied diff and the build exit status for every one, because `git diff --stat` is EMPTY for a file git does not track yet, and a mutation that never applied — or that failed to build — reads as a passing test. ## Unreached and owed All under #1593 and listed in the spec's `## Owed`: - **The C ABI carries no `kv_cache_dtype` field.** A C-ABI caller reaches an fp8 KV cache only through a checkpoint that declares one, which the loader does honour on that path. - **An fp8 KV cache is served by the SLOW attention kernels only** (the section above). Owned by `KV-FP8` under #1593. - **The heterogeneous per-layer fp8 arm is SIZING-only.** G11 gates the arithmetic; the only architecture that populates `per_layer_attn_specs` is Gemma-4, which is one of the 16 that refuse the store, so a Gemma-4 run gets a correctly halved pool and then a named refusal — the intended order. - **16 architectures refuse rather than route, at 17 call sites:** `glm4`, `minicpm`, `opt`, `gemma`, `gemma2`, `gemma3`, `gemma4` (two sites), `commandr`, `phi`, `phi3`, `muse_glimmer`, `stablelm`, `qwen3_vl`, `olmo2`, `granite` and `nemotron_h_device`. - **15 of those 16 refuse with a message naming neither fp8 nor the flag.** Thirteen have their own `"<arch>: KV cache must be bf16 or f32"` guard; `gemma4` dies earlier still, inside `vt::CastF32`, with a message that names no architecture at all. Only `qwen3_vl` reaches the store guard, and only `nemotron_h_device` names fp8 to the operator. Widening those 15 is the same edit that routes them, so it is owed with the bullet above. - **The resolver accepts one document upstream REFUSES**, where the nested `quant_algo` IS present and is neither FP8-family nor NVFP4, and upstream's config convertor raises instead of injecting. The `quant_algo is not None` guard itself is now mirrored (round 3 above), so this is the one remaining arm. Recorded, argued and unreachable for the #1574 subject; porting the convertor's weight-half validation is its own row. - **No engine auto-selects fp8 KV for the #1574 subject,** so the campaign types the flag on both sides and the checkpoint-declaration path this row builds is gated by G1 and G10 rather than exercised by the benchmark it was built for. - **No weight loader extracts `k_scale`/`v_scale`,** so the two checkpoint-loaded arms of the resolver are unit-gated and unreached, and the scale is per-engine rather than per-layer. - **`--calculate-kv-scales` is refused, not implemented** (upstream deprecates it for v0.19). - **Every CUDA case in this feature is still UNEXECUTED,** for the reason W2 recorded: no `nvcc` and no device in this session, and a CPU build compiles zero `.cu` translation units. - **G12's read-side exactness is ASSERTED for LAYER 0 only** (`scale_exact` decodes `bf16.buf[0]`, 320 of the 640 elements each run stores), and layer 1 carries `3/320` elements that dequantize `7.62939e-06` apart between the two invariance runs, in e4m3's subnormal region where the covariance argument does not hold. The logit equality is empirical there, absorbed below f32, and a change to the fixture, the layer count, the thread count or the accumulation order could redden a CORRECT tree with a defect-shaped message. - **The read comparison closes the SCALE half of the read and not the ROUTING half.** Two fp8 runs cannot see a defect that is a function of bytes and indices; a V-from-the-K-page read and a V read that drops the in-page offset both PASS the whole file. Closing that class needs a REFERENCE comparison. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
.agents/issue-index.mdis append-only and carriesmerge=union, butcheck-agent-record.pykeyed uniqueness on the issue NUMBER alone. In anappend-only log an update IS an append, so that key permitted one statement per
issue for the life of the repository, and it red
mainwhenever one lane filedan issue and another lane fixed it. This keys uniqueness on the PAIR (issue
number, owning row) instead: a hand-off between two rows is a record, an
identical append is still a duplicate.
The rule was a leftover. It arrived in
8dd6508dawhen the intake surface wasan editable keyed table inside
.agents/roadmap_v1.md, where one row per issuewas the whole design.
51e0cb5b1moved the table here, made it append-only andgave it
merge=union; uniqueness by number survived that move unexamined.This no longer cures a red, and says so
6354755ba(#1742) landed the other repair first: it deleted theENG-HF-MODEL-DOWNLOADrow for issue 1649 as an argued append-only exception.mainis green oncheck-agent-record.pyby its own deletion, verified byrunning main's checker against main's index. Nothing here reverts that. The
merge takes main's index wholesale and the deleted row stays gone.
What this carries is the class rather than the instance. A deletion is a
per-instance cost that recurs every time two lanes touch one issue; the narrowed
key makes the next hand-off an ordinary append. Issue 1731, which asked for the
repair, is already CLOSED by
6354755ba, so this deliberately claims noclosure. Whether the preventive half is still wanted now that the red is gone
is the operator's call, and it is the one open question on this PR.
The argument is not hypothetical, because this branch's own index is the first
live case. After the merge, issue 1731 appears at
:644underENG-RECORD-CONFLICT-SURFACES, the row that filed it, and at:646underGATE-ISSUE-INDEX-OWNER-KEY, the row that fixed it, with zero duplicate(number, owner)pairs. Main's checker refuses this branch's index withissue #1731 listed twice; this branch's checker passes it. The hand-off the old keycould not express is the one this change had to make about itself.
What the refusal still catches
The corruption it has actually caught in the field is unaffected. Issue 1619
measured the
merge=uniondriver duplicating a row BYTE-IDENTICALLY at twolines, which
git merge-treecalled clean andcheck-issue-index-append-only.pypassed because a duplicate is an addition. A copied LINE carries its owner with
it, so the pair still collides. Given up deliberately: a MISTYPED owner on a
second row, which no reading of the file can separate from a legitimate
hand-off. Rejected deliberately: a cap on rows per issue, which is the
"limit an entry, not a shared file" anti-pattern at the scale of one issue, and
an owner-ID existence check, which would red
mainon day one because unplacedgate rows are legitimate owners.
Red-first
Against
origin/main's UNMODIFIED checker, on one fixture, both checkers drivenside by side:
The branch's suite against main's checker is the red-first tree: 113 tests, 5
failures — the three cases added here, the real index, and the preamble
consistency case, which is red there by construction because the index carries
the new preamble paragraph and main's
INDEX_PREAMBLEdoes not.Mutation
scripts/check-agent-record.pyis7abe4aa4b3a8b0d776364207396be146e58dd9d34067b9f258c106a96e12d593before andafter every mutation. Each hash below is of the exact bytes named beside it, so
a reader can reproduce it with one substitution.
first = seen.get(key)→first = Nonecf8eea17...key = (number, row_id)→key = (number, None)a14213fc...fcddaa08...Disjoint sets are the property worth having: one proves the refusal still
refuses, the other proves the narrowing is what admits the hand-off, and neither
alone shows both.
Two instrument failures are recorded because each would have read as a pass. The
import check reported
AttributeError: 'NoneType' object has no attribute '__dict__'on the PRISTINE file, because@dataclassresolves__module__through
sys.modulesand the module was not registered there; every mutantwould have been reported as "does not build". It is now validated in both
directions before use. Separately the harness was killed by a timeout part-way
through the third mutation and left the tree mutated; the restoration is proved
by hash, not by the harness having finished.
The spec's two previously recorded mutation hashes could NOT be reproduced from
the mutation text printed beside them, and one gates row labelled a mixed-tree
number as an
origin/mainnumber. Both are corrected here rather thanre-quoted, and the gates table now names which of three trees each number came
from.
Records corrected in flight
Issue 1733 duplicates 1731 by 17 minutes and 22 seconds and was closed
NOT_PLANNED. Its bold claim that the repair "is NOT blocked" is false, and thereason is an instrument artifact re-measured here rather than relayed:
scripts/check-issue-index-append-only.py:50-51diffs a COMMIT range, so anuncommitted deletion is invisible to it. The same deleted bytes give rc 0
uncommitted and rc 1 committed. Credit for both findings goes to the
row/FIX-ISSUE-INDEX-1649-DUPlane.Repaired after the fresh review
The review PASSED the design. 16 constructed shapes confirmed the issue 1619
byte-identical-duplicate case still reds and the two predicate mutations still
red disjoint sets, so nothing below re-litigates the key. Three findings are
repaired instead.
The row was in breach of the rule it is about.
AGENTS.mdrequires theissue in three places that agree, and this row had it in none. The spec named
issue 1731, which #1742 CLOSED; the index carried no row for the open issue; and
this body never mentioned it. Issue 1745 was filed against this branch by name
and is now in all three. Issue 1731 stays named through the spec as the first
instance and as the red that no longer stands.
A guard was asserted in prose that does not exist for most of the index. §4
of the spec and the docstring of
test_a_dashed_row_and_an_owned_row_are_not_a_duplicateboth said the oppositeorder "is already gated: an added dashed row raises the unowned count and reds
the
UNOWNED_HIGH_WATERratchet". It does not, whenever any spec lists theissue under
## Owed, becausecheck_issue_indexcounts a dashed row onlyif row_id is None and number not in owed. Re-measured here rather thanrelayed, on this tree: 328 owed numbers, and 226 of 621 rows carry BOTH an owner
and an owed issue. Probing the real index with a dashed row for issue 168 leaves
this branch's checker silent at rc 0, with the unowned count still 33 against a
high water of 33, while main's checker reds it under the OLD number-only key and
never the ratchet. The contrast row, issue 618, which no spec owes, reds this
branch's checker with
34 rows name no owner, above the recorded 33. The indexwas restored byte-for-byte, sha256
4e80b8ba...before and after. The exemptionitself is
AGENTS.mdworking as written, so both sentences are corrected andthe missing ORDERING rule is filed as issue 1749 and listed under
## Owed:building it here would be a second checker semantic in a change that owes one.
Nothing proves the CLI runs
check_issue_index. Its sole production callsite is unguarded inside
main(). Replacing it withpass(checker sha2567abe4aa4...toe98c80ad..., import-checked first so a mutant that fails toBUILD cannot read as a pass) leaves
tests/scripts/test_agent_record.pyat 113tests OK, unmoved, and makes the command accept a byte-identical duplicate row
at rc 0 — the exact corruption issue 1619 measured, which the pristine tree
refuses at rc 1. Inherited, not introduced:
git log -S'check_issue_index(errors)'on that file returns one commit,
51e0cb5b1, and this change touches only thebody of the function. Filed as issue 1750 and owed, not repaired here.
The docstring repair moved two symbols other records cite. The record-anchor
ratchet caught it in the same run with
RECORD ANCHOR REGRESSION in bucket 'stale': 33 > baseline 31: the correction added six lines aboveRecordAnchorRatchet, andENG-RECORD-ANCHOR-RATCHETcited the old lines. Thetwo citations advance to where the symbols now are and the baseline is
untouched. The three appended index rows carry no
path:linecitation at all,deliberately: the index cannot be edited, so an anchor written into it can never
be repaired when the cited file moves, which is exactly what the two issue 1649
anchors cost this row. They name symbols, and the line numbers live in the spec.
Gates
Every number re-measured at the repair head, after the last edit rather than at
the parent.
check-agent-record.pyrc 0,agent record OK: ENGINE=170 MODEL=377 QUANT=84 KERNEL=57 BACKEND=85 ANCHOR-ROT=37;check-issue-index-append-only.pyrc 0;
tests/scripts/test_agent_record.py113 tests OK;tests.scripts.test_agent_record.IssueIntakeTable11 tests OK;scripts/agent-preflight.sh --stagedrc 0 with 96 ok, 0 skipped, 0 failed.UNOWNED_HIGH_WATERneeds no adjustment: 33 before and after, 33 actual, delta0, because the #1745 row names an owner and both dashed rows are owed by the
spec in the same commit. Owed numbers 328 before, 330 after, and the two added
are 1749 and 1750 and nothing else.
git merge-base --is-ancestor origin/main HEADexits 0 atBASE_SHA=6354755ba570848c9f8f1e1fb47d732833346c26withRANGE_COUNT=8, so the trailer green is not the vacuousTRAILER_BEHINDarm.Index rows: main 620, this branch 624, counted on both sides because the union
driver has silently dropped a tail row on a merge git called clean, and
git diff origin/main --numstat -- .agents/issue-index.mdis11 0, additionsonly.
Closes #1745
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]