feat(MODEL-DSV4-EXL3): the real DeepSeek-V4 DSA geometry loads, and the forward refuses it by name (#1970)

.agents/specs/model-dsv4-exl3.md

@@ -888,9 +888,14 @@ compressor width `deepseek_v4.cpp` already documents at the `dsa_dense` comment
 
 So the loader REFUSES BY NAME on those three shapes and names the residual. It
 does not improvise, and it does not widen the host slot to a shape the forward
-would then mis-index: `Gemm`'s host arm is a `MatVec` with no length check, so a
-`[2*hd, H]` buffer read as `[hd, H]` is a silently wrong number, which is the
-`.agents/verification.md` failure this project exists to avoid.
+would then mis-index. WITHDRAWN AS WRITTEN by #1970, and recorded here because
+this sentence is where the claim started: it said `Gemm`'s host arm is a `MatVec`
+with no length check, so a `[2*hd, H]` buffer read as `[hd, H]` is a silently
+wrong number. `deepseek_v4.cpp:413` is an unconditional `VT_CHECK` and `Gemm`'s
+keep-quant arm checks the shape too, so what that produces is an ANONYMOUS
+`vt: MatVec weight size mismatch` naming no tensor and no layer. Refusing by name
+buys a DIAGNOSTIC over that, which is still the `.agents/verification.md` concern,
+and it is not the difference between wrong tokens and a refusal.
 
 **The obvious fix is wrong and the reason is worth recording.** The GGUF arm
 dodges the same geometry by setting `dsa_dense = (be.gguf != nullptr)` and
@@ -1928,6 +1933,30 @@ which is precisely how this landed green locally in the first place.
   entry is for; it is deliberately NOT attached to
   [#1923](https://github.com/mudler/vllm.cpp/issues/1923), because that issue is
   the loader defect and W1c closes it.
+- **The real artifact's DSA geometry now LOADS, and the forward REFUSES on it —
+  [#1970](https://github.com/mudler/vllm.cpp/issues/1970), option C of
+  [#1961](https://github.com/mudler/vllm.cpp/issues/1961).** This SUPERSEDES the
+  dense-MLA-policy entry above: the answer is not a shared dense-MLA selector,
+  because dense MLA is not upstream's attention on a `cr > 0` layer at any
+  sequence length ([#1964](https://github.com/mudler/vllm.cpp/issues/1964)), so
+  routing the EXL3 arm there would have been a wrong-but-plausible path rather
+  than a policy. The loader now derives every DSA width as upstream does
+  (`coff = 1 + (compress_ratio == 4)`, `vllm/models/deepseek_v4/compressor.py:247-248`)
+  and `AttentionBlock` refuses BY NAME when a materialized width is not the one
+  its arithmetic indexes. What stays OWED is the DSA composition itself — the
+  `coff`-overlapped window with `head_offset` role selection, boundary-only
+  emission, a compressed KV cache beside a SWA(128) raw cache, the indexer on
+  `qr` over compressed rows, one joint softmax over the union. **No row owns that
+  port**; `MODEL-DSV4-EXL3` carries it here until one does, and it needs the
+  cache topology [#1960](https://github.com/mudler/vllm.cpp/issues/1960) and
+  [#1925](https://github.com/mudler/vllm.cpp/issues/1925) are scoping. Also owed
+  and NOT closed by #1970: the GGUF arm's `dsa_dense` still runs the same wrong
+  attention on 41 of 43 real layers (#1964, excluded from #1970's scope), the
+  `cr == 128` EXL3 layers pass the width check while their `win = 2` pooling is
+  still not upstream's 128-wide boundary-emitted compressor, and the
+  `indexer.wq_b` input-space defect (`x` where upstream uses `qr`) is real at any
+  geometry. Design, anchors and mutations in
+  [`specs/dsv4-dsa-loader-accept-forward-refuse.md`](dsv4-dsa-loader-accept-forward-refuse.md).
 - **Real-checkpoint residency for the coalesced tower — W2.** W1b copies each
   TP1-coalesced linear into host owner buffers. That is right for the fixture
   and for W2's byte-parity gate, and it is ~100 GB on the real 216-expert

include/vllm/model_executor/models/deepseek_v4.h

@@ -159,14 +159,23 @@ struct DeepseekV4LayerHostWeights {
   std::vector<float> attn_sink;       // [n_heads]
   std::vector<float> wo_a;            // [n_groups, o_lora_rank, in_per_group]
   std::vector<float> wo_b;            // [H, n_groups*o_lora_rank]
-  // DSA Lightning-Indexer (indexer layers only; empty otherwise).
-  std::vector<float> idx_wq;     // [index_n_heads*index_head_dim, H]
-  std::vector<float> idx_wk;     // [index_head_dim, H]
-  std::vector<float> idx_wproj;  // [index_n_heads, H]
-  // DSA compressor (compressor layers only; empty otherwise).
-  std::vector<float> comp_wgate;        // [head_dim, H]  (produces the pool score)
-  std::vector<float> comp_ape;          // [compress_ratio, head_dim]
-  std::vector<float> comp_norm_weight;  // [head_dim]
+  // DSA compressor + Lightning-Indexer (those layers only; empty otherwise).
+  //
+  // TWO GEOMETRIES MEET IN THESE SLOTS, and the shapes below are the LOADED ones
+  // (#1970). The EXL3 loader materializes each at the width upstream DERIVES for
+  // the layer — `coff = 1 + (compress_ratio == 4)`, `compressor.py:247-248`, and
+  // `wq_b`'s natural `q_lora_rank` K, `attention.py:721-726`. `AttentionBlock`
+  // indexes the COLLAPSED synthetic geometry instead (`comp_wgate` as
+  // `[head_dim, H]`, `comp_ape` as `[compress_ratio, head_dim]`, `idx_wq` as
+  // `[index_n_heads*index_head_dim, H]`, `idx_wk` as `[index_head_dim, H]`), so
+  // where the two differ it REFUSES BY NAME rather than reading either. They
+  // coincide exactly where `coff` is 1 — every `compress_ratio != 4` layer.
+  std::vector<float> idx_wq;     // [index_n_heads*index_head_dim, q_lora_rank]
+  std::vector<float> idx_wk;     // [coff*index_head_dim, H]
+  std::vector<float> idx_wproj;  // [index_n_heads, H]  (not widened upstream)
+  std::vector<float> comp_wgate;        // [coff*head_dim, H]  (the pool score)
+  std::vector<float> comp_ape;          // [compress_ratio, coff*head_dim]
+  std::vector<float> comp_norm_weight;  // [head_dim]  (compressor.py:288)
   // MoE router: learned gate + (non-hash) noaux_tc bias OR (hash) tid2eid table.
   std::vector<float> gate_weight;  // [n_routed_experts, H]
   std::vector<float> gate_bias;    // [n_routed_experts]  (non-hash layers)

src/vllm/model_executor/models/deepseek_v4.cpp

@@ -645,6 +645,105 @@ std::vector<float> Slice(const std::vector<float>& v, int64_t off, int64_t len)
 }
 
 // ── 512-wide MLA attention block (W3 + W4 primitives) : [T,H] -> [T,H] ────────
+// #1970 — THE LENGTH CHECK THE DSA PATH NEVER HAD.
+//
+// `AttentionBlock` indexes the DSA tensors at the COLLAPSED synthetic geometry:
+// `comp_wgate` as [head_dim, hidden_size], `comp_ape` as [compress_ratio,
+// head_dim], `idx_wq` as [index_n_heads*index_head_dim, hidden_size], `idx_wk`
+// as [index_head_dim, hidden_size]. Since #1970 the EXL3 loader materializes
+// them at the width the REAL artifact stores — upstream's
+// `coff = 1 + (compress_ratio == 4)` (vllm/models/deepseek_v4/compressor.py:247-248)
+// — so the two can now disagree.
+//
+// AND A DISAGREEMENT HERE IS ANONYMOUS, NOT SILENT. Be exact about what this
+// buys, because overstating it is the defect #1964 was filed for. `Gemm`'s host
+// arm is a `MatVec` whose size assertion is UNCONDITIONAL — `deepseek_v4.cpp:413`
+// is a plain `VT_CHECK`, a throw rather than an `assert`, so `NDEBUG` does not
+// remove it — and its keep-quant arm checks the shape too. A [2*head_dim,
+// hidden_size] weight read at a [head_dim, hidden_size] stride therefore does NOT
+// produce a plausible wrong number. It throws
+//
+//     vt: MatVec weight size mismatch at deepseek_v4.cpp:413
+//
+// which names no tensor, no layer, no geometry and no missing capability, from
+// the middle of a forward, on a checkpoint that loaded successfully.
+//
+// So this is a DIAGNOSTICS improvement, and that is the whole of it: it replaces
+// an anonymous crash with a precise named refusal, listing EVERY mismatched
+// tensor with both counts and naming the composition that is missing. It is not
+// the difference between wrong tokens and a refusal, and it must not be described
+// as one.
+//
+// EVERY mismatch is collected and reported together, not just the first. A
+// refusal that stopped at the first would make the remaining checks
+// unfalsifiable: deleting any one of them would still throw on an earlier one,
+// so a mutation could not tell a live check from a dead one.
+void RequireDsaGeometryOrRefuse(const DeepseekV4LayerHostWeights& L,
+                                const DeepseekV4Params& p, int64_t layer,
+                                bool is_comp, bool is_indexer) {
+  const int64_t H = p.hidden_size;
+  const int64_t hd = p.head_dim;
+  std::string bad;
+  auto want = [&](const char* tensor, const char* indexed_as, size_t got,
+                  int64_t expect) {
+    if (static_cast<int64_t>(got) == expect) return;
+    bad += "\n    - attn." + std::string(tensor) + ": this forward indexes it as " +
+           indexed_as + " = " + std::to_string(expect) +
+           " elements, the checkpoint carries " + std::to_string(got);
+  };
+  if (is_comp) {
+    const int64_t cr = p.compress_ratio(layer);
+    want("compressor.ape", "[compress_ratio, head_dim]", L.comp_ape.size(), cr * hd);
+    want("compressor.wgate.weight", "[head_dim, hidden_size]", L.comp_wgate.size(),
+         hd * H);
+    want("compressor.norm.weight", "[head_dim]", L.comp_norm_weight.size(), hd);
+  }
+  if (is_indexer) {
+    const int64_t inh = p.index_n_heads;
+    const int64_t ihd = p.index_head_dim;
+    want("indexer.wq_b", "[index_n_heads*index_head_dim, hidden_size]",
+         L.idx_wq.size(), inh * ihd * H);
+    want("indexer.compressor.wkv.weight", "[index_head_dim, hidden_size]",
+         L.idx_wk.size(), ihd * H);
+    want("indexer.weights_proj.weight", "[index_n_heads, hidden_size]",
+         L.idx_wproj.size(), inh * H);
+  }
+  VT_CHECK(
+      bad.empty(),
+      std::string("DeepseekV4 forward: REFUSING the DSA path on layer ") +
+          std::to_string(layer) +
+          " — the checkpoint carries this layer's DSA tensors at a geometry this "
+          "forward does not implement. Reading the widened `comp_wgate` at the "
+          "width it DOES index throws an anonymous `MatVec weight size mismatch` "
+          "from inside the forward (deepseek_v4.cpp:413) that names none of this. "
+          "(That is the message the REAL geometry produces, because `comp_wgate`'s "
+          "Gemm runs first. A `comp_ape`- or `comp_norm_weight`-only mismatch "
+          "instead throws `ape size mismatch` / `rms_weight size mismatch` from "
+          "CompressorSaveScoreApe / CompressorPoolNorm "
+          "(deepseek_v4_compressor.cpp:23,54) — equally anonymous.) Refusing on:" + bad +
+          "\n  WHAT IS MISSING: upstream's DSA composition. The extra width is "
+          "`coff = 1 + (compress_ratio == 4)` "
+          "(vllm/models/deepseek_v4/compressor.py:247-248), and its two halves "
+          "are the two OVERLAPPING compression windows a token belongs to — a "
+          "role a row acquires only relative to the window gathering it "
+          "(common/ops/fused_compress_quant_cache.py:164-183), never recoverable "
+          "from the tensor alone. Reaching it needs the coff-overlapped window "
+          "with head_offset role selection, emission at boundary tokens only "
+          "((position + 1) % compress_ratio == 0) into a SEPARATE compressed KV "
+          "cache beside a sliding-window raw cache, and the indexer's query "
+          "projected from `qr` (q_lora_rank) instead of the hidden state "
+          "(vllm/models/deepseek_v4/attention.py:721-726, :835). None of that is "
+          "implemented here, and dense MLA is NOT a substitute for it at any "
+          "sequence length "
+          "(https://github.com/mudler/vllm.cpp/issues/1964).\n"
+          "  The loader accepts this geometry ON PURPOSE, so every NON-DSA "
+          "capability of the artifact is reachable rather than blocked behind a "
+          "path none of them use (MODEL-DSV4-EXL3 option C, "
+          "https://github.com/mudler/vllm.cpp/issues/1970). The DSA port itself "
+          "is OWED and has no owning row — see "
+          ".agents/specs/dsv4-dsa-loader-accept-forward-refuse.md `## Owed`.");
+}
+
 std::vector<float> AttentionBlock(const DeepseekV4LayerHostWeights& L,
                                   const DeepseekV4GgufLayerWeights* Lq,
                                   const DeepseekV4Params& p,
@@ -678,6 +777,13 @@ std::vector<float> AttentionBlock(const DeepseekV4LayerHostWeights& L,
   const bool is_indexer = p.has_indexer(layer) && !dsa_dense;
   const bool is_comp = p.has_compressor(layer) && !dsa_dense;
 
+  // Gated on the predicates above, so it fires only where this forward is about
+  // to READ one of these tensors. A layer that does not enter the DSA path reads
+  // none of them, and the GGUF arm (`dsa_dense`) enters it on no layer at all —
+  // so that arm's behaviour is byte for byte unchanged by #1970, and its own
+  // separate defect stays owed under #1964.
+  RequireDsaGeometryOrRefuse(L, p, layer, is_comp, is_indexer);
+
   // 1. q [T,nh,hd] and raw kv latent [T,hd] (num_key_value_heads=1 MLA). The MLA
   //    linears (wq_a, wq_b, wkv) run the keep-quant GEMM (Gemm) — the whole batch
   //    at once — then the per-token RMSNorm(q_norm/kv_norm) + per-head RoPE.