spec(MODEL-MM-GLM53-FLASH): GLM-5.3-Flash is an architecture no oracle implements and no device here can hold (#1998)

.agents/issue-index.md

@@ -759,3 +759,4 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#2040](https://github.com/mudler/vllm.cpp/issues/2040) | `ENG-HYBRID-PLACEMENT` | W3c: replace W3b's hand-written per-model round trip with ONE shared seam every architecture routes through, exploiting that every MoE block already has the same `(Dev, weights, params, [T,H] dh, T)` shape — and correct `docs/ENVIRONMENT.md` and `docs/FEATURES.md`, which W3b made false by leaving them saying the knobs place nothing | bug |
 | [#2046](https://github.com/mudler/vllm.cpp/issues/2046) | `ENG-HYBRID-PLACEMENT` | `qwen3_5.cpp` kept private `Dev`/`DBuf`/`MakeTensor`/`Reshape` copies instead of the shared `dense_device_glue.h` set — the off-framework divergence its own `ResidentWeight` comment records, where a repair reached 25 model files and not this one. The private types also had INTERNAL LINKAGE, which is what forced the MoE placement seam to carry a glue-templated second spelling; migrating collapses it back to one | bug |
 | [#2050](https://github.com/mudler/vllm.cpp/issues/2050) | `ENG-HYBRID-PLACEMENT` | Laguna's FFN is host-orchestrated token-at-a-time — per-token host rows, the router on the host through `MatmulNK`, and a host scalar combine loop — so a device-shaped MoE entry wrapping those loops would put it in the placement seam's wired list while moving nothing and adding a round trip: supported to read, a regression to measure. The real repair is a device-resident batched FFN, which is a model rework with a performance gate | gap |
+| [#1998](https://github.com/mudler/vllm.cpp/issues/1998) | `MODEL-MM-GLM53-FLASH` | **`zai-org/GLM-5.3-Flash` declares `Glm5NextForConditionalGeneration` / `glm5_next`, an architecture vLLM implements at NO revision, whose only admissible oracle is transformers and which NOTHING on this fleet can execute.** Read live 2026-08-26: `git grep "Glm5\|glm5_next"` returns ZERO hits at the parity pin `555967922` AND at vLLM `origin/main` = `c71f6f8a81` (2026-08-26 08:42 -0700), whose `vllm/models/` package holds `common, deepseek_v32, deepseek_v4, dots3_note, inkling, kimi_k3, minimax_m3` and no `glm5next`. That is ABSENCE from vLLM `main`, not staleness in our pin, so a pin advance — 348 commits of reconciliation — reaches nothing here. [vllm#53906](https://github.com/vllm-project/vllm/pull/53906) "[Model] add GLM-5.3-Flash support" (opened 2026-08-26T14:12:00Z, head `933876c388fb`, 85 files, +12,511/-540) would register `Glm5NextForCausalLM`, `Glm5NextForConditionalGeneration` and `Glm5NextMTPModel`, and is **OPEN, unmerged, `mergeable: false`** — an unmerged PR is not a revision and is INADMISSIBLE as an oracle, the same rule this index already applied to SGLang #36497 on the qwen4-exp row; it is cited only as evidence of upstream shape. SGLang implements nothing either (PR #36507 open; code search 0) and the trap there is that its **cookbook** PRs #36440 and #36513 MERGED at 14:00:16Z and 14:39:29Z — documentation landed ahead of the code, so the model card links a recipe SGLang `main` cannot run. llama.cpp implements nothing (code search 0; PR #27752 open; our oracle pinned at `b10451`), and vllm-omni implements nothing. **The sole admissible reference is transformers**, implementing commit `eb4d9e2a64` (2026-08-26T14:26:40Z, transformers#48342), and the first RELEASE carrying it is **`v5.16.1`** — bounded rather than assumed by fetching `models/glm5_next/modeling_glm5_next.py` at each tag: HTTP **200** at `v5.16.1`, **404** at `v5.16.0`, **404** at `v5.15.1`. Our transformers pin is 5.14.1, so this row needs a lane-scoped second pin expiring when vLLM registers `glm5_next`; note it is `5.16.1` where the `MODEL-MM-QWEN4-EXP` lane is `5.16.0`, one day apart because `Qwen4Exp` merged before the 5.16.0 cut and `Glm5Next` after it — two lanes, two releases, which is what a lane pin is for. **Not the blocked GLM-5 row:** `MODEL-TEXT-deepseek-v2-glm-moe-dsa-for-causal-lm` (`GlmMoeDsaForCausalLM`, `BLOCKED`, `CLAIM-GLM-DSA-LATEST-DEEPSEEK`) is DeepSeek-V3.2 verbatim at 753.9B; `glm5_next` is a different `model_type`, a different parameter count, a different attention topology and has a vision tower. **The architecture:** 45 layers, hidden 4096, vocab 154880, 1,048,576 context, `layer_types` = 34 `linear_attention` (KDA) + 11 `deepseek_sparse_attention` at 3,7,…,43; MoE 288 routed + 1 shared, top-8, `moe_intermediate_size` 2048, sigmoid/`noaux_tc`, `n_group=topk_group=1` (which makes the group stage a NO-OP), `routed_scaling_factor` 2.5, clamped SwiGLU `swiglu_limit` 10.0 in FIVE places; mHC `hc_mult` 4 / `hc_sinkhorn_iters` 20 / `hc_eps` 1e-6; vision 24L/1024/patch **14**/merge 2/temporal 2/out 4096. **The text stack has NO rotary anywhere.** `qk_rope_head_dim: 0`, `mla_use_nope: true`, no `rope_theta`, no `rope_scaling`; the reference *deletes* the inherited `rope_parameters` field and passes `position_embeddings=None` to every layer, and **`indexer_rope_interleave: true` is a VESTIGIAL flag** the indexer override ignores — implementing it because the config names one is implementing a bug. Position comes only from the KDA decay and short convs. **HEAVY REUSE — and the two obvious reuses are WRONG in one place each, in ways a token gate cannot see.** (1) The KDA forget gate takes the OTHER branch: `gate_lower_bound: -5.0` is not None, so it computes `-5.0 * sigmoid(exp(A_log) * (f_b(f_a(x)) + dt_bias))`, whereas our `src/vllm/model_executor/models/kimi_kda.cpp:60` implements Kimi-Linear's `-exp(A_log) * softplus(g + dt_bias)`. Different functions of the same inputs, both smooth, both negative, both fluent — and the sign of `decay_rate` differs too, so getting it wrong flips decay to growth. (2) The mHC head collapse is `hidden_streams.mean(dim=2)`, an UNWEIGHTED mean whose own docstring says "Unlike DeepSeek-V4", where our `src/vllm/model_executor/models/deepseek_v4_mhc.cpp:168` `HcHeadCollapse` is the V4 weighted-gated collapse; the checkpoint carries **no `hc_head.*` tensors at all**, which settles it. **(3) The k-pool indexer is unobservable at or below 2048 tokens:** with `index_topk: 2048` a shorter context selects every candidate, the selection is the identity, the pooling is invisible and any short-prompt gate passes over a broken indexer — the same shape as the trap recorded on the qwen4-exp row. The pooling is **learned, per channel, not a mean**: 128 independent 4-way softmaxes over a learned `[4,128]` intra-pool position embedding plus a cached `[128,4096]` gate score, pools aligned to the first VALID token (so the grid is batch-dependent), a pool must be COMPLETE to be a candidate and is visible iff its LAST member is, `select_k = min(index_topk/index_kpool, P) = min(512, P)`, expanded back to raw token indices and widened to `index_topk + kpool - 1` = **2051** by an unscored ragged tail. Its cache is 257 floats/token/layer, not the parent's 128. **Structural blocker:** `MlaBlockDims::Validate` REFUSES `qk_rope_head_dim == 0` (`src/vllm/model_executor/layers/attention/mla_attention.cpp:90-93` requires every dim > 0, `:95-99` requires it even); Kimi-Linear is NOT the same case — it sets `mla_use_nope` while keeping `qk_rope_head_dim: 64`, so the rope slice still occupies the cache row (`include/vllm/model_executor/models/kimi_linear.h:86,:88`). NoPE does buy one thing: MLA weight absorption becomes trivially valid, since there is no rope slice to keep out of it. **NOTHING FITS, and no GGUF exists.** Measured live by HTTP RANGE over all 62 safetensors headers, payload never fetched: 76,108 tensors, `F8_E4M3` 314,396,639,232 + `BF16` 6,926,096,640 elements = **321,322,735,872 real parameters** plus 19,484,766 F32 block scales; routed experts are **311.65B = 97.0%**, so any GGUF arm's mixed bpw is the experts' bpw to within a percent. On disk against ~119.63 GiB on GB10: `zai-org/GLM-5.3-Flash` FP8 e4m3 block-128x128 **305.78 GiB (2.56x over)**, `zai-org/GLM-5.3-Flash-BF16` **598.53 GiB (5.00x)**, `unsloth/GLM-5.3-Flash-FP8` 305.82 GiB, `LibertAIDAI/GLM-5.3-Flash-NVFP4` **181.32 GiB (1.52x)**. **All four repos named `*-GGUF` contain ZERO `.gguf` files** (`unsloth`, `AtomicChat`, `aj9o9`, `vcruz305` — READMEs and images); a repository name is not an artifact. **`gateable = no` on MEMORY, and the consequence is the scoping result: no oracle can execute this model on any device this project reaches, so NO end-to-end token gate exists or can exist here** — recorded as visible debt, not waived. What replaces it is a tiny-shape CPU reference oracle instantiated from the pinned transformers, which is a real oracle for the NUMERICS and is explicitly NOT oracle gateability for the model. Developer-set W7 target is a low GGUF quant that fits the DGX; arithmetic (not measurement) with experts at the named type and the other 3% at Q6_K, bpw taken from our own reader's block traits at `src/vllm/model_executor/model_loader/gguf_reader.cpp:200`: Q8_0 318.0 GiB, Q6_K 245.5, Q4_K 170.6, IQ3_XXS 118.1 (~1.5 GiB left — not a margin), **Q2_K 102.6 (~17 GiB left)**, IQ2_S 100.3, IQ2_XXS 82.3, IQ1_S 64.0; KV is small by design at ~11.7 KiB/token (1.5 GiB at 128K, ~11.4 GiB at the full 1M) though #1963/#1966 mean that arithmetic is re-derived against the runner, not trusted. **Q2_K is the recommended first arm and the reason is producibility, not quality: i-quants need an importance matrix, an importance matrix needs a forward pass, and a forward pass needs 181 GiB — the dependency is CIRCULAR on this fleet.** K-quants need none. And since llama.cpp has no `glm5_next`, the converter is ours to author, with no reference GGUF to diff against and no llama.cpp floor for the arm. Layer 45 is the MTP block — 888 tensors, 7.43B params, 2.31% of the model, DSA/MLA rather than KDA and carrying no `hc_*` tensors — present in the checkpoint and discarded by the reference at `modular_glm5_next.py:1235`; owed, not implemented, and skipping it in the converter is ~2.3 GiB of headroom. Live seam contention to rebase onto rather than fork: [#1971](https://github.com/mudler/vllm.cpp/pull/1971) (DSA geometry) and [#1977](https://github.com/mudler/vllm.cpp/pull/1977) (DSv4 KV multicache). Spec: [`specs/glm5-next-flash.md`](specs/glm5-next-flash.md), 8 waves, W0-W2 and W4 CPU-gateable and W3/W5-W8 needing a GPU. No product code lands under the spec pull request | feature |

.agents/model-matrix.md

@@ -81,14 +81,15 @@ Rollup by lifecycle state (must equal the detailed per-state row counts):
 | SPIKE | 9 |
 | BLOCKED | 5 |
 | DONE | 3 |
-| READY | 4 |
+| READY | 5 |
 | GATING | 1 |
-| **Total** | **378** |
+| **Total** | **379** |
 
-Engaged architectures (the 54 non-`INVENTORIED` rows):
+Engaged architectures (the 55 non-`INVENTORIED` rows):
 
 | Support | Architecture | Family / example | Status | Row |
 |---|---|---|---|---|
+| 🚧 | `Glm5NextForConditionalGeneration` | GLM-5.3-Flash (321.32B total / ~18B active, natively multimodal; 34 KDA linear-attention + 11 DeepSeek-sparse MLA layers, mHC residual manifold, 288+1 expert MoE, 24-layer ViT) | **SPEC ONLY.** New architecture vLLM implements at NO revision -- absent from the pin `555967922` AND from `main` `c71f6f8a81`, so a pin advance does not reach it; vllm#53906 is OPEN and inadmissible. Sole admissible oracle is transformers, lane-pinned `v5.16.1` (the first release carrying `glm5_next`; `v5.16.0` is 404). `gateable = no` on MEMORY: the smallest published artifact is NVFP4 at 181.32 GiB against ~119.63 GiB on GB10, so no oracle can run this model on any device here and NO end-to-end token gate is reachable. Zero `.gguf` files exist in any of the four `*-GGUF` repos and llama.cpp has no `glm5_next`, so the converter is ours to author. Spec plans 8 waves; no product code has landed | `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` |
 | 🚧 | `Qwen4ExpForConditionalGeneration` | Qwen3.8-Flash-Next (180B total / 6B activated, image-text-to-text) | **SPEC ONLY.** New architecture vLLM does NOT implement at any revision; split oracle by developer direction 2026-08-26 (transformers for the algorithm, vLLM ops for the optimized path). Nothing published fits ~119 GB on GB10, so `gateable = no` and the quantized arms are load-bearing; no GGUF exists and llama.cpp has no `qwen4_exp` either. NO token, NO speed. [#1978](https://github.com/mudler/vllm.cpp/issues/1978) | `MODEL-MM-qwen4-exp-qwen4-exp-for-conditional-generation` |
 | ✅ | `Qwen3ForCausalLM` | Qwen3 dense (0.6B/1.7B/4B/32B) | near-tie-robust token-exact 16/16 on 0.6B+4B vs vLLM 0.25.0; NVFP4A16 (W4A16) dense quant also gated; c1 every-axis speed parity, c8 decode residual; async-serving device token-ids mirror ported (`ROW-SERVE-ASYNC-DENSE-MIRROR`, #31 fix into the shared dense `EmbedInto`) — `test_qwen3_dense_async_serving` RED→GREEN; sibling scope CLOSED (#323): `60e71a0e` fixed the eager path; `DenseDecodeGraphForward` ran first and replayed against stale HOST ids, so it now declines while the mirror is live and falls back to the proven eager path. Async gate 7/7 across Qwen3-0.6B/4B + Llama/Mistral/InternLM2 | `MODEL-TEXT-qwen3-qwen3-for-causal-lm` |
 | ✅ | `Qwen3MoeForCausalLM` | Qwen3-Coder-30B-A3B (MoE) | STRICT token-exact 6/6 vs vLLM 0.25.0; 11/16 speed-grid cells at/above graphed vLLM, c1/c2 residual | `MODEL-TEXT-qwen3-moe-qwen3-moe-for-causal-lm` |
@@ -523,6 +524,7 @@ Transformers compatibility is capability-driven and excluded from finite counts.
 | `MODEL-MM-cohere-asr-cohere-asr-for-conditional-generation` | `CohereAsrForConditionalGeneration` | `registry.py:571-574`; `vllm/model_executor/models/cohere_asr.py::CohereAsrForConditionalGeneration` | conditional generation / audio+image | MM processor; encoder/merge; cross-attention; audio/ASR frontend; vision encoder | ☐ required | `INVENTORIED` | none | unassigned |
 | `MODEL-MM-nemotron-parse-nemotron-parse-for-conditional-generation` | `NemotronParseForConditionalGeneration` | `registry.py:575-578`; `vllm/model_executor/models/nemotron_parse.py::NemotronParseForConditionalGeneration` | conditional generation / audio+image | MM processor; encoder/merge; cross-attention; audio/ASR frontend; vision encoder | ☐ required | `INVENTORIED` | none | unassigned |
 | `MODEL-MM-whisper-whisper-for-conditional-generation` | `WhisperForConditionalGeneration` | `registry.py:579`; `vllm/model_executor/models/whisper.py::WhisperForConditionalGeneration` | conditional generation / audio+image | MM processor; encoder/merge; sliding-window attention; cross-attention; audio/ASR frontend | ☐ required | `INVENTORIED` | none | unassigned |
+| `MODEL-MM-glm5-next-glm5-next-for-conditional-generation` | `Glm5NextForConditionalGeneration` (`model_type: glm5_next`; campaign row `MODEL-MM-GLM53-FLASH`) | **NOT IN vLLM AT ANY REVISION** -- deliberately written with no pinned module/class target, the same convention `MODEL-MM-qwen4-exp-*` follows, so the at-the-pin static invariants are untouched. Read live 2026-08-26: `git grep "Glm5\|glm5_next"` returns ZERO hits at the pin `555967922` AND at `origin/main` `c71f6f8a81` (2026-08-26 08:42 -0700), whose `vllm/models/` package holds `common, deepseek_v32, deepseek_v4, dots3_note, inkling, kimi_k3, minimax_m3` and no `glm5next`. That is ABSENCE, not pin staleness, so advancing the parity pin -- 348 commits of reconciliation -- reaches nothing here. [vllm#53906](https://github.com/vllm-project/vllm/pull/53906) (opened 2026-08-26T14:12:00Z, head `933876c388fb`, 85 files, +12,511/-540) would register `Glm5NextForCausalLM`, `Glm5NextForConditionalGeneration` and `Glm5NextMTPModel`, and is **OPEN, unmerged, `mergeable: false`** -- an unmerged PR is not a revision and is inadmissible as an oracle, the same rule applied to SGLang #36497 on the qwen4-exp row; it is cited only as evidence of upstream shape. SGLang (PR #36507 open, cookbook #36440/#36513 MERGED ahead of the code), llama.cpp (PR #27752 open, our pin `b10451`) and vllm-omni likewise implement nothing. **ONE row and not three:** none of the three names is registered at any vLLM revision, and the only architecture a published artifact declares is `Glm5NextForConditionalGeneration`. | conditional generation / image + video | model loader/forward; GDN/linear-attention state (KDA); MLA/latent KV; DSA sparse indexer; FusedMoE/grouped GEMM; mHC residual manifold; MM processor + ViT encoder/merge; FP8 block-wise quant; GGUF k-quant | [glm5-next-flash](specs/glm5-next-flash.md) | `READY` | **SCOPING ONLY, 2026-08-26 (issue [#1998](https://github.com/mudler/vllm.cpp/issues/1998)): the spec and its records are the whole change; NO product code has landed.** Architecture read from `config.json` live: 45 layers, hidden 4096, vocab 154880, 1,048,576 context; `layer_types` = 34 `linear_attention` + 11 `deepseek_sparse_attention` (at 3,7,...,43); MLA is **fully NoPE** (`qk_rope_head_dim: 0`, `mla_use_nope: true`, `qk_nope_head_dim: 256`, `v_head_dim: 256`, `kv_lora_rank: 512`, `q_lora_rank: 1536`) and `text_config` carries NO `rope_theta` and NO `rope_scaling` -- the transformers reference *deletes* the inherited `rope_parameters` field and passes `position_embeddings=None` to every layer, and `indexer_rope_interleave: true` is a VESTIGIAL flag the indexer override ignores, so the text stack has no rotary anywhere. DSA indexer adds a **learned per-channel k-pool** (`index_kpool: 4`, `index_kpool_compress`, `index_kpool_always_select_tail`): 128 independent 4-way softmaxes over a learned `[4,128]` intra-pool position embedding, pools aligned to the first VALID token (batch-dependent), `select_k = min(index_topk/index_kpool, P) = min(512, P)`, expanded back to raw indices and widened to `index_topk + kpool - 1` = **2051** by an unscored ragged tail. mHC `hc_mult: 4`, `hc_sinkhorn_iters: 20`, `hc_eps: 1e-6`. MoE 288 routed + 1 shared, top-8, `moe_intermediate_size` 2048, sigmoid/`noaux_tc`, `n_group=topk_group=1`, `routed_scaling_factor` 2.5, clamped SwiGLU `swiglu_limit` 10.0 in FIVE places. Vision 24L/1024/patch **14**/merge 2/temporal 2/out 4096. **HEAVY REUSE, and the two obvious reuses are WRONG in one place each.** (1) The KDA forget gate takes the OTHER branch: `gate_lower_bound: -5.0` is not None, so it computes `-5.0 * sigmoid(exp(A_log) * (f_b(f_a(x)) + dt_bias))`, where our `src/vllm/model_executor/models/kimi_kda.cpp:60` implements Kimi-Linear's `-exp(A_log) * softplus(g + dt_bias)` -- different functions of the same inputs, both fluent, and the sign of `decay_rate` differs too. (2) The mHC head collapse is `hidden_streams.mean(dim=2)`, an UNWEIGHTED mean whose own docstring says "Unlike DeepSeek-V4", where our `deepseek_v4_mhc.cpp:168` `HcHeadCollapse` is the V4 weighted-gated collapse; the checkpoint carries no `hc_head.*` tensors at all, which settles it. **Third trap: the k-pool selection is unobservable at or below 2048 tokens** -- with `index_topk: 2048` a shorter context selects every candidate, so any short-prompt gate passes over a broken indexer. **Structural blocker:** `MlaBlockDims::Validate` REFUSES `qk_rope_head_dim == 0` (`src/vllm/model_executor/layers/attention/mla_attention.cpp:90-93` requires every dim > 0, `:95-99` requires it even); Kimi-Linear is NOT the same case, it sets `mla_use_nope` while keeping `qk_rope_head_dim: 64` so the rope slice still occupies the cache row (`include/vllm/model_executor/models/kimi_linear.h:86,:88`). **HW: NOTHING FITS.** Measured live by HTTP RANGE over all 62 safetensors headers (no payload fetched): 76,108 tensors, `F8_E4M3` 314,396,639,232 + `BF16` 6,926,096,640 elements = **321,322,735,872 real parameters**, routed experts **311.65B = 97.0%**; on disk `zai-org/GLM-5.3-Flash` FP8 305.78 GiB (2.56x GB10), `-BF16` 598.53 GiB (5.00x), `LibertAIDAI/...-NVFP4` 181.32 GiB (1.52x). **All four `*-GGUF` repos contain ZERO `.gguf` files.** `gateable = no` on MEMORY: no oracle can execute this model on any device this project reaches, so **no end-to-end token gate exists or can exist here** -- recorded as visible debt, replaced by a tiny-shape CPU reference oracle instantiated from the pinned transformers, which is an oracle for the NUMERICS and is NOT oracle gateability for the model. Layer 45 is the MTP block (888 tensors, 7.43B params, 2.31%), present in the checkpoint and discarded by the reference; owed, not implemented. | `-` |
 
 ## MODEL-AUDIO - Audio encoder components (not vLLM registry architectures)