spec(MODEL-MM-QWEN4-EXP): Qwen3.8-Flash-Next is a new architecture vLLM does not implement, so the port splits its oracle

[localai-bot]

Qwen/Qwen3.8-Flash-Next was released on 2026-08-24 and declares
Qwen4ExpForConditionalGeneration / model_type: qwen4_exp. Nothing in this tree
matched it. This lands the row's records and its spec; no product code.

The name is misleading and the misreading is the expensive one. Qwen3.8-27B is,
per specs/qwen38-27b-bf16-gate.md, the Qwen3.6-27B shape retrained with exactly one
config key changed. That precedent does not extend here: the card calls this "the
architecture that will underpin Qwen4", and it diverges from qwen3_5 in four
load-bearing places.

vLLM implements nothing. Read live 2026-08-26 at origin/main = 6a5e8f5979:
no qwen4* path, no registry.py entry, and a repository-wide GitHub search for
qwen4 returns zero results. vllm-omni likewise. That is absence from vLLM main,
not staleness in 555967922, so advancing the parity pin does not reach it. What
exists is transformers#48337 "Add Qwen4Exp model", merged 2026-08-26, and SGLang
#36497, still open and therefore inadmissible.

So the row runs a split oracle, on developer direction of 2026-08-26: transformers
for the algorithm, vLLM ops for the optimized path. That is not a compromise. The
transformers reference states its own limits in code — Qwen4ExpTextQSAIndexer.forward
loops in Python over (batch_idx, query_idx) and carries the comment "we only allow
eager and sdpa" — so porting it as written yields a correct model at an indefensible
speed, while AGENTS.md's mirror-vLLM polarity still binds every primitive vLLM does
implement. The spec resolves each component against exactly one of the two, and an
implementer who cannot name the oracle for the line they are writing has found a gap
in the spec rather than a licence to choose.

The op survey is the substance of this change, and it moved two conclusions.

Most of the model is already here. Qwen4ExpTextModel inherits from
Qwen3_5MoeTextModel and leaves rotary, MLP, experts, TopK router and the entire
vision tower unchanged — class Qwen4ExpVisionModel(Qwen3_5MoeVisionModel): pass.
GDN is an exact hit on our Triton-AOT gate: the config's 128 / 128 / 16 / 48 against
the specializations cuda_gdn.cu pins to K=V=128, Hg=16, H in {48,32}. Exactly two
components have no vLLM op at all, and they are the two we must author ourselves: the
PLE dilated depthwise conv (git grep dilation over vLLM's layers/mamba/ returns
nothing) and the hashed n-gram embedding.

QSA's twin in vLLM is MiniMax-M3, not DeepSeek-V4. DSA is an MLA indexer; QSA is
plain GQA, 24 Q heads over 2 KV, head_dim 256. models/minimax_m3/common/indexer.py
is vLLM's non-MLA block-sparse case and its own docstring describes QSA's shape:
scores KV blocks with index heads, selects top-k blocks, owns a side cache of one
index-key vector per token. common/ops/index_topk.py supplies the block-score kernel
and a bitonic top-k, and the pooled-key build is
deepseek_v4/nvidia/ops/sparse_attn_compress_cutedsl.py's
SparseAttnCompressNormRopeStoreC4Kernel, which already carries compress_ratio.
This tree has a working DSA indexer, which makes DSA the path an implementer reaches
for first, and it is the wrong one. Recording that is most of why this spec exists.

Two structural consequences beyond the module list, both of which touch code this row
does not otherwise go near. The residual stream is hc_count * hidden_size = 4 x 2560
= 10240 wide through the whole stack, with a low-rank read gate and per-branch scalar
write gate around both attention and MLP, so the per-layer loop and every residual
buffer change. And number_of_conv_states is 3 on a PLE layer, plus the indexer side
cache, which lands on the same KV-cache seam #1963 and #1966 are moving.

Nothing published fits. Against roughly 119 GB usable on GB10: BF16 is ~360 GB,
the official FP8 ~180 GB, RadixArk/...-NVFP4 ~128 GB, and unsloth/...-GGUF is a
README with zero weight files. No GGUF exists and no tool can produce one, because
llama.cpp has no qwen4_exp either, so the standing k-quant requirement means
authoring the architecture here and stating plainly that those arms have no llama.cpp
oracle. The architecture supplies its own lever: the per-token n-gram cost is 16
lookups of 160 dims, so 51 GB of the 180 GB — 28% of the model — is a table touched 16
times per token, and making it non-resident is the intended design point rather than a
workaround. RadixArk reached the same split independently. gateable is therefore
no, on memory rather than software.

Two decisions were put to the developer as explicit accept-or-reject, and both are
settled and recorded in place rather than left open.

The first is the oracle. oracles/transformers.md pins transformers to 5.14.1, tied
to whatever the pinned vLLM environment resolves so that environment cannot hold two
transformers at once, and 5.14.1 does not contain Qwen4Exp. The lane-scoped second
pin is accepted, on the argument that the invariant guards a vLLM environment against
drifting from its transformers and here there is no vLLM implementation to drift from,
with expiry the moment vLLM registers qwen4_exp. It resolves to a real release rather
than a branch SHA, which was not the expected outcome: the merge landed at 12:03:40Z on
2026-08-26 and v5.16.0 published at 12:35:15Z. That was bounded rather than assumed —
fetching models/qwen4_exp/modeling_qwen4_exp.py at each tag gives HTTP 200 at
v5.16.0 and HTTP 404 at v5.15.0, so 5.16.0 is the first release carrying the
architecture and therefore the tightest pin available. The version string stays
UNMEASURED: it is the release proven to contain the model, not a
transformers.__version__ read off a running oracle, and gateable stays no.

The second is the first runnable arm: a Q4_K_M backbone with the n-gram table
non-resident, about 76 GB. Q8_0 was raised and does not fit at ~191 GB, and no partial
split reaches 119 GB while keeping the backbone at 8 bits. Q4_K_M throughout fits on
paper at ~109 GB but leaves roughly 10 GB for KV and activations on a model with 262144
native context, which is not a margin. This promotes the non-resident table from a note
to a first-class W6 deliverable and splits that wave in two, because it is not free:
GB10 is unified memory, so the existing host-pinned offload seam does not by itself
solve it there and the mechanism has to be disk-backed or genuinely unloaded. W6b is
the unknown, and the spec says to spike it before scheduling rather than design around
an unmeasured cost.

The MODEL row ratchet moves 377 -> 378 with its justification appended in the
existing log format, re-derived off the matrix rather than carried forward. One row
and not two: the MTP head is a mtp block inside the same text config, not a
separately registered architecture, so this is not the IndexTTS-2.5 or dots3-note
shape that moved the pin by two. The at-the-pin static invariants (324/373/356/310/261)
are unchanged, because the Upstream cell carries no pinned module/class target.

The component deep-dives, and two corrections to the above

Folds five parallel component investigations into the row spec and corrects two claims
the first draft got wrong. Every load-bearing finding below was re-verified against the
source before it was written down; the agents' reports are inputs, not evidence.

The headline correction. The spec claimed "QSA's structural twin in vLLM is MiniMax-M3,
NOT DeepSeek-V4", reasoning that QSA is plain GQA rather than MLA and so had to map onto
the non-MLA block-sparse case. That reasoning rested on reading MLAAttentionSpec as an
MLA claim. It is not one: M3's own indexer cache uses it while M3 is plain GQA, and the
comment beside it says why, "Key-only: MLAAttentionSpec budgets one vector/token (not 2x
for K+V)". It is a budget shape. With that prop removed the argument collapses, and nine
independent structural matches with DeepSeek-V4 remain, compress_ratio == 4 literally
the same number: MQA index with one key head at dim 128, relu(q.k) summed over index
heads, 1/sqrt(head_dim), one score set per query token with no head axis, the
(position+1) % COMPRESS_RATIO == 0 boundary, RMSNorm on the pooled key, RoPE at the
block-start position, candidates counted as visible // compress_ratio, and one stored
state per four tokens through MLAAttentionSpec(tokens_per_state=compress_ratio).

M3 is a different algorithm, not a worse fit. Verified: its score is tl.max(qk, axis=1)
over 128 raw token dots with no pooling, no relu and no head reduction; it asserts
num_idx_heads == num_kv_heads with the comment "no topk index reduce", so it emits one
block set per KV head where QSA emits one per token; and SPARSE_BLOCK_SIZE = 128 is
welded to the KV page size on both the score and the attend side. It contributes exactly
one thing, a wiring precedent: a plain-GQA model can own a key-only side cache and a
private indexer backend.

The genuinely new work is the consumer, and nothing upstream supplies it. Every DSv4
sparse consumer attends to compressed KV; M3's attend to raw tokens only at page
granularity; QSA attends to raw tokens selected at ratio-4 granularity. Two silent
failures follow. Wiring QSA's top-k into a DSv4 sparse-MLA consumer attends a pooled key
and value and still emits plausible tokens, and a short-prompt gate cannot catch it,
because below indexer_budget every candidate is selected. ## Gates now requires at
least one prompt past 2048 tokens of context for exactly that reason. And
SparseAttnCompressNormRopeStoreC4Kernel does not mean-pool despite the name: it is a
learned softmax pool over an overlapping window of eight, driven by a score channel this
checkpoint does not have.

The second correction is smaller and would have cost an afternoon: vLLM's grouped
RMSNorm is on RMSNormGated, not the plain RMSNorm, whose only related knob is
var_hidden_size, a prefix reduction that cannot express per-group norms.

The chosen arm does not load today, and the blocker is neither the offload nor the
budget. This tree cannot keep a gather table quantized at all: KeepQuantKDim returns
-1 for kEmbeddingTable and the qwen3_5 loader asserts it by name, so a quantized
n-gram table expands to bf16 and 51.2B parameters become 102.4 GB of anonymous memory.
The reason was already recorded in a header comment, "a gather, not a GEMM ... A
quantized-gather op is a follow-up row", and no such row exists. Separately,
moe_intermediate_size = 640 and hc_lowrank = 320 are Q4_K-illegal on their reduction
dims, and our GGUF reader has no entry for ggml type ids 3, 6, 7 or 20, so a stock
llama-quantize -Q4_K_M file would fail at header parse. We author the converter, so
the fix is Q4_0. ENG-WEIGHT-OFFLOAD does not help and must not be budgeted for: it
moves zero bytes today and is documented inert on unified memory. The tier that works
already ships and the 2.4T model proves it. W6 is split into a/b/c accordingly.

Sizing survives: backbone ~67.7 GiB, whole process ~73.5 GiB of 119.631 at 32K single
stream. The design works because per-token demand is at most 64 KiB of reads against the
2.4T expert lane's 6.95 GB/token.

The n-gram derivation was verified against the published checkpoint rather than trusted:
reconstructing the splitmix64 chain at the default seed 1234 reproduces the
layer_multipliers buffer read out of the safetensors payload, and the head vocab sizes
and offsets match entry for entry. Three silent C++ divergence sites are now named, the
worst being that the shard reassembly must be numeric, since a lexicographic key sort
gives shard_0, shard_1, shard_10 and permutes a 95 GiB table. Confirmed from the index:
128 contiguous shards, and the PLE sits on decoder layer 1.

Also recorded: vLLM has no dilated 1-D convolution anywhere, a confirmed negative with
zero hits in mamba/, csrc/ and tests/; the PLE conv is strided history at lags 9/6/3/0;
its signed-sqrt gate clamps before the square root, so the output floor is 1e-3 and tiny
scores are amplified rather than squashed; and Qwen4ExpTextModel has no final RMSNorm,
which the natural copy of our DeepSeek-V4 tail would wrongly add.

The ratchet test, and the anchors it shifted

The 377 -> 378 pin bump changed scripts/check-agent-record.py with no evidence in
tests/scripts/test_agent_record.py, and check-pr-size.py refused it by name: a
governance-checker change owes semantic mutation evidence. The gate was right and the
red was this branch's, not noise.

test_qwen4_exp_row_is_inside_the_model_ratchet follows the convention the dots3-note
and recipe-backfill tests set, naming the row the bump was taken for so that a count
raised to silence a broken parse cannot look identical to a count raised because a row
landed. It additionally pins that there is exactly ONE qwen4_exp row, because the MTP
head is an mtp block inside the same text config rather than a separately registered
architecture — so this is not the by-two shape IndexTTS-2.5 and dots3-note each took.

Adding it shifted that file by 45 lines and staled the two anchors
ENG-RECORD-ANCHOR-RATCHET keeps into it, taking the rot from 31 to 33 STALE. Repaired
to the true lines rather than raising the budget, which the baseline file forbids in
terms. That row's own record already describes this exact failure — an edit to the very
file the row cites — which is what the row exists to measure. Isolated rather than
assumed: the anchor ratchet passes on pristine origin/main, so both new entries were
this branch's.

origin/main moved from a4f8ada to 993b62e during review, so the gate was rerun on
the merged tree rather than on the tree this branch was cut from.

Tracked by #1978.

Gates: check-agent-record ok, check-model-checklist ok,
tests/scripts/test_agent_record.py + test_check_model_checklist.py 130 passed,
agent-preflight.sh --staged ok.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]