feat(MODEL-MM-QWEN4-EXP): W4 — Qwen Sparse Attention with a GATHER consumer, counted at the key-row read

[localai-bot]

Qwen Sparse Attention: the indexer, its side cache, and a gather consumer.

The consumer shape is the point of this wave. llama.cpp #27739 records, with the
mechanism named, that a sparse mask over a dense cache costs the same as dense
attention under CUDA flash attention, because flash_attn_mask_to_KV_max scans
backwards and stops at the first tile that is not all -inf — so a mask that keeps any
late key pays the whole dense prefix. #27742 is mask-only and therefore buys
correctness without decode speed. A mask-shaped port here would pass a token gate and
silently forfeit the long-context lever the row exists to preserve.

QSA maps onto DeepSeek-V4's C4 indexer lane, not MiniMax-M3, and the first reading
of this row said the opposite. That reasoning rested on MLAAttentionSpec being an MLA
claim; it is not — M3's own indexer cache uses it while M3 is plain GQA, with the
comment "Key-only: MLAAttentionSpec budgets one vector/token (not 2x for K+V)". Remove
that prop and nine structural matches with DSv4 remain, compress_ratio == 4 literally
the same number. Two things are deliberately NOT inherited: DSv4's weights_proj, for
which QSA has no tensor (its weight is the constant 1/sqrt(128)), and its
GPT-J-style trailing-span RoPE, where QSA uses interleaved mRoPE over the leading
64 dims — the halves are swapped end for end.

The first review FAILED this change, and the finding is the one worth recording.
keys_visited was assigned sel.size(), and the test compared it to
SelectedCount(sel) — the same quantity, from the same input, by the same rule. The
assertion observed the index buffer and never the walk. The proof is a mutation that
computes a dot product against every one of the kv_len cached rows, i.e. exactly
QsaMaskedAttention's cost, while still reporting the sparse figure: it passed 12
cases / 7251 assertions, including the case named "the GATHER touches only the
selected rows". The code was a real gather; nothing in the tree kept it one.

The counter is now incremented at the point of the key-row read, in both consumers.
The mask was made counted too — leaving it assign-not-count would have put the two
counters in different units and made the comparison between them meaningless. That
mutation is now RED, as is one that merely reads each selected row twice, which the
old instrument also could not see because it was not a read count even for the correct
implementation.

The instrument's limits are declared rather than implied, and the re-review
measured both. A walk-and-skip consumer that iterates all positions without reading the
unselected key rows stays green: the cost #27739 measures is the QK dot and the K/V
tile traffic, and a per-row predicate is about one byte against 256 bytes of key
traffic, so a counter firing on it would be measuring a loop index — the defect the old
counter had. And no in-function instrument can refuse a fabricated number: a dense
body with a hand-written sparse assignment is green by construction. Both are stated in
the header.

Four more gate defects the review found, all closed. The sub-budget "bit-identical to
dense attention" case compared the gather to itself — both sides called
QsaGatherAttention — so scaling its output by two left it green; want now comes
from an independent masked walk over the full causal prefix. Two config refusals masked
each other, because the probe value was odd and either guard alone still threw, leaving
the upstream one untested. The mean pool's normalisation was unobservable, since
k_layernorm is RMS-scale-invariant and /4 is exact — a hand-derived case now probes
where the norm is eps-dominated and linear, with rope neutralised, so every expected
value is exact. And the selection-budget refusal cannot be reached in isolation, so its
case asserts which refusal fires, the only available discriminator.

Two free oracles carry the correctness argument, and neither needs weights. Below
indexer_budget + compress_ratio - 1 cached tokens every candidate is selected, so the
gather must be bit-identical to dense attention — asserted with == on floats, not
a tolerance, over all 2051 rows. Above it, selected token index sets must equal the
transformers reference, including the ragged tail, which is always attended. The
goldens come from running unmodified transformers 5.16.0 with
apply_rotary_pos_emb intercepted rather than reimplemented, so the pooled-key RoPE
position is captured from the oracle's own execution; perturbing the upstream module
moves the goldens, which is how that was proved rather than assumed.

A corollary this wave imposes on every later gate: any QSA token gate must exceed
2048 tokens of context. Below the budget a correct implementation and one attending
pooled keys instead of real tokens are indistinguishable, and the wrong one still emits
plausible output.

This slice lands unreached. There is no Qwen4ExpTextModel yet; the commit body,
the spec ## Owed and the index row each name that, the owning row and #1978. No token
number, no speed number; the speed axis opens at G4, after the GGUF arm.

Tracked by #1991, under #1978.

Gates: test_qwen4_exp_qsa 14 cases / 7263 assertions; agent-preflight.sh exit 0;
commit style, trailers and index append-only all OK. A fresh review returned FAIL, a
different implementer repaired it, and a fresh scoped re-review returned PASS, with
every repair proved by re-running the mutation that used to survive and the two
declared limits measured rather than asserted.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [Claude Code]