Skip to content

MODEL-MM-QWEN4-EXP: the spec's settled-decisions list states the REFUTED QSA mapping and points W4/W5 at the wrong port #2049

Description

@localai-bot

The row spec .agents/specs/qwen4-exp-flash-next.md contradicts itself on the single
most expensive decision in the row, and the stale half sits in the section written
specifically so a fresh implementer does not re-derive it.

## Why this needs a spec before code, item 2, currently reads:

  1. QSA's twin in vLLM is MiniMax-M3, not DeepSeek-V4. See ## Design. Building
    it on the DSA/MLA path is the wrong port, and DSA is the path an agent reaches
    for first because this tree already has it.

That is the row's FIRST reading, and the campaign refuted it. The same file says the
opposite twice:

  • Port map: "DeepSeek-V4 C4 indexer lane ... NOT MiniMax-M3, see below"
  • Design heading: "QSA maps to DeepSeek-V4's C4 indexer lane, NOT to MiniMax-M3"

as do the model-matrix row and #1978, both of which record the correction and the
reason: the original argument rested on treating MLAAttentionSpec as an MLA claim,
and it is not one — it is a per-state BUDGET shape, which MiniMax-M3 (itself plain
GQA) also uses. Nine independent structural matches tie QSA to DeepSeek-V4 including
compress_ratio == 4, relu-summed MQA scoring, RoPE at the block-start position and
tokens_per_state = compress_ratio.

Why this is load-bearing rather than cosmetic. W4 is the QSA wave and W5 wires it.
Item 2 does not merely misname a reference — it issues an instruction ("Building it on
the DSA/MLA path is the wrong port") that is now exactly backwards, in the one section
that tells a fresh implementer to trust it and not re-derive. An implementer who reads
top-down hits the stale item before the corrected Design section.

The two readings are also not equally wrong in effect: M3 scores by max over 128 RAW
dots, emits one set per KV head, and welds SPARSE_BLOCK_SIZE = 128 to the KV page
size, so building QSA on it forces a page size of 4 and breaks tl.dot, whose tile
needs >= 16.

Found while reviewing the W6a merge (#2019). Pre-existing on main — W6a did not
introduce it — so it is filed rather than silently folded into that wave.

Fix: restate item 2 to match the Design section and the matrix row, keeping the
warning shape (the DSA/MLA reflex is still the trap, just in the other direction).

Owned by row MODEL-MM-QWEN4-EXP, spec .agents/specs/qwen4-exp-flash-next.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions