record(MODEL-MM-QWEN4-EXP): a speed denominator exists, and the target is to beat llama.cpp at concurrency

.agents/specs/qwen4-exp-flash-next.md

@@ -500,11 +500,39 @@ Usable budget on GB10 is about 119 GB. Read live from the HF API, 2026-08-26:
 | `Qwen/Qwen3.8-Flash-Next` BF16 | ~360 GB (`BF16 = 179,999,981,424` params) | no |
 | `Qwen/Qwen3.8-Flash-Next-FP8` (official) | ~180 GB | no |
 | `RadixArk/Qwen3.8-Flash-Next-NVFP4` | ~128 GB; NVFP4 backbone with the n-gram table kept at **FP8, 51.2 GB** | no, over budget before KV |
-| `unsloth/Qwen3.8-Flash-Next-GGUF` | **README only, zero weight files** | does not exist |
-
-No GGUF exists and no existing tool can produce one, because llama.cpp has no
-`qwen4_exp` architecture either. Per AGENTS.md the quantized arms are a standing
-requirement, so this row owes them and owes authoring the arch on our side.
+| `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S | **67.56 GiB**, 3 shards | **YES, and it is the ONLY published artifact that does** |
+
+**CORRECTED 2026-08-26.** This table previously read "README only, zero weight files
+-- does not exist", and that was true when it was written and false a few hours later.
+The repository was populated at 13:32Z with `UD-IQ1_S/Qwen3.8-Flash-Next-UD-IQ1_S-0000{1,2,3}-of-00003.gguf`,
+72,546,461,344 bytes = **67.56 GiB**, read from the files' own headers:
+`general.architecture = "qwen4exp"`, `split.tensors.count = 1224`,
+`general.file_type = 24`. **It fits GB10 with roughly 52 GiB of headroom**, where every
+safetensors artifact does not fit at all. The whole shape of this row's `## Work
+breakdown` follows from that, which is why the correction is called out rather than
+quietly applied.
+
+Its metadata independently confirms this spec's own n-gram derivation to the digit:
+`qwen4exp.ple.layer_multipliers = [23703573157769, 20109073645365, 8052911324071]`,
+`qwen4exp.ple.head_vocab_sizes` starting `[20000003, 20000023, 20000033, ...]`, and
+`qwen4exp.ple.layers = [1]` (0-based) corroborating the one-indexed conversion.
+
+**Two things in OUR tree stop us loading it**, and both are W6a's
+([#1989](https://github.com/mudler/vllm.cpp/issues/1989)): our GGUF reader has no
+`case 20`, so the IQ4_NL that file uses for `ffn_down_exps` and for the n-gram table
+fails at header parse; and `KeepQuantKDim` returns `-1` for `kEmbeddingTable`, so a
+quantized gather table expands to bf16 and 51.2B params become 102.4 GB.
+
+It carries **no MTP weights** — zero `nextn`/`mtp` tensors of 1224 — while the
+safetensors repo has 31. That is [#1993](https://github.com/mudler/vllm.cpp/issues/1993)'s
+problem and `docs/USAGE.md` must say so beside the arm.
+
+**A revision and a per-shard sha256 are still OWED.** The repo's `lastModified` moved
+again after this row first read it, which is exactly the re-quantized-in-place case
+AGENTS.md "Say which weights, and from where" names; a repo id alone is not a pin.
+
+llama.cpp still has no *merged* `qwen4_exp` architecture -- two competing PRs are open
+or withdrawn -- so authoring our own converter remains owed for arms nobody publishes.
 
 **The architecture hands us the lever.** Its card argues n-gram embedding is "more
 amenable to offloading than MoE", and the arithmetic agrees: the per-token cost is
@@ -709,9 +737,56 @@ No token gate is claimable until an arm runs. In order:
 4. **G3, quantized arms.** Per arm, with the lower-bound requirement this repository
    places on quantized gates, and with the missing-llama.cpp-oracle limitation stated
    in the result rather than omitted.
-5. **Speed: nothing.** No throughput, latency or memory number is admissible from
-   this row until G2 passes. There is no vLLM denominator for this model, so when a
-   speed axis does open, the spec must first say what the denominator is.
+5. **G4, speed against llama.cpp at its pin.** A denominator now exists and the
+   earlier "there is no denominator" clause is superseded: `llama-cpp` is a registered,
+   pinned, `gateable = yes` oracle whose scope is "GGUF k-quant speed and memory floors,
+   **quant-matched against the same weights**", and `unsloth/Qwen3.8-Flash-Next-GGUF`
+   UD-IQ1_S is one published artifact both engines can run. The axis opens when W6a makes
+   that file loadable, and not before: no throughput, latency or memory number is
+   admissible from this row until then.
+
+   **The target is binding** (developer, 2026-08-26): faster than llama.cpp on this
+   model, *especially at high concurrency*. Therefore:
+
+   - **A concurrency LADDER is the headline, not a point.** c = 1, 4, 8, 16, 32 at
+     minimum. A c=1 result neither confirms nor refutes this target.
+   - **Prefill and decode reported separately**, because input length splits them and an
+     aggregate hides which lever moved.
+   - Memory is an axis: peak RSS and peak device bytes at each concurrency.
+   - llama.cpp runs in its production configuration. A handicapped denominator is not a
+     result, and this repository already has the `--enforce-eager` precedent for how that
+     goes wrong.
+   - Identical artifact, prompts, token counts, sampling and concurrency; idle host;
+     reproduced with a same-binary A/B.
+
+### Where the speed is expected to come from, and what would forfeit it
+
+Four levers, from a source study of the two llama.cpp implementations (#27742 open,
+#27739 closed by courtesy). **Both are UNMERGED**; each item is a reading of a pinned SHA
+and not a measurement. Recorded here because three of them constrain waves that have not
+started.
+
+1. **Continuous batching and paged KV — the concurrency lever.** This engine mirrors
+   vLLM's scheduler and block manager; llama.cpp's server allocates fixed parallel slots.
+   That gap grows with concurrency rather than shrinking, which is where the target aims.
+2. **The QSA consumer — the long-context lever, and the one this row can forfeit by
+   accident.** #27739 records that a sparse **mask** over a dense cache costs the same as
+   dense attention under CUDA flash attention, because `flash_attn_mask_to_KV_max` only
+   scans back to the first tile that is not all `-inf`. #27742 is mask-only and so buys
+   correctness without decode speed. **W4 builds the gather, not the mask.**
+3. **N-gram table residency.** #27742 makes the table CPU-resident by tensor class
+   regardless of `-ngl`, so every token's 16 gathers are host work. At IQ4_NL the table is
+   ~28.8 GB inside a 67.56 GiB file against ~119.6 GiB usable, so it can be
+   device-resident and quantized. At batch B that is 16xB uncoalesced random gathers, so
+   host-versus-device here is a scaling difference, not a constant. W6a's residency
+   decision must be **asserted**, not defaulted.
+4. **The hyper-connection write-back.** Both PRs materialise the rank-1 update as a
+   `repeat_4d` + `mul`: 96 materialised `[2560, 4, T]` broadcasts per forward at 48
+   layers x 2 sites. W3 leaves the fused seam reachable rather than building it.
+
+**No ceiling may be declared** if a first measurement disappoints. An apparent
+same-artifact limit is an unresolved implementation difference with a next traceable
+hypothesis, every time.
 
 ## Evidence required