[void-mckenzie]

Part of #671 (the DeepSeek-V4 item in #794). The Vision-Exp text trunk pairs with its mmproj; the 0731 text-only release has no image token and refuses the pairing.

What it does

• gguf_mm_config derives the flat vision fields from the mmproj (this family has no vision_config section); rope theta 10000 / max_n_token 384 / max_wh_ratio 8 are family constants llama.cpp hardcodes in clip.cpp and its converter asserts.
• iter_mmproj_weights loads the tower under visual.: q|k|v cat (ggml-org ships the attention block fused, unsloth split; both dialects load), gate-before-up w1 cat, the patch embedding flattened back to the Linear kernel.
• Text side: the trunk's exp_probs_b_vl biases feed the gate's image-row routing; the hash layers' converter-dropped text bias zero-fills (never applied by the reference or FreeToken's Gate).
• DSV4MMProcessor._image_processor() returns None: no transformers class exists, the preprocessing is FreeToken-local.
• Refusals, each matching where llama.cpp also refuses: wrong projector, non-SwiGLU tower, output-width mismatch, missing image token (0731 pairing), a trunk without exp_probs_b_vl (converted before llama.cpp #28154), eps other than 1e-6.

tok_compare

unsloth UD-Q2_K_XL vs the HF Vision-Exp tokenizer: 0/13 edge cases, 0/126,527 vocab texts, 0/20,000 combos.

Check sweep (scripts/gguf_check, header stubs of every Vision-Exp GGUF repo on HF: 19 repos, 109 split sets)

Each repo checked with its own mmproj via FREETOKEN_MMPROJ (quants sit in subfolders next to a root mmproj, so folder-local auto-discovery cannot fire in that layout).

Result	Files	Notes
pass, images served	10	all unsloth (template renders the placeholder)
template renders no image placeholder	29	ggml-org-lineage templates (ggml-org, prometheusAIR, vultr, DevQuasar, ddh0, huihui + mirrors, QuantaPlanta, cvgro, ANGELOSEGRETO): tower and weights pass every other check, image requests 400 at tokenize, text unaffected. These files serve images on llama-server, which bypasses the template entirely (image parts become a random per-restart media marker that any template passes through and mtmd splices on); FreeToken's v1 relies on the template rendering the placeholder, so the report is the honest gap. A marker-style splice is the follow-up hook
pre-#28154 trunk refused	54	xautonomics (46, also per-layer), qtum, BahamutRU; the "reconvert" message
fork projector spelling refused	4	tarruda's mmproj writes deepseek4_vision, unknown to llama.cpp too
tokenizer lost the image token	4	apetersson / audreyt abliterated exports
sidecars / unsupported	8	LoRA exports, metadata-only files

Nothing crashes and nothing mis-loads silently.

Real-file verification (RTX 5090 32 GB, Ryzen 9 7950X3D, 187 GiB RAM, driver 580.95.05)

unsloth DeepSeek-V4-Flash-Vision-Exp-UD-Q2_K_XL + mmproj-BF16.gguf: tower bit-exact against the HF checkpoint (267/267 tensors, 43/43 bias_vl, 3/3 tid2eid). Served end to end with

ft serve --gpu 0 --model-path .../DeepSeek-V4-Flash-Vision-Exp-UD-Q2_K_XL-00001-of-00003.gguf \
  --mmproj .../mmproj-BF16.gguf --max-seq-len-override 32768 --moe-cache-mem 6GiB --memory-ratio 0.82

Reads OCR text out of a probe image, answers two images in order, a repeated image prompt hits the radix (512 cached tokens), 0 errors in the log. Through the real processor, the image block's length cycles 114-117 tokens with its insertion offset (lead pads to the 4-token compression stride) and the offset enters the radix hash, so the same image at a shifted position is another sequence. The ggml-org pairing passes the full gguf_check dry load reporting only its template gap (not served live; its text path is the code the unsloth run exercised); --mmproj on the 0731 trunk stops startup with the pairing error.

[jason-fxz]

Thanks! Checked against the HF Vision-Exp checkpoint: every BF16 mmproj tower is bit-exact (split and fused qkv), bias_vl / tid2eid match, and 16 image probes on the UD-Q2_K_XL GGUF give the same prompt tokens and answers as the HF checkpoint. Two things to fix:

1. An older Vision-Exp text file (no exp_probs_b_vl) next to its mmproj no longer starts, even with --mm-disable vision / --text-model-only, e.g. BahamutRU/DeepSeek-V4-Flash-Vision-Exp-Q8_0-MXFP4-Q3_K-Q2_K-mixed-GGUF, which serves text on main. The check in parse_gguf_config (deepseek_v4/gguf.py:129) raises NotImplementedError, which auto-discovery does not skip, and iter_gguf_weights re-parses a shim that still has the vision fields. As in GGUF vision towers (mmproj): survey and plan #794, an auto-discovered mmproj the model cannot use should be skipped (text only) and only --mmproj should fail, so the check belongs in gguf_mm_config as a ValueError.
2. docs/models.md:130: FreeToken has no --chat-template flag. If we want one, it should be its own PR.

The ggml-org template gap is fine to leave documented here.

[void-mckenzie]

Both fixed in 3304d95: the exp_probs_b_vl check moved into gguf_mm_config as a ValueError, so an auto-discovered mmproj on an old trunk is skipped to text only (BahamutRU serves text again; only --mmproj fails), and the docs note no longer mentions --chat-template.

[jason-fxz]

Re-checked 3304d95: the BahamutRU file now skips its mmproj with a warning and serves text (default, --mm-disable vision, --text-model-only), and --mmproj with it fails at startup. The 16 image probes still give the same prompt tokens and answers as the HF checkpoint. LGTM, thanks!