GLM-5.3-Flash has no GGUF and no tool can make one: author the `glm5_next` converter and gate its k-quant encoders byte-for-byte against llama.cpp b10451 (W7a of #1998)

[localai-bot]

W7a of #1998. Spec: .agents/specs/glm5-next-flash.md, §W7.

zai-org/GLM-5.3-Flash (Glm5NextForConditionalGeneration / glm5_next) has no artifact that fits any device this project reaches. Read live 2026-08-26 over the HuggingFace API and the 62 safetensors headers by HTTP RANGE: FP8 e4m3 305.78 GiB, BF16 598.53 GiB, LibertAIDAI/...-NVFP4 181.32 GiB, against ~119.63 GiB usable on dgx:gpu0. All four repositories named *-GGUF contain zero .gguf files. So every GPU gate on this row is blocked behind an artifact that does not exist and that no upstream tool can make: llama.cpp at our pin b10451 enumerates LLM_ARCH_GLM4, LLM_ARCH_GLM4_MOE, LLM_ARCH_GLM_DSA and no GLM5_NEXT — and its src/models/glm-dsa.cpp cites zai-org/GLM-5.2, a different model. gguf-py cannot help either: gguf.quants.Q2_K implements dequantize_blocks and no quantize_blocks, so upstream Python has no k-quant encoder at all.

This issue owns the converter and its gate, and NOT the artifact. Producing the artifact needs the 300–600 GiB checkpoint staged, disk, and a box; that is owed under O7 and is the second half of W7.

Delivered

• scripts/convert-glm5-next-gguf.py — safetensors → GGUF for glm5_next, arch string glm5next. Streams shard by shard (headers first, mmap slices second) so peak RSS is one tensor rather than one shard; FP8 e4m3 block dequant against the weight_scale_inv grid the checkpoint declares (weight_block_size: [128, 128]); per-expert tensors stacked into ffn_{gate,up,down}_exps.weight; the layer-45 MTP block and every shared_head.* dropped, following the reference's own _keys_to_ignore_on_load_unexpected and the glm4_moe_lite_registry.cpp:21-26 precedent.
• Metadata carrying the parameters the port hinges on, spelled at the pin: glm5next.kda.gate_lower_bound (= -5.0, gguf-py/gguf/constants.py:264 @ b10451 — the value that selects the sigmoid forget-gate branch over the softplus one our Kimi-Linear KDA implements, where even the sign of decay_rate differs), glm5next.kda.head_dim, the indexer geometry including index_kpool = 4 (the config class default is 16), the mHC triple with hc_eps 1e-6 kept distinct from rms_norm_eps 1e-5, the MoE routing, and layer_types as the authoritative per-layer schedule. No rope.freq_base is written: the text stack is NoPE end to end.
• Ported Q2_K, Q6_K and Q8_0 encoders, byte-identical to llama.cpp b10451 over a frozen golden. Two traps changed bytes and are recorded in the source: nearest_int is the 12582912.0 add-and-mask trick and not round, and C roundf rounds half away from zero where np.rint rounds half to even.
• tests/scripts/test_convert_glm5_next_gguf.py — 50 assertions on a synthetic tiny-shape checkpoint plus an independent in-test GGUF reader. No real checkpoint, no GPU.
• Refusals by name for every unimplemented arm: the i-quants, because an imatrix needs a forward pass, a forward pass needs 181 GiB, and the dependency is circular on this fleet (O5); Q3_K/Q4_K/Q5_K, because their encoders are not ported and shipping an ungated encoder is worse than refusing.

Exact arithmetic, computed by running the converter's own type resolver over the real topology (1719 output tensors, 313.89B parameters carried after the MTP block is dropped from 321.32B):

arm	weights	mixed bpw	of 119.63 GiB
q2_k (experts Q2_K, rest Q6_K)	100.35 GiB	2.746	fits, ~17.7 GiB left
q6_k	239.89 GiB	6.565	no
q8_0	310.67 GiB	8.502	no
bf16	584.67 GiB	16.000	no

At 128K context and one sequence: KV 1.43 GiB (11 MLA layers x 512 latent x 2 B + an 11 x 64 B indexer side cache = 11,968 B/token), KDA recurrent state 0.14 GiB (64 x 128 x 128 x 4 B x 34 layers) plus conv states. That leaves ~17.7 GiB for activations, allocator overhead and page cache. Arithmetic, not measurement: #1963 and #1966 record the KV byte accounting being wrong by 48x, so W5 re-derives this from the runner rather than trusting the table.

What is NOT reached. The emitted GGUF is not loadable by this tree: glm5next is not registered and general.architecture dispatch (src/vllm/entrypoints/model_loader.cpp:1000) has no entry for it. That wiring is W1's, owned by row MODEL-MM-glm5-next-glm5-next-for-conditional-generation and tracked by #1998. The converter itself is reached as a command-line path and the gate enters through it.

Owed by this issue when it closes: O7, artifact production — staged weights, disk, and a box, plus the sha256 and peak RSS the spec's §Evidence requires. O8, the Q3_K/Q4_K/Q5_K encoders, if an arm ever needs them.