[danielhanchen]

Problem

The prebuilt nightly picks the newest upstream tag older than six hours. Today that is b12012 (b12014, b12015 and b12017 age in next). Six pins no longer merge onto it, so the nightly stops at the merge step:

• Inkling: upstream moved the KV-cache rotation constants to per-layer GPU buffers and took vocab pre-type 66.
• iq1-narrow: upstream added GGML_TYPE_F8_E4M3 = 43 on the type-count line.
• cuda-moe-prompt-kernels: upstream added the RDNA4_M family in mmq.cu.
• dsv41-mxfp8-pin: MXFP8 used type id 43, which upstream F8_E4M3 now owns.
• Metal NVFP4 (third-party branch): F8_E4M3 exclusions in ggml-metal-device.m.
• PLaMo-3 support: upstream merged the same work.

One more pin (the Gemma 4 chat parser fix) is already in b12012.

Change

Each conflict is resolved in its own pin branch with a Fix merge conflicts merge of the upstream tag, keeping upstream verbatim and re-applying only the pin's intent:

pin	old	new	resolution
Inkling	3582c5e	e916936	keep upstream kv-cache code and the pin's set_input_pos_rel_flat; port inkling.cpp to per-layer self_k_rot[il]; Inkling pre-type moves to 67 (internal, not stored in GGUF)
iq1-narrow	97611a1	722adcd	keep upstream F8_E4M3 plus IQ1 types 64..66, GGML_TYPE_COUNT = 67
cuda-moe-prompt-kernels	02aa369	ece7705	keep the pin's MoE tile pick and add GGML_CUDA_CC_IS_RDNA4_M to its RDNA branch
dsv41-mxfp8-pin	80f0f5a	b339e83	MXFP8 moves to type 63. gguf.cpp and gguf-py read a legacy type-43 tensor as MXFP8 when its on-disk span (measured from sorted data offsets) fits only MXFP8; when the span fits both (small tensors at alignment above 32) the rest of the file decides (another type-43 tensor that fits only one type, a type-63 tensor, general.file_type 42), and a file that still cannot be told apart is refused instead of guessed. general.alignment is cast to a Python int, so NumPy 2 no longer overflows

Metal NVFP4: the third-party pin is replaced by the same change carried on b12012 in an unslothai branch (pin 347, a01140e, original author's commit kept); upstream's F8_E4M3 exclusions stay verbatim.

Dropped: the PLaMo-3 pin and the Gemma 4 parser pin (both in upstream). feature-checks.json drops their unchecked entries.

Tests

• llama_pinset.py sim on b12012, b12014, b12015 and b12017: all 25 pins merge, pin_contract reports all 25 intact, merge_checks clean. On current master the same six pins are refused on all four tags.

• CPU verify on b12012 and b12017: gate build (llama test-llama-archs test-backend-ops test-mtmd-impl) ok, feature_matrix 16 / 16 on CPU. Assembled b12012 tree: test-llama-archs 147 / 147, test-chat passed.

• CUDA compile (sm_89) of the full assembled b12012 tree including the server: clean.

• Inkling numeric gate on an RTX PRO 6000 (sm_120), UD-IQ1_S, wikitext-2 -c 512 --chunks 20: old pin, new pin and new pin merged with upstream master all give PPL 39.727228, mean KLD 0.000000, top-1 100%.

• MXFP8: headers of the published DeepSeek-V4.1 UD-Q3_K_XL / UD-Q4_K_XL / UD-Q8_K_XL shard 8 and the MXFP8 engram file read the type-43 tensor as MXFP8; synthetic old-format MXFP8, new-format MXFP8 and real F8_E4M3 files read back with the right type and identical dequantized values.

• CUDA on an L4 (sm_89, nvcc 13.0), assembled b12012 tree vs plain b12012 on the same machine: test-backend-ops MUL_MAT 2627 / 2627 (base 2490 / 2490), MUL_MAT_ID 1027 / 1027 (931 / 931), PAD 37 / 37, GET_ROWS 349 / 349 (216 / 216), full suite 19605 / 19605 (19209 / 19209), 0 failures on either side; every case that passes on b12012 passes with the pins. All 13 MUL_MAT_ID_FUSION cases pass on both, including prompt-sized ones (128 experts, 8 used, n 512). feature_matrix --gpu: all 16 features demonstrated on GPU.

• CUDA on 2x T4 (sm_75, nvcc 12.8), same two trees: test-backend-ops MUL_MAT 2627 / 2627, MUL_MAT_ID 1027 / 1027, PAD 37 / 37, GET_ROWS 349 / 349, MUL_MAT_ID_FUSION 13 / 13, full suite 19605 / 19605 (base 19209 / 19209), 0 failures and no base-passing case lost. feature_matrix --gpu 16 / 16.

• Two-GPU perplexity on the T4s (wikitext-2, -c 512 --chunks 20 -ngl 99, every cell repeated bitwise): Qwen3-0.6B Q8_0 and Q4_K_M and granite-3.1-3b-a800m Q4_K_M (MoE) give identical PPL on both trees for one GPU and for -sm layer across both GPUs, and -sm layer equals one GPU. -sm row no longer exists on CUDA in b12012 (both trees refuse it the same way).

• ROCm on gfx1151 (Radeon 8060S), Linux and Windows 11, HIP builds of both trees: test-backend-ops on ROCm0 MUL_MAT 2627 / 2627, MUL_MAT_ID 1027 / 1027, PAD 37 / 37, GET_ROWS 349 / 349, MUL_MAT_ID_FUSION 13 / 13, full suite 19581 / 19581 (base 19185 / 19185), 0 failures and no base-passing case lost on either OS. feature_matrix --gpu 16 / 16 on Linux (Windows needs the .exe fix in a separate PR).

• After the MXFP8 reader fix and the Metal NVFP4 re-add (26 pins): llama_pinset.py sim on b12012 and on every tag from b12024 (the current six-hour-aged base) to b12032 merges all pins, pin_contract intact, merge_checks clean; CPU verify on b12012, b12024 and b12032 gate build ok, feature_matrix 16 / 16.

• Legacy MXFP8 reader: 23 synthetic cases (alignment 32 / 64, reversed header order, explicit general.alignment, old-format MXFP8, type 63, real F8_E4M3, evidence and refusal paths) pass in Python on NumPy 1.26 and 2.5 and in C++ with identical dequantized values (the previous pin passes 15 / 23 and 10 / 23). Headers of all 68 published DeepSeek-V4.1 GGUF files: only UD-Q3_K_XL / UD-Q4_K_XL / UD-Q8_K_XL shard 8 carry type 43, all at the default alignment, and both readers still read them as MXFP8.

Not covered

• With the experimental -sm tensor split on 2x T4, the MoE model's PPL differs between the trees (8.9745 vs 8.9192, inside the +/- 0.35 error bar; dense models match bitwise). Not bisected.
• Only sm_75 and sm_89 for CUDA and gfx1151 for ROCm; not the multi-arch prebuilt build or more than 2 GPUs.
• The Metal NVFP4 pin is not yet compiled with xcrun metal or run on Apple hardware (g++ syntax check of its host C++ files only). Vulkan, Windows CUDA and ARM builds were not run.
• Newly converted MXFP8 files store type 63, which stock upstream does not read.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-30T21:36:10.499527Z We couldn’t complete this review. Comment @codex review to try again.	6bad20d	PR opened
🔒 Security Review	✅ Completed 2026-10-30T21:38:15.506889Z	6bad20d	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.