[danielhanchen]

Why

The Metal NVFP4 kernels the prebuilt carried came from a third-party branch that no longer merges onto upstream b12012 (upstream added F8_E4M3 exclusions next to the NVFP4 ones in ggml-metal-device.m). Without them, NVFP4 MUL_MAT, MUL_MAT_ID and GET_ROWS are refused by the Metal backend on b12012 and fall back off the GPU on macOS.

Change

The same NVFP4 Metal change, carried on top of upstream b12012 (this branch's base) with a Fix merge conflicts merge, keeping the original author's commit in the history:

• 11 Metal files, the same diff as the original apart from one line of conflict context.
• ggml-metal-device.m: upstream's F8_E4M3 exclusions stay verbatim; only the NVFP4 exclusions are removed.
• ggml-metal-device.cpp: the NVFP4 case keeps smem = 32*sizeof(float). Upstream dropped it only for MXFP4, whose kernel no longer uses a threadgroup lookup table; the NVFP4 kernel still does.

Tests

• llama_pinset.py sim with this pin added to the pin set on b12012 and on every tag from b12024 to b12032: all pins merge, pin_contract intact, merge_checks clean.
• CPU verify of the assembled tree on b12012, b12024 and b12032: gate build ok, feature_matrix 16 / 16.
• g++ syntax check of the Metal host C++ files of the assembled tree: 0 errors.

Not covered

• Not yet compiled with xcrun metal or run on Apple hardware. A macOS build plus test-backend-ops -b MTL0 for MUL_MAT, MUL_MAT_ID and GET_ROWS on NVFP4 is the remaining check.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T10:47:36.999355Z	a01140e	PR opened
🔒 Security Review	✅ Completed 2026-11-01T10:48:45.863442Z	a01140e	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a01140e16b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Guard source-row reads in the NVFP4 matvec tail

For single-vector NVFP4 multiplication where the weight matrix's row count is not divisible by four, the dispatch rounds up to a full threadgroup of two SIMD groups with two rows each, but this loop reads all those rows unconditionally. For example, a three-row matrix reads a nonexistent fourth row; a one-row matrix reads three nonexistent rows. The bounds check in the output loop occurs after these reads, and ggml_backend_metal_buffer_type_get_alloc_size() does not reserve extra weight rows. This can cause out-of-bounds GPU reads and Metal validation failures for otherwise valid tensor shapes, including the MUL_MAT_ID path that reuses this implementation. Clamp source rows or bound this loop before accessing xb.

Useful? React with 👍 / 👎.