Repository navigation
prism-v6: fix Q2_0_g128 ARM link failure and Metal shader compile - #113
Merged
Merged
Conversation
github-actions
Bot
added
documentation
…8 dot type_traits_cpu references ggml_vec_dot_q2_0_g128_q8_0 unconditionally, but the aarch64 section of arch-fallback.h never aliased it to the generic implementation, so every ARM build of prism-v6 failed to link. Every other arch without a native g128 dot already carries this exact line.
kernel_mul_mv_q2_0_g128_f32_impl lost its template<int nr0, typename args_t> prefix in the g128 port, so the Metal library failed to compile at runtime (unknown type args_t, undeclared nr0) on every Apple device, for every model. Restores the same declaration shape as the q2_0 g64 twin. Verified on M5 Pro: test-quantize-fns green, test-backend-ops MUL_MAT 45/45 g128 cases + MUL_MAT_ID/GET_ROWS/CPY green, --pure Q2_0_G128 real-model greedy output byte-identical to g64, tg128 +0.7% at 5.6% smaller file.
khosravipasha
force-pushed
the
prism-v6-g128-arm-metal-fixes
branch
from
August 11, 2026 01:36
dc56f19 to
c5aae60
Compare
Collaborator
|
@bri-prism fixed the history mess I made, is that correct now? seems only two line changes. |
There was a problem hiding this comment.
Pull request overview
Fixes Q2_0_G128 CPU linking on ARM and Metal shader compilation.
Changes:
- Adds the ARM generic dot-product alias.
- Restores the missing Metal template declaration.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
ggml/src/ggml-cpu/arch-fallback.h |
Adds the ARM fallback alias; RISC-V remains unresolved. |
ggml/src/ggml-metal/ggml-metal.metal |
Restores the Q2_0_G128 kernel template. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| #define ggml_gemm_q8_0_4x8_q8_0_generic ggml_gemm_q8_0_4x8_q8_0 | ||
| #elif defined(__aarch64__) || defined(__arm__) || defined(_M_ARM) || defined(_M_ARM64) | ||
| // quants.c | ||
| #define ggml_vec_dot_q2_0_g128_q8_0_generic ggml_vec_dot_q2_0_g128_q8_0 |
Q2_0_g128 has a native dot only on x86 (AVX-512-VNNI); every other arch must alias ggml_vec_dot_q2_0_g128_q8_0_generic -> ...q8_0 so the symbol the CPU traits reference resolves. The RISC-V block was the last one missing it (GENERIC/aarch64/POWERPC/loongarch64/s390x/wasm already had it), which left RISC-V builds with an undefined-symbol link failure. Addresses Copilot review on #113. Now every non-x86 arch is covered; x86 keeps the native impl.
MasterCool4389
pushed a commit
to MasterCool4389/Built_PrismML-ENG-CUDA_llama.cpp
that referenced
this pull request
Sep 18, 2026
…ismML-Eng#113) * ggml-cpu: add missing aarch64 generic-fallback alias for the Q2_0_g128 dot type_traits_cpu references ggml_vec_dot_q2_0_g128_q8_0 unconditionally, but the aarch64 section of arch-fallback.h never aliased it to the generic implementation, so every ARM build of prism-v6 failed to link. Every other arch without a native g128 dot already carries this exact line. * metal: fix Q2_0_g128 mul_mv kernel template declaration kernel_mul_mv_q2_0_g128_f32_impl lost its template<int nr0, typename args_t> prefix in the g128 port, so the Metal library failed to compile at runtime (unknown type args_t, undeclared nr0) on every Apple device, for every model. Restores the same declaration shape as the q2_0 g64 twin. Verified on M5 Pro: test-quantize-fns green, test-backend-ops MUL_MAT 45/45 g128 cases + MUL_MAT_ID/GET_ROWS/CPY green, --pure Q2_0_G128 real-model greedy output byte-identical to g64, tg128 +0.7% at 5.6% smaller file. * ggml-cpu: add Q2_0_g128 generic-fallback alias for RISC-V Q2_0_g128 has a native dot only on x86 (AVX-512-VNNI); every other arch must alias ggml_vec_dot_q2_0_g128_q8_0_generic -> ...q8_0 so the symbol the CPU traits reference resolves. The RISC-V block was the last one missing it (GENERIC/aarch64/POWERPC/loongarch64/s390x/wasm already had it), which left RISC-V builds with an undefined-symbol link failure. Addresses Copilot review on PrismML-Eng#113. Now every non-x86 arch is covered; x86 keeps the native impl. --------- Co-authored-by: Pasha Khosravi <khosravipasha@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two one-line fixes for the Q2_0_g128 port on prism-v6:
ggml-cpu/arch-fallback.h: add the missing aarch64 alias mappingggml_vec_dot_q2_0_g128_q8_0to its generic implementation. Without it every ARM build of prism-v6 fails to link, sincetype_traits_cpureferences the symbol unconditionally. Every other arch without a native g128 dot already carries this line.ggml-metal.metal: restore thetemplate<int nr0, typename args_t>prefix onkernel_mul_mv_q2_0_g128_f32_impl. It was dropped in the port, so the Metal library failed to compile at runtime on every Apple device, for every model.Why
Closes out the "Metal: ported, UNVERIFIED" line on the prism-v6 status board. The port could not previously build on ARM or run on Metal at all.
How verified (M5 Pro)
test-quantize-fnsgreen including q2_0_g128test-backend-ops: MUL_MAT 45/45 g128 cases OK across 3 backends; MUL_MAT_ID, GET_ROWS, CPY green (covers the mul_mv 8-sub-block and nl=8 risk spots)--pure Q2_0_G128, greedy output byte-identical to the same model at g64; tg128 280.8 vs 278.9 t/s (+0.7%) at a 5.6% smaller file