graph: six fusions for the hybrid-attention decode graph (-25% ops/token; Metal 2B +7.7%, 27B +2.6% decode; CUDA 9B +2.5%) - #165

graph: six fusions for the hybrid-attention decode graph (-25% ops/token; Metal 2B +7.7%, 27B +2.6% decode; CUDA 9B +2.5%)#165

khosravipasha merged 9 commits into

prismPrismML-Eng/llama.cpp:prismfrom

perf/metal-fwht-signedPrismML-Eng/llama.cpp:perf/metal-fwht-signedCopy head branch name to clipboard