SVE 256 bit implementation of gemm_q6_K_8x8_q8_K kernel

[anubhavfujitsu]

Overview

This PR adds 256 bit SVE (Scalable Vector Extensions) implementation for ggml_gemm_q6_K_8x8_q8_K kernel using i8mm and vector instructions.

Additional information

By running a Q6_K quantized model of Llama-3.2-3B-Instruct, I checked the generation output.
I also verified that the perplexity matches between the NEON and SVE implementations.

Perplexity:

SVE (This PR)	NEON (OSS)
11.7587 +/- 0.46169	11.7506 +/- 0.46151

The perplexity difference between NEON and this PR is 0.008 which is significantly less than the error/standard deviation terms (~0.46 for both).

The command used to measure the perplexity is:

./llama-perplexity -m model.gguf -f wikitext-2-raw/wiki.test.raw --chunks 20

Performance Check

This PR Improves the Prompt Eval time (TTFT) of LLM Inference between ~20 to 40%, as compared to NEON (Original Version) for Llama-3.2-3B-Instruct-Q6_K.

The performance was measured on Graviton3 @ 32 cores.
Performance is improved as follows. The value is tokens/second.

Task	Threads	NEON (Original)	SVE 256 (This PR)	Speedup
PP128	4	47.53	68.88	1.4492
PP128	8	89.79	130.27	1.4508
PP128	16	156.85	216.08	1.3776
PP128	32	246.35	314.25	1.2756

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: No

Co authors: @abhijain1204fujitsu

[ggml-gh-bot]

Hi @anubhavfujitsu, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.