[0.3] Make AMD EPYC Turin a Tier-1 CPU-only optimization target #12

Goal

Make AMD EPYC Turin / 5th Gen EPYC a Tier-1 reference platform for Quantum Runtime CPU-only inference and placement tuning.

This is a concrete specialization of the CPU-first policy in #11. The generic Runtime API remains hardware-neutral, but Turin-class EPYC systems receive first-class benchmark, topology and kernel-selection coverage because CPU-only performance is a primary product goal.

Priority

Turin is a release-quality reference platform, not merely a compatibility target.

A Runtime backend or upstream update that substantially regresses representative Turin CPU-only workloads should be treated as a performance regression even if the same update improves GPU paths.

Hardware characteristics to model

Runtime host discovery and calibration should represent, where available:

• Zen 5 / Zen 5c core class
• physical cores vs SMT threads
• AVX2 and AVX-512 capability
• AVX-512 VNNI/BF16-related backend capability where exposed
• socket count and CPU package topology
• NUMA nodes per socket (NPS)
• CCD/L3 locality where exposed by the OS
• memory channel population and measured memory bandwidth
• memory capacity and page size
• NVMe topology relative to NUMA nodes
• accelerator presence, but without making GPU availability mandatory

Do not select an optimization path from CPU family name alone. Runtime must confirm actual OS-visible capabilities and benchmark the host.

Turin-specific tuning matrix

Benchmark and retain latest-known-good results for at least:

• SMT on vs off where operator policy permits
• NPS/topology variants visible to Linux
• NUMA-local vs interleaved allocation
• thread pinning/affinity strategies
• physical-core-first scheduling
• prefill thread/batch/ubatch tuning
• decode thread tuning
• huge-page policy where supported and measurably beneficial
• mmap/page-cache behavior
• quantized matrix kernels
• MoE expert placement and prefetch
• RAM pressure and file-backed cold-weight behavior

The Runtime must not automatically change BIOS settings. It may detect/report recommended settings and benchmark the currently active configuration.

Backend candidates

Evaluate through the latest-known-good performance policy rather than hardcoding one implementation:

• llama.cpp/ggml optimized x86 CPU kernels
• AMD ZenDNN or other supported AMD-optimized paths where they materially improve inference
• BLIS/OpenBLAS or other matrix libraries where they improve prefill without harming decode
• future AMD-specific upstream optimizations with compatible licensing

The selected path may differ between prefill and decode.

Benchmark profiles

Create repeatable CPU-only benchmark profiles covering multiple unrelated model families and architectures, for example:

• small dense model
• medium dense model
• Gemma 4 E4B reference profile
• large MoE profile such as Gemma 4 26B-A4B or another supported MoE fixture
• at least one Llama/Qwen/Mistral-family model fixture

Record at minimum:

prefill_tokens_per_second
decode_tokens_per_second
time_to_first_token
resident_ram_bytes
file_backed_bytes
memory_bandwidth_class
cpu_threads
numa_policy
backend
backend_version
quantization
context_length

Real-weight benchmarks may be optional hardware jobs, while scheduler/metadata fixtures remain normal CI.

NUMA policy

Turin systems can expose substantial NUMA/topology differences, especially at high core counts or multiple sockets.

Runtime should support explicit placement policies such as:

auto
local
interleave
single_numa_node
socket_local

auto must be benchmark-driven. A high-core-count EPYC system should not blindly spread threads and memory across NUMA domains if a more local placement is faster for the selected workload.

Memory-first behavior

Turin's memory bandwidth/capacity should be treated as a major inference resource.

For CPU-only execution:

CPU compute
   |
NUMA-local RAM hot working set
   |
file-backed mmap/page cache
   |
optional NVMe cold tier

The planner should measure whether the workload is compute-bound or memory-bandwidth-bound and adapt thread count accordingly. More threads are not automatically faster once memory channels are saturated.

MoE priority

MoE CPU-only and CPU-heavy hybrid placement is a first-class Turin benchmark scenario.

Measure separately:

• shared experts
• routed experts
• active top-k experts
• expert locality
• expert prefetch
• expert working-set size
• RAM bandwidth pressure

A placement profile that activates fewer cores but improves memory locality may be preferable to maximum-thread execution.

Acceptance criteria

• Turin/5th Gen EPYC is documented as a Tier-1 CPU-only reference platform
• host profile captures EPYC/Zen5-relevant CPU and NUMA capabilities without relying on model-name guesses
• Runtime exposes measured core/thread/NUMA/memory placement data
• CPU-only conformance covers at least one dense and one MoE workload on the Turin hardware track
• prefill and decode are tuned independently
• thread-count calibration detects memory-bandwidth saturation instead of always using all logical CPUs
• NUMA-local/interleaved strategies are benchmarkable and persisted in the host profile
• backend updates can be blocked when Turin CPU-only correctness or performance regresses beyond policy tolerance
• no GPU is required for the Turin reference path
• benchmark results remain tied to backend/version/quant/context/profile so stale tuning data is not reused incorrectly

Relationship

Depends on and specializes #11 (CPU-first execution and tiered memory placement).

Feeds the generic backend/capability fusion in #8 without adding AMD-specific behavior to the public Runtime API.