[zihaomu]

Relates to #541.

Add two optional native HIP M=1 block-FP8 dense decode and routed MoE decode kernels for R9700. They use BF16 activations and FP8 weights (W8A16), inherit FT LinearKernel / MoEKernel packing, and consume existing ExpertView banks. FTW compatibility, CPU experts, hybrid routing, shared experts and cache ownership remain with FT.

FREETOKEN_USE_ARK=0 is the default. Enabling it selects only measured eager shapes in the exact-environment profile; unsupported inputs retain independent FT primitives. Graph stays FT. Dense M>1 stays FT W8A8; MoE prefill and other operator families are outside this PR. Both HIP units and the Torch bridge build from packaged source.

Validation

• 41 passed, 42 skipped across ARK, FP8 MoE, FTW and quant-config tests. Coverage includes output/input/alias contracts, routing/scales, bias, actual native dispatch, fallback, bank/Kernel.apply/FTW and direct Graph replay with updated inputs.
• Actual Qwen3 calls: 96 native dense kernels and 48 pairs of native MoE gate/down kernels in one decode step. All 144 same-input model-layer comparisons pass; max NRMSE 0.000735.
• Isolated packaged-source smoke: all 16 files match; both kernels execute after a fresh HIP build.
• Independent bounded generation: FT 7/8, ARK 7/8, no new failure; both fail the decimal-comparison case. This is not a standardized accuracy evaluation.

Eager model A/B

Qwen/Qwen3-30B-A3B-FP8, local FTW, BF16 activation/KV, offload, 20GiB cache, concurrency 1, 256 generated tokens. Three fresh-process paired rounds, alternating service/context order, with per-context warmup. Baseline: FT compute at fe5a5ec9c5644e266527de5afb9f1156b9bdcabb, switch off. Final cleaned sources were remeasured; formal services have no profiler or diagnostic wrapper.

Input tokens	FT median tok/s	ARK median tok/s	Change
512	31.41	40.27	+28.21%
32768	31.89	39.77	+24.71%

Round	512 FT / ARK	32768 FT / ARK
1	31.63 / 40.27	31.76 / 39.77
2	31.39 / 40.49	32.02 / 40.21
3	31.41 / 39.00	31.89 / 38.18

All three pairs improve at both lengths. All 12 formal requests independently verify usage, 256 nonempty token events and zero prefix reuse. Decode tok/s is 255 / (last token time - first token time). Free-running continuations differ; results include changed cache traffic and shared-host variation. No Graph or sustained-CPU hybrid speedup is claimed.

Environment and commands

One Radeon AI PRO R9700 32GiB (gfx1201, 23:00.0), one EPYC 9334 socket (0-31,64-95, memory interleave 2-3) on a shared dual-socket host. Torch 2.14.0+rocm10.1.0 / HIP 7.16.26385, Triton 3.8.0; Ubuntu 24.04.1, kernel 7.0.0-38, amdgpu 7.1.9.31600000. Other environments require recalibration.

python -m pytest tests/kernels/test_ark_fp8.py tests/kernels/test_fp8_blockscale_moe.py tests/checkpoint/test_ftw_weights.py tests/models/test_quant_config.py -q
python -m benchmarks.ark.calibrate --output /tmp/ark-calibration.json --write-profile python/freetoken/kernel/ark/common/validated_shapes.json

Expose one R9700. Run the following with FREETOKEN_USE_ARK=0 and =1, using identical 512/32768-token completion prompts, temperature 0, ignore_eos and 256 output tokens:

FREETOKEN_USE_ARK=1 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 \
taskset -c 0-31,64-95 numactl --interleave=2-3 ft serve \
  --model-path /path/to/Qwen3-30B-A3B-FP8-FTW --dtype bfloat16 \
  --attention-backend triton --moe-strategy offload --moe-cpu-threads 17 \
  --moe-cache-mem 20GiB --moe-cache-scope unified --moe-cache-split equal \
  --num-tokens 33058 --page-size 1 --max-seq-len-override 33024 \
  --max-prefill-length 8192 --max-running-requests 1 --cuda-graph-max-bs 0 \
  --max-output-tokens 256 --memory-ratio 0.9 --expert-load serial \
  --cache-type naive --enable-cache-report --sampling-defaults none \
  --reasoning-parser off --host 127.0.0.1 --port 19673

Contracts, build and selection.