Repository navigation
HIP Flash Next: a shared expert wider than the routed experts cannot load #1599
Description
On AMD Ryzen AI MAX+ 395, Radeon 8060S gfx1151 (40 CUs, 128 GB unified memory, measured 225.6 GB/s read), Ubuntu 26.04, archive HIP 7.1 (clang 21), TensorFold 7c068895, TensorFold/Qwen3.8-Flash-Next-MLX-4bit-MTP.
The fused routed+shared expert stack (zig/src/core/quant/experts.zig) requires the shared expert to have the routed experts' bit width and group size. oQ's default quantizes the shared expert at 8 bits over 4-bit routed experts, and the HIP loader then refuses the checkpoint ("does not stack with the shared expert"); the only workaround is re-quantizing the shared expert down to 4 bits.
Proposal
Load a shared expert whose format differs from the routed experts as its own projections and run it beside the routed launch, keeping the fused stack when the formats match. Each tensor keeps its own bit width and group size; no arithmetic change for checkpoints that already load.
Activity
ashhart commentedon Oct 31, 2026
Thanks @gilby. #1612 is this fix. Loading a shared expert in its own format beside the routed launch, and keeping the fused
stack when the formats match, is the right shape.
The gate:
to plain, concurrent equal to solo and resumed equal to fresh;
Please also state the shared expert's add order against the routed sum in the PR.