[chrisqianz]

What

Adds loading and serving support for nvidia/Qwen3.8-Flash-Next-FP4-Local, a QAD MIXED_PRECISION ModelOpt export of the already-supported Qwen4ExpForConditionalGeneration architecture. The existing NVFP4 release of the same family loads fine; this checkpoint differs in three on-disk encodings, each handled in the subsystem that owns it (no new mechanisms, no upstream behaviour change for other checkpoints):

1. Routed experts: CSF slab scales. Scales arrive as *.weight_scale.nvfp4_csf_fixed (U8) + *.weight_scale.nvfp4_csf_exceptions (U32) instead of plain e4m3 tensors. nvfp4_banks.py decodes the slab layout (per-row base + 4-bit offsets, exceptions overwritten by exact e4m3 bytes) into the usual weight_scale piece, CPU-side. nvfp4_expert_spec() auto-selects the CSF kind set when the index contains *.nvfp4_csf_fixed. Also adds the missing unsigned integer dtypes (U16/U32/U64) to _ST_DTYPE in the parallel expert loader.
2. Activation scales read out of band. This checkpoint keeps all 73728 expert input_scale tensors in the last shard, so requiring them in the stream made the first finished piece wait for the last shard while ~65 GB piled up unread (the "Loading experts" bar froze at 93%). The CSF reader now takes weights and scales from the stream and fetches activation scales per shard itself, keeping the queue shallow.
3. PLE n-gram table in FP4. ple_embedding_dtype: nvfp4 tables are packed e2m1 (80-byte rows for a 160-wide head) with per-16 e4m3 scales and a global weight_scale_2 scalar. load_ple_table loads the packed rows + scale rows into two host banks; both PLE runtimes (CPU LUT in GpuResidentTable, Triton gather in PinnedUVATable) decode e2m1 inline, with bit-exact cross-oracle tests; the disk backend rejects packed tables with a clear error and the engine falls back to the pinned backend with a warning.

Notes

• The dense side (MXFP8 projections storing their block scales as weight_scale) already works via the storage aliases added by fix(qwen4_exp): read weights under the checkpoint's quant config #759; this PR contains no dense-path changes.
• The CSF slab and packed-table byte layouts follow NVIDIA's public converter tooling for this checkpoint (read as a format reference only); the implementation is FreeToken-native.
• Supersedes feat(models): add nvidia/Qwen3.8-Flash-Next-FP4-Local checkpoint support #708 (first draft, closed).

Fixes #817

Tested

• CPU test matrix on Python 3.13 / torch 2.14.1+cu130 / transformers 5.18.0: tests/models/qwen4_exp/ + tests/layers/test_nvfp4_moe.py 92 passed (CUDA-gated tests skipped; test_ple_disk needs the _row_store extension). Plus tests/models/test_models_loader.py 12 passed with fix(models): find index-only checkpoints that keep shards in a subdirectory #819.
• End-to-end serving on RTX 5090 D 32GB (driver 595.84), Ubuntu 22.04.5, Intel Core Ultra 7 265K, 125 GB RAM:

ft checkpoint --model ~/models/NVIDIA-Qwen3.8-Flash-Next-FP4-Local/ --out ~/models/FTW-Qwen3.8-Flash-Next-FP4-Local/
ft serve --model ~/models/FTW-Qwen3.8-Flash-Next-FP4-Local/ --moe-strategy offload --moe-cache-auto --moe-prefill-hit-d2d --max-running-requests 1 --kv-reserve-tokens 262144 --ple-backend pinned --host 0.0.0.0 --served-model-name Qwen3.8-Flash-Next-FP4-Local --enable-cache-report

Ready in ~3 min from a cold start (~40 s expert build on a clean host), chat completions return tokens, radix cache hits reported.