[Bug] CUTEDSL (b12x) NVFP4 MoE on SM120/SM121: weight-scale conversion underflows into E4M3 subnormals, 8-10% error per MoE layer #21461

System info

• nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc1 (commit 0372799), FlashInfer 0.6.18 b12x kernels
• DGX Spark, GB10 (SM121), 128 GB unified memory
• Model: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 rev 59b87216 (moe_config.backend: CUTEDSL, as the model card's DGX Spark recipe sets)
• The code is unchanged on current main (tensorrt_llm/_torch/moe/fused_moe/quantization.py, NVFP4CuteDslB12xFusedMoEMethod.transform_weights).

What happens

The CUTEDSL backend gives 8.1-9.5% relative error per MoE layer on this checkpoint's real routed experts (layer 1: hidden 1024, intermediate 2688, 512 experts, top-22, relu2, W4A4 NVFP4) at 1, 2, 4, 8, 17 and 64 tokens. FlashInfer's static kernel fed ModelOpt's original scales gives 0.2-0.3% on the same inputs.

Error is relative to b12x's moe_reference_nvfp4 arithmetic. CUTEDSL serves batches under 64 tokens, which covers every decode step and every speculative verify step; prefill goes to CUTLASS.

Root cause

transform_weights multiplies the E4M3 block scales by weight_scale_2 (about 2e-5 on this checkpoint) and casts the product back to E4M3:

w1_sf_fp8 = (w1_sf_fp8_norm * bcast1).to(torch.float8_e4m3fn)

The products (about 0.004) land in E4M3's subnormal range (below 2^-6) and lose most of their precision. On layer 1, 0.8% of FC1 block scales become exactly 0 and the mean scale error is 14.1%; layers 45 and 87 show 8.1-8.4% mean scale error (CPU-only check).

A newer FlashInfer alone does not fix this, since the conversion happens before the kernel. Keeping the original block scales and passing weight_scale_2 through the alphas roughly halves the error in our layer-level test (to 2-6%), so it is only a partial fix.

Impact and workaround

On our small quality suite the end-to-end effect was within noise (GSM8K 46/50 CUTEDSL vs 48/50 CUTLASS, HumanEval 30/30 both, reasoning off), so we don't claim a measured quality drop. Workaround: moe_config.backend: CUTLASS. CUTEDSL was not affected on B200 in our runs; this path is the SM12x b12x one.

Repro

Standalone script (synthetic experts, or --checkpoint <path> --layer 1 for real ones): compares b12x through TRT-LLM's conversion (wrapper-trt), with unmodified scales (wrapper-exact), and each forced backend against one reference.

python3 b12x_moe_repro.py --checkpoint /models/super-vl-nvfp4 --layer 1 --rows 1,8,64 --activations relu2

Script: https://gist.github.com/jvpflum/00cce032d1662cb25d38367553bbee94

Generated by Claude Code