Add NVFP4 per-token quantization recipe#3045

Open

cael-ling wants to merge 20 commits into

NVIDIA:mainfrom

cael-ling:feature/nvfp4-per-token-recipe

Commits on May 27, 2026

Add NVFP4 per-token GEMM, fused grouped amax, cast, tests and benches

cael-ling
and
zhongbozhu
committed
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed

Commits on May 28, 2026

Add optional col-wise RHT to NVFP4 per-token amax+quant (single + grouped)

cael-ling
and
zhongbozhu
committed
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed

Commits on May 29, 2026

Commits on May 30, 2026

Commits on May 31, 2026

Commits on Jun 2, 2026

Commits on Jun 4, 2026

Commits on Jun 8, 2026

Add opt-in 2D weight quantization for the NVFP4 per-token recipe

committed

Commits on Jun 11, 2026

Add docs and Megatron-Core example for the NVFP4 per-token recipe

committed

Commits on Jun 15, 2026