Repository navigation

5080deploy 1/9: merge upstream #408, nvfp4 KV cache quantization - #9

Open

gdevenyi wants to merge 22 commits into

mainfrom

5080/01-nvfp4-kv-408

Open

5080deploy 1/9: merge upstream #408, nvfp4 KV cache quantization#9
gdevenyi wants to merge 22 commits into
mainfrom
5080/01-nvfp4-kv-408

Commits on Sep 2, 2026
- feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype fp8)
  ArqAlice
  committed
  03fb043
Commits on Sep 4, 2026
- fix(kernels): give the V tensor its own row pitch in the fp8 KV store
  
  MT-z
  and
  claude
  committed
  811ccee
- Merge pull request #1 from MT-z/fix/kv-fp8-vstore-pitch
  ArqAlice
  authored
  de5d250
- Merge branch 'FlashML-org:main' into feat/fp8-quantization
  ArqAlice
  authored
  e9aac86
Commits on Sep 5, 2026
Commits on Sep 6, 2026
- perf(kernel): apply the fp8 KV dequant scale after the dot, not to the tile
  
  naerymdan
  and
  claude
  committed
  6a66ee6
- perf(kernel): size the extend tile from the KV cache element size
  
  naerymdan
  and
  claude
  committed
  ba148eb
Commits on Sep 7, 2026
Commits on Sep 8, 2026
- feat(kvcache): support nvfp4 latent dsa kv
  ArqAlice
  committed
  1deebb1
Commits on Sep 13, 2026
- Merge branch 'FlashML-org:main' into feat/nvfp4-kv-quantization
  ArqAlice
  authored
  1bb1259
Commits on Sep 23, 2026
- merge #408: nvfp4 kv quantization
  gdevenyi
  committed
  84fbdc4
- test(kvcache): give the qsa fp8 pool fakes the mrope flag main now reads
  gdevenyi
  committed
  a88f041