Repository navigation

feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype… - #354

Open

ArqAlice wants to merge 14 commits into

FlashML-org:mainfrom

ArqAlice:feat/fp8-quantization

Open

feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype…#354
ArqAlice wants to merge 14 commits into
FlashML-org:mainfrom
ArqAlice:feat/fp8-quantization

Commits on Sep 2, 2026
- feat(kvcache): store the KV cache as fp8 e4m3 codes (--kv-cache-dtype fp8)
  ArqAlice
  committed
  03fb043
Commits on Sep 4, 2026
- fix(kernels): give the V tensor its own row pitch in the fp8 KV store
  
  MT-z
  and
  claude
  committed
  811ccee
- Merge pull request #1 from MT-z/fix/kv-fp8-vstore-pitch
  ArqAlice
  authored
  5b9efc5
- Merge branch 'FlashML-org:main' into feat/fp8-quantization
  ArqAlice
  authored
  3e5bbdd
Commits on Sep 5, 2026
Commits on Sep 6, 2026
- perf(kernel): apply the fp8 KV dequant scale after the dot, not to the tile
  
  naerymdan
  and
  claude
  committed
  05861fb
- perf(kernel): size the extend tile from the KV cache element size
  
  naerymdan
  and
  claude
  committed
  73ca76c
Commits on Sep 7, 2026