Repository navigation

5080deploy 9/9: merge upstream #389 (+#385), load-time FP8 dense projections - #17

Open

gdevenyi wants to merge 296 commits into

5080/08-moe-stats-231from

5080deploy

Open

5080deploy 9/9: merge upstream #389 (+#385), load-time FP8 dense projections#17
gdevenyi wants to merge 296 commits into
5080/08-moe-stats-231from
5080deploy

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 17, 2026
Commits on Sep 19, 2026
Commits on Sep 20, 2026
- fix(server): clamp /v1/models context_length to the allocated KV pool
  
  oscar-investmatic
  and
  claude
  committed
  b9b8f1b
Commits on Sep 21, 2026
- fix(moe): keep eager fallback and a bounds guard in the prefill invalidation helper
  
  chrisqian
  authored and
  alvarorsouza-arch
  committed
  f9947db
- test(moe): check the prefill invalidation kernel against the eager reference on cuda
  
  chrisqian
  authored and
  alvarorsouza-arch
  committed
  dc9216c
Commits on Sep 24, 2026
- fix(runtime): correct overlap token limits and cache safety (#546)
  
  jason-fxz
  and
  andy-yang-1
  authored
  cee23bf
Commits on Sep 25, 2026
- refactor(ple): extract a shared disk row store (#548)
  jason-fxz
  authored
  f5b9700
Commits on Sep 26, 2026
- feat(rocm): add RDNA3 and RDNA4 runtime foundation (#132)
  
  authored
  0d652e7
Commits on Sep 28, 2026
- fix(engine): reject fa attention backend below sm_90 at config time
  SanjayS45
  committed
  c2091c6
- fix(engine): drop the tutorial comment on the fa sm_90 gate
  SanjayS45
  committed
  f29be24
Commits on Sep 29, 2026
- fix(server): keep a silent stream alive and notice a client that left
  YevheniiKotyrlo
  committed
  6e63b8f
- perf(qsa): reduce block top-k candidates in levels past 16 splits (#571)
  YevheniiKotyrlo
  authored
  5031e3d
Commits on Sep 30, 2026
- feat(install): record the install root as FREETOKEN_HOME in environment.d (#576)
  jason-fxz
  authored
  eba7120
Commits on Oct 1, 2026
Commits on Oct 3, 2026
- fix(checkpoint): refuse converting into a directory that already holds an FTW
  Avicennasis
  committed
  1d5ce54
Commits on Oct 4, 2026
- feat(moe): serve NVFP4 and MXFP4 experts fused through the expert banks (#601)
  jason-fxz
  authored
  189c8ad
Commits on Oct 5, 2026
Commits on Oct 6, 2026
- fix(server): leak no semaphores when a signal stops the server
  YevheniiKotyrlo
  committed
  a8498c7
Commits on Oct 7, 2026
- perf(moe): Improved `fast_index_copy_multi_jit` for NVLink-C2C hosts (#610)
  SuperHotDogCat
  authored
  c1a504a
- perf(moe): spin-wait the cpu moe pool between layers (#619)
  
  jason-fxz
  and
  shubham3-ucb
  authored
  1c183e7
- feat(deepseek-v41): support text and vision serving (#603)
  
  jason-fxz
  and
  andy-yang-1
  authored
  127c856
- ci: auto-label amd issues and prs by title (#620)
  jason-fxz
  authored
  377a3bc
Commits on Oct 8, 2026
Commits on Oct 9, 2026
Commits on Oct 10, 2026
Commits on Oct 11, 2026
- fix(sampling): apply presence and frequency penalties (#527)
  taking-lying-flat
  authored
  c818639
Commits on Oct 12, 2026
- feat(cpu-moe): serve block-fp8 experts on the cpu and hybrid backends (#662)
  
  authored
  95ac86f
Commits on Oct 13, 2026
Commits on Oct 14, 2026
- refactor(cpu-moe): split the executor into weight formats and gpu sync backends (#670)
  
  jason-fxz
  and
  zihaomu
  authored
  2010271
- perf(moe): split hybrid decode routes inside the ensure kernel (#685)
  jason-fxz
  authored
  f02fbb1
Commits on Oct 15, 2026
Commits on Oct 16, 2026
Commits on Oct 17, 2026
Commits on Oct 18, 2026
Commits on Oct 19, 2026
Commits on Oct 20, 2026
Commits on Oct 21, 2026
- perf(engine): make the capture-time cuda graph upload switchable with FREETOKEN_GRAPH_UPLOAD
  
  gdevenyi
  and
  claude
  committed
  77874d2
- perf(qsa): score prefill rows in tiles of 16 query rows per program
  
  gdevenyi
  and
  claude
  committed
  aab6bdd
- test(fp8): pin a dynamic fp8 linear row against the other rows of its batch
  
  gdevenyi
  and
  claude
  committed
  fa5e0ca
- fix(fp8): scale dynamic fp8 activations per row so batched sequences stay independent
  
  gdevenyi
  and
  claude
  committed
  6c381c3

Commits

Commits on Sep 17, 2026

Commits on Sep 19, 2026

Commits on Sep 20, 2026

Commits on Sep 21, 2026

Commits on Sep 24, 2026

Commits on Sep 25, 2026

Commits on Sep 26, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 1, 2026

Commits on Oct 3, 2026

Commits on Oct 4, 2026

Commits on Oct 5, 2026

Commits on Oct 6, 2026

Commits on Oct 7, 2026

Commits on Oct 8, 2026

Commits on Oct 9, 2026

Commits on Oct 10, 2026

Commits on Oct 11, 2026

Commits on Oct 12, 2026

Commits on Oct 13, 2026

Commits on Oct 14, 2026

Commits on Oct 15, 2026

Commits on Oct 16, 2026

Commits on Oct 17, 2026

Commits on Oct 18, 2026

Commits on Oct 19, 2026

Commits on Oct 20, 2026