Repository navigation

5080deploy: upstream #676 on-demand short prefills (+ two fixes), 1080deploy and chatdnp picks - #86

Open

gdevenyi wants to merge 19 commits into

5080/40-rebuild-9c635a15from

5080/41-r5-additions

Open

5080deploy: upstream #676 on-demand short prefills (+ two fixes), 1080deploy and chatdnp picks#86
gdevenyi wants to merge 19 commits into
5080/40-rebuild-9c635a15from
5080/41-r5-additions

Commits on Oct 13, 2026
- test(moe): a short prefill chunk loads its experts through the slot cache, bit-identical to the stream
  YevheniiKotyrlo
  committed
  aae638d
- feat(moe): load a short prefill chunk's experts on demand through the slot cache
  YevheniiKotyrlo
  committed
  689f3d1
- test(moe): a dense model drops the demand-prefill threshold
  YevheniiKotyrlo
  committed
  96c7310
- test(moe): the demand-prefill threshold is refused off GPU decode
  YevheniiKotyrlo
  committed
  a8f24b1
- merge #676: feat(moe): load a short prefill chunk's experts on demand through the slot cache
  
  gdevenyi
  and
  claude
  committed
  8392c72
- fix(kernel): size triton's autotune flush buffer to the device l2
  gdevenyi
  committed
  b880e7c
- test(utils): drive the async coalesced-pull test with asyncio.run
  gdevenyi
  committed
  47afdaf
- fix(engine): keep multimodal placeholder ids out of the repetition-penalty prompt mask
  gdevenyi
  committed
  c0190aa
- fix(engine): apply the repetition penalty from device state, not a per-step prompt upload
  gdevenyi
  committed
  3af6e25
- fix(server): clarify unsupported response_format error text
  
  Siarhei Berdachuk
  authored and
  gdevenyi
  committed
  1e32ddb
- fix(mm): classify image fetch errors and re-check redirect targets
  
  Avicennasis
  authored and
  gdevenyi
  committed
  9586885
- test(engine): cover presence/frequency penalties on the greedy and sampled paths
  gdevenyi
  committed
  cbe217b
- perf(model): fuse hc combine with the next block's RMSNorm via hc_combine_norm
  
  Eva (port)
  authored and
  gdevenyi
  committed
  d8d261b
- fix(kernel): bit-exact rmsnorm reduction tree in hc_combine_norm
  
  Eva (port)
  authored and
  gdevenyi
  committed
  7ec5a11
- test(kernel): bitwise pin for hc_combine_norm against the split path
  
  Eva (port)
  authored and
  gdevenyi
  committed
  c039107
- fix(server): soft-accept OpenAI response_format json_object/json_schema
  
  Siarhei Berdachuk
  authored and
  gdevenyi
  committed
  0aa00e4
- refactor(server): drop response_format system-hint injection
  
  Siarhei Berdachuk
  authored and
  gdevenyi
  committed
  97758b6
- test(engine): drop the pre-volta sampling penalty test, whose torch fallback this branch does not carry
  
  gdevenyi
  and
  claude
  committed
  941fe3b
- fix(moe): align on triton past sgl_kernel's 1024-expert limit
  
  gdevenyi
  and
  claude
  committed
  8eb353b

Commits

Commits on Oct 13, 2026