Repository navigation

feat(ds41): two-box expert-parallel serving - #826

Merged

davide221 merged 42 commits into

mainfrom

feat/ds41-cluster

Oct 12, 2026

Merged

feat(ds41): two-box expert-parallel serving#826
davide221 merged 42 commits into
mainfrom
feat/ds41-cluster

Commits on Oct 8, 2026
- perf(ggml-cuda): fused, bit-exact DeepSeek V4.1 decode kernels
  
  mrciffa
  and
  claude
  committed
  3d44f15
- perf(moe): fused routed combine and q8_1 inputs for the hybrid expert owners
  
  mrciffa
  and
  claude
  committed
  3525192
- fix(spec): the verify-width controller prices widths it has not measured
  
  mrciffa
  and
  claude
  committed
  af2e85f
- feat(cluster): two-box runtime with a raw-RDMA decode exchange
  
  mrciffa
  and
  claude
  committed
  d8f5c54
- feat(ds41): expert-parallel DeepSeek V4.1 across two boxes
  
  mrciffa
  and
  claude
  committed
  9f350f0
- perf(ggml-cuda): one launch for an RDNA4 Q8_0 gate/up pair with SwiGLU-DS4
  
  mrciffa
  and
  claude
  committed
  228b5bf
- perf(ggml-cuda): grouped ROCmFP2 expert products use the prefetching loop
  
  mrciffa
  and
  claude
  committed
  40c4808
- fix(ggml-cuda): CUDA builds of the gate/up pair and the grouped prefetch
  
  davide221
  and
  claude
  committed
  4f2c693
- test(ds4): the whole-row-lane slice is bit-exact on HIP, close on CUDA
  
  davide221
  and
  claude
  committed
  a49bc3e
- fix(cluster): worker exit, two unparsed flags, help text, watchdog at 0
  
  davide221
  and
  claude
  committed
  53d673d
Commits on Oct 9, 2026
- fix(cluster): bounded fast-reduce handshake, stall clock, RDMA-error release
  
  davide221
  and
  claude
  committed
  bcbfc8c
- fix(ds4): ranks agree on fast reduce and the drafter; unaligned slices fail the load
  
  davide221
  and
  claude
  committed
  4e6cdcb
- refactor(ds4): drop dead cluster helpers and a prefill opt-in that measured slower
  
  davide221
  and
  claude
  committed
  8108b66
- fix(ggml-cuda): the DS4 HC boundary barrier survives CUDA graph replay
  
  davide221
  and
  claude
  committed
  3a89de1
- Merge main into feat/ds41-cluster
  
  mrciffa
  and
  claude
  committed
  8461443
- fix(ggml-cuda): a single-token decode no longer aborts the RDNA4 gate/up pair
  
  mrciffa
  and
  claude
  committed
  ab496e1
- perf(ds41): two-box prefill pipelines its bands when they are long enough
  
  authored and
  davide221
  committed
  5306ca5
Commits on Oct 11, 2026
- perf(ds41): two-box prefill runs the R9700's experts on the grouped kernels too
  
  mrciffa
  and
  claude
  committed
  11a92e0
- perf(ggml-cuda): ROCmFP2 expert prefill on the RDNA4 matrix cores
  
  mrciffa
  and
  claude
  committed
  ffc9649
- perf(cluster): an 8-bit prefill exchange (opt-in)
  
  mrciffa
  and
  claude
  committed
  b387125
- perf(ds41): cluster prefill exchanges 8-bit blocks by default
  
  mrciffa
  and
  claude
  committed
  5e94487
- perf(ggml-cuda): a prefill band's HC mix in one pass over the residual
  
  mrciffa
  and
  claude
  committed
  c8d1267
- perf(ds41): a cluster band's Engram rows cross the link once
  
  mrciffa
  and
  claude
  committed
  60364d6
- Merge main into feat/ds41-cluster
  
  davide221
  and
  claude
  committed
  974e5be
- fix(cluster): rank 0 decodes in the mode it broadcast
  
  davide221
  and
  claude
  committed
  6a28b6a
- refactor(ds4): drop the verify stage-batch and graph-prebuild experiments
  
  davide221
  and
  claude
  committed
  cb63056
- fix(cluster): shard the shared expert by default
  
  davide221
  and
  claude
  committed
  3645375
- fix(ds4): verify-width costs carry over per context range
  
  davide221
  and
  claude
  committed
  fcfe8d5
- fix(cluster): one wire format and one lean rule for every fast-reduce path
  
  davide221
  and
  claude
  committed
  190676b
- fix(cluster): a failed request answers before rank 0 exits
  
  davide221
  and
  claude
  committed
  e28cf0f
- fix(ds4): async Engram keys only with the hybrid graph's broadcast
  
  davide221
  and
  claude
  committed
  9c53761
- fix(ggml-cuda): masked routes, resident split boundary, exact sink column
  
  davide221
  and
  claude
  committed
  e5f6405
Commits on Oct 12, 2026
- refactor(ggml-cuda): remove ggml_backend_cuda_keepalive
  
  davide221
  and
  claude
  committed
  46e9fbe
- fix(ggml-cuda): fuse into MMVQ only where mul_mat would run MMVQ
  
  davide221
  and
  claude
  committed
  71c78c3
- fix(ggml-cuda): skinny F16 matvec on HIP RDNA4 only
  
  davide221
  and
  claude
  committed
  617c75e
- refactor(ds4): drop verify input timing, pass the vocab split to the builder
  
  davide221
  and
  claude
  committed
  baac939
- refactor(ds4): drop kill switches whose on path is the default
  
  davide221
  and
  claude
  committed
  2144c38
- docs(ds4): drop benchmark numbers and repeated attribution from comments
  
  davide221
  and
  claude
  committed
  3e6bf26
- refactor(cluster): drop dead DS4 cluster helpers and research comments
  
  davide221
  and
  claude
  committed
  0972e72
- refactor(cluster): remove the --cluster-verify-hash probe
  
  davide221
  and
  claude
  committed
  de9b185
- refactor(cluster): drop unused head state, the HTTP view header and stale switches
  
  davide221
  and
  claude
  committed
  effc0c8
- refactor(cluster): one decode exchange path in the fast reduce
  
  davide221
  and
  claude
  committed
  1996c81

Commits

Commits on Oct 8, 2026

Commits on Oct 9, 2026

Commits on Oct 11, 2026

Commits on Oct 12, 2026