feat: add kimi-k3 (kimi-linear) serving support - #308

Merged

Andrewxu313 merged 46 commits into

mainfrom

tairanxu/k3-prefill-model

Aug 12, 2026

Commits on Aug 12, 2026

feat(kimi-linear): model-side serving stack
TairanXU
committed
007b6fa
docs: NUMA dev-header prerequisite and troubleshooting guide
TairanXU
committed
73493f3
build(kernels): package conv1d wrapper in wheel
TairanXU
committed
5de6d41
fix(kimi-linear): M1 model-side fixes (F1-F4, FA3 import, KDA slots)
TairanXU
committed
ec7b35c
fix(kimi-linear): M1 planner overrides (F6)
TairanXU
committed
69697d8
fix(kimi-linear): remove spurious silu in _kda_project gate stacks
TairanXU
committed
aa82835
fix(kimi-linear): guard MoE router gate against empty DP ranks
TairanXU
committed
e52172e
fix(kimi-linear): use the checkpoint's own chat template
TairanXU
committed
0e25962
fix(kimi-linear): apply no-silent-fallback policy to tokenizer
TairanXU
committed
96ebdfc
feat(kimi-linear): resident-EP decode wiring and D2 tolerance tests
TairanXU
committed
be8b201
feat(kimi-linear): wrapper KDA state pools become manager views
TairanXU
committed
435f628
test(kimi-linear): use indexed device in M5.1 tests
TairanXU
committed
041fed9
feat(kimi-linear): bucketed decode-graph adapter (phase A)
TairanXU
committed
af7c68c
test(kimi-linear): compare KDA state per slot, not whole pool
TairanXU
committed
51d8ee1
feat(kimi-linear): install decode-graph adapter in eager mode too
TairanXU
committed
78dde83
fix(kimi-linear): include K-cache geometry in the capture signature

SeedEverything328
authored and
TairanXU
committed
2091dcf
test(kimi-linear): drive a K-cache geometry change end to end

SeedEverything328
authored and
TairanXU
committed
120331c
test(kimi-linear): shrink the K cache so the pointer is reused

SeedEverything328
authored and
TairanXU
committed
a98533b
test(kimi-linear): give the fake KV cache shrink headroom

SeedEverything328
authored and
TairanXU
committed
a58272f
feat(k3): weight-name reconciler and MXFP4 tensor map
TairanXU
committed
8e451ff
feat(k3): vendored tokenizer metadata, pinned chat rendering
TairanXU
committed
01f9c54
feat(k3): M2 strict config, eager prefill decoder, CPU parity suite
TairanXU
committed
aed70e0
test(k3): judge the KDA kernel by fla's own metric
TairanXU
committed
ad63774
test(k3): derive Part D's memory bound from the chunk model

SeedEverything328
authored and
TairanXU
committed
5930122
test(k3): make Part E test composition, not kernels

SeedEverything328
authored and
TairanXU
committed
7193002
test(k3): share kernels into Part B; repair the seam test

SeedEverything328
authored and
TairanXU
committed
c4dc674
fix(k3): packed (varlen) prefill — a plain causal mask corrupts it
TairanXU
committed
b4316c3
feat(kimi-linear): stream attn/kda/shared; require num_experts
TairanXU
committed
6f3d6ab
fix(kimi-linear): make the streamed-ring plan reachable without is_k3
TairanXU
committed
597f6bc
feat(kimi-linear): serving MoE runs K3's experts in the latent space
TairanXU
committed
11fafcc
fix(kimi-linear): look the skeleton up by its checkpoint name
TairanXU
committed
8ddb97c
feat(k3): experts that fit the MXFP4 tensors the engine serves
TairanXU
committed
eb582a1
feat(k3): wire block attention residuals into the serving path
TairanXU
committed
e61ea06
test(k3): verify the MXFP4 expert seam end to end
TairanXU
committed
a72f89b
fix(k3): refuse decode CUDA graphs under block attention residuals
TairanXU
committed
7c736ad
test(k3): judge the MXFP4 expert chain against BF16 truth
TairanXU
committed
2aa1b36
docs(k3): point the repack docstring at the test that actually gates it
TairanXU
committed
9a09602
feat(k3): integrate the four serving-path tracks
TairanXU
committed
5628754
feat(kimi-linear): model half of the K3 block-residual carry
TairanXU
committed
4e7a39a
fix(kimi-linear): cast conv1d weights to activation dtype for K3
TairanXU
committed
f79ec96
perf(kimi-linear): kill the KDA conv1d layout round-trip
TairanXU
committed
7ab26e6
perf(kimi-linear): tile KimiMLP over tokens
TairanXU
committed
4ffeeb1
perf(kimi-linear): preallocate the block_residual buffer
TairanXU
committed
8bb7ff7
perf(kimi-linear): segment the KDA prefill
TairanXU
committed
5d3d3eb
perf(kimi-linear): integrate the five K3 prefill-memory fixes
TairanXU
committed
bd289b3
test(k3): pin the block_residual release over a carried prefill
TairanXU
committed
711117f