Repository navigation

Sparse decode attention: per-token reference lifecycle for async self-speculation - #157

Open

kunkunblueberry wants to merge 24 commits into

InferMatrixLab:mainfrom

kunkunblueberry:kunkun/rfc156-sparse-decode-async

Open

Sparse decode attention: per-token reference lifecycle for async self-speculation#157
kunkunblueberry wants to merge 24 commits into
InferMatrixLab:mainfrom
kunkunblueberry:kunkun/rfc156-sparse-decode-async

Commits on Oct 27, 2026
- sparse decode attention: per-token reference lifecycle for async self-speculation
  kunkunblueberry
  committed
  e2f9f73
Commits on Oct 28, 2026
Commits on Oct 29, 2026
- Merge sparse decode attention ordinary decode and synchronous speculation
  kunkunblueberry
  committed
  efe5bc7
- Batch equal-length sparse decode rows and retain scalar oracle
  
  0z5a
  authored and
  kunkunblueberry
  committed
  0aaab79
- Preserve scalar cuBLAS reduction shapes when batching reference rows
  
  0z5a
  authored and
  kunkunblueberry
  committed
  903e101
- Prepare batched views once before identical per-row bmm calls
  
  0z5a
  authored and
  kunkunblueberry
  committed
  a7859a5
- sparse decode attention: restore the dense-output contract in the batched path
  kunkunblueberry
  committed
  e4f8988
- benchmarks: dense-baseline context sweep and memory breakdown
  kunkunblueberry
  committed
  f292b76
Commits on Oct 31, 2026
Commits on Nov 1, 2026