Repository navigation

Add opt-in loop-level speculative execution - #133

Draft

YuDeng0102 wants to merge 22 commits into

InferMatrixLab:mainfrom

YuDeng0102:feat/loop-level-speculative

Draft

Add opt-in loop-level speculative execution#133
YuDeng0102 wants to merge 22 commits into
InferMatrixLab:mainfrom
YuDeng0102:feat/loop-level-speculative

Commits on Oct 18, 2026
- feat: add opt-in loop-level speculative pipeline
  yudeng
  committed
  aaa31de
- feat: add greedy early rejection pruning
  yudeng
  committed
  9586b4a
- docs: record rebased loop speculative benchmark
  yudeng
  committed
  9c0dd53
- bench: expose CUDA graph batch limit
  yudeng
  committed
  f3a6a1d
- docs: use graph256 result for c128 benchmark
  yudeng
  committed
  dbaea32
- docs: record graph256 sampling benchmark
  yudeng
  committed
  7200a75
- docs: clarify loop speculative usage and results
  yudeng
  committed
  2ce6d21
- docs: explain issue scope and pruning limits
  yudeng
  committed
  853b653
- docs: add loop pipeline schedule diagram
  yudeng
  committed
  41bf7a6
- test: document issue 43 loop validation
  yudeng
  committed
  e9e5ef1
- docs: clarify sampling results and throughput units
  yudeng
  committed
  ada1e4e
- docs: clarify aggregate decode throughput
  yudeng
  committed
  e27f9f5
- test: record filtered sampling errors
  yudeng
  committed
  3415844
Commits on Oct 20, 2026
- docs: correct memory units and summary
  yudeng
  committed
  68c724b
Commits on Oct 24, 2026
- Merge main into loop-level speculation
  
  0z5a
  and
  claude
  committed
  80f849d
- Run loop-level speculation as a planned submit/collect round
  
  0z5a
  and
  claude
  committed
  0388ea0
- Pair benchmark trials and report interval estimates
  
  0z5a
  and
  claude
  committed
  1d7c401
- Expose the CUDA graph cache limit in the benchmark
  
  0z5a
  and
  claude
  committed
  c29bad4
- Restore the filtered top-p/top-k rejection-sampling test
  
  0z5a
  and
  claude
  committed
  2a38a99
- Document the loop round, its cost model and RTX 5090 measurements
  
  0z5a
  and
  claude
  committed
  80a737d
Commits on Oct 25, 2026
- docs: keep loop diagram local
  yudeng
  committed
  3f10ece
Commits on Oct 29, 2026
- Merge upstream main and preserve loop-level speculative mode
  yudeng
  committed
  83690cd

Commits

Commits on Oct 18, 2026

Commits on Oct 20, 2026

Commits on Oct 24, 2026

Commits on Oct 25, 2026

Commits on Oct 29, 2026