Skip to content

fix: release completed KDA prefill buffers - #595

Closed
simone-chen wants to merge 1 commit into
simonec/vllm031-scale-qualificationfrom
simonec/vllm031-kda-buffer-lifetime
Closed

simone-chen wants to merge 1 commit into
simonec/vllm031-scale-qualificationfrom
simonec/vllm031-kda-buffer-lifetime

Conversation

@simone-chen

Copy link
Copy Markdown
Contributor
  • Kimi KDA prefill retained the completed convolution input while allocating independent core inputs, and retained FlashKDA output/state/workspace buffers into the next point.
  • Release the convolution input and all three views after its timing/logging completes. Clear native buffers in each point’s cleanup, including after failures.
  • Preserve native workspace sizing, callback grids, dispatch, input generation, timing settings and partial-row/failure accounting. Oversized native workloads still fail explicitly; this change does not make the whole grid fit on B200.
  • Stacked on fix: adapt vLLM 0.31 collector runtime contracts #588 (simonec/vllm031-scale-qualification); no runtime or collection-request pins changed.

Validation

  • 58 vLLM/SGLang KDA contract tests passed; Ruff lint and formatting passed.
  • Weak-reference lifetime tests execute the collector with both prefill backends and verify continuation after a core failure, unchanged timing parameters and retained partial convolution rows.
  • Mutation checks fail when either cleanup is removed: four failures for retained convolution inputs; two FlashKDA failures for retained native buffers.
  • GPU collection has not run for this change. Full CI will be requested for the exact PR head.

Signed-off-by: Simone Chen <simonec@nvidia.com>
@coderabbitai

coderabbitai Bot commented Oct 26, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@simone-chen

Copy link
Copy Markdown
Contributor Author

/ok to test 1f7451a

@simone-chen

Copy link
Copy Markdown
Contributor Author

Superseded by consolidated recurrent/MoE compatibility PR #605, stacked on attention #604. The MoE cleanup is preserved once, including the duplicate in #603. Communication remains #591. Original branches and campaign commit references are preserved; closing without merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant