feat(dsa): reuse full-layer top-k on shared GLM-5.2 decode layers - #311
Merged
Conversation
Andrewxu313
force-pushed
the
tairanxu/glm52-model
branch
from
August 12, 2026 21:23
301f639 to
ed8b545
Compare
Wire GLM-5.2 DSA indexer top-k reuse into the eager (non-CUDA-graph) decode selector. In build_glm5_dsa_flashmla_inputs, skip the indexer-K compute and aux-KV update on shared layers (indexer is None), reuse type(wrapper)._dsa_prev_topk_indices when skip_topk, and publish it from full layers when next_skip_topk. Hoist index_topk to a local so the sparse gather stays None-safe on shared layers. GLM-5 has no such schedule (dsa_layer_skips_topk is False for every layer), so every new branch is dead for it and its behavior is unchanged. The CUDA-graph segment builders still dereference the indexer unconditionally and are handled separately.
Andrewxu313
force-pushed
the
tairanxu/glm52-kernel
branch
from
August 12, 2026 21:24
d2797d7 to
4a26a49
Compare
Andrewxu313
marked this pull request as ready for review
August 12, 2026 21:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Wire GLM-5.2 DSA indexer top-k reuse into the eager (non-CUDA-graph) decode
selector. In
build_glm5_dsa_flashmla_inputs, skip the indexer-K compute andaux-KV update on shared layers (
indexer is None), reusetype(wrapper)._dsa_prev_topk_indiceswhenskip_topk, and publish it fromfull layers when
next_skip_topk.index_topkis hoisted to a local so thesparse gather stays None-safe on shared layers.
Why
GLM-5.2 runs its DSA indexer on the full layers only; the shared layers carry
no indexer weights and reuse the previous full layer's top-k. This is the
selector-side half of that reuse. GLM-5 has no such schedule
(
dsa_layer_skips_topkis False for every layer), so every new branch is deadfor it and its behavior is unchanged. The CUDA-graph segment builders still
dereference the indexer unconditionally and are handled separately. Stacked on
the model-side hooks in #310 — review/merge that first.
Type of Change
modelkernelcorefixinfradocsFile changes
batchgen/attention/dsa/glm5_decode_selector.pyChecklist
model/kernelPR does not touch the scheduling/scaffolding layer (§2.5–§2.6).debug_*/scratch_*/tmp_*scripts,BATCHGEN_*env-guards, strayprint(), committed artifacts, orCo-Authored-By(§1/§4). (bench_*benchmarks are fine.)tests/; touched modules'MODULE.mdupdated if the public API changed (§2).bash .github/workflows/scripts/check-pr-hygiene.shlocally (against thetairanxu/glm52-modelbase); the CI hygiene check is green.🤖 Generated with Claude Code