Skip to content

feat(dsa): reuse full-layer top-k on shared GLM-5.2 decode layers - #311

Merged
Andrewxu313 merged 1 commit into
mainfrom
tairanxu/glm52-kernel
Aug 12, 2026
Merged

feat(dsa): reuse full-layer top-k on shared GLM-5.2 decode layers#311
Andrewxu313 merged 1 commit into
mainfrom
tairanxu/glm52-kernel

Conversation

@Andrewxu313

Copy link
Copy Markdown
Contributor

What

Wire GLM-5.2 DSA indexer top-k reuse into the eager (non-CUDA-graph) decode
selector. In build_glm5_dsa_flashmla_inputs, skip the indexer-K compute and
aux-KV update on shared layers (indexer is None), reuse
type(wrapper)._dsa_prev_topk_indices when skip_topk, and publish it from
full layers when next_skip_topk. index_topk is hoisted to a local so the
sparse gather stays None-safe on shared layers.

Why

GLM-5.2 runs its DSA indexer on the full layers only; the shared layers carry
no indexer weights and reuse the previous full layer's top-k. This is the
selector-side half of that reuse. GLM-5 has no such schedule
(dsa_layer_skips_topk is False for every layer), so every new branch is dead
for it and its behavior is unchanged. The CUDA-graph segment builders still
dereference the indexer unconditionally and are handled separately. Stacked on
the model-side hooks in #310 — review/merge that first.

Type of Change

  • model
  • kernel
  • core
  • fix
  • infra
  • docs

File changes

File Δ Note
batchgen/attention/dsa/glm5_decode_selector.py mod skip indexer-K on shared layers; reuse/publish carried top-k; None-safe sparse gather

Checklist

  • Every changed file traces to this task; the diff is surgical and one concern (§3).
  • Changed files stay within the declared type's allowlist — a model/kernel PR does not touch the scheduling/scaffolding layer (§2.5–§2.6).
  • The File changes table lists every changed file and matches the diff (§2.5).
  • No debug_*/scratch_*/tmp_* scripts, BATCHGEN_* env-guards, stray print(), committed artifacts, or Co-Authored-By (§1/§4). (bench_* benchmarks are fine.)
  • Tests added/updated under tests/; touched modules' MODULE.md updated if the public API changed (§2).
  • Ran bash .github/workflows/scripts/check-pr-hygiene.sh locally (against the tairanxu/glm52-model base); the CI hygiene check is green.

🤖 Generated with Claude Code

@Andrewxu313
Andrewxu313 force-pushed the tairanxu/glm52-model branch from 301f639 to ed8b545 Compare August 12, 2026 21:23
Wire GLM-5.2 DSA indexer top-k reuse into the eager (non-CUDA-graph) decode
selector. In build_glm5_dsa_flashmla_inputs, skip the indexer-K compute and
aux-KV update on shared layers (indexer is None), reuse
type(wrapper)._dsa_prev_topk_indices when skip_topk, and publish it from full
layers when next_skip_topk. Hoist index_topk to a local so the sparse gather
stays None-safe on shared layers.

GLM-5 has no such schedule (dsa_layer_skips_topk is False for every layer), so
every new branch is dead for it and its behavior is unchanged. The CUDA-graph
segment builders still dereference the indexer unconditionally and are handled
separately.
@Andrewxu313
Andrewxu313 force-pushed the tairanxu/glm52-kernel branch from d2797d7 to 4a26a49 Compare August 12, 2026 21:24
@Andrewxu313
Andrewxu313 changed the base branch from tairanxu/glm52-model to main August 12, 2026 21:24
@Andrewxu313
Andrewxu313 marked this pull request as ready for review August 12, 2026 21:24
@Andrewxu313
Andrewxu313 merged commit 31d5d0f into main Aug 12, 2026
1 check passed
@Andrewxu313
Andrewxu313 deleted the tairanxu/glm52-kernel branch August 12, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant