TopKValuesIndicesRowKernel costs 708 us/step; a radix top-k does it in 40 us

[localai-bot]

From the #1857 kernel head-to-head: our per-step selector TopK (TopKValuesIndicesRowKernel, 8 rows x 248320 vocab, K=16) costs 708 us; SGLang's radix top-k does the equivalent in 40 us. −0.7 ms/step. Mirror the radix approach (FlashInfer's radix top-k is the upstream reference). Owner: SPEC-DFLASH2. Smallest of the three #1857 levers; ride it with whichever wave touches the selector next.

[localai-bot]

Two corrections of record for this issue, from the fresh review of #1929. Neither
is edited into the title or the index row, because a GitHub title is not history
this repository rewrites and .agents/issue-index.md is append-only under
merge=union. ## Risks/decisions D2 of .agents/specs/dflash2-spec-decode.md
carries both.

1. The number in the title and the body is 708 us; the measurement is 683 us. #1857's
artifact-verified kernel table reads TopKValuesIndicesRowKernel at 683
us/step for the production 8 x 248320 at K=16, against SGLang's
RadixTopKKernel_Unified at 40 us. Every statement in the branch, the spec and
.agents/kernel-matrix.md uses 683.

2. This is the THIRD-largest per-step lever of FOUR, so "Smallest of the three
#1857 levers" in the body above is superseded. On
#1857's corrected table the adverse per-step deltas are FP8 +3.04 ms
(#1866), draft attention +2.29 ms, this top-k +0.65 ms, and GDN +0.38
ms. .agents/specs/dflash2-draft-block-fa2.md already put it this way: "#1866
(FP8 GEMM algo selection, +3.04) is the largest and #1867 (radix TopK, +0.65) the
third." The branch's earlier "fourth-largest" was its position in a numbered
prose list whose item 1 is the attention retraction rather than a lever.

And one finding this issue should carry, because it changes what the fix is.
The first revision of the port sized the kernel's shared candidate buffer at 2048
entries, on the argument that the round-0 candidate set is the columns sharing
the k-th largest value's exponent. Round 0's digit is key >> 24 -- the sign bit
plus the top SEVEN of eight exponent bits -- so one bucket spans TWO adjacent
exponents, a 4x value range. Measured on the production rows, at 248320 columns
and K = 16, that bucket holds 92701 to 93938 columns: 45x the buffer, and 5.7x
FlashInfer's own 16K ceiling. So the compaction did not engage on this shape at
all, and the claim that "only two of the four rounds touch global memory" was
false here. The same rows hold 452 to 522 one round later, which is what the
branch's two-stage compaction now uses.

Still owed, and why this issue stays open: nothing has compiled under nvcc
and no timing exists. ## Owed O34 owns the device run, and O35 now ranks the
candidate set FIRST among the levers that run should read -- the arm each row
takes, then the time.