[davidtai]

Refs #1661.

Draft. The branch is being restructured; the current head is an older squashed commit and will be replaced.

Scope

The gfx11 HIP path for Qwen3.8-Flash-Next, measured on gfx1151, limited to changes whose output bytes equal the base path's.
The new head will be a series of self-contained commits, one per change, each with its own tests, its kernel-contract
registration (gates 1-4 test type and gate-5 record) and its evidence, so each commit can become its own pull request if
the maintainers prefer.

The rounding-class kernels and the numerics=exact|inexact switch of the earlier squashed commit are no longer part of
this pull request. Under numerics=inexact they did not meet CONTRIBUTING's rules: their FP64 error is larger than the
exact path's, and their output depends on how many rows run together, so a resumed turn and a decoder beside a long
prompt did not equal their fresh and solo runs. The evidence is in #1661, recorded as findings to fix before any later
proposal.

Planned content (the series)

• Prompt: dense trunk products and routed experts (gate+up with SwiGLU, down) on WMMA tiles with the existing matrix
  tile's bytes; the DeltaNet prompt recurrence at half a value head a block and its prework in one launch; the indexer
  at 4 lanes a score; a prompt's picks a wave a row and its route plan in three launches; the router tile.
• Decode and verify: router, pick and plan in one launch; the HC norm and mix with 16-byte accesses and the MoE combine
  inside the next HC norm; QSA decode prep in one launch; stream-tile work for 1 to 8 rows; top-k in parts; a side
  stream that reads the next experts' down codes at one-row decode (reviewed by the expert-kernel owner).
• Speculation (lossless): verify windows up to 8 rows with a held DeltaNet state and a keep pass; depth priced from
  forwards timed at load; a gated draft chain; sampled streams draft the target's keyed choice.

Exactness (numerics as the base path)

Recorded on an AMD Strix Halo (gfx1151) machine: drafted == plain (greedy and sampled), solo == together, decoders
unchanged by a long prompt, resumed == fresh through rewinds with up to 10,240 cached tokens, cancellation, and one logits
hash across prompt chunks of 512, 2,048, 8,192 and 20,480 rows. The Flash Next greedy hash equals the base path's
(243ecea88fe50d9f). Full receipts come with the series.

Known finding carried from upstream

Upstream's select_body (the MoE pick) is not deterministic for a row that holds a NaN, and leaves pick slots unwritten
for a row with fewer than k numbers. The new pick kernels keep its bytes, so their contract specs record the same known
finding.

Results

Performance and memory receipts for this scope (cold prefill 2K to the native window, serial and drafted decode, peak
memory, against current main) are running and will be posted with the series. Earlier numbers and charts were measured
with the rounding-class kernels; they describe the fork's numerics=inexact path and are kept in #1661 with that label.

Not run

Metal and CUDA device tests (no such hardware). HIP devices other than gfx1151 are covered only by the limits check.

ATC: issue #1661 is claimed, with anonymous session 220947162292872.

🤖 Generated with Claude Code

[AbrahamCete]

Hi @davidtai, a heads-up that #1694 overlaps with the stream-tile part of this series. It is the stream-tile work I mentioned in question 4 of #1563. Both change zig/kernels/hip/tiles/stream.hpp in the same two ways, and they don't merge cleanly:

• wide rounds put a column block's row blocks next to each other on the grid's x;
• a chunk's code pairs are made once for every row of the block.

Two things from my side that may help when you split the series:

• I tried naming the layout in a StreamSides field read by stream_tile first. On gfx1151 it doubled the stream kernels' VGPRs (4x4: 140 -> 256) and cost about 10% on wide verify rounds. Mine derives the layout from the grid instead (launches/stream_layout.zig).
• It has a GPU test that requires the interleaved and reference layouts to write the same bytes. The test covers plain, relu2, pair, routed and grouped products at 5 to 32 rows. There is also a golden digest for the 4-bit BF16 code pairs.

On Qwen3.8-27B it gives +30% aggregate at 8 streams and +47% at 16, with the bytes unchanged. I'm happy to rebase onto your series, or for yours to build on this one, whichever the maintainers prefer.