[diwakergupta]

What this changes

This is the rest of #1474 that #1465 doesn't have, as @ashhart asked when closing #1474. It is stacked on #1465: review the last two commits.

• The drafter's EXL3 head over its two vocabulary spans. On qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465, DFlash2 projects all 248,320 columns of a dense EXL3 target's head. It now projects only the two spans it proposes from, {0..98,304} and {248,032..248,320}.
  • exl3.Weight.columns(from, n) (zig/src/cuda/exl3.zig) is a view of whole 128-column blocks of the stored matrix that keeps the parent's strides. It has a host test.
  • The .x3 arm of head() (cuda_draft_frame.zig) takes the first span, and for the tail the last exl3.wide_blocks (64) blocks, from column 240,128, dropping the first 7,904 columns when packing. It shares the affine head's projection and cuMemcpy2DAsync packing.
  • Both views plan their K split as the whole head does (one split, eight warps), so the logits keep qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's bits. exl3.plan() gives a shape one split from 64 blocks up, and wide_blocks names that threshold. The tail reads about 30 MB more a round than a 384-column tail would, and still skips ~545 MB.
• exl3 in the family's formats (cuda_native.zig), and the exl3.md and cuda.md lines that said the native engine serves no EXL3 pack.
• qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's refusal for MoE EXL3 targets is kept in openDrafter and openMtp.

Receipt

• Environment. RTX 3090 Ti (24 GB, sm_86). turboderp/Qwen3.8-27B-exl3 branch 3.50bpw (rev 8351c54e), z-lab/Qwen3.8-27B-DFlash2 (rev 50307d4c) at 4 bits, bf16 KV cache, --context 32768. Both binaries came from zig build native -Dsm=86, one from qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465 (c127b644) and one from this branch. Each ran twice, alternating (qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465, this, qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465, this), on a restarted server.

• Exactness. tools/bench_concurrent.py --levels 1 --tokens 384 --reps 3 --alone --serial, with the default code and chat cells (greedy and sampled) and --prompts with two long prompts (5,080 and 2,278 tokens):

  • Drafted == "draft": false and every repeat == its first run, in every cell on both binaries: 0 unequal, 0 failed.
  • token_sha256 of this branch equals qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's in 144/144 replies.

• Draft speed. Per-stream tok/s (each run a median of 3) and tokens per verify round, runs 1 / 2:

  Cell	qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465 tok/s	this PR tok/s	mean	qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465 tokens/round	this PR tokens/round
  code, sampled	184.3 / 184.6	187.7 / 185.2	+1.1%	6.74 / 6.74	6.74 / 6.62
  chat, sampled	114.8 / 114.9	115.3 / 115.5	+0.5%	4.17 / 4.17	4.17 / 4.17
  code, greedy	178.2 / 178.2	181.3 / 181.5	+1.8%	6.40 / 6.40	6.40 / 6.40
  chat, greedy	107.9 / 108.0	109.0 / 109.1	+1.0%	3.92 / 3.92	3.92 / 3.92
  long prompts, sampled	102.1 / 101.9	103.4 / 103.5	+1.4%	4.63 / 4.63	4.63 / 4.63
  long prompts, greedy	91.4 / 92.1	91.4 / 88.1	−2.2%	4.41 / 4.47	4.41 / 4.36

  • In run 1, every cell's tokens per round equals qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's.
  • Run 2 differs in two cells. Tokens per round in the long greedy cell also varies between qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's own two runs (4.41 / 4.47), so that count isn't fixed from run to run on either binary.
  • When the tokens per round are equal, a round is ~0.6 ms shorter (median round gap 36.5 → 35.8 ms on code) and decode is 0.5–1.8% faster.
  • With the 384-column tail (the first revision), long greedy was 4.41 → 4.36 in both runs, −3.5%.

• Prompt speed: n/a. The head projection runs only in drafter rounds.

• Tests. The full checklist suite (zig build test test-kernel-contract test-golden test-server-security test-pull-security fuzz-cortheon test-cuda-host): exit 0, 444/444 build steps succeeded, as on qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465. New host tests: a column view keeps the parent's strides and moves to its first 128-column block, and Draft frames: an EXL3 head's span views plan their K split as the whole head does. The second fails with the 384-column tail (expected 1, found 8). It runs in test-qwen35-draft-frame-host, whose the real hold adapter … test aborts on my machine on qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465 as well; the other 11 pass. lean_new.py against qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465: 0 new problems.

• Not run. Metal (no Mac). GB10 / sm_121. MoE EXL3 targets, which still refuse a drafter.

Checklist

• Every token still goes through the lane rounds.
• Drafted output equals "draft": false, and this branch's tokens equal qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's. Concurrent vs solo was not rerun here (one stream): no concurrent path changed.
• No precision traded for speed: the views read the same stored words and plan as the whole head, so the proposals' logits keep qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's bits.
• No GPU kernel changed.
• Prompt processing unchanged.
• Platforms: CUDA sm_86 only.
• New tests fail before the change, pass after it.
• Comments and docstrings are one line.
• No personal data, local paths or AI attribution lines.

[ashhart]

Thanks @diwakergupta, this is what we invited when we closed #1474: Weight.columns with its host test, the .x3 arm of head(), exl3 in formats with the doc lines, and #1465's MoE EXL3 refusal kept in openDrafter and openMtp. Merged onto current main, fmt, lean (no new findings), test-exl3-cuda-host with your column-view test and the qwen3_5 draft-frame, formats and admission steps pass.

One change before we run it:

1. The −3.5% long-prompt greedy cell comes from the tail view's K split. plan() (zig/src/cuda/exl3.zig:63) depends on the column count. The 98,304-column first span plans like the full head (one split, eight warps), but the 384-column tail from cuda_draft_frame.zig:164 plans eight splits and four warps, so its logits, and some proposals, change. For this head, any view of 64 blocks or more plans like the full head. If the tail view takes the last 8,192 columns (from 240,128), its logits keep qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's bits and every cell's tokens per round should match qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's. That reads about 30 MB more a round at the 6-bit head, against ~545 MB still skipped, and still fits d.logits. Please add a host test that the tail view plans as the full head does (red with today's 384 columns), and rerun the speed table.

A heads-up: this PR and #1475 add tests at the same place in exl3.zig, so whichever lands second needs a trivial rebase that keeps both.

Then we'll run the dense EXL3 pack with DFlash2 on a GB10 against #1465: token hashes, drafted == plain, concurrent == solo at two and four streams, tokens per round and per-stream speed.

[diwakergupta]

Thanks @ashhart, that was the cause. Pushed f056dbb5:

• The tail view now takes the last 64 blocks, from column 240,128. exl3.wide_blocks names plan()'s 64-block threshold, which plan() now uses too. Both span views plan as the whole head does: one split, eight warps.
• The new host test Draft frames: an EXL3 head's span views plan their K split as the whole head does builds a 248,320 × 5,120 head and checks both views against the whole head's plan. With the 384-column tail it fails with expected 1, found 8.

I reran the speed table on the 3090 Ti, alternating #1465 and this branch twice. It's in the description.

• Tokens: 144/144 hashes equal qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's, and drafted == "draft": false in every cell.
• Tokens per round: in run 1 every cell matches qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's.
  • In run 2, two cells differ. The long greedy cell also varies between qwen3_5 cuda: dense EXL3 checkpoints (Qwen3.8-27B), with DFlash2 drafts #1465's own two runs (4.41 / 4.47), so its count isn't fixed from run to run on either binary.
  • The long greedy mean is −2.2%, from one run. With the 384-column tail it was 4.36 in both runs.
• Speed: where tokens per round are equal, rounds are ~0.6 ms shorter and decode is 0.5–1.8% faster.

On this machine test-qwen35-draft-frame-host's the real hold adapter … test aborts at a cuLaunchKernel inside gemvSeg, and it does the same on #1465. The other 11 pass, the new one included. The full checklist suite: 444/444 steps, and lean shows no new findings.

[ashhart]

Thanks @diwakergupta, that's the fix:

• The 64-block tail view plans as the whole head does.
• wide_blocks only names plan()'s existing threshold.
• The new test goes red on the 384-column tail.

All 144 hashes equal #1465's, which is the result we wanted. Tokens per round also moved between #1465's own runs, so
we'll compare them over repeated runs.

Next, after #1465, we'll run the dense pack with DFlash2 on a GB10 against #1465: token hashes, drafted against plain,
concurrent against alone at two and four streams, tokens per round, and per-stream speed. We've started the CI runs that were waiting on our approval.

[diwakergupta]

Hi @ashhart, checking in on the GB10 comparison against #1465. CI on f056dbb5: desktop and zig-windows were cancelled, not failed. If you can rerun them on a fresh merge with main, that would clear them. Is anything else needed from me here?