perf(GFX1100-TG200): raise Qwen3.5-4B Q4_K_M gfx1100 decode to 76.6 tok/s - #1936

Draft

ghazni101 wants to merge 77 commits into

mudler:mainfrom

ghazni101:row/GFX1100-TG200

Commits on Aug 25, 2026

spec(GFX1100-TG200): commit the 200 tok/s campaign spec
ghazni101
committed
592afd3
measure(GFX1100-TG200): T1 re-prices the tip -- 40.65 tok/s median, busy 8.64 + gap 2.08 ms/tok
ghazni101
committed
767d369
measure(GFX1100-TG200): T2a splits the GdnPostConv symbol -- the grid=1 pathology was a profiler artifact
ghazni101
committed
f758643
research(GFX1100-TG200): rank vLLM/SGLang mechanisms against our measured budget
ghazni101
committed
5e57df7
record(GFX1100-TG200): reject the pointer-keyed quant cache -- allocator recycling breaks it
ghazni101
committed
2034e17
perf(GFX1100-TG200): merged keep-quant gate_up -- one quant GEMM per MLP instead of two
ghazni101
committed
bdc8114
perf(GFX1100-TG200): T2b flips ROCm support_static_graph_mode -- decode-graph capture engaged
ghazni101
committed
64f38f1
record(GFX1100-TG200): session-state note appended to t2b evidence (hindsight 500s)
ghazni101
committed
68316bd
perf(GFX1100-TG200): T3a ports the f32-query decode-GQA attention arm -- and fixes its LDS underallocation
ghazni101
committed
8a2ee61
record(GFX1100-TG200): t3a evidence session-state note (hindsight 500s)
ghazni101
committed
239c106
test(GFX1100-TG200): T4a lands the red-first ROCm quant-dot gate
ghazni101
committed
9df8f0a
perf(GFX1100-TG200): T4a adds the VT_GEMV_MMVQ=1 K-quant decode GEMV arm -- bit-exact vs the CPU oracle
ghazni101
committed
205df07
perf(GFX1100-TG200): T4a folds activation quant into the MMVQ GEMV prologue and widens the gate to engine dtypes
ghazni101
committed
9ce2055
record(GFX1100-TG200): T4a evidence -- decode GEMV lever closed negative, garbage-fast instance #2
ghazni101
committed
554e01d
fix(GFX1100-TG200): T4a repairs the MMVQ arm -- m-gates the whole dispatch and makes the GEMV bit-equal to the baseline kernel
ghazni101
committed
60dac6c
record(GFX1100-TG200): T4a repair-round evidence -- lever adopted at +13.2% median with byte-identical engine outputs
ghazni101
committed
baf97fe
test(GFX1100-TG200): T4a repair-2 adds host-side dispatch-route counters and the F1/F2 routing-witness gate cases
ghazni101
committed
0ebe869
perf(GFX1100-TG200): T4a lever-B1 makes the fold crossover tunable behind VT_GEMV_MMVQ_FOLD_MAX
ghazni101
committed
28cbc6d
record(GFX1100-TG200): T4a lever-B1 evidence -- fold-crossover re-tune closed negative, 512 already optimal
ghazni101
committed
26e14bb
record(GFX1100-TG200): T4a lever-B2 attributes all 21.6 Cijk calls/tok to the two f32-out GDN BA projections
ghazni101
committed
5140bd1
test(GFX1100-TG200): T4a lever-B2 adds the red-first f32-out decode-skinny gate (VT_SKINNY_BF16=1)
ghazni101
committed
6c55cdf
perf(GFX1100-TG200): T4a lever-B2 adds the VT_SKINNY_BF16=1 f32-out decode-skinny arm
ghazni101
committed
8dcac46
test(GFX1100-TG200): T4a lever-B2 restores the routing-witness case preamble lost in the red-first edit
ghazni101
committed
6218d9b
test(GFX1100-TG200): T4a lever-B2 corrects the routing-witness expectations to per-arm deltas
ghazni101
committed
9f5d9da
record(GFX1100-TG200): T4a lever-B2 adopts the VT_SKINNY_BF16=1 f32-out decode-skinny arm at +8.7% median
ghazni101
committed
cecceb7
record(GFX1100-TG200): T4a lever-B2 closes the loop -- ON-arm capture shows the starved Cijk tile fully displaced
ghazni101
committed
a17a209
test(GFX1100-TG200): T4a lever-B2 makes the TRUE-unset routing window observe getenv()==NULL
ghazni101
committed
aab871f
record(GFX1100-TG200): T4a lever-B2 re-pins the default-routing witness against reviewer mutation M-A
ghazni101
committed
242e8da
attribution(GFX1100-TG200): lever-C maps all 97 QuantizeQ8KK decode launches to producers
ghazni101
committed
c574c7c
test(GFX1100-TG200): lever-C adds red-first witnesses for the fused norm-quant epilogue
ghazni101
committed
cfbba00
feat(GFX1100-TG200): lever-C fuses Q8_K activation quant into the RmsNorm epilogue
ghazni101
committed
25f511f
record(GFX1100-TG200): lever-C adopts VT_NORM_QUANT_FUSED=1 at +7.3% median, byte-identical
ghazni101
committed
79a992d
perf(GFX1100-TG200): T5a vectorizes the shared Q8_K quant superblock body
ghazni101
committed
65d8555
perf(GFX1100-TG200): T5b extends the f32-Q DecodeGqa arm to head_dim 128
ghazni101
committed
53eb36a
record(GFX1100-TG200): T5b evidence — attention routing hole closed, 69.8 tok/s median
ghazni101
committed
bd6551f
record(GFX1100-TG200): T5c closed negative — MMVQ nontemporal weight loads wash
ghazni101
committed
580c185
perf(GFX1100-TG200): T6a adds the warp-per-row cooperative GDN scan arm
ghazni101
committed
314dd01
record(GFX1100-TG200): T6a evidence — cooperative scan adopted at +4.6% median
ghazni101
committed
b8608a2
perf(GFX1100-TG200): T6b adds the warp-per-item cooperative attn preamble arm
ghazni101
committed
0a7ac39
record(GFX1100-TG200): T6b evidence — cooperative preamble adopted at +4.6% median
ghazni101
committed
3945b7f
record(GFX1100-TG200): session-close attribution — 76.6 tok/s median position
ghazni101
committed
1cee023
record(GFX1100-TG200): classify the campaign env knobs kernel-internal
ghazni101
committed
9d028b7
Merge upstream main into row/GFX1100-TG200
ghazni101
committed
4793e87
record(GFX1100-TG200): T7 evidence — COALK load topology closed wash, budget re-ranked
ghazni101
committed
bfaff5b
perf(GFX1100-TG200): T8 adds a cooperative single-row rmsnorm arm
ghazni101
committed
2c30b3e
record(GFX1100-TG200): T8 evidence — cooperative rmsnorm adopted at +3.2% median
ghazni101
committed
90ae363
perf(GFX1100-TG200): T9 gives the gated norm a per-row block
ghazni101
committed
36a6b87
record(GFX1100-TG200): T9 evidence — cooperative gated norm adopted at +2.6% median
ghazni101
committed
23f1858

Commits on Aug 26, 2026

Commits on Aug 27, 2026