The Qwen family on Vulkan: the KHR kq tile, the OS-room auto arm, and the three fixes the walk found

[borisbat]

Behavior change: the Vulkan resident plan's auto arm now sizes against the room the OS reports on the adapter, and device allocations carry no memory-priority tag - a model that needed a pin to arm may arm on its own, and one that used to spill to demoted memory now decodes at full rate.

Why. Every Qwen text model on the box (27 GGUF files, 0.5B to 48 GB) was measured one at a time against llama.cpp on the same card: three families were silently declined or derailed (the qwen2 projection bias, a hybrid KV mirror one byte past 4 GiB, a missing add_bos default), and on a KHR-only cooperative-matrix device every K-quant prefill ran at a third of llama.cpp's rate (221 against 677 t/s on the 27B).

What changes.

• The resident drivers serve the qwen2 q/k/v projection bias (rope stage and per-op attention chain) and a K-quant deltanet out plane; the KV binding cap sums per-layer rows.
• A BPE vocab without add_bos_token follows llama.cpp's per-pre default.
• The auto arm reads the OS's adapter room through a hand-bound vk_os_video_memory; allocations drop the priority tag; the grid GEMVs stage codebooks from a device buffer; the codebook scale row is two words.
• The SPIR-V emitter lets a kernel call a [spirv_decode] method directly: a struct parameter spills to a local, a plane-element argument passes its index.
• The kq prefill tile gains a KHR cooperative-matrix arm for all thirteen formats; its two staging arrays are the whole workgroup footprint (the edge-store bounce rides the weight array), which is one workgroup more per SM.
• Twin kernels collapse onto templates and shared bases; the census maps the Vulkan and Metal twins.

Observable behavior.

• Qwen2.5, qwen2moe and the 27B's deltanet out plane on the resident driver: declined -> served.
• unsloth Qwen3.8-27B: a leading <|endoftext|> derails -> correct output.
• KHR-only device (DASLLAMA_COOPMAT=mm): 4B Q4_K_M prefill 1564 -> 3051 t/s (0.72x of llama.cpp's KHR arm, from 0.37x), 27B UD-IQ4_XS 221 -> 395 on the kq tile before the slab fix; its re-measure after the fix goes with the next PR, where the KHR kernels move again.
• 27B UD-IQ4_XS decode with 1.7 GB demoted: 4.2 -> 23.5 t/s.
• iq2s GEMV 84 -> 388 GB/s; the untouched kernels' SPIR-V is byte-identical.

Where to look. khr_tile and the templates in dasllama_vulkan_classes.das, the element-index rail in spirv_emit.das, the auto arm in dasllama_gpu_resident.das, the bias rope kernels.

Validation, claims, ledger

Validation

• The model-free suite: 74 files green (two reds found by its run and fixed here: the OS video memory seam carried a facade underscore with no facade, and the tier want test asserted an env-backed answer its own comment forbade); the stocked suite: 42 files, none failed, 140 cells skipped for absent models and fixtures (whisper, the tower oracles, the large tier).
• Preflight, lane by lane after a red fast tier (one architecture doc at 301 lines): lint, docs, tests-cpp green; the interpreter lane 14268 of 14284 with one file over the lane's 60 s per-file cap - tests/watchdog/test_watchdog.das, 157 s alone on this box (its twenty-thousand-line chatty-child cell 92 s on the interpreter), untouched by this branch, and CI runs no such cap; the JIT lane 14138 of 14152 with the same file as its one red; the utils lane: every one of its thirteen suites green (the ast-fuzz suite for the first time on this layout - it spawned a hardcoded bin/daslang, fixed here to the running binary), yet the run_utils_tests rule reports exit -1 after the last suite's SUCCESS under msbuild alone - the same daspkg command exits 0 from Bash, from cmd.exe and through a pipe; CI runs the rule on Linux; the AOT lane skips locally by design (the DLL-flavor host pins the runtime), so the full AOT sweep below stands in for it.
• The full AOT sweep (test_aot, ~1080 TUs): 13233 of 13250, the seventeen others twelve skips and five reds in tests/module_cache/test_default_cache_path.das - its child-spawning cells run the host binary as the child, which under the AOT host runs no script and prints no cache marker; they now skip on --use-aot through the gate the file's other two cells already carried (after the rebuild: one cell passes, seven skip, none fail; the interpreter still runs all eight).
• The codex round on the final shape returned two P1 findings, both real and both fixed here: the deltanet conv kernel staged a whole position row into an 8192-float slab while the admitted conv width had grown to 16384 (the 27B's 10240 wrote past it) - it now stages only the q/k region it norms (2 kd, asserted against DN_MAX_QK at the three capacity checks) and streams the v region through; and the add-rms row stash, doubled to 32 KB for the 27B's 5120, past what a 32 KiB workgroup-memory device holds with the reduce's partials - ruled not a target: the tier's floor is the 48 KiB every kernel here is sized against, stated once in the GEMM architecture doc, and the stash stays at 8192.
• The KHR arm ran on the NVIDIA card with NV_coopmat2 disabled (DASLLAMA_COOPMAT=mm); the thirteen kernel cells hold 0 of 89600 off with their edge tiles, the hybrid parity file 10 of 10 on the KHR tile, the Vulkan serving census reaches every format's sdot4, cm2 and KHR classes through the 1B vehicles under every coopmat mode (the KHR classes 32 to 330 dispatches each; the census's idle warnings 98 -> 36, the rest the L tiles and the q8 mul_mm tiles the 40-token census prompts never pick, and the MoE and attention variants above the small tier).
• The KHR modules carry only the KHR cooperative-matrix capability; a KHR workgroup holds the block base and two 2560-word arrays (dumped through DASLLAMA_VK_SPV_DUMP, read with spirv-dis).
• Of the 133 kernels the kernel suite creates, dumped at the pre-harvest commit and at HEAD: the thirteen KHR classes re-emit (the rewrite); the three grid families' sdot4 batch and GEMV kernels and the requant pair differ by one helper emitted as a call instead of inline (their scale decode, the store helper), the deltanet step by spelling its out_f32 test as != 0; the other 110 are byte-identical.
• The qwen2 and hybrid resident files pick their session codec from the box's mirror codec, so under DASLLAMA_VK_KV32=1 the same cells run on the f32 mirrors (the qwen2 file 3 of 3 there, the bias rope's f32 twin served).
• The dupes report at the final shape, triaged over the functions this branch adds: the two Vulkan probes re-authored their synthetic-plane generators word for word, now one shared fixture module (harness/_vk_probe_fixture.das); every other exact match is the tier's seat idiom (a one-line installer, getter or invoke forwarder per seat, the same shape as its neighbours by design) or a shape coincidence across domains (a two-value predicate, a name table), left as they are.
• Audits: one review-md auditor per binding checklist (eight discovered plus the kind-routed companions by hand), the tdd auditor in its own worktree, the style auditor per file cluster; every violation fixed in code or in the checklist itself.

Claims - stated, not tested

• The KHR tile on an AMD or Intel part: only the NVIDIA card with cm2 disabled was available; a break would read as a cell failure or a decline log on that device.
• The OS-room auto arm was measured on the NVIDIA WDDM driver only; on other drivers vk_os_video_memory answers 0 and the plan falls back to the probed budget.

Not done

• The integer MMQ tile for the KHR arm (followup_vulkan item 42): prototypes cap at 11 TFLOP/s; the emitter owes unroll-at-emission first (plans/shader_emitter_followups.md item 2).
• The Metal twins (followup_metal.md sec. 6) - a Mac session.
• The image mint's two-copy footprint (followup_general item 120); Qwen1.5-MoE-A2.7B on the per-op tier (the board's row).
• Gemma and the remaining families.

[Copilot]

🔵 Needs a closer look

The change set is very large and affects core GPU/Vulkan serving behavior, shader emission semantics, and many tests/docs, so it needs careful human validation (and the PR body should remove the prohibited Claude Code footer/link).

Pull request overview

This PR extends dasLLAMA’s Vulkan resident path and shader emission/test infrastructure to better serve the Qwen family on Vulkan (including KHR cooperative-matrix paths), while tightening or expanding test coverage and updating the related architecture/review documentation.

Changes:

• Add/extend SPIR-V emitter support for direct [spirv_decode] method calls (including struct-parameter spill behavior) and cover it with new fixtures + golden/census/fail-closed tests.
• Add Vulkan OS video-memory “room” query plumbing (os_video_memory) plus an integration test, and route the value into dasLLAMA’s residency planning behavior.
• Expand dasLLAMA model/kernel tests and supporting utilities for Qwen2/Qwen3.* + K-quant/iQ formats (including scale-row packing changes) and update associated checklists/docs.

Notable process issue: the PR description currently includes a claude.com/claude-code “Generated with” footer/link, which is explicitly disallowed by the repo’s PR hygiene guidance (skills/internal/make_pr.md).

File summaries

File	Description
utils/internal/ast-fuzz/test_ast_fuzz.das	Use running binary path for fuzz
utils/dasllama-server/REVIEW.md	Checklist routing note for tests
tests/spirv/test_struct_param.das	New SPIR-V direct-call tests
tests/spirv/test_golden.das	Add new golden fixtures
tests/spirv/test_fail_closed.das	Add new fail-closed checks
tests/spirv/test_census.das	Add new opcode census inputs
tests/spirv/REVIEW.md	Strengthen SPIR-V test checklist
tests/spirv/_spirv_common.das	Add spdirect/spelem fixtures
tests/spirv/_gen_golden.das	Emit new golden outputs
tests/spirv/_fail_closed/_fc_decode_twoplanes.das	New fail-closed fixture
tests/spirv/_fail_closed/_fc_decode_mixed.das	New fail-closed fixture
tests/spirv/_fail_closed/_fc_decode_index64.das	New fail-closed fixture
tests/spirv/_fail_closed/_fc_decode_helper_copy.das	New fail-closed fixture
tests/msl/REVIEW.md	Reference shared emitter checklist
tests/module_cache/test_default_cache_path.das	Skip module-cache tests under AOT
skills/internal/make_pr.md	Clarify targeted gate reruns
plans/shader_emitter_followups.md	Document unroll-at-emission followup
plans/REVIEW.md	New plans folder checklist
modules/REVIEW_SHADER_EMITTERS.md	Rewrite shared emitter checklist text
modules/dasVulkan/utils/vulkan2rst.das	Group os_video_memory in docs
modules/dasVulkan/tests/REVIEW.md	Add integration-test numeric comparison rule
modules/dasVulkan/tests/integration/test_os_video_memory.das	New OS video-memory integration test
modules/dasVulkan/REVIEW.md	Route kernel bodies to shared checklist
modules/dasVulkan/daslib/vulkan_boost.das	Add public os_video_memory helper
modules/dasVulkan/ARCHITECTURE.md	Document OS video-memory mechanism
modules/dasUnitTest/unit_test.das.inc	Reformat embedded unit_test blob
modules/dasSpirv/REVIEW.md	Add OpVariable ordering rule
modules/dasSpirv/ARCHITECTURE.md	Document direct decode call lowering
modules/dasMetal/ARCHITECTURE.md	Note Vulkan-only direct decode calls
modules/dasLLVM/REVIEW.md	Checklist routing note for tests
modules/dasLLAMA/tests/test_vulkan_tier.das	Extend attention tests for bias path
modules/dasLLAMA/tests/test_tokenizer.das	Add BOS-default behavior test
modules/dasLLAMA/tests/test_kquant.das	Add iQ two-word scale-row gather test
modules/dasLLAMA/tests/test_kernel_coverage.das	Expand Vulkan serving census vehicles/modes
modules/dasLLAMA/tests/test_gpu_tier.das	Align gpu_want_auto with env export
modules/dasLLAMA/tests/test_gpu_serving_declines.das	Add decline/servability gates (bias, out plane, plan room)
modules/dasLLAMA/tests/test_gpu_resident_qwen2.das	New resident Qwen2 bias parity test
modules/dasLLAMA/tests/test_gpu_resident_hybrid.das	Adjust hybrid parity to mirror codec + tags
modules/dasLLAMA/tests/test_gguf_quant.das	Add ggml type sizing/name tests
modules/dasLLAMA/tests/run.das	Register new resident qwen2 test
modules/dasLLAMA/tests/REVIEW.md	Refine checklist update rules
modules/dasLLAMA/tests/REVIEW_KERNEL_CELLS.md	Clarify geometry/kargs update obligations
modules/dasLLAMA/tests/CLAUDE.md	Update suite census documentation
modules/dasLLAMA/tests/_vkd_toy.das	Add direct-decode dispatch fixture
modules/dasLLAMA/tests/_model_tier.das	Add mirror_kv_dtype helper
modules/dasLLAMA/REVIEW.md	Clarify measurement/logging rules
modules/dasLLAMA/REVIEW.das	Bump image layout stamp
modules/dasLLAMA/REVIEW_UPSTREAM.md	Retitle/clarify upstream naming policy
modules/dasLLAMA/REVIEW_PLACEMENT.md	Tighten placement rules for transforms/kernels
modules/dasLLAMA/REVIEW_MEASUREMENT.md	Clarify provenance and decision rules
modules/dasLLAMA/REVIEW_IMAGE.md	Clarify mint invariants (no double model)
modules/dasLLAMA/REVIEW_GPU.md	Tighten kernel/template and role rules
modules/dasLLAMA/REVIEW_GPU_VULKAN.md	Add Vulkan-specific rules (KHR, decode call form, grid staging)
modules/dasLLAMA/REVIEW_GPU_RACE.md	Clarify binding/order and timing rules
modules/dasLLAMA/README.md	Split Vulkan docs into GEMM companion
modules/dasLLAMA/performance/REVIEW.md	Clarify record-hygiene + recipe change rule
modules/dasLLAMA/performance/model_specs.das	Update Qwen3.5 Q4_K_M recipe
modules/dasLLAMA/HOW_TO_ADD_A_FORMAT.md	Update Vulkan iQ device-row + KHR arm guidance
modules/dasLLAMA/harness/REVIEW.md	Add vk_dispatch role silencing rule
modules/dasLLAMA/harness/_vk_probe_fixture.das	New shared Vulkan probe fixture
modules/dasLLAMA/followup_metal.md	Add Metal twins census plan section
modules/dasLLAMA/followup_general.md	Add image mint double-hold finding
modules/dasLLAMA/ENVIRONMENT.md	Flip vk_memprio default; clarify cm2 knobs
modules/dasLLAMA/dasllama/dasllama_vulkan_seams.das	Add bias placeholder binding in seam
modules/dasLLAMA/dasllama/dasllama_math_vulkan.das	Install new Vulkan resident seats
modules/dasLLAMA/dasllama/dasllama_layout.das	Pack iQ device scale rows (two-word)
modules/dasLLAMA/dasllama/dasllama_image.das	Serve streamed build if image save declines
modules/dasLLAMA/dasllama/dasllama_gguf.das	Add ggml sizing/type name helpers
modules/dasLLAMA/dasllama/dasllama_gemm_schema.das	Add kq_dev_ssb + two-word-row policy
modules/dasLLAMA/dasllama/dasllama_env.das	Change defaults/docs for Vulkan env knobs
modules/dasLLAMA/dasllama/dasllama_common.das	Add mb_delta formatting helper
modules/dasLLAMA/dasllama/dasllama_bpe.das	Add per-pre BOS default table
modules/dasLLAMA/benchmarks/REVIEW.md	Clarify race vs attribution rules
modules/dasLLAMA/ARCHITECTURE.md	Re-route Vulkan sections to new companion doc
modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md	Extend measurement axes descriptions
modules/dasLLAMA/ARCHITECTURE_GPU.md	Update Vulkan role table + seats list
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_RESIDENCY.md	Update residency plan with OS-room sizing
modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_DECODE.md	Document bias + out-plane KQ behavior
modules/dasLLAMA/ARCHITECTURE_GPU_QUANT_PLANES.md	Note two-word device scale rows
modules/dasLLAMA/ARCHITECTURE_ENGINE.md	Document BOS default + conversion accounting

Review details

• Files reviewed: 100/103 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

It changes multiple high-impact GPU/runtime subsystems (Vulkan residency sizing, shader emission/test infrastructure, tokenizer defaults) and also includes a PR-body policy violation (Claude Code footer link) that should be removed before merge.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/spirv/REVIEW.md:5

• Line 5 starts a new Markdown bullet ("- apply …") even though the preceding line doesn’t introduce a list item, which makes the checklist render oddly. This reads like it was meant to be a continuation of the previous sentence, similar to tests/msl/REVIEW.md.

• Files reviewed: 100/103 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

It touches core GPU residency/serving behavior and shader emission semantics across many files, which warrants careful human validation beyond an automated review.

Review details

• Files reviewed: 100/103 changed files
• Comments generated: 1
• Review effort level: Lite