Skip to content

feat(qwen3.8): qwen3.8-dflash2:27b, Qwen3.8-27B with the DFlash 2 draft - #916

Merged
tawei-amd merged 5 commits into
mainfrom
feat/qwen38-dflash2
Nov 2, 2026
Merged

tawei-amd merged 5 commits into
mainfrom
feat/qwen38-dflash2

Conversation

@tawei-amd

Copy link
Copy Markdown
Collaborator

Summary

qwen3.8-dflash2:27b: Qwen3.8-27B decoded with z-lab's DFlash 2 draft instead of the built-in MTP head.

  • Catalog: tag qwen3.8-dflash2 / 27b, base_model: qwen3.8-mtp:27b, own package Qwen3.8-27B-DFlash2-NPU2 (weights only: dflash_weight.q4nx 1.39 GB, config.json, LICENSE), xclbin_folder: Qwen3.8-27B-NPU2 so it runs the base model's kernels (no xclbin copy).
  • model_list.hpp: resolve_source always forced xclbin_folder to the dependant's own name; it now keeps one the entry sets. New catalog tests (explicit xclbin_folder; the shipped qwen3.8-dflash2 entry).
  • Wrapper: Qwen3_8DFlash2 : Qwen3_8MTP (only names the draft), family qwen3.8-dflash2 registered on the same engine library (RegisterFlm<qwen3_8mtp_npu>), CI family tables.
  • Engine library (libqwen3_8mtp_npu.so / .dll, XRT and HRX): from IRON PR AMD-ROCm-Internal/FastFlowLM_IRON#136, CI run 41346212324; sha256 below. The engine loads the draft package from draft_model_path and the DFlash2 drafter replaces the MTP head (FLM_DFLASH=0 keeps the MTP head).
  • Docs: model card in qwen.md, decode benchmark in qwen3.8_results.md, notes in the request-parameters / prefix-cache / OpenAI pages. No new page, so no sidebar entry.

Verification

  • Libraries are the IRON CI build, verified locally before vendoring: same outputs and speeds as the local build (10 prompts x 160 tokens, 16-case behaviour matrix).
  • Decode (greedy, ten prompts): 3.74 tok/s vs 3.23 for qwen3.8-mtp:27b (1.16x, 0.87x to 1.47x per prompt).
  • Output equals the MTP tag's on 14 of 15 greedy comparisons (the other differs at a near-tie); multi-turn, max_tokens 1 to 16, think on, tools, an image, a 4.5k-token prompt and cancel + retry behave as on qwen3.8-mtp:27b.
  • test_model_downloader (catalog / base_model tests) passes, built standalone with g++.

Not in this PR

  • src/model_info.json entry and the HF / ModelScope upload of Qwen3.8-27B-DFlash2-NPU2 (needed before flm pull qwen3.8-dflash2:27b works): after the upload, from the HF tree API.
  • Windows installer entries: none needed (same qwen3_8mtp_npu.dll).

🤖 Generated with Claude Code

Vendored libraries (IRON PR Check run 41346212324)

c578de45346c36dfcebf45f38fb77a387f9e497aae5fc327ec1a9e894f28c278  src/lib/xrt/libqwen3_8mtp_npu.so
444cbcd24de97a46241ff5ba1e3e98e3809445d35d72ae2805a6e93eff2b4721  src/lib/hrx/libqwen3_8mtp_npu.so
d0ad732e3ba6ce8096d190922c6e25b3db87c5be07783af0433dc35c6f2f053a  src/lib/xrt/qwen3_8mtp_npu.dll
f4b2c9fb6777bc765f744d1964a4c6d1d3c520aaf8772ac53b1d140f6a1422c2  src/lib/xrt/qwen3_8mtp_npu.lib
83f23e72ca00c50d9b457c3996aff8a2da70f5a423f64cc0b7b03194f06e577f  src/lib/hrx/qwen3_8mtp_npu.dll
3ea183f2dbfb8db4a20c966389d959e2eca083121b910d9e4edc3048e24b6969  src/lib/hrx/qwen3_8mtp_npu.lib

Only the Linux XRT library ran locally (HRX and Windows are exercised by this PR's CI). The tag cannot be pulled until the HF / ModelScope upload.

tawei-amd and others added 5 commits November 1, 2026 12:22
A catalog tag whose base_model is qwen3.8-mtp:27b and whose own package (Qwen3.8-27B-DFlash2-NPU2, weights only) is the
draft: the same engine library and the base's xclbins, Qwen3_8DFlash derives from Qwen3_8MTP and only names the draft
differently. model_list.hpp keeps an explicit xclbin_folder of a dependant entry (it always forced the tag's own name, which
would need a copy of every xclbin), with catalog tests; docs (model card, benchmark, request/prefix-cache notes) and the
CI family tables.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…after

From IRON PR #136 PR Check run 41346212324 (XRT and HRX, Linux and Windows); sha256 in the PR. Verified locally on Linux XRT before vendoring. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tawei-amd
tawei-amd marked this pull request as ready for review November 2, 2026 01:39
@tawei-amd
tawei-amd merged commit 0d6c78f into main Nov 2, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant