Repository navigation
feat(qwen3.8): qwen3.8-dflash2:27b, Qwen3.8-27B with the DFlash 2 draft - #916
Merged
Merged
Conversation
A catalog tag whose base_model is qwen3.8-mtp:27b and whose own package (Qwen3.8-27B-DFlash2-NPU2, weights only) is the draft: the same engine library and the base's xclbins, Qwen3_8DFlash derives from Qwen3_8MTP and only names the draft differently. model_list.hpp keeps an explicit xclbin_folder of a dependant entry (it always forced the tag's own name, which would need a copy of every xclbin), with catalog tests; docs (model card, benchmark, request/prefix-cache notes) and the CI family tables. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…after From IRON PR #136 PR Check run 41346212324 (XRT and HRX, Linux and Windows); sha256 in the PR. Verified locally on Linux XRT before vendoring. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
qwen3.8-dflash2:27b: Qwen3.8-27B decoded with z-lab's DFlash 2 draft instead of the built-in MTP head.qwen3.8-dflash2/27b,base_model: qwen3.8-mtp:27b, own packageQwen3.8-27B-DFlash2-NPU2(weights only:dflash_weight.q4nx1.39 GB,config.json,LICENSE),xclbin_folder: Qwen3.8-27B-NPU2so it runs the base model's kernels (no xclbin copy).model_list.hpp:resolve_sourcealways forcedxclbin_folderto the dependant's own name; it now keeps one the entry sets. New catalog tests (explicitxclbin_folder; the shipped qwen3.8-dflash2 entry).Qwen3_8DFlash2 : Qwen3_8MTP(only names the draft), familyqwen3.8-dflash2registered on the same engine library (RegisterFlm<qwen3_8mtp_npu>), CI family tables.libqwen3_8mtp_npu.so/.dll, XRT and HRX): from IRON PR AMD-ROCm-Internal/FastFlowLM_IRON#136, CI run 41346212324; sha256 below. The engine loads the draft package fromdraft_model_pathand the DFlash2 drafter replaces the MTP head (FLM_DFLASH=0keeps the MTP head).qwen.md, decode benchmark inqwen3.8_results.md, notes in the request-parameters / prefix-cache / OpenAI pages. No new page, so no sidebar entry.Verification
qwen3.8-mtp:27b(1.16x, 0.87x to 1.47x per prompt).qwen3.8-mtp:27b.test_model_downloader(catalog / base_model tests) passes, built standalone with g++.Not in this PR
src/model_info.jsonentry and the HF / ModelScope upload ofQwen3.8-27B-DFlash2-NPU2(needed beforeflm pull qwen3.8-dflash2:27bworks): after the upload, from the HF tree API.qwen3_8mtp_npu.dll).🤖 Generated with Claude Code
Vendored libraries (IRON PR Check run 41346212324)
Only the Linux XRT library ran locally (HRX and Windows are exercised by this PR's CI). The tag cannot be pulled until the HF / ModelScope upload.