refactor(qwen35): split weights.rs layer types + loaders into weights/layers.rs - #969
Open
CAICAIIs wants to merge 12 commits into
Open
refactor(qwen35): split weights.rs layer types + loaders into weights/layers.rs#969CAICAIIs wants to merge 12 commits into
CAICAIIs wants to merge 12 commits into
Conversation
Split the 683-line config.rs into a config/ module with model/tokenizer/tp submodules (k3-style mod.rs entry + config/), reducing config.rs to a re-export. Behavior unchanged. cargo check + clippy --all-targets -D warnings + config tests (14/14) green. Signed-off-by: CAICAIIs <3360776475@qq.com>
…try types Rebuild the config split into a directional ownership model instead of a flat file redistribution: - model.rs: deserialize a RawConfig into a validated, TP-agnostic Config35 via TryFrom, with typed ConfigError variants for every cross-field and kernel-AOT rule. - tp.rs: validate TensorParallelConfig at construction (private fields, so invalid world_size/rank is unrepresentable) and derive a validated LocalGeometry from (Config35, TP config, cuda-graph mode). Downstream accepts LocalGeometry instead of re-deriving shards from a raw rank pair. - error.rs: typed ConfigError (thiserror) used across model + TP boundaries. - tokenizer.rs: own the frontend compatibility schema explicitly instead of hiding unread fields behind allow(dead_code). - mod.rs: explicit exports (no glob re-exports); tests co-located in the owning module, fixture + table-driven mutations. Downstream (weights, decode_buffers, batch_decode, batch_decode_graph, prefill, tp_executor) now threads LocalGeometry for all shard math. Signed-off-by: CAICAIIs <3360776475@qq.com>
… re-export recurrent.rs reads GDN_AOT_*_DIM and LINEAR_CONV_MAX_KERNEL_DIM from crate::config; re-export them from config/mod.rs. ConfigError is referenced only inside the config submodules, so the crate-level re-export is dead. Signed-off-by: CAICAIIs <3360776475@qq.com>
…nused tp() accessor The FrontendAddedToken schema is an explicitly-owned fail-closed contract whose fields exist only to shape the typed parse; attach a justified allow(dead_code) instead of letting -D warnings reject it. LocalGeometry::tp() is unused now that downstream takes LocalGeometry directly. Signed-off-by: CAICAIIs <3360776475@qq.com>
Signed-off-by: CAICAIIs <3360776475@qq.com>
…paths Expose RawConfig at the config boundary so sibling test modules can deserialize fixtures, and reference RawConfig via crate::config paths that resolve inside both the real crate and the isolated harness. Signed-off-by: CAICAIIs <3360776475@qq.com>
Reaching into the private model module for a fixture weakens the validated boundary and pulls RawConfig into the crate surface. Build the test config through the real Config35::from_file entry point with a tempdir, and stop re-exporting RawConfig (lib-only clippy flags it as unused). Signed-off-by: CAICAIIs <3360776475@qq.com>
…n_layers Signed-off-by: CAICAIIs <3360776475@qq.com>
…/layers.rs Move FullAttentionLayer/LinearAttentionLayer/LayerKind/MLP35/TransformerBlock35 and the weight-loading helpers (full_attention_gated_q_shard_range/load_*_shard_if_needed) out of the 921-line weights.rs into a flat weights/layers.rs. Layer types + fields are pub(crate) (forward/* reads them as crate::weights::X) and the root re-exports them; helpers are used by the Qwen35Model impl via use layers::*. weights.rs 921 -> ~800. cargo check + fmt + test --lib 95/95 green. Signed-off-by: CAICAIIs <3360776475@qq.com>
CAICAIIs
force-pushed
the
refactor/qwen35-weights-a
branch
from
August 27, 2026 05:18
2dd4a8c to
d7c4e73
Compare
…ometry split Upstream pegainfer-project#966 reworked the qwen35 TP boundary to validated LocalGeometry in place; this branch had already moved the same loaders into weights/layers.rs carrying the LocalGeometry signatures. Keep the moved-heir versions in weights/layers.rs and leave weights.rs clean; qwen35 crate content is otherwise identical to the PR head. Signed-off-by: CAICAIIs <3360776475@qq.com>
…r load() Follow-up to the weights/layers.rs split: give the loader side of the split a functional shape and prune a test that only mirrored arithmetic. - WeightSource owns the tensor source plus the loop-invariant TP shard ranges (gated-q rows, k/v rows, o_proj cols, MLP intermediate); every loader names only its tensor. The per-layer constructors (FullAttentionLayer::load / LinearAttentionLayer::load / MLP35::load / TransformerBlock35::load) live next to their containers in layers.rs, so the weights.rs load loop is one TransformerBlock35::load call per layer and the ~200-line inline match is gone (weights.rs 796 -> 498 lines). - tune_decode_gemm_algos' eight near-identical filter_map sample blocks collapse into sample_mats() over two LayerKind variant accessors. - Delete the single-consumer GatedQShardRange struct; the shard-range helper returns a tuple and its adjacency test moves into layers.rs with a shared cfg(test) config fixture (weights/fixture.rs). - Retire mlp_tp2_uses_matching_gate_up_rows_and_down_cols: it asserted 2*intermediate arithmetic over LocalGeometry::shard_range (core-owned API). The gate/up-vs-down row match it guarded is now structural (one WeightSource.intermediate range feeds all three) and covered by the TP2 HF golden gates. Evidence (4x A100-40GB, sm_80, Qwen3.5-4B): cargo check/clippy --all-targets -D warnings clean; lib tests 99 passed; hf_golden_gate TP1 2/2 and TP2 (--ignored) 2/2; e2e_scheduler passed. Signed-off-by: CAICAIIs <3360776475@qq.com>
…st fixture Review pass over the loader split, trimming wrapping that paid no rent: - LayerKind::load was a pure forwarding hop; the full/linear match now lives in TransformerBlock35::load, so a block load is one call, not two. - weights/fixture.rs was a module serving a single test after the mlp arithmetic test retired; the two helpers re-join layers.rs's test module and the file is gone. Evidence (A100-40GB, sm_80, Qwen3.5-4B): cargo check/clippy --all-targets -D warnings clean; lib tests 99 passed; hf_golden_gate TP1 2/2 and TP2 (--ignored) 2/2. Signed-off-by: CAICAIIs <3360776475@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Split the layer container types + weight-loading helpers out of the 921-line
pegainfer-qwen35/src/weights.rsintoweights/layers.rs.weights.rskeeps the model + its loading/inference impl. Behavior unchanged.Structure
Validation
cargo check— passcargo clippy --all-targets -- -D warnings— pass (matches CI)Note
Independent slice (does not depend on the scheduler stack). Adds
#![allow(clippy::wildcard_imports)]at crate level for the submodule glob.