split_single_dyn_dim: add bucket_by_optimals to cut dyn-shape compile…#5028
split_single_dyn_dim: add bucket_by_optimals to cut dyn-shape compile…#5028chun-wan wants to merge 1 commit into
Conversation
|
Thank you for your contribution! Since this is an external pull request, a maintainer must review PR and add the "ok-to-test" label if it is approved for testing. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #5028 +/- ##
===========================================
+ Coverage 92.89% 92.89% +0.01%
===========================================
Files 603 603
Lines 32448 32482 +34
===========================================
+ Hits 30140 30174 +34
Misses 2308 2308
🚀 New features to boost your workflow:
|
|
This is the lower-risk half of #4911, split out per review feedback so it can land independently. It contains only the compile-time bucketing: split_single_dyn_dim{.bucket_by_optimals} (a pass parameter, driven directly in tests) plus select_module's smallest-compatible-bucket dispatch with host padding. The per-instance dispatch cache from #4911 is intentionally not here — it'll be a separate follow-up PR. All CI is green except security_gate (fork-only). @pfultz2 @CharlieL7 could you review? Thanks! |
1da50bb to
bfd2499
Compare
… cost Add a bucket_by_optimals field to split_single_dyn_dim: emit one submodule per (min, optimals..., max) instead of one per integer in [min,max], cutting compile time and .mxr size from O(max-min) to O(|optimals|). select_module dispatches to the smallest compatible bucket at runtime (ref pads inputs on the host; GPU callers pre-pad). GPU/Ref targets read MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS and forward it to the pass field. Adds split_single_dyn_dim and select_module bucket-dispatch tests. Split out of ROCm#4911; the per-instance select_module dispatch cache is a separate follow-up PR.
bfd2499 to
6b9ae29
Compare
|
This is the low-risk, bucket-only split out of #4911: just It collapses dynamic-shape compile from O(max−min) → O(|optimals|) — measured on a customer model: @CharlieL7 @kahmed10 could one of you review when you have a chance? 🙏 |
split_single_dyn_dimgains abucket_by_optimalspass field. When true anda dynamic dimension has non-empty
optimals, it emits one static submoduleper (min, optimals..., max) instead of one per integer in [min, max].
select_module::compute()first tries an exact-shape match, thenfalls back to the smallest compatible submodule (same type/rank, every bucket
dim >= the input dim). On the ref backend the input is zero-padded into the
bucket-sized buffer on the host; GPU callers pre-pad.
MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALSonce and forward itto the pass field. The field is a normal pass parameter, so unit tests set it
directly via
run_pass(...)rather than env vars.test/split_single_dyn_dim_test.cpp(bucketed enumeration) andtest/ref/select_module.cpp(exact match / round-up-with-host-pad /too-big-throws).
select_moduledispatch cache (a pureruntime optimization) is a separate follow-up PR.