Skip to content

split_single_dyn_dim: add bucket_by_optimals to cut dyn-shape compile…#5028

Open
chun-wan wants to merge 1 commit into
ROCm:developfrom
chun-wan:feat/dyn-dim-bucket-by-optimals
Open

split_single_dyn_dim: add bucket_by_optimals to cut dyn-shape compile…#5028
chun-wan wants to merge 1 commit into
ROCm:developfrom
chun-wan:feat/dyn-dim-bucket-by-optimals

Conversation

@chun-wan

@chun-wan chun-wan commented Jul 1, 2026

Copy link
Copy Markdown
Contributor
  • split_single_dyn_dim gains a bucket_by_optimals pass field. When true and
    a dynamic dimension has non-empty optimals, it emits one static submodule
    per (min, optimals..., max) instead of one per integer in [min, max].
  • At runtime select_module::compute() first tries an exact-shape match, then
    falls back to the smallest compatible submodule (same type/rank, every bucket
    dim >= the input dim). On the ref backend the input is zero-padded into the
    bucket-sized buffer on the host; GPU callers pre-pad.
  • GPU/Ref targets read MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS once and forward it
    to the pass field. The field is a normal pass parameter, so unit tests set it
    directly via run_pass(...) rather than env vars.
  • Tests: test/split_single_dyn_dim_test.cpp (bucketed enumeration) and
    test/ref/select_module.cpp (exact match / round-up-with-host-pad /
    too-big-throws).
  • Split out of Reduce dynamic-shape compile cost and select_module dispatch overhead #4911; the per-instance select_module dispatch cache (a pure
    runtime optimization) is a separate follow-up PR.

@chun-wan
chun-wan requested review from a team and causten as code owners July 1, 2026 19:11
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Since this is an external pull request, a maintainer must review PR and add the "ok-to-test" label if it is approved for testing.

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #5028      +/-   ##
===========================================
+ Coverage    92.89%   92.89%   +0.01%     
===========================================
  Files          603      603              
  Lines        32448    32482      +34     
===========================================
+ Hits         30140    30174      +34     
  Misses        2308     2308              
Files with missing lines Coverage Δ
src/include/migraphx/op/select_module.hpp 90.91% <100.00%> (+4.37%) ⬆️
src/include/migraphx/split_single_dyn_dim.hpp 100.00% <ø> (ø)
src/split_single_dyn_dim.cpp 98.65% <100.00%> (+0.19%) ⬆️
src/targets/ref/target.cpp 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chun-wan

chun-wan commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

This is the lower-risk half of #4911, split out per review feedback so it can land independently. It contains only the compile-time bucketing: split_single_dyn_dim{.bucket_by_optimals} (a pass parameter, driven directly in tests) plus select_module's smallest-compatible-bucket dispatch with host padding. The per-instance dispatch cache from #4911 is intentionally not here — it'll be a separate follow-up PR. All CI is green except security_gate (fork-only). @pfultz2 @CharlieL7 could you review? Thanks!

@chun-wan
chun-wan force-pushed the feat/dyn-dim-bucket-by-optimals branch from 1da50bb to bfd2499 Compare July 9, 2026 03:07
… cost

Add a bucket_by_optimals field to split_single_dyn_dim: emit one submodule
per (min, optimals..., max) instead of one per integer in [min,max], cutting
compile time and .mxr size from O(max-min) to O(|optimals|). select_module
dispatches to the smallest compatible bucket at runtime (ref pads inputs on
the host; GPU callers pre-pad). GPU/Ref targets read
MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS and forward it to the pass field.
Adds split_single_dyn_dim and select_module bucket-dispatch tests.

Split out of ROCm#4911; the per-instance select_module dispatch cache is a
separate follow-up PR.
@chun-wan
chun-wan force-pushed the feat/dyn-dim-bucket-by-optimals branch from bfd2499 to 6b9ae29 Compare July 14, 2026 13:34
@chun-wan
chun-wan requested review from CharlieL7 and kahmed10 July 14, 2026 15:05
@chun-wan

Copy link
Copy Markdown
Contributor Author

This is the low-risk, bucket-only split out of #4911: just MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS in split_single_dyn_dim + a smallest-compatible-bucket fallback in select_module (no freeze pass, no dispatch cache). Just rebased onto latest develop; all code checks pass, only security_gate (infra) is red.

It collapses dynamic-shape compile from O(max−min) → O(|optimals|) — measured on a customer model: max=10244.3s, max=1000005.1s, vs the legacy path not finishing in 150s.

@CharlieL7 @kahmed10 could one of you review when you have a chance? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant