Add reference examples and guidance (especially for multi-device scenarios) for ValidateCompiledModelCompatibilityInfo #29168
Open
adrastogi wants to merge 4 commits into
Open
Add reference examples and guidance (especially for multi-device scenarios) for ValidateCompiledModelCompatibilityInfo #29168adrastogi wants to merge 4 commits into
adrastogi wants to merge 4 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR clarifies the intended semantics of OrtEpFactory::ValidateCompiledModelCompatibilityInfo—especially for multi-device EP scenarios—by documenting the required “per-device verdict then worst-of fold” behavior, and updates the example plugin EP + tests to match that guidance.
Changes:
- Expanded
ValidateCompiledModelCompatibilityInfodocumentation to define multi-device interpretation and required verdict folding rules (with guidance for “no opinion” vs “bad artifact” cases). - Updated the example plugin EP’s compatibility validation to compute per-device compatibility and combine results using the documented reduction.
- Added end-to-end and unit tests covering single-device, “no opinion”, and multi-device folding behavior.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/test/autoep/test_model_package.cc | Adds tests that exercise the public compatibility API against the example plugin EP, including multi-device folding behavior. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.cc | Refactors the example EP factory’s compatibility validation to evaluate per-device and combine verdicts per the documented rules. |
| onnxruntime/test/autoep/library/example_plugin_ep/compatibility_combine.h | Introduces a shared helper to combine per-device compatibility verdicts using the documented identity + worst-of rule. |
| include/onnxruntime/core/session/onnxruntime_ep_c_api.h | Adds detailed guidance and a normative algorithm for multi-device compatibility validation and result combination. |
Copilot's findings
- Files reviewed: 4/4 changed files
- Comments generated: 3
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
skottmckay
reviewed
Jun 25, 2026
| * \param[in] num_devices Number of entries in `devices`. | ||
| * \param[in] num_devices Number of entries in `devices`. May be 0 when no device-specific context is available; | ||
| * in that case evaluate `compatibility_info` against the EP's own configuration and do NOT | ||
| * dereference `devices`. |
Contributor
There was a problem hiding this comment.
Do we ever call this with zero devices? OrtApis::GetModelCompatibilityForEpDevices requires at least one.
Comment on lines
+2806
to
+2807
| * All devices provided belong to the same execution provider instance that this factory creates. The set represents | ||
| * the devices the EP would run the model on *together* (e.g., multi-adapter or multi-GPU scenarios), NOT a menu of |
Contributor
There was a problem hiding this comment.
Maybe a concrete example would help. e.g OV EP has OrtEpDevice instances for CPU, GPU and NPU. How many calls are we expecting to this API and with what combinations?
Comment on lines
+2816
to
+2818
| * Required implementation when num_devices > 1 (a "best of any device" result is NOT permitted -- a single verdict | ||
| * cannot convey which device it applies to, so ORT would otherwise be told a model is runnable on a set that | ||
| * contains a device it cannot run on): |
Contributor
There was a problem hiding this comment.
Wouldn't the user provide the same set of OrtEpDevice instances when running the model and that's a hard requirement?
e.g. say the devices were CPU and NPU. model is optimal for NPU. it's up to the EP which devices it actually uses at runtime, so as long as I have the optimal device in that set why should I downgrade the rating due to extra devices as I am not forced to use those?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ValidateCompiledModelCompatibilityInfois underspecified in that it does not provide enough guidance for EP implementors on how to handle multi-device cases. This change attempts to fill that gap by defining the intended prioritization of the various compatibility states. It also updates the reference implementation in the test plugin EP.Motivation and Context
This change is being made in response to customer feedback.