fix(config): skip winml quantization for EPs that quantize internally - #1242
Merged
Conversation
Auto-precision was keyed on the device alone, so every NPU host resolved to
w8a16. On an AMD host the auto-selected EP is VitisAI, which quantizes to XINT8
itself and cannot consume a winml-produced QDQ graph: it partitions the
QuantizeLinear/DequantizeLinear and IsNaN nodes back to CPU and then aborts
inside xir with
Check failed: attrs_.count(key) > 0 Attrs doesn't contain attribute data
That abort is a native fail-fast, so the process dies with 0xC0000409 instead of
the build reporting an error. tests/e2e/test_eval_e2e.py was killed on its very
first test and none of its 46 cases ran.
The policy already existed, but only inside the eval harness:
scripts/e2e_eval/run_eval.py worked around it by passing --no-quant. The CLI
itself still produced w8a16 for every user on an AMD NPU.
Make auto-precision EP-aware. EPS_WITH_INTERNAL_QUANT in utils/constants.py is
now the single source of truth, consumed by both resolve_precision and the eval
harness. When it matches, resolve_precision sets PrecisionPolicy.skip_quantization
and leaves precision at "auto" -- winml makes no precision choice, the EP does.
Both policy consumers honor the flag by applying exactly what --no-quant does
(quant = None), so no fake precision value is invented.
Warn at WARNING level when this happens: the artifact stays unquantized and the
accelerator applies its own scheme at load time, which differs from every other
EP. An explicit --precision still forces a winml quantization pass.
Drop the 13 per-test require_not_ep("vitisai") guards in
tests/e2e/test_eval_e2e.py. They were added one at a time as models hit the
abort, and the first test was never guarded, so the module took the whole
process down on AMD anyway. These tests now run for real.
Verified on an AMD Ryzen AI NPU host: tests/e2e/test_eval_e2e.py goes from
crashing on test 1 to 41 passed / 5 skipped / 0 failed, with zero
VitisAI-related skips. Full unit suite: 7897 passed.
Review follow-up: the previous commit only covered an explicit --ep and the
fully-automatic path. It missed the common "pinned device, omitted EP" case.
`_resolve_policy_target` short-circuits when the device is concrete, returning
`(device, None)`, so `resolve_precision` saw `ep=None` and fell through to the
w8a16 NPU default. The EP was only deduced afterwards, when computing
`compile_provider`. On an AMD-only host that deduction yields VitisAI, so
`winml config -m ... --device npu` (and `resolve_quant_compile_config(device=
"npu")`) still produced the QDQ graph VitisAI cannot consume.
Confirmed on an AMD Ryzen AI host before this change:
winml config -m microsoft/resnet-50 --device npu
-> quant: weight_type=uint8 activation_type=uint16 mode=static
Resolve the effective EP once, up front, and derive both the auto-precision
decision and `compile_provider` from it so the two can never disagree. The same
command now emits the skip-quantization warning and writes `quant: null`.
Add coverage for the pinned-device path: an AMD-only host skips quantization, a
QNN host keeps w8a16, and the build-facing `resolve_quant_compile_config`
wrapper produces no quant config.
Also move the typing-only `EPNameOrAlias` import into a TYPE_CHECKING block
(CodeQL flagged it as unused, since it is referenced only from the quoted
`cast()` that ruff's TC006 requires).
…ions
Second review follow-up, same class of gap as the previous commit but on the
harness side.
`run_eval.py --device npu` with `--ep` omitted left `_should_skip_winml_quant`
looking at `None`, so `_build_jobs()` kept expanding the quantized NPU fan-out
and `_resolve_precision()` returned `w8a16`. `_run_build()` forwards that as an
explicit `--precision`, which suppresses the product-side auto-precision policy
added in the previous commit, and `--no-quant` was not injected either. The
harness could therefore still hand VitisAI the QDQ graph this PR exists to
avoid.
Deduce the effective EP the same way the product does and take it into account
before any policy decision:
_deduce_ep_for_device(device) -> default_ep_for_device on a concrete device
_effective_ep(ep, device) -> explicit --ep, else the deduced one
`_should_skip_winml_quant` now takes the device, and all five call sites pass
it. `--device auto` deliberately does not deduce: the harness forces no
precision there, so the product resolves the device itself and its own
auto-precision policy already applies.
Confirmed on an AMD Ryzen AI host:
_should_skip_winml_quant(None, "npu") False -> True
_resolve_precision("npu", None) "w8a16" -> None
Adding the deduction made five existing tests depend on whichever EP the host
registers (they flipped on an AMD box). Pin it with an autouse fixture so the
suite stays deterministic everywhere; the new tests patch it explicitly to
exercise both an AMD-only and a QNN host.
DingmaomaoBJTU
approved these changes
Jul 29, 2026
DingmaomaoBJTU
left a comment
Collaborator
There was a problem hiding this comment.
LGTM — solid fix. Centralizing the internal-quant EP policy into EPS_WITH_INTERNAL_QUANT in constants.py and deducing the effective EP before the auto-precision decision is the right approach. The skip_quantization flag on PrecisionPolicy makes the caller contract explicit. Test coverage is comprehensive (mocked host deduction for AMD vs QNN, build.py integration, e2e skip removal). Minor nit: the _deduce_ep_for_device.cache_clear() calls in test teardown could be wrapped in try/finally to avoid stale cache if an assertion fails mid-test, but the autouse fixture already handles the main path. Approved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tests/e2e/test_eval_e2e.pydoes not fail on an AMD Ryzen AI host — it kills the pytest process on its very first test, so none of its 46 cases run:The same reproduces straight from the CLI, with no test harness involved:
Root cause
_AUTO_PRECISIONinconfig/precision.pyis keyed on the device only:On an AMD host the auto-selected NPU EP is VitisAI, so
--precision autoproduces a winmlw8a16QDQ graph. VitisAI quantizes to XINT8 inside the EP and is not meant to consume that graph: it partitions the quantize/dequantize andIsNaNnodes back to CPU, then aborts inside xir. The abort is a native fail-fast, so the process dies rather than the build reporting an error.This policy was already established — but only inside the eval harness.
scripts/e2e_eval/run_eval.pycarried its own_EPS_SKIP_WINML_QUANTand worked around the problem by passing--no-quanttowinml config/winml build(see #1226). The product itself still handed every AMD NPU user aw8a16graph.Verification that the unquantized track is the fix, same model and host:
w8a16(the auto default)0xC0000409accuracy 1.0, results writtenChange
utils/constants.py: addEPS_WITH_INTERNAL_QUANTas the single source of truth for "this EP runs its own quantization pipeline".run_eval.pynow consumes it instead of defining its own copy.config/precision.py:resolve_precisionsetsPrecisionPolicy.skip_quantizationwhen the resolved EP matches, and leavesprecisionat"auto"— winml makes no precision choice here, the EP does. No fake precision value is invented.config/build.py: both policy consumers (_apply_device_precision_policy,resolve_quant_compile_config) honor the flag first and apply exactly what--no-quantdoes (quant = None).Warn at
WARNINGlevel so the deviation is visible to end users:tests/e2e/test_eval_e2e.py: drop the 13 per-testrequire_not_ep("vitisai")guards. They had been added one at a time as individual models hit the abort;test_image_classificationwas never guarded, so the module took the whole process down on AMD regardless. These tests now run for real.An explicit
--precisionis unaffected — onlyautois redirected.--precision autow8a16w8a16--epon a QNN hostw8a16--precision w8a16w8a16Verification
On an AMD Ryzen AI NPU host:
pytest tests/e2e/test_eval_e2e.py -m e2e→ 41 passed, 5 skipped, 0 failed (was: process killed on test 1, 0 cases completed). Zero VitisAI-related skips; the 5 remaining arerequire_ep("qnn")×3 andrequire_not_ep("migraphx")×2.pytest tests/unit→ 7897 passed, 77 skipped, 3 xfailed.winml config -m microsoft/resnet-50 --device npu --ep vitisai→ warning shown, generated config has"quant": null.ruff check/ruff format --checkclean.Notes for follow-up (not in this PR)
vaip-ort-api.cppemit oneE ... Invalid NodeIndexline per graph node, twice (808 lines for a 404-node ViT; 98.6% of the full e2e log). Inference is correct and the tests pass — it is pure log noise from the EP, but it buries real test output. Worth an upstream report to AMD.41 passed, 5 skippedis printed, the process hits0xC0000005inMicrosoft.Windows.AI.MachineLearning.dllteardown. That is winml perf: post-result Windows 0xC0000005 in Microsoft.Windows.AI.MachineLearning.dll teardown #1233, reproduces on Intel CPU EP too, and is unrelated to this change.