feat(optimize/analyze): preview applicable optimizations and check their produced-operator support

docs/commands/analyze.md

@@ -28,6 +28,7 @@ $ winml analyze [options]
 | `--run-unknown-op` / `--no-run-unknown-op` | | flag | disabled | For operators not in the rule database, build a minimal ONNX graph and run it on the target EP locally to determine support. Enable when local EP libraries are available. |
 | `--save-node` | | `partial\|unsupported` | *(none)* | Save partial or unsupported node subgraphs to disk for further investigation. Can be specified multiple times: `--save-node partial --save-node unsupported`. |
 | `--optim-config` | | `PATH` | *(none)* | Save the auto-discovered optimization config (merged across all analyzed EPs) to a JSON file. |
+| `--check-optim` / `--no-check-optim` | | flag | disabled | For each optimization that would change the model, check whether the operators it *introduces* (its added/modified nodes) are supported on the resolved EP/device. Answers "if I apply this optimization, will its output run on my target?" before you commit to it. |
 
 ## How it works
 
@@ -102,6 +103,12 @@ Enable local execution for operators not in the rule database:
 $ winml analyze --model model.onnx --ep qnn --device NPU --run-unknown-op
 ```
 
+Check whether the operators each applicable optimization would introduce are supported on the target:
+
+```bash
+$ winml analyze --model model.onnx --ep qnn --device NPU --check-optim
+```
+
 ## Common pitfalls
 
 - **Omitting `--ep` uses `auto` (inferred from local availability)** — to analyze every EP regardless of what is installed, pass `--ep all`. Specify `--ep <name>` when you know your target hardware.

docs/commands/optimize.md

@@ -22,6 +22,7 @@ $ winml optimize [options]
 | `--verbose` | `-v` | flag | off | Enable verbose output. |
 | `--list-capabilities` | `-l` | flag | off | Print all registered optimization capabilities grouped by category and exit. Add `--verbose` for descriptions and ORT names. |
 | `--list-rewrites` | | flag | off | Print all available pattern-rewrite families with their source-to-target mappings and exit. |
+| `--check-optim` | | flag | off | Analyze which optimizations would apply to `--model` and the nodes/constants they affect, then exit **without writing any output**. Probes every capability independently. |
 | *(dynamic)* | | flag | *(per capability)* | Each registered capability generates a `--enable-<name>` / `--disable-<name>` pair. Run `--list-capabilities` to see the full current list. Examples: `--enable-gelu-fusion`, `--disable-constant-folding`. Pattern-rewrite flags follow the form `--enable-<source-slug>-<target-slug>`; run `--list-rewrites` to discover all names. |
 
 ### Configuration precedence
@@ -85,6 +86,39 @@ Discover pattern-rewrite families and their flag names:
 $ winml optimize --list-rewrites
 ```
 
+Check which optimizations apply to a model before committing to a run (writes nothing):
+
+```bash
+$ winml optimize -m bert-base-uncased.onnx --check-optim
+```
+
+```text
+Input: bert-base-uncased.onnx
+--check-optim — analyzing applicable optimizations (no output written).
+
+Loading model...
+Probing 61 optimization capabilities... (this can take a while on large models)
+
+2 applicable optimization(s):
+
+--enable-matmul-add-fusion  (matmul)
+  Fuse MatMul+Add operations into single kernel
+  nodes: -1 ~1
+    removed: MatMul (1)
+      - MatMul 'mm'
+    modified: Gemm (1)
+      - Gemm 'mm/MatMulAddFusion'
+
+--enable-clamp-constant-values  (surgery)
+  Clamp extreme float constants (e.g., -inf -> -1e3) to prevent quantization issues
+  constants: ~1
+      - BIG
+
+Enable any of the above with its --enable-* flag (dependencies are auto-enabled).
+```
+
+Add `-v` to list every affected node and constant instead of a sample.
+
 ## Common pitfalls
 
 - **`--model` is required for actual optimization** — it can be omitted only when using `--list-capabilities` or `--list-rewrites`. Missing `--model` in any other case raises a usage error.

src/winml/modelkit/analyze/core/runtime_checker.py

@@ -161,6 +161,7 @@ def op_support(
         run_unknown_op: bool = False,
         save_node_types: set[str] | None = None,
         on_node_result: Callable | None = None,
+        node_output_filter: set[str] | None = None,
     ) -> list[PatternRuntime]:
         """Check operator-level runtime support.
 
@@ -185,6 +186,12 @@ def op_support(
                   level enum. Call ``.value`` to get the string, e.g.
                   ``"supported"``, ``"partial"``, ``"unsupported"``,
                   ``"unknown"``.
+            node_output_filter: Optional set of output tensor names. When
+                provided, only nodes that produce at least one of these tensors
+                are checked; all other nodes are skipped. This lets callers
+                restrict the check to a specific subset of the graph (e.g. the
+                operators an optimization introduced) without paying to check
+                every node.
 
         Returns:
             List[PatternRuntime]: Runtime results for each operator pattern
@@ -228,6 +235,11 @@ def op_support(
         # Use tqdm for progress unless caller provides a callback
         iterator = nodes if on_node_result else tqdm.tqdm(nodes)
         for node in iterator:
+            if node_output_filter is not None and not (
+                node_output_filter.intersection(node.output)
+            ):
+                # Node is outside the requested subset — skip it.
+                continue
             node_start = time.perf_counter()
             result = query.run_for_node(
                 node,

src/winml/modelkit/analyze/optim_output.py

@@ -0,0 +1,244 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""EP/device support of the operators an optimization would produce.
+
+This bridges the optimizer's dry-run applicability analysis with the analyze
+runtime-support rule data. For each applicable optimization it takes the model
+that optimization would produce, then checks whether the operators the
+optimization *introduces* (its added and modified nodes) are supported on a
+target execution provider and device — answering "if I apply this optimization,
+will its output run on my target?".
+
+The check is universal and architecture-agnostic (CARDINAL RULE #1): the set of
+produced operators is derived entirely from the concrete graph diff carried on
+each :class:`~winml.modelkit.optim.CapabilityFinding`; nothing about specific
+operators or model architectures is hardcoded.
+"""
+
+from __future__ import annotations
+
+import logging
+from collections import Counter
+from dataclasses import dataclass, field
+from typing import TYPE_CHECKING, cast
+
+from .models.onnx_model import ONNXModel
+from .models.support_level import SupportLevel
+
+
+if TYPE_CHECKING:
+    from collections.abc import Iterable
+
+    from onnx import ModelProto, NodeProto
+
+    from ..optim import CapabilityFinding, NodeRef
+    from ..utils.constants import EPName
+
+
+logger = logging.getLogger(__name__)
+
+# Ordering used to summarize a mix of per-operator support levels into a single
+# "worst" level. Higher rank = more concerning for the target.
+_SEVERITY: dict[SupportLevel, int] = {
+    SupportLevel.SUPPORTED: 0,
+    SupportLevel.UNKNOWN: 1,
+    SupportLevel.PARTIAL: 2,
+    SupportLevel.UNSUPPORTED: 3,
+}
+
+
+@dataclass(frozen=True)
+class ProducedOperatorSupport:
+    """Support of a single operator an optimization would introduce.
+
+    Attributes:
+        op_type: The produced operator's op type (e.g. ``"Gemm"``).
+        label: Human-readable node label (op type + name/output).
+        change: Whether the node is ``"added"`` or ``"modified"`` by the
+            optimization.
+        support: Support level of the operator on the target EP/device.
+        reason: Optional reason string from the runtime check.
+    """
+
+    op_type: str
+    label: str
+    change: str
+    support: SupportLevel
+    reason: str | None = None
+
+
+@dataclass
+class OptimizationOutputSupport:
+    """Per-optimization view of its produced operators' target support.
+
+    Attributes:
+        name: Capability name (kebab-case, e.g. ``"matmul-add-fusion"``).
+        enable_flag: CLI flag that turns the optimization on.
+        category: Capability category (e.g. ``"matmul"``).
+        description: Human-readable capability description.
+        pipe_name: Name of the pipe that owns the capability.
+        operators: Per-operator support for every produced (added/modified) node.
+        error: Non-empty when the support check could not be completed.
+    """
+
+    name: str
+    enable_flag: str
+    category: str
+    description: str
+    pipe_name: str
+    operators: list[ProducedOperatorSupport] = field(default_factory=list)
+    error: str | None = None
+
+    @property
+    def worst_support(self) -> SupportLevel:
+        """Most concerning support level across produced operators.
+
+        Returns ``SUPPORTED`` when there are no produced operators to check.
+        """
+        if not self.operators:
+            return SupportLevel.SUPPORTED
+        return max((op.support for op in self.operators), key=lambda level: _SEVERITY[level])
+
+    def support_counts(self) -> dict[SupportLevel, int]:
+        """Return a count of produced operators per support level."""
+        return dict(Counter(op.support for op in self.operators))
+
+
+def _produced_node_refs(
+    finding: CapabilityFinding,
+) -> list[tuple[str, NodeRef]]:
+    """Return ``(change_kind, NodeRef)`` pairs for a finding's produced nodes.
+
+    Produced nodes are those an optimization introduces: newly added nodes and
+    nodes modified in place. Removed nodes are excluded — they no longer exist
+    after the optimization, so their target support is irrelevant.
+    """
+    pairs: list[tuple[str, NodeRef]] = []
+    pairs.extend(("added", ref) for ref in finding.added_nodes)
+    pairs.extend(("modified", ref) for ref in finding.modified_nodes)
+    return pairs
+
+
+def _check_one(
+    finding: CapabilityFinding,
+    produced_model: ModelProto,
+    ep: EPName,
+    device: str,
+    model_path: str,
+) -> OptimizationOutputSupport:
+    """Check target support for the operators one optimization would produce."""
+    from .core.runtime_checker import RuntimeChecker
+
+    result = OptimizationOutputSupport(
+        name=finding.name,
+        enable_flag=finding.enable_flag,
+        category=finding.category,
+        description=finding.description,
+        pipe_name=finding.pipe_name,
+    )
+
+    produced = _produced_node_refs(finding)
+    if not produced:
+        return result
+
+    # Output tensor names uniquely identify produced nodes within the graph.
+    produced_outputs = {out for _, ref in produced for out in ref.outputs}
+
+    try:
+        onnx_model = ONNXModel.from_onnx_model(produced_model, model_path)
+        checker = RuntimeChecker(ep=ep, device=device, model=onnx_model)
+        # Only evaluate the produced nodes; skip the rest of the graph.
+        runtimes = checker.op_support(
+            node_output_filter=produced_outputs,
+            on_node_result=lambda _r: None,
+        )
+    except Exception as exc:  # pragma: no cover - defensive, keeps analyze alive
+        logger.warning(
+            "Could not check produced-operator support for '%s' on %s/%s: %s",
+            finding.name,
+            ep,
+            device,
+            exc,
+        )
+        result.error = str(exc)
+        return result
+
+    # Map each produced output tensor to its support result.
+    support_by_output: dict[str, tuple[SupportLevel, str | None]] = {}
+    for runtime in runtimes:
+        node = _matched_node(runtime)
+        outputs = tuple(node.output) if node is not None else ()
+        classification = runtime.result.classification
+        reason = runtime.result.reason
+        for out in outputs:
+            support_by_output[out] = (classification, reason)
+
+    for change, ref in produced:
+        support, reason = _lookup_support(ref, support_by_output)
+        result.operators.append(
+            ProducedOperatorSupport(
+                op_type=ref.op_type,
+                label=ref.label(),
+                change=change,
+                support=support,
+                reason=reason,
+            )
+        )
+
+    return result
+
+
+def _matched_node(runtime: object) -> NodeProto | None:
+    """Best-effort extraction of the ONNX node behind a ``PatternRuntime``."""
+    pattern_match = getattr(runtime, "pattern_match", None)
+    skeleton = getattr(pattern_match, "skeleton_match_result", None)
+    matched_nodes = getattr(skeleton, "matched_nodes", None)
+    if matched_nodes:
+        return cast("NodeProto", matched_nodes[0])
+    logger.debug(
+        "Could not extract matched node from runtime (type=%s); produced operators "
+        "correlated via this runtime will report UNKNOWN support",
+        type(runtime).__name__,
+    )
+    return None
+
+
+def _lookup_support(
+    ref: NodeRef,
+    support_by_output: dict[str, tuple[SupportLevel, str | None]],
+) -> tuple[SupportLevel, str | None]:
+    """Resolve a produced node's support via any of its output tensors."""
+    for out in ref.outputs:
+        if out in support_by_output:
+            return support_by_output[out]
+    return SupportLevel.UNKNOWN, "not evaluated"
+
+
+def check_optimization_output_support(
+    optimization_outputs: Iterable[tuple[CapabilityFinding, ModelProto]],
+    ep: EPName,
+    device: str,
+    model_path: str,
+) -> list[OptimizationOutputSupport]:
+    """Check produced-operator support for applicable optimizations on a target.
+
+    Args:
+        optimization_outputs: ``(finding, produced_model)`` pairs from
+            :func:`winml.modelkit.optim.iter_optimization_outputs`. Materialize
+            the iterator once and reuse it across targets — the dry-run that
+            produces these pairs is target-independent, so only the support
+            lookup needs to run per EP/device.
+        ep: Target execution provider (canonical name).
+        device: Target device string (e.g. ``"NPU"``).
+        model_path: Path to the source model, used for external-data resolution.
+
+    Returns:
+        One :class:`OptimizationOutputSupport` per applicable optimization, in
+        the same order as ``optimization_outputs``.
+    """
+    return [
+        _check_one(finding, produced_model, ep, device, model_path)
+        for finding, produced_model in optimization_outputs
+    ]