Add WinUI DevTools: live inspection, binding diagnosis and source-anchored UI comments - #943
Nikola Metulev (nmetulev) wants to merge 44 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ilters - -a falls back to title matching (preferring ApplicationFrameHost frames) when the matched process owns no top-level window, and fails instead of reporting an empty target - inspect --interactive includes Document and writable-value elements - --type/--root/--class-name on every single-selector ui command; invoke accepts them without --action Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…d examples - `winapp ui --help` renders plain text: golden path, -a/<selector> definitions, and commands grouped by task (2,741 bytes redirected, down from 6,538) - every ui command gets plain-text help with 1-3 placeholder examples and a one-line global options pointer; examples are data (IHelpExamples) and are parse-tested - `winapp ui <unknown>` fails with exit 1 (even with --help) before sandbox routing, suggesting up to two commands; --json emits the error envelope with `suggestions` - `find` and `tree` are aliases for `search` and `inspect` - root help points agents at `winapp ui --help` - a missing option value ends with a pointer to --help instead of the full help - inspect's footer example uses set-value for editable-only elements Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ghten unknown-command detection - `-a` falls back only to ApplicationFrameHost frames; a process with no window yet keeps the process-scoped target so `wait-for` and `status` behave as before - bare `winapp ui` still prints the group help after "Required command was not provided" - only the first unmatched token counts as a command, so `ui -a Notepad` and `ui -- inspect` report the normal parse error instead of an unknown command Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
| { | ||
| } |
| decimal.TryParse(requested, NumberStyles.Integer, CultureInfo.InvariantCulture, out var b) && a == b, | ||
| "Double" => double.TryParse(actual, NumberStyles.Float, CultureInfo.InvariantCulture, out var a) && | ||
| double.TryParse(requested, NumberStyles.Float, CultureInfo.InvariantCulture, out var b) && | ||
| double.IsFinite(a) && double.IsFinite(b) && a == b, |
Build Metrics ReportValidation did not pass. Artifacts were uploaded before validation finished; check the workflow run before using them. Binary Sizes
.NET Test Results (TRX reports)Other suites are reflected in the overall validation status above. ✅ 9361 passed, 41 skipped out of 9402 tests in 1431.8s (+1486 tests, +81.5s vs. baseline) Test Coverage✅ 86% line coverage, 80.1% branch coverage · CLI Startup Time67ms median (x64, Try This BuildInstalls the MSIX for your architecture, replacing any previously installed build. Needs the GitHub CLI — the command offers to install it and sign you in if it is missing. & ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) 943Switching between builds often?Put the tool on your PATH once: & ([scriptblock]::Create((irm https://raw.githubusercontent.com/microsoft/winappCli/main/scripts/winapp-pr.ps1))) -AddToPathThen this build is just: winapp-pr 943Run Updated 2026-09-30 15:27:12 UTC · commit |
d25242b to
d69d6cb
Compare
Triage fixes (groups 1 and 2)Seven commits on top of the original squash. Each fix has a regression test that fails without it. Live checks ran against AI Dev Gallery and FixedPicking text with no source picks the authored control. Hover and click share the path; Just my XAML off keeps the raw hit. A comment on an element that still has no source says
Comment anchors. A unique declaration or
Rerun while the app is running Identical Smaller fixes Also fixed: the comments pane says "not linked to source" for unanchored comments and pluralizes the footnote correctly; Layout adorners is a UIA toggle ( Group 2: Not reproducedQuick-edit panel off-screen. On a 150% single monitor, the panel stays inside the work area for a full-width TextBox and a Gallery expander, with the Popup unconstrained. The existing placement tests already cover the flip, shift and height cap. I added the element and viewport to the Also
|
|
Quick-edit panel off-screen: not a bug. I reproduced the reported setup: a window as wide as the monitor, then picks on a full-width element and on one near the bottom, at 150% (monitor 3840×2160 px, work area 3840×2088 px). The panel stays inside the work area: The original report compared physical coordinates with a logical screen size. |
Group 3 decisions implementedFour commits. Each change has a test that fails without it. "Before" is the pre-fix build; "after" is from this branch, run on Pin (kept) is a toolbar toggle. It uses the same accent on-state as Pick and Layout adorners, and the corner dot is gone. Theme toggle (kept on the toolbar). Tooltip "Toggle the inspected app between Light and Dark themes" is now "Switch app theme". Inspector Properties pane. Order is now: type and name, ancestors, source line + Open, then the filter and the grid. The box model and the offered/took explanation move into a Layout section after the grid. It is collapsed by default, and its state is remembered across runs (
No
Guide split. Unchanged: |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…hored UI comments `winapp run --devtools` injects an inspection agent into a WinUI 3 app: an in-app overlay (pick, quick edit, comments), an inspector window, and a `winapp devtools` command group for agents (inspect, search, get/set property, get-source, diagnose-binding, comments). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Keep walking past a nearer generated scope that does not reference the target
(a UserControl whose content the page supplies) instead of reporting unavailable.
- Enrol windows held in static App fields (App.MainWindow { get; private set; }).
- Resolve non-public members for {x:Bind}, which compiles against its owner.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…omments A hit with no source (a Button's string content, a TextBox placeholder, a nav item label) now promotes to its nearest ancestor with authored source, the same as a template part. Hover and click share this path; Just my XAML off keeps the raw hit. A comment saved on an element that still has no source reports 'Saved. Not linked to source.' in the panel. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- A declaration that matches exactly one source declaration, or a unique x:Name, is a confirmed anchor; runtime instance counts no longer weaken it. - Drop 'via type' candidates when a declaration or x:Name match exists. - Show the historical creation location only when there is no current match, and never print an empty line number. - A comment without a source file is weak, requires confirmation, and says it is not linked to source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Comments pane: an unanchored comment says 'not linked to source' instead of 'not on this screen' and is not counted as off screen; the footnote agrees in number. - Layout adorners is a ToggleButton, so UI Automation exposes its state; the tooltip no longer says 'Show' while it is on. - The precedence chain shows a null default as empty instead of the raw handle '0'. - The inspector's extra-width note appears only when HorizontalAlignment is not Stretch. - Log the quick-edit panel's element and viewport for placement triage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…arnings - When staging fails because the app is still running, name its PID(s) by matching the process image to the staged path, instead of a generic access-denied error. - Collapse identical source-location warnings into one entry with a count, and say to rebuild when the build output does not match the XAML sources. - run --devtools --detach ends with 'Next: winapp devtools inspect -a <pid>' and no longer prints 'running bytes are not verified'. - devTools fields in run --json are camelCase like the rest of the new fields. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- set-property warns when it replaces a {Binding} or overrides an x:Bind (replacedBinding in JSON).
- get-property/set-property take the property positionally; -p still works.
- search falls back to the whole tree when nothing in your XAML matches (items created
from data), and no longer claims displayed text is not searched.
- inspect --filter says when matches may be below --depth.
- comments: -a alias; list --status footer counts what was hidden; --on is hidden
where it is always rejected.
- Relaxed JSON escaping so captured XAML is readable; 'properties' instead of 'propert(ies)'.
- Hide the healthy-binding scope disclaimer from human diagnose-binding output.
- Binding-owner e2e: static-held windows diagnose; unique x:Name comments confirm.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Pinned uses the same accent on-state as Pick and Layout adorners and is a ToggleButton, so UI Automation reports its state. The corner dot, which read as a notification badge, is removed. - The theme button's tooltip is 'Switch app theme' so it is not clipped at the edge. - Move the settings helpers to DevToolsSettings.h so the inspector can use them. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The Properties pane shows the header (type and name, ancestors, source line and Open), then the filter and the grid. The box model and the offered/took explanation move into a Layout section after the grid, collapsed by default and remembered across runs. Source-verification notes are one line; the full reason is on hover and exposed to screen readers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…inding - winapp devtools JSON (attach, list, live commands, errors) uses processId like winapp ui and winapp target. run --json keeps ProcessId; --pid and -a are unchanged. - The set-property binding warning ends with 'Restart the app to restore the binding.' Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
devtools.md starts with numbered steps and two screenshots, then covers reviewing comments with an agent, finding elements, bindings, live changes, refreshing, the terminal workflow and failures. devtools-advanced.md covers Sandbox, attaching to a running app, window and subtree targeting, comment storage and the protocol. Links in usage, Sandbox docs and skills point at the right page. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DevTools.negotiate (and the legacy hello reply) report processId instead of pid, matching the CLI payloads. The protocol schema and the CLI's negotiate parser change with it; there is no alias because the protocol has not shipped. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Generalize #942's ui help renderer and unknown-command handling to any compact help group; winapp ui output is unchanged apart from one pointer line. - winapp devtools --help: purpose, the main workflow, what -a and <selector> mean, commands grouped by task, and when to use winapp ui instead. Every devtools command has examples, stored as data and parsed in tests. Unknown devtools commands suggest the closest one. - Root help gets a short 'Start here' block for winapp ui and winapp devtools. - A winapp ui command whose target has the DevTools agent prints one tip line on stderr (not with --json or --quiet). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
#942 added the options to ui record; the hand-written argument builder did not forward them, which failed the generated-options sync test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The winui-devtools skill now matches requests to change, translate or restyle a running UI without editing source; winapp-ui-automation says it reads, clicks and types in any app and defers live WinUI edits to winui-devtools. The UI automation and DevTools docs mention the new help and the ui tip. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
75f8ae2 to
537ebca
Compare
A Type argument cannot be serialized for discovery, so the two rows were discovered as one case and ran as two. CI's CLI shard 2 requires the reported total to match discovery exactly, which failed on #942 and here. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round 4 skeptical retest: AI Dev GalleryBuild: installed
Setup:
Picking:
Claims
New issues, by severityN1 (High for the agent loop): a comment on an unnamed element needs confirmation again after the agent makes the requested change. Repro on a copy of
Result:
N2 (Medium): rerunning with
N3 (Medium, unconfirmed): hover and click disagreed once with Just my XAML on.
N4 (Low-Medium): a pick on a NavigationViewItem label selects the template's
N5 (Low):
Bottom line: Every round-3 High is fixed for the main flows:
What still breaks the agent loop is N1: an agent that implements a comment on an unnamed element is then asked to confirm it. Behind that are the offline |
DevTools discoverability from the CLIThis PR is now stacked on #942, and
|
After an agent implements a comment on an element without x:Name, the declaration no longer matches exactly. When the element keeps its tree position and resembles the captured declaration more than any other element of its type, it is now a strong match, and type-only candidates are dropped. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
comments add --file X --name Y now captures the authored identity and line when Y names exactly one element in X, so the comment is strong and prints no line-less historical location. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A --devtools run over a running instance either failed at staging, or launched an instance the running one absorbed, with a jargon error and the PID of an exited process. It now stops before registering, names the running PID, and says to close it. When re-registering a changed package closes a running instance, run says so in one line. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A pick on a NavigationViewItem restyled in the app's own XAML selected the template's Border, because it has authored source. Promotion now also skips declarations inside a <ControlTemplate>, up to the element the developer declared. Hover re-picks when Just my XAML changes, so a cached highlight no longer disagrees with the click. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Source info was backfilled 100 elements per request, so search and inspect said the agent had not finished classifying for several calls after navigating. Backfill now continues in UI-thread batches of 100 until done or 250 ms have passed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Agents reuse -a across devtools commands, but list and get rejected it as a parse error. With -a <pid>, they now read the comments of that running DevTools app's project. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- The theme button's accessible name matches its tooltip. - get-source on an unverified build says to rebuild in plain language. - diagnose-binding on a literal says the property has no binding. - Piped devtools list keeps one line per app. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Round-4 fixes (head
|
PasswordBox.Password was readable in plain text through get-property, the precedence chain, query fields, the inspector and binding diagnosis. Every property-chain read now replaces the value of a property named *Password with <redacted> before any caller sees it. Property rows mark it redacted and read-only, and JSON keeps the key with "redacted": true. Binding diagnosis redacts its source, resolved and target values, and writes to such a property are refused so nothing echoes it back. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The diagnostics chain reports a null value, including an empty String such as AccessKey="", as the handle text "0". Property rows showed that "0" while the precedence chain showed "". Rows and the quick-peek read now show "" and keep valueState "null". Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Nothing has shipped, so DevTools.negotiate now reports protocolVersion "0" alongside experimental: true, matching the legacy hello reply. It moves to "1" together with dropping experimental once the protocol is stable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A refused whole-app query write reported only truncated: true. The query result now carries truncatedBy (for example tree-changed, stale-node, unknown-child or time-budget), and the human summary prints it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The inspector keeps its snapshot model, but re-reads live values when the selected element is picked or clicked again, when a property row or its editor is opened, and when a quick-peek text editor gains focus. Nothing is re-read over an open editor, typed text or an unconfirmed write. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Presentation-prep findings (head
|
The toolbar, highlight, selection chrome, pick catcher and comment markers now live in IXamlDiagnostics2::GetUiLayerForXamlRoot's layer for each XamlRoot. They draw above app content, in-window popups and dialogs, and stay out of the app's tree and layout. Each chrome Popup remains the fallback host, chosen per XamlRoot when any step of the layer lookup fails. - UIA does not walk into the layer. An unparented zero-size Popup per XamlRoot holds a proxy whose automation peer reports the layer content, so screen readers and winapp ui keep the same tree, names, roles and toggle states. - The quick peek's windowed Popup opens unparented on the XamlRoot, and the pick catcher keeps only a border around it: windowed popup input is hit-tested against the layer too. - Tab cycles within the toolbar and Esc reaches the pick handler, as they did inside a Popup. The chrome follows the app content's theme, which the layer does not inherit. - Overlay.getState reports host (uiLayer, popup or none). The internal Internal.overlayHost forces the fallback for tests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Lifting the windowed quick-peek Popup out of the selection markup broke FindName lookups for its status and error text, so the comment failure status never showed (e2e probe). The pick catcher's cut-out around the open peek is what lets the windowed popup receive input, so the Popup now stays where it was and is only seeded out of the census. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… open A windowed Popup whose parent is in the diagnostics UI layer makes the framework list the layer's chrome under that popup's window, so UIA clients resolved the toolbar to the peek and 'winapp ui focus' on it failed with foreground_not_target (e2e probe). The peek's Popup now opens unparented on the XamlRoot again; names inside it are looked up from its content when the selection root no longer reaches them. The peek's comment status was looked up from the binding-diagnosis root, which is only set when the element has binding rows, so unbound elements never showed Saving/Saved/Unsaved status. It now uses the selection root. The probe checks that status on an unbound element. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
While the toolbar is shown, Ctrl+Shift+F12 in the app expands it and focuses its first action; Esc from the toolbar returns focus to the app element that had it (a following Esc still leaves pick mode). The chord is ignored while the toolbar is hidden, including --no-overlay. Plain F12 is avoided because Windows reserves it to break into an attached debugger. The overlay probe focuses a fixture button, sends the chord and Esc, and checks where UIA reports focus. The toolbar carries the shortcut as its AcceleratorKey and tooltip; the guide and skill document it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Esc did not return focus in the CI probe, and it did not reproduce locally. The shortcut captured the return target only when the toolbar's focus counter read zero, which depends on every LostFocus arriving. The shortcut and Esc now check whether the focused element is inside the toolbar instead. The overlay probe keeps the app's DevTools log as evidence so a repeat failure shows which step failed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
92f4205 to
26d9905
Compare





Adds WinUI DevTools to winapp.
winapp run --devtoolslaunches a WinUI 3 app with an in-app overlay so a developer can pick an element, tweak it, and leave comments anchored to the XAML that produced it. The newwinapp devtoolscommand group exposes the same live tree to the terminal and to agents: inspect, search, read/set properties, diagnose bindings, and read or resolve the comments.Usage
What's in it
winapp devtoolscommand group (inspect,search,get-property,set-property,diagnose-binding,comments, ...).winapp/ui-comments.json--on sandbox)winapp devtools --helpwith the workflow, examples for every command, and a root-helpStart herepointer;winapp uiprints a tip when its target has DevToolsdocs/guides/devtools.mdanddevtools-advanced.md, plus updated usage docs and agent skillsTriage fixes in this PR
Fixed (details and before/after in the PR comments):
diagnose-bindinghandles non-public, page-owned and window-owned x:Bindx:Names are confirmed comment anchors; cleaner anchor outputset-propertywarns when it replaces a bindingNot a bug: the "quick-edit panel off-screen" report mixed physical and logical coordinates; the panel stays inside the monitor work area (see comments).
Validation
CI is green. Each fix has a regression test, and fixes were checked live against AI Dev Gallery and
samples/winui-app.