Skip to content

agent: enable built-in computer use on Windows - #1383

Merged
AnthonyRonning merged 1 commit into
masterfrom
agent-windows-computer-use
Oct 30, 2026
Merged

AnthonyRonning merged 1 commit into
masterfrom
agent-windows-computer-use

Conversation

@AnthonyRonning

@AnthonyRonning AnthonyRonning commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Turns on built-in computer use (embedded cua-driver) for Maple Work on Windows. This is best effort: it has not been tested by hand on Windows yet. Refs #1295.

What changes

  • build.rs now sets embedded_cua on Windows as well as macOS and Linux.
  • Cargo.toml adds Windows to the cua-driver-sdk dependency. It shares the macOS section, so a future SDK bump still touches two places, not three.
  • No other code changes. The pinned SDK (trycua/cua 2a397557, 0.38.0) already has a Windows backend: UI Automation, SendInput and Windows.Graphics.Capture. Upstream builds and ships it for x64 and arm64.
  • Cargo.lock is unchanged. platform-windows was already resolved in it.

Consent on Windows

Windows has nothing like the macOS permission prompts. Any app running as the user can read UI Automation, send input and capture windows. So the requirement list is empty, and Settings shows the card as Ready with Enable. Computer use stays off until the user switches it on there, and each task still has its own switch. Those two switches are the only consent step. The docs now say this in so many words.

Known Windows limits (now in docs/embedded-cua.md)

  • Admin windows: Maple runs as a normal app, so it can't drive windows that run as administrator, or the UAC prompt.
  • Covered and UWP windows: capturing them needs Windows 10 1903 or later.
  • Video recording: needs ffmpeg on PATH, the same as on Linux.
  • Session 0: the SDK refuses to start outside a normal logged-in desktop session.

Verification

  • cargo metadata --locked passes, so the lockfile needs no change.
  • scripts/audit-sqlcipher-dependencies.py (cargo-deny advisories and bans) passes for aarch64-apple-darwin, x86_64-unknown-linux-gnu and x86_64-pc-windows-msvc.
  • The pre-commit hook passes on macOS. On macOS and Linux the change is only comments and an equivalent cfg.
  • Windows CI, with the ci:windows label added:
    • agent-windows: clippy -D warnings on Windows, compiling platform-windows and the cua code. Passes.
    • agent-full-windows: the native Windows test suite, 2,365 passed and 0 failed. The test binary links the SDK, and the agent::cua::driver tests that only build under embedded_cua ran on Windows. 43 minutes.
    • agent-windows-updater: passes.
  • Not run: a packaged installer build (ci:package), and anything that drives a real Windows desktop.

Manual test after rollout

  1. Settings > Integrations: the computer-use card shows Ready. Turn on Enable.
  2. In a task with computer use on, ask it to list open windows, take a screenshot, click and type in Notepad.
  3. Press Stop during a run. The run should cancel cleanly and the next run should bind again.

🤖 Generated with Claude Code

The pinned cua-driver-sdk already carries a Windows backend (UI
Automation, SendInput, Windows.Graphics.Capture). Turn on embedded_cua
for Windows and depend on the SDK there. Windows grants accessibility,
input and capture to any process at the user's integrity level, so the
requirement list stays empty and enabling computer use in Settings is
the only consent step; the docs now say so, plus the Windows limits.

Refs #1295

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AnthonyRonning AnthonyRonning added the ci:windows Run Windows checks on this PR label Oct 30, 2026
@AnthonyRonning
AnthonyRonning marked this pull request as ready for review October 30, 2026 06:14
@AnthonyRonning
AnthonyRonning merged commit a6e9efa into master Oct 30, 2026
45 checks passed
@AnthonyRonning
AnthonyRonning deleted the agent-windows-computer-use branch October 30, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:windows Run Windows checks on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant