iphone-use

Computer-use, but for the iPhone — let AI agents (and your browser) see and drive a real phone.

English · 简体中文

iphone-use-promo-en.mp4

_{2-minute demo · 中文版}

iphone-use lets an AI agent see and operate a real iPhone: read the screen as text, tap, swipe and type, and get told plainly when an action did not land. It works on apps that have no API, including banking and payment apps that hide their screens from capture.

A daemon on your Mac runs WebDriverAgent on a USB-connected iPhone and exposes it as:

• an HTTP API for agents and scripts (/agent/*),
• an MCP server with 21 tools for Claude Code, Claude Desktop and other MCP clients,
• a web page and a native iOS app for people (live screen, tap, type).

Nothing touches the Mac's own screen, cursor or focus.

Install

You need macOS 15+, full Xcode signed in to a development team (a free Personal Team works), an iPhone with Developer Mode on and trusted over USB, and brew install libimobiledevice.

curl -fsSL https://raw.githubusercontent.com/leeguooooo/iphone-use/main/install.sh | sh
~/.iphone-use/setup-wda.sh           # builds and starts WDA; keep the phone unlocked

Then open http://<mac-ip>:44321/phone and log in with the password the installer printed. ~/.iphone-use/setup-wda.sh doctor explains a USB, trust, VPN or signing problem; iphone-use upgrade updates everything later.

Use it from an agent

The installer also installs the agent skill. For MCP clients:

{
  "mcpServers": {
    "iphone-use": {
      "command": "/Users/YOU/Applications/iPhoneUse.app/Contents/MacOS/iphone-use-mcp",
      "env": { "PHONE_REMOTE_URL": "http://127.0.0.1:44321", "PHONE_REMOTE_TOKEN": "<agent token>" }
    }
  }
}

Or call the HTTP API directly:

AUTH="Authorization: Bearer $TOKEN"; CTL="X-Phone-Control: 1"
curl -s -H "$AUTH" $HOST/agent/elements                     # the screen as text + a snapshot token
curl -s -H "$AUTH" -H "$CTL" -X POST "$HOST/agent/input?return=delta" \
  -d '{"type":"tap","element":7,"snapshot":"…"}'            # tap, then what changed (~4 s)

Every answer says whether the action was applied, not sent, or unknown, and whether a retry is safe. Taps on covered or hidden elements and writes that did not stick are refused or reported instead of passing silently.

Flows: do a task once, replay it with no model

Tasks you repeat are kept as flows: reviewed per-app scripts that replay without a model, so they cost no tokens. The official registry is leeguooooo/iphone-use-flows; record a flow in the browser's 流程 panel and publish it there with a PR.

MCP=~/Applications/iPhoneUse.app/Contents/MacOS/iphone-use-mcp
$MCP flow update                            # sync the registry (sha256-checked)
$MCP flow list                              # what is available
$MCP flow run health/export-all-zh-cn       # replay one; failures say which step and why

More

• Full guide: browser and iOS app, flows and the flow registry, lifecycle, configuration, security, development.
• Agent API reference · MCP tools · Architecture · WDA setup
• Security in one line: the password protects port 44321 only; WDA's own ports on the phone are unauthenticated, so use a trusted network (details).
• Issues and ideas: GitHub issues.

License

MIT

The *-use family

Small, composable CLIs that give an AI agent hands on one real thing. Same shape everywhere: curl … install.sh | sh to install, JSON on stdout, and a companion skill that teaches your agent to drive it — each repo's own README says how to install its skill, since the right way differs per project and per release.

Repo	Gives your agent
chrome-use	A real browser — logged-in sessions, forms, scraping, screenshots
mail-use	Email — read, search, send, triage across Gmail / QQ / 163 / any IMAP
wechat-use	WeChat on macOS — send messages, query contacts and history
discord-use	Discord — messages, channels, forums, webhooks (REST-only, Rust)
cookie-use	Many logged-in accounts per site — capture, switch, apply sessions
profile-use	Your personal profile, safely — fill signup / KYC / checkout forms
bitwarden-use	Bitwarden / Vaultwarden — headless passkey (FIDO2) login
chatgpt-use	Your ChatGPT subscription as a coding-agent backend — no API key
computer-use	The macOS desktop itself
pixcake-use	Read-only PixCake probing — snapshot / diff / SQLite inspection

Author

Built by 郭立 (Guo Li / leeguoo) — leeguoo.com · GitHub · X · more tools in the *-use family.