Skip to content

feat: native XCTest device runner replaces WebDriverAgent - #142

Merged
leeguooooo merged 10 commits into
mainfrom
feat/native-runner
Oct 6, 2026
Merged

leeguooooo merged 10 commits into
mainfrom
feat/native-runner

Conversation

@leeguooooo

Copy link
Copy Markdown
Owner

Replaces WebDriverAgent with our own XCTest runner (runner/IPhoneUseRunner). It serves the same routes on device ports 8100/9100, so the daemon's client is unchanged.

Why: WDA was the slow part of every step.

Measured on an iPhone 13 over USB:

runner agent-device
/source 0.08–0.135 s ~0.7 s
tap + settle 1.0 s 2.8–3.3 s
  • Through the daemon, a label tap with an observed delta takes 1.7–2.1 s, against 4.2 s with WDA.
  • MJPEG streams at 27–28 fps.

What changes:

  • runner/ adds the Xcode project:
    • accessibility snapshots go through the private XCAXClient with a fixed attribute set;
    • touches are synthesized with XCSynthesizedEventRecord and skip the quiescence wait;
    • the HTTP server is built on NWListener;
    • MJPEG is served on 9100.
    • build.sh, ci-check.sh and unit-check.sh build and check it.
  • setup-wda.sh builds and launches the runner, then hands it to the same launchd supervisor as before. The bundle id is unchanged, so a profile that already exists is reused.
  • install.sh ships the runner sources (release asset iphone-use-runner.tar.gz); uninstall.sh removes them.
  • Release and PR workflows run runner/ci-check.sh.
  • scripts/runner-compat.py checks every route the daemon uses against a live runner; scripts/runner-smoke.py is a quick smoke test.
  • The guide and README now describe the runner.
  • Fix: setup-wda.sh had stopped parsing under macOS /bin/bash 3.2, because an apostrophe in a heredoc inside $(…) opened a quote that never closed. The launchd supervisor uses /bin/bash, so it failed on every start. CI now runs /bin/bash -n instead of Homebrew bash.

Hardware validation (iPhone 13, named instance through install.sh and setup):

  • setup takes 23 s with the cached product, and the daemon reports drivable=true;
  • runner-compat.py --mutate passes every route, and MJPEG runs at 27 fps;
  • through the daemon: home, launch, scroll, a label tap with delta (including the covered-centre reveal) and a nested tap into 关于本机 all work.

…P API

A single long-running XCTest method (RunnerTests.testServe) hosts an
HTTP/1.1 server on port 8200 (IPU_RUNNER_PORT) and handles one JSON
request per connection serially on the main thread.

- /source reads the foreground app through the private XCAXClient
  snapshot request with nine attributes, a depth ladder and frontier
  re-rooting, serialized in WDA's /source JSON node shape; falls back
  to XCUIApplication.snapshot()
- tap/swipe/longpress/type via XCSynthesizedEventRecord, with
  XCUICoordinate / typeText fallbacks
- home, launch, apps/active, screenshot, alert get/tap, window/size,
  shutdown
- all XCTest quiescence waits disabled process-wide

Private-API pieces are adapted from callstack/agent-device (MIT).
runner/build.sh builds for a generic iOS device with build-for-testing,
signing with team 6ZPXG4KVVS through the ASC API key (env or the WDA
LaunchAgent plist) or unsigned as a compile check, and prints the
.xctestrun path. scripts/runner-smoke.py checks the endpoints and times
/source, /screenshot and an optional tap against 127.0.0.1:8200.
…ient

Every WDA route WdaClient (crates/server/src/wda.rs) uses, with or without
a /session/:sid prefix and the same request/response shapes: session and
status, appium/settings, source, screenshot, window/size, W3C actions
(pointer + key sources), element finds (accessibility id, class name,
predicate string, class chain, ...), element reads and actions, element
gestures, picker wheel select, buttons, apps launch/list, keys, keyboard
dismiss, url, lock state, and the alert routes with WDA's 404 shapes.

Element finds evaluate the locators over the private AX tree instead of
XCUI queries; element ids map to live accessibility elements that are
re-snapshotted on every read, and a vanished one answers 404 stale
element reference. /status and /wda/locked are answered off-main.
The HTTP API defaults to 8100 and an MJPEG stream in WDA's multipart
format listens on 9100 (IPU_RUNNER_MJPEG_PORT, 0 disables), so existing
relays and daemon config work unchanged. Frames are captured on their own
thread only while a client is connected, through testmanagerd's JPEG
screenshot request, scaled with ImageIO, and follow the mjpeg* appium
settings; /status reports the achieved fps.
- build.sh --device <udid> registers the phone in the team's provisioning
  profile during a signed build (build only, nothing is installed)
- scripts/runner-compat.py exercises every WdaClient route plus the MJPEG
  stream against a running runner; read-only unless opted in
- runner/unit-check.sh tests HTTP parsing, locators and W3C action
  parsing on the Mac with a stub bridge
- README covers the WDA routes, MJPEG, approximations and ports
Drop the host app so the phone gets exactly one app, iPhoneUse-Runner,
the same footprint as WebDriverAgent (free provisioning profiles cap the
number of installed apps). The UI-test target keeps its name, so
-only-testing:IPhoneUseRunnerUITests/RunnerTests/testServe still selects
it, and setup can pass a team-specific PRODUCT_BUNDLE_IDENTIFIER.

The runner now logs ServerURLHere->http://<wifi-ip>:<port><-ServerURLHere
once its listener is ready, the marker setup-wda.sh waits for.
…bDriverAgent

setup-wda.sh no longer clones, pins, patches or icon-injects
WebDriverAgent. It builds runner/IPhoneUseRunner with build-for-testing
into <state>/runner-build (per instance) and launches it with
test-without-building -xctestrun ... -only-testing:IPhoneUseRunnerUITests/
RunnerTests/testServe. The runner listens on device ports 8100/9100 and
prints WDA's ServerURLHere marker, so the relays, the daemon wiring and the
status protocol are unchanged.

- sources: IPU_RUNNER_SRC > persisted > a repo checkout's runner/ >
  ~/.iphone-use/runner (laid down by install.sh); refused unless owned by
  this user and not writable by others
- product cache keyed on the source hash, signing identity (team, bundle,
  API key or Xcode account), device, Xcode/SDK and deployment override
- process identity accepts the runner argv; the two WebDriverAgent forms
  stay recognised so the first stop after an upgrade can still end an old
  WDA runner
- device selection, instances, lock wait/backoff, trust, DDI, UI
  automation, ASC/Xcode-account signing, supervisor, pause/resume/stop/
  status/doctor, blocker codes and the daemon-parsed messages are kept;
  user-facing wording says device runner
- tests follow the new flow; the icon-injection test is replaced by a
  runner build/launch test, and scripts/test-runner-suite.sh runs every
  device-free test with a throwaway HOME
- release-binaries.yml packages runner/ as iphone-use-runner.tar.gz
  (+ .sha256) and, like pr-checks.yml, runs runner/ci-check.sh: an
  unsigned iOS build of the runner (CODE_SIGNING_ALLOWED=NO), its
  device-free unit check, and coherence checks on the scheme, test id,
  app name, device ports and readiness marker setup-wda.sh relies on
- install.sh lays the sources down at ~/.iphone-use/runner from a local
  checkout or the verified release asset (only runner/ entries, no
  links), inside the install transaction: the previous tree is restored
  on failure and dropped on commit. The curl|sh bootstrap pins the asset
  next to the other release helpers; named instances share the tree.
- uninstall.sh recognises the runner process (started from the state
  directory) and removes runner-build/, the runner state files and, with
  the last instance, the shared sources
- the project uses XcodeGen's xcode14_0 format (objectVersion 56) so
  older Xcode and the CI image open it
README, guides (en/zh), the skill and the setup-pitfalls page describe the
iphone-use device runner: setup builds it from ~/.iphone-use/runner, the
WDA name survives only in status fields and environment variables, and
WDA_RUNNER_ICON is gone (IPU_RUNNER_SRC and WDA_RUNNER_REBUILD are new).
runner/README.md gains how setup-wda.sh manages the runner and the
measured MJPEG rate; two installer hints say device runner.
@coderabbitai

coderabbitai Bot commented Oct 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 46 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a9733cdc-f9de-477a-9a0f-e65596576c3f
📥 Commits

Reviewing files that changed from the base of the PR and between 6cdb58f and d9abac7.

⛔ Files ignored due to path filters (1)
  • runner/IPhoneUseRunner/IPhoneUseRunner.xcodeproj/project.xcworkspace/contents.xcworkspacedata is excluded by !**/*.xcworkspace/contents.xcworkspacedata
📒 Files selected for processing (47)
  • .github/workflows/pr-checks.yml
  • .github/workflows/release-binaries.yml
  • README.md
  • README.zh-CN.md
  • docs/guide.md
  • docs/guide.zh-CN.md
  • docs/wda-setup.html
  • install.sh
  • runner/.gitignore
  • runner/IPhoneUseRunner/IPhoneUseRunner.xcodeproj/project.pbxproj
  • runner/IPhoneUseRunner/IPhoneUseRunner.xcodeproj/xcshareddata/xcschemes/IPhoneUseRunner.xcscheme
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/IPURBridge.h
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/IPURBridge.m
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/IPhoneUseRunnerUITests-Bridging-Header.h
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/RunnerActions.swift
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/RunnerElements.swift
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/RunnerHTTP.swift
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/RunnerMJPEG.swift
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/RunnerTests.swift
  • runner/IPhoneUseRunner/IPhoneUseRunnerUITests/RunnerWDA.swift
  • runner/IPhoneUseRunner/project.yml
  • runner/README.md
  • runner/build.sh
  • runner/ci-check.sh
  • runner/unit-check.sh
  • runner/unit-check/IPURBridgeStub.h
  • runner/unit-check/IPURBridgeStub.m
  • runner/unit-check/main.swift
  • scripts/release.sh
  • scripts/runner-compat.py
  • scripts/runner-smoke.py
  • scripts/setup-wda.sh
  • scripts/test-install-runner-sources.sh
  • scripts/test-runner-suite.sh
  • scripts/test-setup-wda-asc-signing.py
  • scripts/test-setup-wda-icon-injection.sh
  • scripts/test-setup-wda-lock-backoff.sh
  • scripts/test-setup-wda-lock-wait.sh
  • scripts/test-setup-wda-product-verdict.py
  • scripts/test-setup-wda-runner-build.sh
  • scripts/test-setup-wda-runner-cache.py
  • scripts/test-setup-wda-runner-product.py
  • scripts/test-setup-wda-runner-repair.sh
  • scripts/test-setup-wda-xcode-compat.py
  • scripts/test-uninstall-safety.sh
  • skills/iphone-use/SKILL.md
  • uninstall.sh
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@leeguooooo
leeguooooo merged commit 12d2bf1 into main Oct 6, 2026
2 checks passed
@leeguooooo
leeguooooo deleted the feat/native-runner branch October 6, 2026 16:03
leeguooooo added a commit that referenced this pull request Oct 7, 2026
The runner icon was injected by setup-wda.sh before the device runner
replaced WebDriverAgent (#142) and was dropped with it, so the phone shows
the blank placeholder again. The native build now injects it after the
product validates: the installed iPhoneUse app's AppIcon.icns
(WDA_RUNNER_ICON=auto, the default; none keeps the placeholder; or a local
.png/.icns) is compiled with actool, merged into the runner's Info.plist
(CFBundleIcons > CFBundlePrimaryIcon > CFBundleIconName = AppIcon), and the
app is re-signed inside-out: Frameworks, then PlugIns/*.xctest, then the app
with its own entitlements, then verified. Any failure restores the pristine
signed app from its ditto backup and setup continues without the icon.

The icon's path and content hash join the product cache key, a previously
injected app is discarded before an incremental build (#75), and the
script's interactive build discards it too until it moves to Rust.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant