feat(tap_locator): via:point for custom controls that ignore element clicks

[leeguooooo]

Why: On Xiaohongshu 9.49.1 the 30×30 back button (label 返回, no identifier) acknowledged two XCUIElement clicks from tap_locator and the page stayed put. A coordinate tap at its centre opened the menu. Reported with exact rects by the parallel session.

What: tap_locator gains "via": "point".

• The locator is still proven unique, the same as today.
• Then the centre of the element's live frame is tapped once through W3C actions. The frame comes from the WDA element rect, not the /source rect, which misses inside system sheets.
• The default remains element click, so nothing that works today changes. An invalid via returns 400.
• MCP and flow tap_locator steps carry via.

Not automatic: I chose not to add a coordinate tap after every element click. If the first click did act, a second tap lands on whatever is now under it.

Tests: new daemon integration test: centre (30, 84), no /click, bad via → 400. cargo test -p server and -p iphone-use-mcp pass.

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 49 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 7a6d2e18-3339-4ef2-8ad1-8fbb03cdf366

📥 Commits

Reviewing files that changed from the base of the PR and between 68a721e and 7a7803f.

📒 Files selected for processing (4)

• crates/mcp/src/server.rs
• crates/server/src/http.rs
• crates/server/tests/agent_actions_outcome.rs
• docs/agent-reference.md

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}