brentvatne · Oct 24, 2026
diff --git a/README.md b/README.md
@@ -174,7 +174,7 @@ Pointed to a build, the [quick start](#quick-start) maps every screen of it, wit
 - A simulator build. A device `.ipa`, an `.aab`, an unsigned APK, or an APK without native libraries for the emulator (x86_64 on EAS Simulator, `arm64-v8a` locally on Apple silicon; a universal APK has both) is rejected before a device starts.
 - Platform, bundle id, name, and version are read from the build; you pass nothing else.
 - A run stops after 15 minutes, 40 screens, or 150 actions (`--max-duration-minutes`, `--max-screens`, `--max-actions`).
-- Buttons that delete, pay, sign out, send, or leave the app are never tapped. `--allow-destructive "Reset filters"` allows one by its exact label.
+- Buttons that delete, pay, sign out, send, or leave the app are never tapped. Explore judges a control by its label and its identifier (`btn_delete`, `deleteAccount`), including one tapped by position. `--allow-destructive "Reset filters"` allows one by its exact label.
 - `--hints "…"` tells the model what to focus on or how to sign in ([details](#sign-in-and-other-app-state)).
 - On EAS Simulator the report is hosted, and the link lasts as long as its EAS Hosting deployment (see [report access](docs/private-review.md#hosted-previews)); `--no-upload` keeps it local. On a local simulator it opens in the local Viewer (`--no-open` or `--json` skips that; `view --dir <folder>` opens it later), and `--upload` hosts it, which needs an EAS login. `--json` prints the result for scripts.
 - For a link that stays the same from run to run, pass `--alias NAME` and set `EXPLORE_PREVIEW_TOKEN` to a fixed token of 32 characters from a–z and 2–7 (`LC_ALL=C tr -dc a-z2-7 </dev/urandom | head -c 32`).
@@ -508,7 +508,7 @@ npx @expo/explore explore:binary ./my-app.app \
   --hints "Sign in as explore-test@example.com with {{secret:APP_TEST_PASSWORD}}"
 ```
 
-The model only sees the reference, and Explore types the value. The value is redacted everywhere, and a screen that shows it gets no screenshot, so keep non-sensitive values like a test email as plain text. In CI, store the value as a repository secret instead.
+The model only sees the reference, and Explore types the value. The value is redacted everywhere, and a screen that shows it gets no screenshot, so keep non-sensitive values like a test email as plain text. Explore finds the value in the screen's accessibility tree; a screen that draws it without exposing it there (in a canvas or an image) is not recognised. In CI, store the value as a repository secret instead.
 
 If the sign-in can't be typed (one-time codes, email links, Sign in with Apple or Google), sign in by hand on a session and run `explore:session`. Explore never taps Sign out.
 

diff --git a/skills/explore-binary/SKILL.md b/skills/explore-binary/SKILL.md
@@ -55,7 +55,7 @@ set -a && . ./.env.explore && set +a && npx @expo/explore explore:binary ./app.a
   --hints "Sign in with {{secret:APP_TEST_EMAIL}} and {{secret:APP_TEST_PASSWORD}}, then explore the signed-in app"
 ```
 
-The model only ever sees the reference; Explore types the value into the field. The value is redacted from the map, the report and errors, and a frame that shows it (the filled-in form, or a profile screen that displays the email) is kept without a screenshot. So make only what is sensitive a secret: a dedicated test email can go in the hints as plain text, and then the account screen keeps its picture. A reference to a variable that is not set fails before a session starts (`EXPLORATION_SECRET_MISSING`); provider keys, `EXPO_TOKEN` and CI variables cannot be referenced.
+The model only ever sees the reference; Explore types the value into the field. The value is redacted from the map, the report and errors, and a frame that shows it (the filled-in form, or a profile screen that displays the email) is kept without a screenshot. Explore finds the value in the accessibility tree, so a value drawn without an accessible label (in a canvas or an image) is not recognised. So make only what is sensitive a secret: a dedicated test email can go in the hints as plain text, and then the account screen keeps its picture. A reference to a variable that is not set fails before a session starts (`EXPLORATION_SECRET_MISSING`); provider keys, `EXPO_TOKEN` and CI variables cannot be referenced.
 
 **When a sign-in cannot be typed** (a one-time code, an email link, Sign in with Apple or Google, a captcha): sign in by hand on a session and use `explore:session`. Sign out is never tapped, so the run stays signed in.
 
@@ -64,7 +64,7 @@ The model only ever sees the reference; Explore types the value into the field.
 - Onboarding: `--launch-url` to a screen past it, or hints such as "skip the tour with Skip".
 - Screens that need data: a test account that already has it, and hints naming where it is ("open the first project").
 - Permission prompts are denied. To explore what is behind one, allow it by hand on a session first, then `explore:session`.
-- Purchases, deletes, sends and sign out are blocked. `--allow-destructive "Label"` allows one control by label; a paywall needs an account that already has the entitlement.
+- Purchases, deletes, sends and sign out are blocked, judged by a control's label and its identifier, including a control tapped by position. `--allow-destructive "Label"` allows one control by label; a paywall needs an account that already has the entitlement.
 - A large app: raise the budgets (`--max-duration-minutes`, `--max-screens`, `--max-actions`; see [What happens](#what-happens)) and use the hints to say what to cover first.
 
 ## Already have a session? `explore:session`

diff --git a/src/binary/explorer.ts b/src/binary/explorer.ts
@@ -69,6 +69,8 @@ export type BlackBoxExplorerResult = {
 const DESTRUCTIVE = /\b(delete|remove|erase|reset|wipe|clear all|sign ?out|log ?out|unsubscribe|cancel subscription|deactivate|close account|purchase|buy|subscribe|pay|checkout|send|post|publish|report|block|submit order|confirm order|place order)\b/i;
 // A bare "Map" is usually the app's own map tab or screen; only a named maps app or directions leave the app.
 const EXTERNAL = /\b(open in|safari|chrome|browser|app store|play store|share|mail|call|sms|(apple|google) maps|directions)\b/i;
+/** For the person running Explore, who sees blocked steps in flow and replay results; the model cannot pass it. */
+const ALLOW_HINT = "(Whoever runs Explore can allow a safe control with --allow-destructive and its exact label.)";
 const VOLATILE = /\d|[:%$€£]|\b(am|pm)\b/gi;
 const INTERACTIVE = new Set(["button", "link", "cell", "switch", "tab", "menuitem", "textfield", "securetextfield", "searchfield", "textview", "edittext", "slider", "checkbox", "radiobutton", "segmentedcontrol", "toggle"]);
 
@@ -185,13 +187,20 @@ function isInteractive(node: Record<string, unknown>): boolean {
   return traits.some(trait => /button|link|tab|adjustable/i.test(trait));
 }
 
+const allowed = (label: string, allow: readonly string[]) => allow.some(item => item.toLowerCase() === label.toLowerCase());
+
 export function riskOf(label: string, allow: readonly string[]): ActionRisk {
-  if (allow.some(allowed => allowed.toLowerCase() === label.toLowerCase())) return "safe";
-  if (DESTRUCTIVE.test(label)) return "destructive";
-  if (EXTERNAL.test(label)) return "external";
+  if (allowed(label, allow)) return "safe";
+  // Identifiers join their words (btn_delete, deleteAccount, sign-out), which \b does not split.
+  const words = label.replace(/(\p{Ll}|\d)(\p{Lu})/gu, "$1 $2").replace(/[_.-]+/g, " ");
+  if (DESTRUCTIVE.test(words)) return "destructive";
+  if (EXTERNAL.test(words)) return "external";
   return "safe";
 }
 
+const RISK_ORDER: readonly ActionRisk[] = ["safe", "unknown", "external", "destructive"];
+const worstRisk = (...risks: ActionRisk[]): ActionRisk => risks.reduce((worst, risk) => RISK_ORDER.indexOf(risk) > RISK_ORDER.indexOf(worst) ? risk : worst, "safe");
+
 /** Structure plus stable text, with values and numerals masked, so a clock tick or a counter does not create a new screen. */
 /**
  * A text field's label is whatever it shows: its placeholder while empty (Android sets `hintShowing`), then the typed text.
@@ -660,7 +669,11 @@ export async function runBlackBoxExploration(options: BlackBoxExplorerOptions):
         if (input.frameId !== undefined) {
           const point = { action: "tap" as const, frameId: text(input.frameId, "frameId", 256), x: Number(input.x), y: Number(input.y) };
           const target = input.target !== undefined ? cleanLabel(text(input.target, "target", 80)) : "";
-          return { action: point, kind: "press", label: target || placeUnder(point) || `tap ${point.x.toFixed(2)},${point.y.toFixed(2)}`, risk: target ? riskOf(target, allow) : "unknown" };
+          // The control under the point is judged by its own label and identifier too, not only by the model's description of it.
+          const under = nodeUnder(point), underLabel = str(under?.label);
+          const risk = underLabel && allowed(underLabel, allow) ? "safe"
+            : worstRisk(target ? riskOf(target, allow) : "unknown", ...[underLabel, str(under?.identifier)].flatMap(name => name ? [riskOf(name, allow)] : []));
+          return { action: point, kind: "press", label: target || placeUnder(point) || `tap ${point.x.toFixed(2)},${point.y.toFixed(2)}`, risk };
         }
         if (!selector) throw failure("AGENT_ACTION_INVALID", "tap needs a label, an identifier, or frame coordinates.");
         return { action: { action: "tap", ...selector } as BlackBoxAction, kind: "press", label, risk };
@@ -714,13 +727,12 @@ export async function runBlackBoxExploration(options: BlackBoxExplorerOptions):
     return { action: "tap", frameId: latest.frameId ?? "current", x: (bounds.x + bounds.width / 2) / extent.width, y: (bounds.y + bounds.height / 2) / extent.height };
   };
 
-  /** A coordinate tap the model did not describe, on a node the map has no name for (an icon-font button in a header), is
-   * named by what it is and where it sits: "icon button, top right" says more than the coordinates did. */
-  const placeUnder = (point: { x: number; y: number }): string | undefined => {
+  /** The smallest hittable control at a frame point in the latest tree, whether or not the map has recorded it. */
+  const nodeUnder = (point: { x: number; y: number }): Record<string, unknown> | undefined => {
     const extent = latest ? frameExtent(latest.nodes) : undefined;
     if (!extent) return undefined;
     const x = point.x * extent.width, y = point.y * extent.height;
-    const under = latest!.nodes
+    return latest!.nodes
       .filter(node => {
         const rect = node.rect as { x: number; y: number; width: number; height: number } | undefined;
         return !!rect && node.hittable === true && x >= rect.x && x <= rect.x + rect.width && y >= rect.y && y <= rect.y + rect.height
@@ -730,6 +742,12 @@ export async function runBlackBoxExploration(options: BlackBoxExplorerOptions):
         const ra = a.rect as { width: number; height: number }, rb = b.rect as { width: number; height: number };
         return ra.width * ra.height - rb.width * rb.height;
       })[0];
+  };
+
+  /** A coordinate tap the model did not describe, on a node the map has no name for (an icon-font button in a header), is
+   * named by what it is and where it sits: "icon button, top right" says more than the coordinates did. */
+  const placeUnder = (point: { x: number; y: number }): string | undefined => {
+    const under = nodeUnder(point);
     if (!under) return undefined;
     const role = roleOf(under);
     const label = str(under.label) ?? "";
@@ -805,7 +823,8 @@ export async function runBlackBoxExploration(options: BlackBoxExplorerOptions):
                 : elementUnder(chosen.action, fromScreenId);
               const { action, kind } = chosen;
               const label = named ? named.label : chosen.label;
-              const risk = named && chosen.risk !== "destructive" && chosen.risk !== "external" ? riskOf(named.label, allow) : chosen.risk;
+              // A control allowed by its label is allowed however it is targeted, even when its identifier reads as risky.
+              const risk = named && allowed(named.label, allow) ? "safe" : named && chosen.risk !== "destructive" && chosen.risk !== "external" ? riskOf(named.label, allow) : chosen.risk;
               const startedAt = timestamp();
               const attemptId = `att_${randomUUID().slice(0, 12)}`;
               const matched = map.frontier.find(item => item.status === "pending" && item.screenId === fromScreenId && item.actionKind === kind
@@ -817,8 +836,8 @@ export async function runBlackBoxExploration(options: BlackBoxExplorerOptions):
                   startedAt, completedAt: timestamp(), reason });
                 await persist();
                 return toolResult({ ok: false, blocked: true, reason: risk === "destructive"
-                  ? `"${label}" looks destructive (delete/reset/sign out/purchase/send). Explore does not activate such controls. Choose another control.`
-                  : `"${label}" looks like it opens another app (browser/store/share/mail/maps). Explore does not activate such controls. Choose another control.` });
+                  ? `"${label}" looks destructive (delete/reset/sign out/purchase/send). Explore does not activate such controls. Choose another control. ${ALLOW_HINT}`
+                  : `"${label}" looks like it opens another app (browser/store/share/mail/maps). Explore does not activate such controls. Choose another control. ${ALLOW_HINT}` });
               }
               // The map keeps the reference the model typed; only the device gets the value.
               const typed = action.action === "fill" || action.action === "type" ? { ...action, text: inputs.resolve(action.text) } : action;

diff --git a/test/node/binary-explorer.test.mjs b/test/node/binary-explorer.test.mjs
@@ -1207,3 +1207,52 @@ test("a replay plan's {{secret:NAME}} inputs are read before any device starts a
   assert.equal(map.actionAttempts[0].action.input.value, "{{secret:APP_TEST_PASSWORD}}");
   assert.ok(!JSON.stringify(map).includes("hunter2-test"));
 });
+
+test("risk words are found inside identifiers that join them", async () => {
+  const { riskOf } = await tsImport("../../src/binary/explorer.ts", import.meta.url);
+  for (const label of ["btn_delete", "deleteAccount", "sign-out", "logOutButton", "settings.reset"]) assert.equal(riskOf(label, []), "destructive", label);
+  assert.equal(riskOf("openInSafari", []), "external");
+  for (const label of ["postsTab", "settings_button", "Profile", "MapView"]) assert.equal(riskOf(label, []), "safe", label);
+  assert.equal(riskOf("resetFilters", ["resetFilters"]), "safe", "an allowed identifier");
+});
+
+/**
+ * A screen whose only control is a hittable custom view the map does not record as a control (type Other, as many
+ * custom pressables are): its risk is in its own label and identifier.
+ */
+function iconScreen(icon) {
+  return [{ type: "Window", rect: { x: 0, y: 0, width: 400, height: 800 } }, { type: "StaticText", label: "Inbox" },
+    { type: "Other", hittable: true, rect: { x: 340, y: 40, width: 44, height: 44 }, ...icon }];
+}
+
+test("a coordinate tap is judged by the control under the point, whatever the model calls it", async () => {
+  for (const icon of [{ identifier: "btn_delete" }, { label: "Delete conversation" }]) {
+    const { calls, acted, map } = await harness([["observe", { includeImage: true }], ["act", { action: "tap", frameId: "frame-0", x: 0.9, y: 0.075, target: "Profile" }], ["finish", { summary: "x" }]],
+      { observe: async includeImage => ({ nodes: iconScreen(icon), capturedAt: new Date().toISOString(), ...(includeImage ? { frameId: "frame-0", png: PNG, imageSha256: "d".repeat(64) } : {}) }) });
+    assert.equal(acted.length, 0, JSON.stringify(icon));
+    assert.equal(calls[1].result.blocked, true);
+    assert.match(calls[1].result.reason, /looks destructive.*--allow-destructive/);
+    assert.equal(map.actionAttempts.at(-1).risk, "destructive");
+  }
+});
+
+test("a coordinate tap on a control allowed by its label goes through, even when its identifier reads as risky", async () => {
+  const { acted } = await harness([["observe", { includeImage: true }], ["act", { action: "tap", frameId: "frame-0", x: 0.9, y: 0.075 }], ["finish", { summary: "x" }]],
+    { allowDestructive: ["Reset filters"],
+      observe: async includeImage => ({ nodes: iconScreen({ label: "Reset filters", identifier: "resetFilters" }), capturedAt: new Date().toISOString(), ...(includeImage ? { frameId: "frame-0", png: PNG, imageSha256: "d".repeat(64) } : {}) }) });
+  assert.equal(acted.length, 1);
+});
+
+test("a coordinate tap with no control under the point is still recorded as unknown, not refused", async () => {
+  const { acted, map } = await harness([["observe", { includeImage: true }], ["act", { action: "tap", frameId: "frame-0", x: 0.5, y: 0.5 }], ["finish", { summary: "x" }]],
+    { observe: async includeImage => ({ nodes: iconScreen({ identifier: "btn_delete" }), capturedAt: new Date().toISOString(), ...(includeImage ? { frameId: "frame-0", png: PNG, imageSha256: "d".repeat(64) } : {}) }) });
+  assert.equal(acted.length, 1);
+  assert.equal(map.actionAttempts.at(-1).risk, "unknown");
+});
+
+test("a tap by identifier on a control allowed by its label goes through", async () => {
+  const screen = [{ type: "StaticText", label: "Filters" }, { type: "Button", label: "Reset filters", identifier: "resetFilters" }];
+  const { acted } = await harness([["observe", { includeImage: true }], ["act", { action: "tap", identifier: "resetFilters" }], ["finish", { summary: "x" }]],
+    { allowDestructive: ["Reset filters"], observe: async includeImage => ({ nodes: screen, capturedAt: new Date().toISOString(), ...(includeImage ? { frameId: "frame-0", png: PNG, imageSha256: "e".repeat(64) } : {}) }) });
+  assert.equal(acted.length, 1);
+});