[ahamsel]

Why

In real daily use, WebPilot failed on Gmail in both Chrome and Edge with "Lost control of the tab (target_closed)". The cause was LastPass: it puts its own chrome-extension://…/webclient-infield.html frame into Gmail, and Chrome refuses chrome.debugger on any page that holds another extension's frame. When such a frame appears mid-task, Chrome detaches the debugger. Password managers and tools like Grammarly do this on many sites, so this hits most people.

The same session showed three more problems:

• A follow-up correction ("no my mun email is on gmail provider") got a chat reply instead of a retry.
• On a sign-in page pre-filled by LastPass, fast mode typed an invented placeholder over the saved password, and the planner clicked LOG IN.
• A planner run spent 87s and ended with "I ran out of steps".

What changed

Page-script driver (extension/src/dom-browser.ts, page-scripts.ts)

• When the debugger is blocked (another extension's frame, or another debugger already attached), the tab is driven with chrome.scripting.
• The visible DOM becomes the same snapshot format, with its own d… refs so a stale debugger ref can never hit the wrong element.
• Shadow DOM and same-origin frames are read; password values are masked; clickable rows (Gmail's inbox) get refs with a [clickable] attribute.
• Input is DOM events (execCommand("insertText") for typing).
• A debugger detached mid-task hands over cleanly: reads are redone, and actions that already reached the page count as done.
• The flight log notes the switch.
• Requires scripting plus all-sites host access.

Credentials

• WebPilot never types into password, one-time-code or card fields (fast mode, run-task planner, desktop planner).
• The field writer replies NONE instead of inventing usernames or personal details.
• Clicking "Log in", Next or Continue on a page with a filled-in password field asks first, with a "Hold on — sign in?" card.

Conversations and robustness

• Failed and stopped turns stay in the history, so a correction retries the original request with the fix applied.
• The router is told to act on such corrections instead of asking.
• A guessed start site that doesn't respond falls back to a search.
• Navigations give up on hosts that don't respond (8–10s) instead of hanging.

Planner

• Sees 8,000 characters of page text (it was parsing pages with Jev's 1,500-character limit, so it couldn't read email threads and kept expanding and scrolling).
• Is nudged to wrap up after about a minute.
• When out of steps, answers with what it found, what it couldn't find, and where it looked.

Daily-use loop

• Each answer keeps a local trace: routing, Jev decisions with probabilities and timings, answer checks, planner tool results and errors.
• The panel has 👍/👎 plus a note on each answer, and /feedback … from the chat box.
• Dev bridge commands: last, feedback, targets (what blocks the debugger on a page), --then follow-ups, and a build-id check after reload (Chrome sometimes ignores an extension reloading itself).

Testing

• npm test: 55/55. Includes jsdom tests that run the page scripts from their serialized source (so a function that depends on outside code fails), plus safety and run-task tests.
• npm run extension:smoke: 11/11 in headless Chromium, with a stand-in password manager (scripts/fixtures/frame-injector) and a pre-filled sign-in fixture:
  • blocked_mail_relocation, blocked_buy_cancelled
  • sign_in_asks_first (cancel → never signs in)
  • sign_in_allowed (allow → signs in with the saved password untouched)
  • feedback and trace saved.
• Realistic suite in the user's real browsers (dev bridge): Chrome 9/9 (median 6.1s), Edge 9/9 (median 6.4s).
• Real Gmail with LastPass installed, via page scripts:
  • Chrome: search from the inbox, 10.8s.
  • Edge: inbox question 4.0s; search 5.6s.
  • Edge conversation: "check my email at https://mail.mun.ca…" → "https://mail.mun.ca is not responding." (8.4s), then "no my mun email is on gmail provider" → found it via the MUN label (11.1s).
• npm run build passes.

🤖 Generated with Claude Code