Skip to content

Keep verify proof small and use Playwright's bundled Chromium - #171

Merged
sreejithraman merged 10 commits into
mainfrom
agent/verify-playwright-browser
Nov 9, 2026
Merged

sreejithraman merged 10 commits into
mainfrom
agent/verify-playwright-browser

Conversation

@sreejithraman

Copy link
Copy Markdown
Owner

verify now keeps proof small and short-lived, and scripted browser checks use Playwright's own Chromium. Verification stops filling the disk with copied builds and leaked Chrome app clones.

Why

On 2026-11-09 this Mac's disk hit 96% full and the load average reached 800–990, which stalled a Simulator proof.

  • Old evidence: ~/.t3/artifacts held 240 GiB of seatcommons verification evidence for closed tickets. One run alone was 57 GiB: a DerivedData copy, a source snapshot, two .app bundles, Pods, and every rerun. The actual proof was under 1% of that.
  • Leaked Chrome clones: agent proof scripts launched the installed Google Chrome through executablePath and were killed on timeout. Each killed launch leaks a code-sign copy of Chrome.app under /var/folders, and 126 of them built up.
  • The gap in verify: it said "keep failed attempts…" but never said what proof is, where it goes, or when it goes away.

Changes

  • verify/references/web.md: scripted checks reuse the project's Playwright test when one exists, and otherwise use a library script in run scratch. They launch Playwright's bundled Chromium, with no channel or executablePath. They use channel: 'chrome' only when the check needs the real browser, such as WebMCP. They close the browser in finally and stop on Playwright timeouts rather than being killed. Every web check records the browser and its version.
  • verify/SKILL.md, "Keep proof, not runs":
    • Run scratch comes from mktemp -d and is deleted when the run ends. It holds scripts, snapshots, result bundles, reruns, and any build a later run won't reuse.
    • Kept proof is the verdict table, the captures behind it, the log excerpts it cites, and build identity (commit plus product hash). For reruns it keeps outcome counts, plus captures only for failures behind an open finding.
    • Where proof goes: in a linked worktree on a branch, proof goes in that worktree's private Git dir (git rev-parse --git-path verify-proof). Git never commits it, and removing the worktree deletes it. Other runs keep it in session scratch.
    • Repeated checks: verify offers to turn a check that keeps rerunning into a committed test or a map recipe.
  • verify/references/ios.md, Build output:
    • Only a checkout's sole builder uses its normal DerivedData. Every other run, and any build of a disposable checkout, uses -derivedDataPath in run scratch.
    • Every test run passes -resultBundlePath in run scratch.
  • repo-cleanup: removing a worktree also removes its Xcode DerivedData, matched by WorkspacePath. Housekeeping removes DerivedData whose WorkspacePath no longer exists.
  • simulator-lifecycle: before acquiring a device, it checks free disk and load once, on the machine that runs the device. Below 10% free, or load above twice the CPU count, it holds device work and reports.
  • implement: the Proof report gives the kept proof's absolute path. project-runners.md points to the iOS build rules. SOURCES.md is updated to match.

Evidence

  • python3 scripts/check.py: exit 0 at cddd0d9.

  • Review Fix Loop, 8 passes.

    • Pass 1: Primary, Ponytail and Thermo. The OpenCode and Gemini extras failed (quota, runtime error) and were skipped.
    • Passes 2–7: Primary, Thermo and Adversarial.
    • Pass 7: cancelled as superseded.
    • Pass 8: Primary and Ponytail on the final design. Primary had no findings, and Ponytail's three wording cuts were applied.

    Passes 3–7 built a global proof store: an XDG root, hashed names, a task.json manifest, and cleanup rules. Each pass found new edge cases in it. The final design drops all of that and ties proof to the worktree's lifetime, as the upstream pstack verification skills do: evidence stays in a location the skill names.

  • Dry-run trials on frozen copies, with expectations written first: 7/7, 5/5, 4/4, 6/6, and 4/4 on the final design.

    • Final design (4/4): worktree proof in the git dir; a main-checkout audit kept in session scratch; cleanup after merge, where proof goes with the worktree and DerivedData is removed; a shared-checkout child builds in scratch.
    • Throwaway repo: git worktree remove and git worktree prune both deleted the proof folder, and git status --ignored stayed empty.

Not proven

  • No live agent run has used the new text. Installed skills update only after merge.
  • The source of the plist-less DerivedData folders (147 of 154 on this Mac) is still unknown, and the cleanup rules leave them alone. That's a follow-up.
  • Something still launches branded Chrome and leaks clones today, and its source isn't identified. Clone cleanup in repo-cleanup is a follow-up.

Merge Danger

Two-way door: instructions only, and a revert restores the old text. No proof store exists yet, so there is nothing to migrate. Agents running under the old text keep their ~/.t3/artifacts habits until the installed skills update.

🤖 Generated with Claude Code

sreejithraman and others added 10 commits November 9, 2026 14:19
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Verify splits a run's files into run scratch, deleted when the run ends,
and small kept proof that Repo Cleanup removes once the work lands.
Simulator Lifecycle checks free space and load before device work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…last reader

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…terministically

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…scratch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ically

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Proof now lives and dies with its worktree, so it needs no naming,
manifest, or cleanup rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sreejithraman
sreejithraman merged commit 8f13f75 into main Nov 9, 2026
3 checks passed
@sreejithraman
sreejithraman deleted the agent/verify-playwright-browser branch November 9, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant