diff --git a/SOURCES.md b/SOURCES.md index c61f2f1..6aae717 100644 --- a/SOURCES.md +++ b/SOURCES.md @@ -712,7 +712,8 @@ One automatically discoverable skill owns real-workflow selection, launch, drivi acceptance judgments, evidence, and cleanup. It writes no docs in any repo, except formal check records under the formal reference's conditions: it uses a temporary plan without a map, reads a map when one exists, and reports drift. Adds a read-only doctor before driving, red-first reproduction against the base -revision, rerunnable scripts, and refusal of an assertion with no independent +revision, run-scratch scripts with an offer to turn repeat checks into a test or +recipe, and refusal of an assertion with no independent expectation or way to run it. Formal checks use one conditional reference, also reached when a change touches an existing check. It retains tool selection, code/model correspondence, bounds, negative controls, counterexample replay, @@ -729,7 +730,9 @@ explicit. Concurrent and long checks plan resource ownership; authorized harness isolation changes prove concurrent isolation and independent cleanup. Imported interface criteria retain full requested audit coverage. Web and iOS -checks follow the requested deployment or Simulator/device target. The iOS +checks follow the requested deployment or Simulator/device target. Scripted web +checks launch Playwright's bundled Chromium unless they need the shipping browser, +and close it in the script. The iOS reference reuses project harnesses and prefers XCTest with XCUIAutomation for new iOS UI workflow tests, Swift Testing for new Swift unit tests, and XCTest for performance tests. Suite execution preserves target identity, required-check @@ -739,8 +742,13 @@ automated and needs assertion coverage beyond a successful command. Targeted suites preserve required integration gates, reconcile every selected test and attempt, and require matching runtime evidence for pre-existing failure claims. Terminal and service checks use process or PTY driving, bounded completion, public -protocols, and fresh persisted-output reads. Keeps bounded reproduction attempts, -failed-attempt evidence, authorization, isolation, redaction, run-owned cleanup, +protocols, and fresh persisted-output reads. Separates deleted run scratch from +small kept proof. A linked worktree on a branch keeps that proof in its private Git +directory, which worktree removal deletes; other runs keep it in session scratch for +the session. The iOS reference keeps builds in the build system only for a +checkout's sole agent and sends other builds and result bundles to run scratch. +Keeps bounded reproduction attempts, outcome counts, authorization, isolation, +redaction, run-owned cleanup, and separate defect-reproduction reporting. Authenticated workflows reuse designated test accounts and standing scope, delegate application sign-in and command/API credential use to `agent-login`, @@ -764,8 +772,10 @@ evidence limits. Greetings and attribution instructions are omitted. Accepts merge completion handoffs from `merge`. Cleanup includes verified task-owned development processes, simulator/emulator installs, disposable test -data, and temporary devices created solely for the task. Reused devices are -preserved and shut down only when the task booted them and no consumer remains. +data, the Xcode DerivedData of removed or missing worktrees, and temporary devices +created solely for the task. +Reused devices are preserved and shut down only when the task booted them and no +consumer remains. Cleanup reconstructs process, device, and install ownership from available session history and tool results, with logs, current state, and optional handoffs as supporting evidence. Discovery and cleanup rules live together in @@ -1174,7 +1184,10 @@ than test phones. Adds host-routing, retained-target and shared-requester owners gates plus fresh lease-identity requirements, and links local run/handoff, runner-integration and adoption references, including bounded iOS agent-device helper preparation, bounded failure diagnostics and bridge capability -checks. Failure guidance defers to host recovery instructions and preserves unresolved +checks. Checks free disk space and load once before acquiring a device, on the +machine that runs it, and holds or reports when either is critical. Project +runners follow Verify's iOS build output rules. +Failure guidance defers to host recovery instructions and preserves unresolved ownership conflicts. Owns T3 Device panel routing, returned driver and session flags, and Xcode and platform tool routes for reaching a device. Adds a last-resort, coordinated CoreSimulator restart for repeated iOS boot failures before a grant, guarded by a local script. Runtime lease rules stay with `simlock instructions`. Technical citations use checked upstream source diff --git a/skills/implement/SKILL.md b/skills/implement/SKILL.md index dc5d46c..1247570 100644 --- a/skills/implement/SKILL.md +++ b/skills/implement/SKILL.md @@ -219,8 +219,8 @@ The owner report carries every field that applies: delivery state against it. - **Layers:** each PR, branch, or local artifact, with its link, head, base, revisions, and working-tree state. -- **Proof:** each required assertion's verdict and evidence, and what is blocked - or untested. +- **Proof:** each required assertion's verdict and evidence, the kept proof's + absolute path, and what is blocked or untested. - **Shown:** what you showed and how to open it, with what it needs, such as a login or a running server and its stop action. - **Waits:** pending PRs, live helpers and watchers, and the events they await. diff --git a/skills/repo-cleanup/SKILL.md b/skills/repo-cleanup/SKILL.md index bc3f3a7..a65971b 100644 --- a/skills/repo-cleanup/SKILL.md +++ b/skills/repo-cleanup/SKILL.md @@ -56,6 +56,14 @@ Clean up only resources whose owner and disposal reason are established. A merge task changes, with no retained ignored data. Leave the current worktree in place for Codex. + When you remove a worktree, also remove its Xcode DerivedData folders. These + are the folders in Xcode's DerivedData location, `~/Library/Developer/Xcode/DerivedData` + by default, whose `info.plist` has a `WorkspacePath` inside the removed + worktree: it starts with the worktree's absolute path plus `/`. During + housekeeping, also remove each DerivedData folder whose `WorkspacePath` no + longer exists on disk. It is a regenerable cache, whoever removed the worktree. + Leave every other DerivedData folder, including folders without that key. + Before deleting any branch, recheck its tip and worktree use. For merged work, the tip must still equal the source head verified at merge time or be reachable from the intended base. If neither can be proved, retain the branch. Retain diff --git a/skills/simulator-lifecycle/SKILL.md b/skills/simulator-lifecycle/SKILL.md index b5e17ef..3cfda83 100644 --- a/skills/simulator-lifecycle/SKILL.md +++ b/skills/simulator-lifecycle/SKILL.md @@ -19,6 +19,18 @@ Apply the [run ownership guard](references/workflow.md#choose-the-route-before-a before following reuse, release or cancellation advice: a shared requester may belong to a sibling run. +Check the machine that runs the device once, before you acquire or boot it. Read +free space with `df -h "$HOME"` and the 1-minute load average with `uptime`, and +compare the load with the CPU count (`sysctl -n hw.logicalcpu` or `nproc`). Less than +10% free space, or a load above twice the CPU count, is critical. Then hold the device +work, continue independent work, and recheck when that runs out. If it is still +critical, report both numbers as the blocker. Free space only through +[Repo Cleanup](../repo-cleanup/SKILL.md)'s ownership checks. + +On a remote device host, single-quote the commands so `$HOME` expands there, such as +`ssh 'df -h "$HOME"; uptime; sysctl -n hw.logicalcpu'`. If the host offers no +shell, report the check as not run and continue under the host's own controls. + In T3, acquisition and interactive control must follow its documented route to the same SimLock lease; an ordinary panel session is not proof of that lease. Before acquiring, read [run and handoff conventions](references/workflow.md) for diff --git a/skills/simulator-lifecycle/references/project-runners.md b/skills/simulator-lifecycle/references/project-runners.md index f4d1515..f3af17e 100644 --- a/skills/simulator-lifecycle/references/project-runners.md +++ b/skills/simulator-lifecycle/references/project-runners.md @@ -23,17 +23,19 @@ The runner must: occur. A one-shot detached lease needs a supported renewal owner; choosing a long TTL alone is insufficient. Observe lease loss and stop device-dependent work before any further command. -- Allocate a unique run directory for DerivedData, intermediate app builds, - temporary fixtures and logs. Put retained results outside the disposable - directory. Record created paths and process identities as they are created. +- Allocate a unique run directory for temporary fixtures and logs, and put + Verify's [kept proof](../../verify/SKILL.md#keep-proof-not-runs) outside it. + Record created paths and process identities as they are created. +- Place Xcode DerivedData and result bundles by Verify's + [build output](../../verify/references/ios.md#build-output) rules. - Capture PIDs at spawn and use existing ownership-checked stop tooling. Recheck start identity, executable/workspace and other consumers immediately before stopping. Await all run-owned writers, including test helpers and descendants; a root process exiting is insufficient if it left children running. -- Export required screenshots, logs and result bundles before releasing. Keep - failure evidence. Release through the same interface in teardown even when - acquisition only partly succeeded; reconcile an uncertain grant by request/run - identity before requesting another device. +- Export the device's screenshots and logs that kept proof needs before + releasing. Release through the same interface in teardown even when + acquisition only partly succeeded; reconcile + an uncertain grant by request/run identity before requesting another device. - Bound queued acquisition and use supported cancellation only for a request proven to belong to this run. Killing a waiting CLI process does not necessarily cancel its queued request. Reconcile whether it was cancelled or granted, and release an eventual diff --git a/skills/verify/SKILL.md b/skills/verify/SKILL.md index 399346b..9a4f920 100644 --- a/skills/verify/SKILL.md +++ b/skills/verify/SKILL.md @@ -19,9 +19,47 @@ It owns its prerequisites, verdicts, and repo writes. The steps below apply to p claims. A model doesn't replace them, and ordinary workflows need no new model. Look for existing formal checks by function, not by name. -Keep plans, scripts, and evidence in the session scratch directory. Verify writes no -docs in any repo, including a repo's verification map. If the user asks to create or -change a map, follow [verification maps](../setup-sreestack/references/verification-map.md). +Verify writes no docs in any repo, including a repo's verification map. If the user +asks to create or change a map, follow [verification maps](../setup-sreestack/references/verification-map.md). + +## Keep proof, not runs + +A run leaves two kinds of files. Keep them in separate places from the start. + +**Run scratch** is anything you could produce again: plans, helper and rerun scripts, +per-step snapshots, result bundles, intermediate reruns, and builds no later run +reuses. Create its directory with `mktemp -d`, which puts it under `$TMPDIR`, so the +OS reclaims what a cancelled run leaves. Delete that directory when the run ends, pass +or fail. Only the sole builder in a checkout leaves its builds where the build system +puts them, for its later runs to reuse; +[iOS build output](references/ios.md#build-output) says which runs that covers. Record +a build by its build identity instead of copying it or its inputs, such as source +trees and Pods. + +**Kept proof** is what a reader needs to trust the verdicts: + +- the verdict table; +- the captures behind each verdict, visual for a visual claim: screenshots, + recordings, transcripts, or responses; +- the log excerpts a verdict or finding cites; +- build identity: the commit SHA and the hash of the product installed or tested. + +For repeated attempts, keep the outcome counts, plus captures only for failures behind +a finding that is still open. + +Write each run's kept proof to a new run folder. In a linked worktree on a branch, +where `git rev-parse --path-format=absolute --git-dir --git-common-dir` prints two +different paths and `git symbolic-ref -q HEAD` succeeds, put the folder in that +worktree's private Git directory: + +```sh +proof=$(git rev-parse --path-format=absolute --git-path verify-proof) +mkdir -p "$proof" && mktemp -d "$proof/$(date -u +%Y%m%dT%H%M%SZ)-$(git rev-parse --short HEAD)-XXXXXX" +``` + +Git never commits that folder, and removing or pruning the worktree deletes it. +Otherwise put the folder in the session scratch directory, or in a temporary directory +when the host has none, and say in your report that it lasts only for the session. ## Choose what to prove @@ -92,8 +130,9 @@ options in the repo and the environment. Follow who drives an interface and with which model. The driver reports what it saw, and you keep the verdicts. -Prefer a script you can rerun over steps by hand. Put repeatable setup, driving, and -evidence capture in it, and give the rerun command in your report. +Prefer a script in run scratch over steps by hand. Put repeatable setup, driving, and +capture in it. When the same check runs again on a later commit, offer to turn it into +a committed test or a verification-map recipe. **Run a read-only doctor before you drive.** Run the map's doctor if it has one, then check whatever it doesn't cover, without changing state: @@ -135,8 +174,8 @@ Remove secrets and private data from evidence. When a claim says a behavior changed or a bug is fixed, reproduce the old result before you trust the new one. Run the same check, with the same environment and fixtures, against the base revision, in a disposable checkout if the change is already in place, -and keep both results. Run the doctor there too. Red counts only when the old side runs -and shows the old behavior. A setup error isn't red. +and keep both results in the same run folder. Run the doctor there too. Red counts only +when the old side runs and shows the old behavior. A setup error isn't red. A check that passes on both sides proves nothing: change it until it tells them apart, or mark the claim `untested`. If you can't run the old side, revert the change in a disposable @@ -158,14 +197,15 @@ target exists, and take them from current state. dependencies. Before running them, connect each assertion to a real check, and confirm the target, fixtures, filters, optional checks, and retries. Afterward, reconcile every selected test with its result: actual skips, unmet preconditions, and unfinished tests. -Count a duplicate test once for coverage and keep every attempt's outcome. Exercise +Count a duplicate test once for coverage and record every attempt's outcome. Exercise uncovered assertions yourself. A green command, zero tests, partial results, or a passing rerun can't establish a pass. Keep required broader gates, and report any you didn't run. **When something doesn't change:** Record it before you retry. Retry once, only when -evidence shows the target moved or the interface hadn't settled, and record the retry as -its own attempt. A repeated failure is evidence. After a stuck interface or a surprising -result, check instance health and restore known state, even if the process looks fine. +evidence shows the target moved or the interface hadn't settled, and record the retry's +outcome as its own attempt. A repeated failure is evidence. After a stuck interface or a +surprising result, check instance health and restore known state, even if the process +looks fine. **Intermittent reports:** Choose the attempt count before you start, and use it on both sides. Use about 3 divided by the failure rate, so about 30 attempts for 1 in 10. Restore @@ -201,15 +241,12 @@ or another revision's results don't show that. State the limits of the compariso ## Clean up and report -Keep failed attempts and the screenshots, snapshots, logs, transcripts, or responses -behind each verdict. Visual claims need visual captures. - -Stop only the processes and sessions this run started, and remove their disposable data, -after success or failure. Leave reused services and source fixtures running. Then open -the kept evidence to confirm it survived. +After success or failure, stop only the processes and sessions this run started, remove +their disposable data, and delete the run scratch. Leave reused services and source +fixtures running. Then open the kept proof to confirm it survived. Give each workflow and entry point its actions, expected and observed results, and -evidence location. Give every assertion one verdict: +the kept proof's absolute path. Give every assertion one verdict: - `passed`: the observed result matched. - `failed`: the observed result differed. diff --git a/skills/verify/references/ios.md b/skills/verify/references/ios.md index 21e36b1..c6e1032 100644 --- a/skills/verify/references/ios.md +++ b/skills/verify/references/ios.md @@ -33,6 +33,20 @@ change executable tests when the active task includes implementation, harness repair, or reusable verification setup. Keep test files, test plans, fixtures, and CI configuration in the product repo. +## Build output + +Xcode keys its default DerivedData folder by workspace path, so each worktree and +each disposable checkout gets its own folder, and that folder outlives the checkout. + +- Use the checkout's normal DerivedData only when you are its sole builder: the + only agent running in that checkout, such as a delegated child in its own + isolated worktree. +- Every other run, including any run unsure it is alone and every build of a + disposable checkout such as a red-first base, passes `-derivedDataPath` inside + its [run scratch](../SKILL.md#keep-proof-not-runs), so deleting the run scratch + deletes that build. +- Every test run passes `-resultBundlePath` inside its run scratch. + ## Prepare and run native UI tests Discover the project's UI test target, scheme, test plan, wrapper command, and @@ -51,8 +65,9 @@ delivery path from [authenticated verification](authenticated-verification.md). Map selected test methods to required workflow assertions. Inspect test-plan configurations, selection filters, disabled tests, skips, expected failures, and retries. Reconcile the tests actually executed with the selected assertions under -Verify's main procedure. Retain the runner's exit status and result bundle, -including screenshots, attachments, logs, and every attempt needed for the verdict. +Verify's main procedure. Record the runner's exit status. Export the screenshots, +attachments, and log excerpts a verdict cites from the result bundle into +[kept proof](../SKILL.md#keep-proof-not-runs). Pair public UI actions with explicit outcome assertions. Prefer accessibility identifiers and meaningful labels. Wait for observable states with bounded @@ -103,8 +118,9 @@ hardware is unavailable, report the device-specific assertions as blocked. ## Evidence -Record the command and toolchain, scheme/test plan, build and target identity, -fixture state, selected tests, per-assertion outcomes, and artifact locations. +Record the command and toolchain, scheme/test plan, build identity, target +identity, fixture state, selected tests, and per-assertion outcomes in +[kept proof](../SKILL.md#keep-proof-not-runs). Capture the workflow states needed to support visual claims. Use [interface review](interface-review.md) for relevant visual, Dynamic Type, keyboard, focus, touch-target, and assistive-technology checks. Exercise VoiceOver diff --git a/skills/verify/references/web.md b/skills/verify/references/web.md index 9e36fab..b5e1bcb 100644 --- a/skills/verify/references/web.md +++ b/skills/verify/references/web.md @@ -3,8 +3,8 @@ Use the requested deployment and the browser environment already attached to the task when possible. For a local change, start or locate the development server; for a live site, open its URL. -Reach the exact route or state under test. Record the URL, viewport/device mode, -and relevant console or network evidence. +Reach the exact route or state under test. Record the URL, browser and version, +viewport/device mode, and relevant console or network evidence. In T3 Code, follow the injected collaborative-browser instructions for tool selection and recovery. Start with `preview_status`, then `preview_open` when @@ -15,6 +15,22 @@ resolve the actual URL and port from the running process or app environment. The [T3 runtime instructions](https://github.com/pingdotgg/t3code/blob/c71ca063dfcd8c2e3dcb2bd92adbb2180cbbece3/apps/server/src/provider/T3OrchestrationInstructions.ts) and live tools supply the current interaction contract. +## Scripted browser checks + +Use the project's Playwright Test check when it already has one for this +behavior; its fixtures close the browser. Otherwise write a script with the +Playwright library in [run scratch](../SKILL.md#keep-proof-not-runs). See +[Playwright library](https://playwright.dev/docs/library). + +- Launch Playwright's bundled Chromium, with no `channel` or `executablePath`. + Install it with `npx playwright install chromium`. Use a branded channel such as + `channel: 'chrome'` only when the check needs the shipping browser, such as for + release parity or a Chrome-only feature. Its headless mode differs, so record the + channel with the browser version. See [Playwright browsers](https://playwright.dev/docs/browsers). +- Close the context and browser in a `finally` block. Bound the run with + Playwright's timeouts so it exits through that block. A killed branded Chrome + leaves a copy of its app on disk. + ## Observe and interact - Capture a browser snapshot or inspect the DOM and accessibility tree before @@ -49,9 +65,9 @@ applicable contracts into assertions: - Check supported appearance changes and browser-chrome hints in the actual target mode. Inspect focus zoom and keyboard/input hints on relevant fields. -Record browser/version, device or simulator, orientation, display mode, and input -method. Emulation can establish layout and some input behavior but does not -establish every hardware/browser assertion. Use target hardware when available +Also record the device or simulator, orientation, display mode, and input method. +Emulation can establish layout and some input behavior but does not establish +every hardware/browser assertion. Use target hardware when available for keyboard, chrome, safe areas, gestures, and touch feel; mark unavailable checks `blocked` or `untested` rather than calling them passed. See [Chrome's Device Mode limitations](https://developer.chrome.com/docs/devtools/device-mode#limitations).