Repository navigation
feat(tests): functional UI test harness + client/server paradigm scan (TIGER-386) - #364
Merged
Merged
Conversation
… (TIGER-386)
AI-driven functional user testing for the Tiger UI, so the defect class that
passes a green PHPUnit suite because nothing ever rendered gets caught.
Three layers, each answering a different question:
bin/paradigm-scan.php does the CODE follow client/server? every view, ms
bin/auth-probe.php does the AJAX/JSON contract behave? no browser
scenarios/auth.json does it WORK for a user? intent-level, no selectors
The scan enforces the contract in AGENTS.md ("the UI calls /api, it does not
post to pages") and WEBSERVICES.md §5. First run over 108 view scripts: ZERO
page-POST violations — the paradigm holds.
Two false-positive classes it had to learn, both cost real hits:
- a method="get" search form is NAVIGATION, not a page-POST (3 flagged, all wrong)
- a loop over a view-local array is CONTENT, not data — only foreach ($this->x)
emitting <tr> is a break (8 hits -> 1)
Mutation-tested: synthetic bad views are caught, good ones stay clean, the exit
code flips.
auth-probe reports a third state, GATED, for a check that could not actually run
(reCAPTCHA refusing a tokenless client). Identical captcha refusals would have
made the account-enumeration check pass vacuously, and a vacuous green is worse
than no check.
Dev-only by construction: config.json carries never_target, both probes refuse a
production base URL, and negative-path scenarios use @example.invalid identifiers
so a run can never trip a real account's lockout.
CI is untouched — phpunit.xml scans tests/Unit and tests/Integration only.
Findings filed from the first run: TIGER-383 (server-rendered updates grid),
TIGER-384 (56 inline script/style tags), TIGER-385 (dev reCAPTCHA blocks auth
testing).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013vdyTeoDrGE4hKxt53TsE8
Ran the anonymous auth scenarios against tiger-dev through Playwright MCP. Results, replacing the speculative blocked_by notes: LOGIN-RENDER PASS 0 console errors, no raw i18n keys, no action/method LOGIN-FAIL PASS no navigation; XHR -> 401; in-page alert; password cleared LOGIN-ENUM PASS (new scenario) two unknown identifiers, identical refusal FORGOT PASS no navigation; XHR -> 200; no-enumeration wording RESET-BADTOKEN FAIL -> TIGER-387 The blocked_by notes on LOGIN-FAIL and FORGOT were wrong and are removed: dev runs reCAPTCHA v3 (invisible, score-based), so a real browser passes it and those paths DO execute. Only the headless curl probe is gated. TIGER-385 has been narrowed to say so. UI-AUTH-LOGIN-ENUM is promoted to its own scenario now that the browser can actually reach the credential path — it was previously unreachable, and would have passed vacuously on identical captcha refusals. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013vdyTeoDrGE4hKxt53TsE8
…ev account
Created a purpose-built dev account via `tiger install:admin` (uiprobe@tiger.test,
org "UI Probe", role `user`) so no existing dev test account is touched. The
credential lives in tests/ui/credentials.local.json, which is gitignored and 0600.
LOGIN-OK PASS XHR -> 200, client redirect to /account (the SERVER-chosen
role home, not a hardcoded path); 0 console errors
LOGOUT PASS signed-out card renders; /account afterwards redirects to
/auth/login, so the session is genuinely destroyed
ROLE-BOUNDARY PASS (new) a `user` navigating to /admin is bounced to /account —
ACL refuses without dumping a 403 page
ROLE-BOUNDARY was added because signing in made it reachable: the authenticated
header links the brand to /admin regardless of role, so it was worth proving the
ACL actually stops a non-admin rather than trusting the nav to hide it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013vdyTeoDrGE4hKxt53TsE8
…e 387 fix UI-AUTH-RESET-BADTOKEN now expects the dead-link card (it was the probe's first real find). Added UI-AUTH-RESET-VALID, which is the regression that matters more: a live token must still render the form, redeem, and sign the user in — the guard added for TIGER-387 must not break real reset links. Its third step re-opens the same link to prove single-use is enforced AND surfaced early. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013vdyTeoDrGE4hKxt53TsE8
"No console errors" is the probe's cheapest detector for OUR defects — a dropped pageScripts slot, a bare asset(), a 404 on a module asset all surface there first. Google's reCAPTCHA frame logs a report-only CSP violation into our console on /auth/reset, and a permanent foreign error trains you to stop reading the console at all, which is the actual damage. config.json now carries console_ignore: a pattern plus, required, WHY it is not ours. The CSP one is Google's own policy — tiger-dev sends no CSP header at all, verified with curl; www.google.com serves content-security-policy-report-only and reports to csp.withgoogle.com. The second entry is scoped deliberately: a 401 in the console is EXPECTED when a negative-path scenario provokes one, and must never be filtered on a happy path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013vdyTeoDrGE4hKxt53TsE8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-driven functional user testing for the Tiger UI — the harness for the defect class that passes a green PHPUnit suite because nothing ever rendered.
Three layers
php tests/ui/bin/paradigm-scan.phpphp tests/ui/bin/auth-probe.phpscenarios/auth.json+ thetiger-ui-probeskillStatic is exhaustive but blind to runtime; the browser proves behaviour but only where a scenario exists. Both are needed — that gap is where the TigerImage studio bugs lived.
Client/server conformance: it holds
The scan enforces AGENTS.md ("the UI calls
/api, it does not post to pages") and WEBSERVICES.md §5.Zero page-POST violations across 108 view scripts. The auth screens are textbook:
<form novalidate>with noaction/method,e.preventDefault(),fetch('/auth/login'), JSON back, client-side redirect to the server-chosen target.Two false-positive classes the scanner had to learn
Both cost real hits before the detector was trustworthy:
method="get"search form is navigation, not a page-POST — 3 flagged, all wrong.foreach ($this->x)emitting<tr>is a break — 8 hits down to 1.Mutation-tested: synthetic bad views are caught, good ones stay clean, the exit code flips.
GATED — a deliberate third result state
reCAPTCHA on dev refuses a tokenless client, so 3 auth checks never reach the logic they assert. They report GATED, not PASS: identical captcha refusals would have made the account-enumeration check pass vacuously, and a vacuous green is worse than no check.
First run
PASS: login renders · no raw i18n keys · form is AJAX not page-POST · empty creds refused cleanly · bad reset token fails safely (no 500) · session heartbeat is JSON · logout card renders · every link on it resolves ·
/loginserves the real login screen · reset can't be triggered by a bare GET.Safety
Dev-only by construction —
config.jsoncarriesnever_target, both probes refuse a production base URL, the Playwright MCP server is launched with--blocked-originscovering every prod host, and negative-path scenarios use@example.invalididentifiers so a run can never trip a real account's lockout.CI
Untouched.
phpunit.xmlscanstests/Unitandtests/Integrationonly; coveragesourcecoverslibrary/Tiger,modules,core/controllers. These are CLI scripts, not PHPUnit tests.Findings filed from this run
/api<script>/<style>tags, incl. core's own auth screensAll labelled
functional-user-test+ui-probe.Open
tests/ui/credentials.local.jsonneeds a DEV test account before the authenticated scenarios (successful login, real logout) can run.🤖 Generated with Claude Code
https://claude.ai/code/session_013vdyTeoDrGE4hKxt53TsE8