Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions tests/ui/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ AI-driven functional testing for the Tiger UI. Three layers, each answering a di
| `php tests/ui/bin/auth-probe.php` | does the AJAX/JSON contract behave? | seconds, no browser |
| the `tiger-ui-probe` skill + Playwright MCP | does it WORK for a user? | a real browser |

Browser work is recorded as **scenarios** under `scenarios/` — `auth.json` (sign in, failed sign
in, reset, sign out) and `admin-grids.json` (every admin data grid actually draws). Each scenario is
intent-level with no selectors, and carries a `verified` line naming what the last real run saw.

The static scan is exhaustive but can only see code; the browser proves live behaviour but only
on pages someone wrote a scenario for. Run both — that gap is where the TigerImage studio bugs
lived (three defects that all passed a green PHPUnit suite because nothing ever rendered).
Expand All @@ -23,6 +27,25 @@ scenarios use `@example.invalid` identifiers so a run can never trip a real acco
Copy `credentials.local.json.dist` to `credentials.local.json` (gitignored) and fill in a **dev**
test account. Without it the authenticated scenarios are skipped; everything anonymous still runs.

## Seed first, or a grid assertion means nothing

```
sudo -u apache php tests/ui/bin/seed-dev.php \
--base=https://tiger-dev.webtigers.com --email=uiprobe-super@tiger.test --apply
```

An **empty grid and a broken grid are the same picture** — that is how three grids shipped serving
HTTP 200 with no rows and no console error (TIGER-390). If the probe account's org owns no content,
every grid looks like those three and the only way to tell them apart is a SQL query, which a
scenario cannot run. The seeder gives that org one page, one article, one menu, one (inactive)
snippet and one subscriber, all labelled `UI Probe` and on the reserved `.invalid` domain.

It is a **dry run** unless you pass `--apply`, it is idempotent, and it refuses to run without
`--base`; a `--base` named in `never_target` is refused, and so is a missing `never_target` list.
That guard is explicit on purpose: an install cannot tell you which site it is — on tiger-dev,
`APPLICATION_ENV` is literally `production`, `tiger.site.url` is unset and `gethostname()` returns
an EC2 internal name — so the operator names the target and the list refuses the forbidden ones.

## Exit codes

Both probes exit non-zero on a real failure, so either drops into a script when you want that.
Expand Down
93 changes: 87 additions & 6 deletions tests/ui/bin/paradigm-scan.php
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
'native_submit' => ['Form has no AJAX handler (native submit)','review'],
'rendered_rows' => ['Table rows server-rendered, not from /api','bug'],
'inline_script' => ['<script>/<style> tag in a view, not an asset','batch'],
'datatable_gate' => ['Grid screen never sets useDataTables (lib never loads)','blocker'],
];

/** Views whose page-POST is a DOCUMENTED exception (AGENTS.md: SSO and the like). */
Expand Down Expand Up @@ -60,20 +61,76 @@ function hasAjaxHandler(string $src, string $file, string $root): bool
// view pulls in by theme path, asset()). Follow either and look in the real file — otherwise every
// conforming screen reads as a native-submit form, which is the opposite of the truth.
if (preg_match_all('/(?:pageScript|pageStyle|asset)\([^)]*?[\'"]([^\'"]*\.js)[\'"]/i', $src, $m)) {
$idx = jsIndex($root);
foreach ($m[1] as $a) {
foreach (glob($root . '/**/' . basename($a)) ?: [] as $hit) {
if (preg_match('/fetch\s*\(|TigerButton\.run|XMLHttpRequest/i', (string) @file_get_contents($hit))) { return true; }
}
$found = shell_exec('find ' . escapeshellarg($root) . ' -name ' . escapeshellarg(basename($a))
. ' -not -path "*/vendor/*" -print -quit 2>/dev/null');
if ($found && preg_match('/fetch\s*\(|TigerButton\.run|XMLHttpRequest/i', (string) @file_get_contents(trim($found)))) {
$hit = $idx[basename($a)] ?? '';
if ($hit !== '' && preg_match('/fetch\s*\(|TigerButton\.run|XMLHttpRequest/i', (string) @file_get_contents($hit))) {
return true;
}
}
}
return false;
}

/**
* The JS a view REGISTERS, as source text. The house rule puts behavior in asset files
* ($this->pageScript()), so a question about a screen's JavaScript has to be asked of those files,
* not of the .phtml. Returns the concatenated contents of every .js the view names.
*/
/**
* basename -> path for every shipped .js under the root, built ONCE. The scan asks "what is in the
* file this view registered?" for nearly every view, and a `find` per asset per view turned a
* milliseconds-long scan into a 30-second one. One pass, then pure array lookups.
*/
function jsIndex(string $root): array
{
static $idx = null;
if ($idx !== null) { return $idx; }
$idx = [];
$it = new RecursiveIteratorIterator(
new RecursiveCallbackFilterIterator(
new RecursiveDirectoryIterator($root, FilesystemIterator::SKIP_DOTS),
static function ($f) {
$n = $f->getFilename();
if ($f->isDir()) { return $n !== 'vendor' && $n !== 'node_modules' && $n !== '.git'; }
return substr($n, -3) === '.js';
}
)
);
foreach ($it as $f) {
$b = $f->getFilename();
if (!isset($idx[$b])) { $idx[$b] = $f->getPathname(); }
}
return $idx;
}

function registeredJs(string $src, string $root): string
{
$out = '';
if (!preg_match_all('/(?:pageScript|asset)\([^)]*?[\'"]([^\'"]*\.js)[\'"]/i', $src, $m)) {
return $out;
}
$idx = jsIndex($root);
foreach (array_unique($m[1]) as $a) {
$hit = $idx[basename($a)] ?? '';
if ($hit !== '') { $out .= (string) @file_get_contents($hit); }
}
return $out;
}

/**
* The controller file that owns a view, or '' when there isn't one (a theme view, a layout).
* `modules/system/views/scripts/sites/index.phtml` -> `modules/system/controllers/SitesController.php`;
* `core/views/scripts/auth/login.phtml` -> `core/controllers/AuthController.php`.
*/
function controllerFor(string $rel, string $root): string
{
if (!preg_match('~^(core|modules/[^/]+)/views/scripts/([^/]+)/~', $rel, $m)) { return ''; }
$class = str_replace(' ', '', ucwords(str_replace(['-', '_'], ' ', $m[2])));
$path = $root . '/' . $m[1] . '/controllers/' . $class . 'Controller.php';
return is_file($path) ? $path : '';
}

function lineOf(string $src, int $offset): int { return substr_count(substr($src, 0, $offset), "\n") + 1; }

function scanFile(string $file, string $root): array
Expand Down Expand Up @@ -163,6 +220,30 @@ function scanFile(string $file, string $root): array
'detail' => trim(preg_replace('/\s+/', ' ', $tag))];
}
}

// ---- 5. a grid screen that never opts into DataTables ----
// The admin layout loads jQuery + DataTables + tiger.datatable.js ONLY behind
// `if (!empty($this->useDataTables))`. Without the flag the `tigerDataTable` global is never
// defined, and every grid script opens with a bail-out guard — so the screen serves HTTP 200
// with an empty table and NOT ONE console error. That shape is invisible to the browser probe's
// "no console errors" detector (the guard is doing its job) and to PHPUnit (nothing renders),
// which is why it needs a STATIC check. Three core screens shipped this way (TIGER-390).
//
// Scope, stated honestly: the flag is looked for anywhere in the owning controller, not inside
// the specific action. So a controller that sets it in SOME action passes — a finding here means
// the controller never sets it at all, which is the case that actually shipped and keeps the
// false-positive rate at zero.
$ctrl = controllerFor($r, $root);
if ($ctrl !== '' && strpos($r, '/layouts/') === false) {
$js = $src . registeredJs($src, $root);
if (preg_match('/tigerDataTable\s*\(/', $js)
&& strpos((string) @file_get_contents($ctrl), 'useDataTables') === false) {
$hits[] = ['check' => 'datatable_gate', 'file' => $r, 'line' => 1,
'detail' => 'grid JS calls tigerDataTable() but ' . rel($ctrl, $root)
. ' never sets $this->view->useDataTables'];
}
}

return $hits;
}

Expand Down
139 changes: 139 additions & 0 deletions tests/ui/bin/seed-dev.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
<?php
// SPDX-License-Identifier: BSD-3-Clause
// Copyright (c) 2026 WebTigers. Tiger™ and WebTigers™ are trademarks of WebTigers.
/**
* seed-dev.php — give the probe account's org just enough content that a grid scenario can tell
* "correctly empty" from "broken".
*
* Why this exists: on 2026-10-25 three admin grids were shipping an empty table behind HTTP 200
* with no console error (TIGER-390). The probe org owned no content, so EVERY grid looked exactly
* like those three — and separating the two took a SQL query, which a scenario cannot run. A grid
* assertion is only meaningful when the screen is supposed to have rows.
*
* Runs ON THE SERVER (it needs the booted app + DB), not from a laptop:
* sudo -u apache php tests/ui/bin/seed-dev.php \\
* --base=https://tiger-dev.webtigers.com --email=uiprobe-super@tiger.test [--apply]
*
* Default is a DRY RUN; nothing is written without --apply. Idempotent: every row is matched before
* insert, so a re-run adds nothing. Every row is labelled "UI Probe" and uses the reserved
* `.invalid` domain for addresses, so the seed is identifiable and can never reach a real inbox.
*/

$opts = getopt('', ['base::', 'email::', 'root::', 'config::', 'apply']);
$root = $opts['root'] ?? dirname(__DIR__, 3); // the app root (…/vendor/webtigers/tiger-core → up 3)
$email = (string) ($opts['email'] ?? '');
$apply = array_key_exists('apply', $opts);

if (!is_file($root . '/vendor/autoload.php')) {
fwrite(STDERR, "seed-dev: no app at $root (pass --root=/var/www/tiger-dev)\n");
exit(2);
}
define('APPLICATION_ROOT', $root);
require $root . '/vendor/autoload.php';
(new Tiger_Application($root))->boot();

/**
* Dev only, enforced here and not merely documented. The probes refuse a production base URL; a
* script that WRITES has to refuse a production install, and it checks the thing that cannot be
* talked around — the configured site host, against tests/ui/config.json's never_target list.
*/
// ---------------------------------------------------------------- the dev-only guard
// There is NO reliable way for a Tiger install to tell you which site it is from the inside: on
// tiger-dev, APPLICATION_ENV is literally "production" (dev mirrors prod on purpose), tiger.site.url
// is unset, and gethostname() returns an EC2 internal name. An environment sniff would therefore
// either refuse the install this script exists for, or pass vacuously.
//
// So the operator NAMES the target and the guard refuses a forbidden one — the same --base +
// never_target contract the two probes already use. Three deliberate gates: --base is required,
// never_target must be loadable (fail closed), and nothing is written without --apply.
$base = rtrim((string) ($opts['base'] ?? ''), '/');
if ($base === '') {
fwrite(STDERR, "seed-dev: --base=https://host is required — name the install you believe you are seeding\n");
exit(2);
}
$cfgFile = $opts['config'] ?? (__DIR__ . '/../config.json');
$cfg = is_file($cfgFile) ? (json_decode((string) file_get_contents($cfgFile), true) ?: []) : null;
$never = is_array($cfg) ? ($cfg['never_target'] ?? null) : null;
if (!is_array($never) || !$never) {
fwrite(STDERR, "REFUSING: no never_target list (looked in $cfgFile) — pass --config=<tests/ui/config.json>\n");
exit(2);
}
// Hosts compare EXACTLY, never as substrings: tiger-dev.webtigers.com contains "webtigers.com", so a
// substring test would refuse the install this script is for. Prod and dev differ by one label.
$host = strtolower((string) parse_url((strpos($base, '//') === false ? 'https://' . $base : $base), PHP_URL_HOST));
foreach ($never as $forbidden) {
$needle = strtolower((string) (parse_url(rtrim((string) $forbidden, '/'), PHP_URL_HOST) ?: trim((string) $forbidden, '/')));
if ($needle !== '' && $needle === $host) {
fwrite(STDERR, "REFUSING: $host is a production target (never_target in $cfgFile)\n");
exit(2);
}
}

if ($email === '') { fwrite(STDERR, "seed-dev: --email=<probe account> is required\n"); exit(2); }

$db = Zend_Db_Table_Abstract::getDefaultAdapter();
$who = $db->fetchRow(
$db->select()->from(['u' => 'user'], ['user_id'])
->join(['ou' => 'org_user'], 'ou.user_id = u.user_id', ['org_id'])
->where('u.email = ?', $email)->where('u.deleted = 0')->limit(1)
);
if (!$who) { fwrite(STDERR, "seed-dev: no account + membership for $email\n"); exit(1); }
[$org, $usr] = [(string) $who['org_id'], (string) $who['user_id']];

Tiger_Model_Table::setActor($usr, $org);
$now = date('Y-m-d H:i:s');
$std = ['org_id' => $org, 'deleted' => 0, 'created_by' => $usr, 'updated_by' => $usr,
'created_at' => $now, 'updated_at' => $now];

$report = [];
$mk = function (string $what, string $table, array $where, array $row) use ($db, $apply, &$report) {
$sel = $db->select()->from($table, ['c' => new Zend_Db_Expr('COUNT(*)')]);
foreach ($where as $c => $v) { $sel->where("$c = ?", $v); }
if ((int) $db->fetchOne($sel) > 0) { $report[] = " have $what"; return; }
if ($apply) { $db->insert($table, $row); }
$report[] = ($apply ? ' ADD ' : ' would add ') . $what;
};

// A CMS page the org genuinely OWNS (/cms/page).
$mk('cms page "UI Probe Page"', 'page', ['org_id' => $org, 'slug' => 'ui-probe-page'], $std + [
'page_id' => Tiger_Uuid::v7(), 'title' => 'UI Probe Page', 'slug' => 'ui-probe-page',
'type' => Tiger_Model_Page::TYPE_PAGE, 'locale' => 'en', 'format' => 'html',
'status' => Tiger_Model_Page::STATUS_PUBLISHED, 'published_at' => $now,
'body' => '<p>Seeded for functional UI testing. Safe to delete.</p>',
]);

// An article (/blog/post). An article IS a page row — type='article', NOT 'post'; seeding 'post'
// produced a row no grid could see, which is its own small lesson about guessing a constant.
$mk('blog article "UI Probe Article"', 'page', ['org_id' => $org, 'slug' => 'ui-probe-article'], $std + [
'page_id' => Tiger_Uuid::v7(), 'title' => 'UI Probe Article', 'slug' => 'ui-probe-article',
'type' => Blog_Model_Post::TYPE_ARTICLE, 'locale' => 'en', 'format' => 'html',
'status' => Tiger_Model_Page::STATUS_PUBLISHED, 'published_at' => $now,
'body' => '<p>Seeded for functional UI testing. Safe to delete.</p>',
]);

// A one-item menu (/cms/menu).
$mk('menu "uiprobe"', 'menu', ['org_id' => $org, 'menu_key' => 'uiprobe'], $std + [
'menu_id' => Tiger_Uuid::v7(), 'menu_key' => 'uiprobe', 'source' => 'user', 'sort_order' => 0,
'label' => 'UI Probe Home', 'url' => '/', 'status' => Tiger_Model_Menu::STATUS_PUBLISHED,
]);

// A snippet for the Code Area (/code) — CSS, INACTIVE, draft. Seeding executable PHP that actually
// runs would be reckless in a fixture; the grid only needs a row to draw.
$mk('code snippet "UI Probe snippet" (inactive)', 'code', ['org_id' => $org, 'name' => 'UI Probe snippet'], $std + [
'code_id' => Tiger_Uuid::v7(), 'name' => 'UI Probe snippet', 'language' => Tiger_Model_Code::LANG_CSS,
'description' => 'Seeded for functional UI testing. Inactive on purpose.',
'code' => "/* seeded for UI testing */\n", 'run_location' => 'frontend', 'priority' => 50,
'active' => 0, 'status' => Tiger_Model_Code::STATUS_DRAFT,
]);

// A subscriber (/newsletter/admin) on a reserved, undeliverable domain (RFC 2606).
$mk('newsletter subscriber (@example.invalid)', 'newsletter_subscriber',
['org_id' => $org, 'email' => 'uiprobe-subscriber@example.invalid'], $std + [
'newsletter_subscriber_id' => Tiger_Uuid::v7(), 'email' => 'uiprobe-subscriber@example.invalid',
'name' => 'UI Probe Subscriber', 'status' => Newsletter_Model_Subscriber::STATUS_CONFIRMED,
'consent_source' => 'uiprobe-seed', 'consent_at' => $now, 'token' => Tiger_Uuid::v4(),
'source' => 'uiprobe-seed',
]);

printf("%s — %s (org %s)\n%s\n", $apply ? 'seeded' : 'DRY RUN (pass --apply to write)', $email, $org,
implode("\n", $report));
Loading
Loading