Skip to content

test(e2e): wait for the h2 sections before snapshotting headings - #190

Merged
minuszka merged 1 commit into
mainfrom
fix/accessibility-h2-race
Sep 15, 2026
Merged

minuszka merged 1 commit into
mainfrom
fix/accessibility-h2-race

Conversation

@minuszka

Copy link
Copy Markdown
Owner

Fixes the flaky accessibility.spec.ts:79 test ("a page names itself in one h1, with its sections under it"): the CI run of #184 (2026-09-14) saw it red once on a cold start, green on the push run of the same commit (299/299).

Cause

The Overview page's h1 (.page-title) renders unconditionally on the first paint. The h2 section cards render only after a second, data-dependent render pass (two chained Promise.all fetches in dd-page-overview.ts's _load). The test snapshotted headings right after the h1 text resolved, so a slow-enough cold start could catch the gap: h1 present, 0 h2s, list.filter(h.level === 2).length fails toBeGreaterThan(0) -- exactly the CI failure signature.

Fix

One line: wait for the first h2 to be visible before taking the heading snapshot. No assertion weakened; the "no h3 or deeper" check still runs on the post-wait snapshot.

     await expect(page.locator('.page-title')).toHaveText('Overview');
+    // The h2 cards render after a second, data-dependent pass; a cold start
+    // can catch the h1 alone, which is what made this test flaky in CI (#184,
+    // 2026-09-14). Wait for the sections before taking the heading snapshot.
+    await expect(page.locator('h2').first()).toBeVisible();

     const list = await headings(page);

Diff sha256: f045bc90a1e98d8de16a55fa2e331f04014b049ce1b4ce4080c464fe7fdd0b57.

The other accessibility test that only checks h1 (every public page carries exactly one h1) does not race the same way: on every page type, the h1 is .page-title itself and renders unconditionally, so its presence never depends on the async data that gates the h2 cards.

Proof

Negative control: a scratch copy of the unfixed assertions against overviewStubs() with /api/v1/chainlocks delayed 300ms (delays the profile-dependent second fetch phase, so the h2 cards render later while the h1 does not) reproduced the exact CI failure -- h2 count 0 -- deterministically. Control file sha256: 64730e33d7c7c56adb3f6c7046071dc8d10214abe443b3170c8fc0df948d0a11 (scratch copy, not committed). Adding the same one-line wait to that copy: 10/10 green under the induced delay.

With the fix, real file:

  • npx playwright test e2e/accessibility.spec.ts --repeat-each=5: 90/90 passed.
  • npx playwright test e2e/accessibility.spec.ts --repeat-each=20: 359/360 passed -- the one failure is a different, pre-existing test (the header poll does not take the focus, line 207, uses page.clock.install/fastForward), reproduced in isolation at 1/20 with --workers=1. Unrelated to this change (different locators, different mechanism, fails before the focus assertions even run) and out of scope here; flagging it, not fixing it in this PR.
  • Full suite once: npx playwright test: 299/299 passed.

Not part of this change: A8 (feature_llmq_q60_proto_floor.py, Core) -- a separate, unrelated CI flake.

🤖 Generated with Claude Code

The Overview page's h1 (.page-title) renders unconditionally on first
paint; the h2 cards render only after a second, data-dependent pass.
A cold CI start can catch the gap between the two: the CI run of #184
(2026-09-14) saw the h1 638ms in with 0 h2 sections, while the same
commit's push run was 299/299 green. Reproduced locally against an
unfixed copy of the assertions with a 300ms-delayed chainlocks stub
(h2 count 0, matching the CI failure); the same delayed stub passes
10/10 once the wait is added.

Waits for the first h2 to be visible before taking the heading
snapshot; no assertion is weakened. Verified: 90/90 (repeat-each=5)
and 359/360 (repeat-each=20, one unrelated pre-existing flake in a
different test) on this fix, full e2e suite 299/299.
@minuszka
minuszka merged commit c531260 into main Sep 15, 2026
6 checks passed
@minuszka
minuszka deleted the fix/accessibility-h2-race branch September 15, 2026 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant