Skip to content

test: resolve potentially flaky playwright tests - #66026

Merged
susnux merged 11 commits into
masterfrom
test/playwright-flaky-audit
Oct 30, 2026
Merged

susnux merged 11 commits into
masterfrom
test/playwright-flaky-audit

Conversation

@susnux

@susnux susnux commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Summary

resolve the remaining potential flaky playwright test issues.

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

susnux added 11 commits October 29, 2026 23:44
System tags are server-global, but the systemtags specs cleaned up by
deleting every tag on the instance (`clearTags()` in `afterAll`, and a
mass delete in `beforeEach` of the admin settings spec, which also used
the fixed names `foo` / `bar`). A spec running in parallel lost its tags
mid-test, and the non-forced delete threw when a tag vanished underneath
it.

Generate tag names through a `tagName` fixture that deletes exactly the
tags created under those names after each test.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
`force_language` / `force_locale` were deleted in `beforeAll` and
restored in `afterAll` of the whole personal info spec. With
`fullyParallel`, every worker that runs a test of the file runs those
hooks, so one worker restoring the enforcement made the language and
locale section read-only for another worker still testing it. The
per-test `beforeEach` only narrowed that window.

Toggle the enforcement around the language and locale tests alone and
run them serially, so a single worker owns that state. The restored
values are the defaults of the test server.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
`networkidle` is timing-fragile with background polling. After picking
a locale it could also resolve on the page that was about to reload,
before the reload had even started.

Wait for the reload's `load` event instead, and rely on the retrying
assertion after opening the profile page.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
The session and account fixtures retried a failed login or
`createRandomUser()` exactly once after a fixed 800 ms. The transient
failures are real (logins answered with HTTP 500 twice in a row, user
creation exiting non-zero), and a single fixed-delay retry is not
enough to ride them out.

Share a `retryWithBackoff` helper that tries three times with an
exponential delay, and use it in every fixture that had its own copy of
the single retry.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
`ContactsMenuPage.search()` promised to wait for the server response
but only filled the debounced input, so callers asserted against
whatever the list showed before the search request was even sent.

Wait for the contacts request carrying the search term.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
`isFullyInViewport()` first asserted that the locator resolves to one
element. For a row the virtual list has dropped from the DOM, which is
the strongest form of "not in the viewport", that assertion threw, and
since `expect.poll()` does not catch errors thrown by its callback, the
scrolling specs polling for `false` failed instead of passing.

Report an element that is not rendered as not in the viewport.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
`selectAll()` / `deselectAll()` force-clicked the master checkbox
without checking the outcome. A click landing mid-render could be a
no-op, and the caller only failed later on an unrelated assertion.

Use `check()` / `uncheck()` instead, which assert the resulting
checked state. The data attribute sits on the `<input>` itself.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
`appRow()` filtered table rows by a substring of their text and
silently took the first match in DOM order, so "Files" also matched
"Files sharing". The enable and disable button helpers inherited that.

Match the row by the app name its details link starts with.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
The app management tests enable `testing` and disable
`updatenotification` instance-wide, but only reset that state before
each of their own tests. The last test's changes leaked into every
spec running afterwards on the same server.

Restore both apps to their default state once the spec is done.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
The group share test had the sharee delete the file through the shared
folder right after creating the share. Shares mount into the
recipient's tree asynchronously, so the DAV `DELETE` could hit a 404
before the mount was there.

Poll for the shared file with `waitForShare()` first, as the other
sharing specs do.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
The rename input selects the basename in a `$nextTick` after it is
rendered, but the test read `selectionStart` / `selectionEnd` once as
soon as the input was visible, which could be before the selection was
applied.

Poll the selection range until it matches.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>
@susnux susnux added this to the Nextcloud 36 milestone Oct 29, 2026
@susnux susnux added 3. to review Waiting for reviews tests Related to tests labels Oct 29, 2026
@susnux susnux changed the title Test/playwright flaky audit test: resolve potentially flaky playwright tests Oct 29, 2026
@susnux
susnux marked this pull request as ready for review October 29, 2026 23:20
@susnux
susnux requested a review from a team as a code owner October 29, 2026 23:20
@susnux
susnux requested review from DerDreschner, kristian-zendato, pringelmann and skjnldsv and removed request for a team October 29, 2026 23:20
@susnux
susnux merged commit 30d9cca into master Oct 30, 2026
185 of 195 checks passed
@susnux
susnux deleted the test/playwright-flaky-audit branch October 30, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews AI assisted tests Related to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[tests] Analysis of potential flaky e2e tests (AI generated)

4 participants