fix(tests): resolve Newman fixture paths and honour the CI base_url variable

[rubenvdlinde]

What

Clears all 14 Newman failures on development. No test was skipped, disabled or relaxed, and no application code changed — both defects were in the test harness.

Root causes (one cluster, two defects)

1. Multipart fixture paths could never resolve in CI — classification (c) fixture/seed wrong

The upload requests carried "src": "tests/integration/fixtures/*". Newman resolves a formdata src against --working-dir, which defaults to the process CWD. The shared workflow (ConductionNL/.github → .github/workflows/quality.yml, job newman) does cd server/apps/<app>/<newman-collection-path> and then invokes newman with no --working-dir — so CWD is the collection directory and a repo-root-relative path resolves to nothing.

Newman does not fail on an unresolvable file part; it sends the request without it. The app then correctly answers 400 No file uploaded. This is why all 14 failures presented as "custom token set upload is broken":

Symptom	Actual cause
upload → 400 (want 200)	no file part sent
collision → 400 (want 409)	never got far enough to collide
disallowed selector → 400 (want 422)	no CSS to inspect
malformed JSON → 400 (want 422)	no JSON to parse
list empty, export 404, delete 400	downstream of the failed upload

This only ever worked locally because run-newman.sh passed the repo root as --working-dir; CI never did.

Fix: paths are relative to the collection file (which is CWD in CI), and run-newman.sh pins --working-dir to the same directory so both agree.

2. CI's base_url / admin_user / admin_password were inert — classification (b) collection wrong

CI passes those three snake_case variables; the requests use {{baseUrl}} / {{adminUser}} / {{adminPass}}. Nothing mapped one onto the other, so all three flags did nothing and the suite silently fell back to its collection defaults.

Measured directly: with --env-var base_url=http://localhost:8199, the old collection still sent every single request to localhost:8080. Today the defaults happen to equal what CI passes, so it looked fine — but a run could target a completely different instance and report on it as if it were the one under test. (This bit during investigation: an early measurement silently hit the shared dev instance instead of the disposable replica.)

Fix: a collection-level prerequest script aliases the three names and derives noAuthBase from the effective base URL so the authz host-split stays intact. Environment scope still outranks collection scope, so an explicit --env-var baseUrl= (what run-newman.sh passes) continues to win.

Five numbers

Measured on a disposable pgsql-backed Nextcloud with openregister + nldesign enabled, invoking newman exactly as CI does (cwd = collection dir, snake_case env vars, no --working-dir):

	collections	requests	assertions	passed	failed
Before	1	46	108	94	14
After	1	46	108	108	0

Exit code 0. Identical result via the shipped run-newman.sh. This reproduces CI run 31045984082 / job 92442612396 exactly (same 14 assertions, same order).

Failure proofs

Every fix was shown able to go red:

1. Fixture resolution — hid fixtures/newman-valid.css: 12 of the 14 failures return (the other 2 use the bad-selector and malformed-JSON fixtures, still present). Restored → green.
2. The assertions exercise the app, not just file presence — changed the disallowed-selector status from 422 to 418 in CustomTokenSetController. Confirmed live with a direct curl (STATUS=418) before trusting the run; newman then reported exactly 1 failure and logged [418 I'm a teapot]. Reverted → green.
3. The alias fix — with base_url pointed at a dead port 8199, the old collection still hit :8080 and reported its usual 14; the fixed collection follows to :8199 and fails loudly (50 failed). Proves the variable now actually takes effect.

Also fixed (pre-existing)

README-newman.md claimed "23 requests / 50 assertions" and documented no folder 4 — stale since the custom-token-set family landed. Corrected to 46/108 with the family documented, plus both conventions written down so they cannot drift back.

[github-actions]

Quality Report — ConductionNL/nldesign @ 466f04e

Check	PHP	Vue	Security	License	Tests
lint	✅
phpcs	✅
phpmd	✅
psalm	✅
phpstan	✅
phpmetrics	✅
eslint		✅
stylelint		✅
build		✅
check-manifest		✅
test-l10n		✅
composer			✅	✅ 100/100
npm			✅	✅ 2/2
PHPUnit					✅
Newman					✅
Playwright					✅
Hydra gates					❌

---

Quality workflow — 2026-08-06 06:40 UTC

Download the full PDF report from the workflow artifacts.