Skip to content

fix(newman): fixture src paths are relative to the COLLECTION dir, not the app root - #238

Merged
rubenvdlinde merged 1 commit into
developmentfrom
fix/newman-fixture-paths
Aug 6, 2026
Merged

fix(newman): fixture src paths are relative to the COLLECTION dir, not the app root#238
rubenvdlinde merged 1 commit into
developmentfrom
fix/newman-fixture-paths

Conversation

@rubenvdlinde

@rubenvdlinde rubenvdlinde commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The quality / Integration Tests (Newman) job has been red on development with 14 assertion failures, all in "4. Custom token sets (upload/list/export/delete)". Every upload returned 400 — including the one asserting 200 — and the follow-on assertions then failed on undefined, because there was no body to read.

None of it was an API bug. The three multipart fixtures were declared with a src relative to the app root:

"src": "tests/integration/fixtures/newman-valid.css"

But the shared workflow runs:

cd server/apps/<app>/tests/integration
newman run "$collection" ...

so newman's working directory is tests/integration, and it looked for tests/integration/tests/integration/fixtures/…. The file was never attached, the request arrived with no upload, and CustomTokenSetController::readUpload() correctly answered No file uploaded. with 400.

The endpoint was doing exactly the right thing. The collection was lying about where its fixtures live.

The cascade is worth naming

One unresolvable path produced 14 failures across six named cases — 409-on-collision, 422-on-bad-selector, 422-on-malformed-json, listing, export, delete — because each depends on the token set the first case was supposed to create. Read bottom-up, that failure list looks like a broken feature. It is a single missing file.

Verified

Ran newman against a dead port, so only file resolution is exercised:

result
before unable to load data for "…/fixtures/newman-bad-selector.css", no such file — and the same for newman-malformed.tokens.json and newman-valid.css
after no file-resolution errors at all

Each rewritten path was also checked to resolve from CI's working directory.

Why the collection and not the workflow

Adding --working-dir to the shared workflow would also work, but that workflow's contract is already "cwd is the collection directory" — and a fixture path relative to its own collection is what any other consumer of that contract would expect. A fleet sweep found nldesign is the only repo with app-root-relative src paths, so there is nothing else to bring along.

…t the app root

`quality / Integration Tests (Newman)` has been red on development with 14
assertion failures, all in "4. Custom token sets (upload/list/export/delete)".
Every upload returned 400, including the one asserting 200, and the follow-on
assertions then failed on `undefined` because there was no response body to
read.

None of it was an API bug. The three multipart fixtures were declared as

    "src": "tests/integration/fixtures/newman-valid.css"

— a path relative to the APP ROOT. The shared workflow runs

    cd server/apps/<app>/tests/integration
    newman run "$collection" ...

so newman's working directory IS tests/integration, and it looked for
tests/integration/tests/integration/fixtures/... . The file was never
attached, the request arrived with no upload, and CustomTokenSetController::
readUpload() correctly answered `No file uploaded.` with 400. The endpoint was
doing exactly the right thing; the collection was lying about where its
fixtures live.

The cascade is worth naming: ONE unresolvable path produced 14 failures
spread across six named test cases (409-on-collision, 422-on-bad-selector,
422-on-malformed-json, listing, export, delete), because each subsequent case
depends on the set the first one was supposed to create. Reading the failure
list bottom-up suggests a broken feature; there is a single missing file.

Verified by running newman against a dead port, so only file resolution is
exercised:

    before   'unable to load data for "…/fixtures/newman-bad-selector.css", no such file'
             'unable to load data for "…/fixtures/newman-malformed.tokens.json", no such file'
             'unable to load data for "…/fixtures/newman-valid.css", no such file'
    after    (no file-resolution errors at all)

and by checking each rewritten path resolves from the CI working directory.

Fixing the collection rather than adding `--working-dir` to the shared
workflow: the workflow's contract is already "cwd is the collection
directory", and a fixture path relative to its own collection is what every
other consumer of that contract would expect.
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nldesign @ 5f0c50a

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
composer ✅ 100/100
npm ✅ 2/2
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-06 13:07 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit d7f723c into development Aug 6, 2026
30 of 32 checks passed
@rubenvdlinde
rubenvdlinde deleted the fix/newman-fixture-paths branch August 6, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant