Skip to content

CI: test-backend per-marker steps exit-5 on empty markers (a2a, goals) — no tolerance #82

Description

@izzywdev

Summary (devops / CI)

test-backend (.github/workflows/test.yml) runs one pytest invocation per marker-m unit, -m api, -m database, -m integration, -m rag, -m a2a, -m mcp, -m goals — with no exit-code-5 tolerance. When a marker selects zero tests, pytest exits 5 ("no tests ran"), which fails the step and the whole job.

Two markers are now legitimately empty:

This was masked until now because the -m database step aborted the job first (23 pytest-asyncio errors, since fixed on fix/test-backend-lifespan). With those fixed + the aspirational suites quarantined (#80, #81), the job now reaches these empty-marker steps and fails on exit 5 rather than on any real test failure. Steps unit/api/database/integration/rag pass; mcp passes (quarantined but collected-skipped).

Fix (owned by devops-engineer — outside services/orchestrator)

Either:

  1. Tolerate the empty-marker exit code per step, e.g. python -m pytest tests/ -m "a2a" ... || [ $? -eq 5 ], or
  2. Consolidate the per-marker steps into a single pytest tests/ run (markers then only matter for reporting), or
  3. Drop the dedicated -m a2a / -m goals steps until those markers have runnable tests.

Backend slice cannot resolve this: it owns services/orchestrator/**, not CI workflows, and is instructed not to touch the #76 a2a quarantine. Context: PR #72, branch fix/test-backend-lifespan.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions