Skip to content

test harness: an explicit -m replaces addopts and the acceptance lane's session Playwright fixture poisons every later async test #44

Description

@NetDevAutomate

Symptom

Any explicit -m <expr> on the command line (e.g. pytest -m 'not e2e') replaces the marker expression in addopts (pyproject.toml:98, packages/studyloop/pyproject.toml:95) instead of adding to it. The acceptance, integration, live and uat lanes are then collected and skipped rather than deselected. For tests/acceptance/test_kiro_web_acp_lane.py that difference matters: its not_selected_marker("kiro") skip runs after pytest has already set up the session-scoped fixtures the tests request — --setup-plan shows playwright, browser_type_launch_args, browser_name['chromium'] — and pytest-playwright's playwright fixture calls sync_playwright().start(), which runs an asyncio loop in a greenlet on the main thread.

Every later test that starts its own loop (pytest-asyncio / asyncio.run) then fails with:

RuntimeError: Runner.run() cannot be called from a running event loop

Observed 2026-09-23 on a full run: 178 of 319 failures, across test_acp_transport.py, test_active_session.py, test_active_generation.py, test_session_ws_grace.py, test_session_slot_reconcile.py, test_mcp_stdio_smoke.py, test_pty_transport.py, test_study_*, test_web_schema_init.py … none of which is at fault.

Reproduction (clean main @ c1a28de1)

# probe: asserts no loop is running during a plain sync test
printf 'import asyncio\ndef test_probe():\n    try: loop = asyncio.get_running_loop()\n    except RuntimeError: loop = None\n    assert loop is None, loop\n' > packages/studyloop/tests/test_zz_probe.py
uv run pytest -q packages/studyloop/tests/acceptance/test_kiro_web_acp_lane.py packages/studyloop/tests/test_zz_probe.py
#  -> 1 passed, 3 deselected              (addopts deselects the lane; no fixture runs)
uv run pytest -q -m 'not e2e' packages/studyloop/tests/acceptance/test_kiro_web_acp_lane.py packages/studyloop/tests/test_zz_probe.py
#  -> 1 failed, 3 skipped                 (lane skipped AFTER the session Playwright fixture started)
rm packages/studyloop/tests/test_zz_probe.py

Bisected from the 109 files collected before test_acp_transport.py: only this file poisons the loop. Importing the module outside pytest does not (the loop comes from the fixture, not the import).

Why the file's own guard does not hold

The module comment says the marker skip is evaluated "before any fixture of any scope, so an unselected run never launches the session-scoped Playwright browser". That is true of the browser fixture (function-level request through _acp_auth_context), but pytest still sets up the session-scoped playwright fixture for a skipped test — --setup-plan shows it. The guarantee the comment claims exists only when the lane is deselected, which is what addopts does and what any explicit -m undoes.

Candidate fixes (one is enough)

  1. Deselect rather than skip. A pytest_collection_modifyitems hook in tests/acceptance/conftest.py that removes unselected acceptance items from items (appending to config.hook.pytest_deselected) instead of adding a skip marker. Fixtures of a deselected item never run. Cheapest and matches the file's stated intent.
  2. Do not request session fixtures from an unselected lane. Have _acp_auth_context obtain the browser via request.getfixturevalue("browser") after require_harness("kiro"), so the Playwright fixture chain is only entered when the lane is selected.

Either way, a regression test: the probe above as a real test placed after the acceptance directory in collection order (or a pytester test that runs the lane with -m 'not e2e' and asserts the probe passes).

Not caused by

Any of #36 / #40 / #42 / #41 — reproduced byte-for-byte on a clean origin/main worktree. Found while verifying #43.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions