Skip to content

ci: run the CPU-only pytest suite in GitHub Actions - #428

Closed
rishabhsinha17 wants to merge 1 commit into
ovg-project:mainfrom
rishabhsinha17:ci/pytest-workflow
Closed

rishabhsinha17 wants to merge 1 commit into
ovg-project:mainfrom
rishabhsinha17:ci/pytest-workflow

Conversation

@rishabhsinha17

@rishabhsinha17 rishabhsinha17 commented Aug 6, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Run the CPU-only pytest suite in GitHub Actions on every push and pull request.

Fixes #395.

Problem

The only existing workflow (pre-commit.yml) runs lint/format/mypy and the C++ unit tests, but none of the Python tests under tests/ run in CI, so regressions in the pure-Python paths are only caught when someone runs the right test by hand.

Changes

  • add .github/workflows/python-tests.yml: installs CPU-only PyTorch (--index-url https://download.pytorch.org/whl/cpu) plus requirements.txt and the test-only deps (pyyaml, numpy, aiohttp), with pip caching
  • run pytest tests/ with an explicit, commented ignore list for what cannot run on a CPU runner:
    • test_ElasticMLAMemoryPool, test_kvcache_manager, test_offline_serving, test_paged_allocator_aliasing, test_alloc_kv_cache_alignment, test_hybrid_contiguous_layout, test_ipc_timeout — import or exercise the compiled kvcached.vmm_ops extension, whose build requires a CUDA/ROCm PyTorch (setup.py raises on CPU-only builds)
    • test_sleep_manager — async tests that expect running vLLM/SGLang server endpoints

Everything else runs, including the /dev/shm-backed test_shm_info_tracker tests, which are Linux-only and cannot run on macOS dev machines at all.

Validation

Verified on an actual ubuntu-latest runner via my fork before opening this PR: green run — 81 passed, 1 skipped.

Possible follow-ups: split the vmm_ops-dependent files so their pure-Python tests also run in CI, and add a GPU-runner job for the rest.

Related: #403 proposes a manifest-based classification approach to the same issue; this PR is the minimal single-workflow variant.

The only existing workflow (pre-commit.yml) runs lint/format/mypy and the
C++ unit tests, but none of the Python tests under tests/ run in CI. Add
a workflow that installs CPU-only PyTorch and runs the subset of the
pytest suite that needs neither a GPU nor the compiled vmm_ops extension
(81 tests today), so regressions in the pure-Python paths are caught on
every PR.

Fixes ovg-project#395
@shipiyouniao

Copy link
Copy Markdown
Collaborator

Thanks for working on this. #403 has now been merged and covers the same CPU-only pytest CI goal with exhaustive manifests and omission checks. Since this PR uses an explicit ignore list for the same issue, it appears to be superseded. Could you close this PR, unless there is a non-overlapping follow-up you would like to rebase into a focused change?

@rishabhsinha17

Copy link
Copy Markdown
Collaborator Author

Agreed, closing. I compared the two: #403's manifests are an allowlist with tools/check_test_classification.py enforcing that every discovered test is classified, so it cannot silently drift the way this PR's ignore list would as tests are added. That is the better design and it covers the issue.

One non-overlapping remainder, now folded into #432 rather than here: tests/test_sleep_manager.py is in integration.txt because its async tests need live vLLM/SGLang endpoints. #432 makes them self-contained (they register their own models and degrade gracefully with no server), so it now also moves that file integration -> cpu, adding 9 tests to the CPU run. check_test_classification.py reports total=26, cpu=16, gpu=5, integration=5, and the cpu manifest passes locally apart from the /dev/shm tests that cannot run on macOS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI runs no tests — add a GitHub Actions job for the CPU-only pytest suite

2 participants