ci(rocm): run the capability-table conformance suite without a GPU - #287
Merged
Conversation
Nothing in tests/rocm_tests/ runs on a pull request today -- it all needs an
MI300X or MI355X and a ROCm toolchain. The capability table is the one part
that does not: it is pure data plus routing logic, and its suite already
monkeypatches the architecture rather than querying one.
The only thing standing in the way was the import. flashinfer/__init__.py ends
in `raise RuntimeError("FlashInfer requires either CUDA or ROCm/HIP backend.
Detected CPU-only PyTorch installation.")`, so `from flashinfer import
arch_caps` needs a GPU-capable torch build even though nothing in the file
touches a tensor. Load arch_caps and hip_utils directly instead, under a
synthetic package so their relative imports still resolve, and the suite runs
with pytest as its only dependency.
That also makes an existing contract structural rather than aspirational. Both
modules must stay torch-free at module scope -- hip_utils sits on the
pre-HIP_VISIBLE_DEVICES path in tests/conftest.py -- and a module-scope
`import torch` now breaks the CI job outright instead of regressing quietly.
test_module_does_not_import_torch becomes test_suite_loads_without_torch and
covers both modules rather than arch_caps alone.
The job needs --noconftest: both conftests import torch and flashinfer at
module scope, so collection under tests/rocm_tests/ would fail long before
reaching this file.
Measured, all 53 tests:
pytest-only venv, no torch installed at all 53 passed in 0.43s
ROCm container, --noconftest 53 passed in 0.36s
ROCm container, conftest active, gfx950 GPU 53 passed in 0.59s
Negative control: the pre-change file cannot be collected in the first
environment at all.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR makes the ROCm capability-table conformance suite run in standard CI without requiring a GPU, ROCm toolchain, or even a Torch install, by avoiding flashinfer package import side effects during test collection.
Changes:
- Updated
tests/rocm_tests/test_arch_caps_hip.pyto loadarch_capsandhip_utilsdirectly from source files under a synthetic package, preserving their relative imports while bypassingflashinfer/__init__.py. - Reworked the “no torch at module scope” assertion into a subprocess test that validates the entire suite loads and key symbols/constants are available without importing
torch. - Added a new GitHub Actions workflow to run this single conformance file with
pytest --noconfteston PRs (and pushes toamd-integration).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/rocm_tests/test_arch_caps_hip.py | Adds a synthetic-package loader to import arch_caps/hip_utils without running flashinfer/__init__.py, enabling torch-less CI execution. |
| .github/workflows/arch-caps-conformance.yml | New CI job that installs only pytest and runs the conformance suite with --noconftest on PRs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the capability-table conformance suite run on every pull request, with no GPU and no ROCm toolchain.
Nothing in
tests/rocm_tests/runs at PR time today — it all needs an MI300X or MI355X. The capability table added in #285 is the one part that does not need hardware: it is pure data plus routing logic, and its suite already monkeypatches the architecture rather than querying one. The table is also the thing most likely to drift silently, since a wrong row does not crash — it just quietly grants or denies a backend.What was in the way
Only the import.
flashinfer/__init__.py:287ends in:so
from flashinfer import arch_capsneeds a GPU-capable torch build even though nothing in the file touches a tensor. I confirmed this against a real CPU-only install rather than assuming it — installingtorch==2.13.0+cpuand importing the package raises exactly that error, which rules out the obvious "just install CPU torch in CI" approach.The fix is to load
arch_capsandhip_utilsdirectly from their files, registered under a synthetic package so their relative imports (hip_utils:10at module scope,arch_caps:354inside_live_versions) still resolve. Two unrelated top-level modules would leave those unresolvable and silently degrade_live_versionsto its except-branch, which would makeTestVersionProbeIsCheappass vacuously.Why this is more than a convenience
Both modules must stay torch-free at module scope:
hip_utilssits on the pre-HIP_VISIBLE_DEVICESpath intests/conftest.py, andarch_capsis imported by it. That contract was asserted in one subprocess test and otherwise held by convention. It is now structural — a module-scopeimport torchin either file breaks the CI job outright instead of regressing quietly.test_module_does_not_import_torchaccordingly becomestest_suite_loads_without_torch, covering both modules instead ofarch_capsalone, and asserting the exact precondition the CI job depends on.Test plan
All 53 tests, three environments:
--noconftestModuleNotFoundError: No module named 'flashinfer'), so the suite is not passing there for some unrelated reason.test_detection_runs_once_per_processno longer clears the real_live_versionscache out from under the rest of the session.pre-commit runon both changed files.Notes
--noconftestis required and is not incidental:tests/conftest.pyandtests/rocm_tests/conftest.pyboth import torch andflashinferat module scope, so collection undertests/rocm_tests/would fail long before reaching this file. The suite defines its own fixtures and needs nothing from either conftest;test_suite_loads_without_torchis what keeps that true.Unrelated observation, not fixed here:
.github/workflows/pre-commit.ymltriggers onpush: branches: [main], but this fork's default branch isamd-integration, so that trigger never fires. Thepull_requesttrigger does, which is why pre-commit still runs on PRs.