fix(rocm): resolve the build architecture in one place, not three

[demandal25]

Summary

Three call sites independently answered "what architecture are we building for", and on CDNA4 they disagreed. Measured on a gfx950 host with FLASHINFER_ROCM_ARCH_LIST unset, before this change:

ACTUAL device arch                              gfx950
validate_flashinfer_rocm_arch(arch_list=None)   ['gfx942']   <-- wrong
CompilationContext().TARGET_ROCM_ARCHS          ['gfx950']
resolve_aiter_build_arch()                      gfx950

The JIT validated gfx942 while compiling for gfx950. That check exists to catch "your PyTorch was not built for this architecture", and it was asking about an architecture nobody was building for — vacuous on a PyTorch carrying both, and a spurious hard failure on an arch-specific build carrying only gfx950. Nothing noticed because jit/core.py discards the return value; the comment on jit/core.py:148 even said # ... or defaults to gfx942.

What changed

hip_utils.resolve_target_archs() becomes the single resolver — explicit argument, then FLASHINFER_ROCM_ARCH_LIST, then the architectures actually present, then every supported architecture with a warning. validate_rocm_arch, validate_flashinfer_rocm_arch, CompilationContext and aot_hip all route through it. CompilationContext._auto_detect_archs is deleted — private, no other caller, and the holder of two more return "gfx942" lines.

The last-resort fallback changes from "gfx942" to every supported architecture. On a GPU-less build host the old literal was not conservative, it was a guess that silently produced a gfx942-only artifact. A fat build is slower but correct wherever it lands, and the warning tells the operator how to make it cheap again. No Dockerfile, CI job or Jenkinsfile in this repo sets the variable, so this path is reachable in practice.

Detection uses rocminfo rather than torch.cuda, so the resolver adds no torch dependency to a module that must stay importable without one (the hardware-less conformance job from #287 loads it).

The test that protected the bug

test_defaults_to_gfx942_when_no_env_and_no_arg asserted == "gfx942" for exactly this path. A test that pins a wrong constant is how the constant survives. It now pins the detected architecture, and a new TestResolveTargetArchs covers all four resolution branches plus the property the change exists for — that the validator and the thing emitting --offload-arch agree.

Test plan

• After, same host and script: all four rows report gfx950.
• FLASHINFER_ROCM_ARCH_LIST=gfx942 on a gfx950 box still resolves to gfx942 — cross-compiling stays possible.
• A container with no /dev/kfd resolves to gfx942,gfx950, emits the warning, and does so without importing torch.
• 169 passed across test_hip_utils.py, test_aiter_build_arch_hip.py, test_arch_caps_hip.py on gfx950.
• pre-commit run on all changed files.
• Full suite on gfx950 — running.
• Full suite on gfx942 (MI300X) — running. This changes a build path shared by both architectures, so the CDNA3 result will be posted here before merge rather than inferred. Each run uses a SHA-pinned worktree and its own JIT cache directory; the shim cache is keyed on arch + AITER version only, so a shared cache between branches that disagree about arch resolution would poison exactly the thing under test.

Deliberately not in this PR

aot_hip still publishes the resolved list back into os.environ, a process-global side effect that outlives the call. It is load-bearing rather than incidental: the AITER shim resolves its own build architecture from FLASHINFER_ROCM_ARCH_LIST, and an AOT build has no other channel to tell it what this build targets. Removing it means threading an explicit parameter across the AOT → JIT boundary, which is wider than the resolution bug being fixed here. The value it publishes now comes from the same resolver, and the reasoning is recorded at the call site.

Separately noted while measuring, not addressed: import flashinfer requires a live GPU, not merely a ROCm-enabled torch — jit/env.py:212 calls torch.cuda.get_device_properties(torch.cuda.current_device()) at import time, which raises No HIP GPUs are available on a build host.

[Copilot]

Pull request overview

This PR centralizes ROCm target-architecture resolution into hip_utils.resolve_target_archs() so all build/validation paths (JIT validation, CompilationContext, and AOT packaging) agree on the same --offload-arch set—fixing a prior CDNA4 mismatch where validation could check gfx942 while compilation targeted gfx950.

Changes:

• Added resolve_target_archs() and routed validate_rocm_arch, validate_flashinfer_rocm_arch, CompilationContext, and aot_hip through it.
• Removed CompilationContext._auto_detect_archs and switched detection to the shared resolver.
• Updated/added tests to cover resolution branches and to assert validator/compile-context agreement.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
flashinfer/hip_utils.py	Introduces the unified architecture resolver and wires validators through it.
flashinfer/compilation_context_hip.py	Uses the unified resolver to avoid divergence between compilation and validation.
flashinfer/aot_hip.py	Resolves+publishes the target arch list once, then validates via CompilationContext.
tests/rocm_tests/test_hip_utils.py	Adds targeted tests for the new resolver and updates the prior “defaults to gfx942” assertion.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

flashinfer/aot_hip.py:231

• compile_and_package_modules() sets FLASHINFER_ROCM_ARCH_LIST before validating it via CompilationContext(). If CompilationContext() raises (e.g. unsupported ROCm version / arch), the process environment is left mutated to an invalid value, which can affect subsequent calls in the same process.

    rocm_arch_list = resolve_target_archs()
    os.environ["FLASHINFER_ROCM_ARCH_LIST"] = rocm_arch_list
    CompilationContext()  # validates the resolved list, raising on a bad one
    if verbose:
        print(f"Target ROCm architectures: {rocm_arch_list}")

[demandal25]

Suppressed comment in review 4984984345 — accepted, fixed in 4d1335d.

A CompilationContext() raise left FLASHINFER_ROCM_ARCH_LIST set for the rest of the process; now restored on failure. One correction: the leaked value was canonical, not invalid — a valid list that failed validation, which is why it was subtle.

My first regression test was vacuous (it seeded the variable with the value the resolver returns, so the buggy write was a no-op); rewritten and A/B-ed against the unfixed code.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

flashinfer/aot_hip.py:234

• The comment says the arch list is “resolved once, then validated”, but CompilationContext() re-resolves via resolve_target_archs() (with the env var still unset), so this can re-run detection (extra rocminfo subprocess + duplicate warning) and the validation isn’t guaranteed to be against the exact rocm_arch_list you just computed. Consider temporarily setting FLASHINFER_ROCM_ARCH_LIST to rocm_arch_list just for the validation call, and restoring it on failure; on success you can leave it set to the resolved value.

    # the inputs cannot change in between, so it validates exactly the list
    # published below -- the order costs nothing and stops a raise here from
    # leaving FLASHINFER_ROCM_ARCH_LIST set for whatever runs next in-process.
    rocm_arch_list = resolve_target_archs()
    CompilationContext()  # validates the resolved list, raising on a bad one

[demandal25]

Suppressed comment in review 4985096141 — accepted, fixed in 6e8773c. This was a regression I introduced in 4d1335d, not pre-existing.

Both halves verified: rocminfo_gpu_agents() is not cached (only the two get_*_device_indices helpers are), and two resolves on a GPU-less host emit 2 warnings. Now publishes before validating and restores on failure, so detection runs once. The test pins both properties.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

flashinfer/aot_hip.py:247

• compile_and_package_modules() publishes the resolved arch list to FLASHINFER_ROCM_ARCH_LIST, but CompilationContext() may silently filter that list (unsupported by ROCm version / unsupported by FlashInfer) without raising. If that happens, the env var can still contain unsupported entries (and even keep them first), which can cause the AITER shim (jit/aiter_source.py) to build for a different/invalid architecture than the kernels being packaged. Update the env var to the validated set after CompilationContext succeeds so the process-global side effect reflects what will actually be compiled.

    rocm_arch_list = resolve_target_archs()
    previous_arch_list = os.environ.get("FLASHINFER_ROCM_ARCH_LIST")
    os.environ["FLASHINFER_ROCM_ARCH_LIST"] = rocm_arch_list
    try:
        CompilationContext()  # validates the resolved list, raising on a bad one
    except BaseException:
        if previous_arch_list is None:
            os.environ.pop("FLASHINFER_ROCM_ARCH_LIST", None)
        else:
            os.environ["FLASHINFER_ROCM_ARCH_LIST"] = previous_arch_list
        raise

tests/rocm_tests/test_aot_hip.py:198

• This test creates a temporary build_dir via tempfile.mkdtemp() but never cleans it up, which can leak disk space across repeated test runs. Mirror test_compile_and_package_minimal() by deleting the directory in a finally block.

    with pytest.raises(RuntimeError, match="not recognized"):
        aot_hip.compile_and_package_modules(
            out_dir=None,
            build_dir=Path(tempfile.mkdtemp()),
            project_root=Path(__file__).parent.parent,
            config={
                "fa2_head_dim": [(128, 128)],
                "f16_dtype": [torch.float16],
                "use_sliding_window": [False],
                "use_logits_soft_cap": [False],
            },
            verbose=False,
            skip_prebuilt=True,
        )

[demandal25]

Suppressed comments in review 4985222187 — both accepted, fixed in 7bdc2c9. The first is the best catch on this PR.

• Validation filters rather than raises (gfx900,gfx942 → gfx942, warning only), so the variable could advertise an architecture the packaged kernels were never built for — and the AITER shim resolves its own target from it. Now republishes the validated list, ordered via arch_flags since TARGET_ROCM_ARCHS is a set.
• Leaked mkdtemp() → tmp_path fixture.

Both new tests were A/B-ed against the unfixed code.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Suppressed comments (4)

flashinfer/hip_utils.py:98

• This set comprehension includes every physical supported agent reported by rocminfo, even after the process has been restricted with HIP_VISIBLE_DEVICES. The repository documents that rocminfo ignores that variable (tests/rocm_tests/conftest.py:97-102); on a mixed gfx942/gfx950 host with HIP_VISIBLE_DEVICES selecting gfx950 and a gfx950-only PyTorch wheel, this resolves to both architectures and validate_flashinfer_rocm_arch then fails because PyTorch lacks gfx942. The resolver needs to honor the visible/current device when one is selected, or explicitly require an override for this case.

    detected = sorted(
        {
            arch
            for arch, _ in rocminfo_gpu_agents()
            if arch in FLASHINFER_SUPPORTED_ROCM_ARCHS
        }
    )

flashinfer/hip_utils.py:107

• When FLASHINFER_ROCM_ARCH_LIST is set to a non-empty value that normalizes away (for example ";;" or whitespace), this warning says the variable is "unset" even though it was supplied. That misdirects the operator toward setting a variable that is already present; report that it is unset or contains no usable architecture instead.

    logger.warning(
        "No supported AMD GPU detected and FLASHINFER_ROCM_ARCH_LIST is unset; "
        "building for every supported architecture (%s). This is slower than "
        "targeting one. Set FLASHINFER_ROCM_ARCH_LIST to the architecture you "
        "are building for.",

tests/rocm_tests/test_hip_utils.py:345

• This new test covers validate_rocm_arch, but the existing TestValidateFlashinferRocmArch.test_defaults_to_gfx942_when_no_env_no_arg still mocks validate_rocm_arch and asserts the old hard-coded gfx942. A regression in validate_flashinfer_rocm_arch(arch_list=None) resolving the wrong target would therefore still pass. Update that wrapper test to stub rocminfo_gpu_agents and assert the detected target instead.

    def test_falls_back_to_the_running_device_not_a_hard_coded_arch(self, monkeypatch):
        """With no argument and no env var, follow the hardware.

        This used to assert ``== "gfx942"``, encoding the literal that made
        ``validate_flashinfer_rocm_arch(arch_list=None)`` answer ``gfx942`` on a
        gfx950 device while CompilationContext compiled for gfx950. A test that
        pins a wrong constant is how the constant survives, so it is now pinned
        to the detected architecture instead.

flashinfer/hip_utils.py:98

• When no environment override is present, this path starts a new rocminfo subprocess on every resolve_target_archs() call. The removed _auto_detect_archs() used the cached get_supported_device_indices(), but JIT setup constructs CompilationContext more than once and gen_jit_spec() revalidates for each operation, so a normal multi-op process now repeatedly pays this probe (including its 10-second timeout) and can emit the fallback warning repeatedly. Cache the hardware probe for the process or reuse the already-resolved list while preserving invalidation when visibility changes.

    detected = sorted(
        {
            arch
            for arch, _ in rocminfo_gpu_agents()
            if arch in FLASHINFER_SUPPORTED_ROCM_ARCHS
        }
    )

[demandal25]

Review 4985367370 — 4 suppressed comments; 3 fixed in b2f586d, 1 deferred.

• rocminfo_gpu_agents() now cached. gen_jit_spec validates per module, so this was one subprocess (10 s timeout) per operation — a regression this PR introduced by dropping _auto_detect_archs(), which went through the already-cached get_supported_device_indices().
• Fallback warning no longer says "unset" when the variable was set but normalized away.
• test_defaults_to_gfx942_when_no_env_no_arg stubbed the resolved list out of its own assertion; it now echoes the argument and checks the detected arch.
• Deferred: rocminfo ignoring HIP_VISIBLE_DEVICES. Real, but honouring visibility is a behaviour change to detection, not to this PR's "resolve in one place". Same bucket as the two threads above.

164 passed; pre-commit clean.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (8)

flashinfer/aot_hip.py:235

• The no-environment behavior changed here, but amd-flashinfer-jit-cache/README.md:35-40 still says that AOT always compiles gfx942 and that the environment default is gfx942. A gfx950 host now follows detection, while a GPU-less host targets every supported architecture, so the package build instructions are inaccurate. Update that documentation (and the stale JIT default comment) with the centralized resolver behavior.

    rocm_arch_list = resolve_target_archs()

flashinfer/aot_hip.py:260

• context is used only to validate and derive the environment value; the actual HIP flags later come from the module-global jit.core.current_compilation_context (jit/core.py:406), which was initialized when flashinfer.jit was first imported. If this API is called after changing FLASHINFER_ROCM_ARCH_LIST (or called twice with different targets), context can validate/republish gfx950 while the kernels still compile with stale gfx942 flags; AITER then follows the republished value and no longer matches the packaged kernels. The active JIT context must be rebuilt/updated for this target before generating specs, or target changes must be rejected.

    rocm_arch_list = ",".join(
        flag.removeprefix("--offload-arch=") for flag in context.arch_flags
    )
    os.environ["FLASHINFER_ROCM_ARCH_LIST"] = rocm_arch_list

flashinfer/hip_utils.py:98

• The deleted _auto_detect_archs() caught all probe failures and fell back, but this new direct call is not protected. rocminfo_gpu_agents() only converts FileNotFoundError and timeouts to an empty result, so an unavailable executable that raises another OSError (for example PermissionError) now aborts resolution instead of reaching the documented GPU-less fallback. Catch the probe's OSError here or broaden the helper's failure handling.

    detected = sorted(
        {
            arch
            for arch, _ in rocminfo_gpu_agents()
            if arch in FLASHINFER_SUPPORTED_ROCM_ARCHS
        }
    )

flashinfer/hip_utils.py:64

• The no-argument behavior documented by this new resolver is now detection/env dependent (and can fall back to both supported architectures), but flashinfer/jit/core.py:148 still says arch_list=None “defaults to gfx942”. That comment is on a call site now routed through this resolver, so it should be updated in this change to avoid documenting the behavior this PR removes.

    Step 4 replaces a hard-coded ``"gfx942"`` that three call sites reached
    independently. On a CDNA4 host that literal was not a conservative default
    but a wrong answer: ``validate_flashinfer_rocm_arch(arch_list=None)``
    returned ``{"gfx942"}`` on a gfx950 device while ``CompilationContext``
    compiled for gfx950, so the check that exists to catch "your PyTorch was not
    built for this architecture" was validating an architecture nobody was
    building for. Vacuous on a PyTorch carrying both; a spurious hard failure on
    an arch-specific build that carries only gfx950.

tests/rocm_tests/test_aot_hip.py:213

• This new test hard-codes gfx950 but runs CompilationContext against the real PyTorch extension flags. On a supported gfx942 host with an arch-specific PyTorch build, validation filters gfx900 and then raises because PyTorch does not advertise gfx950, so the test fails before checking the AOT environment behavior. Stub _get_rocm_arch_flags (as the resolver-agreement test does) or choose the host's target so this test is hermetic.

    monkeypatch.setenv("FLASHINFER_ROCM_ARCH_LIST", "gfx900,gfx950")

flashinfer/aot_hip.py:231

• The explanation here is now stale: rocminfo_gpu_agents() is decorated with functools.cache, and publishing the environment before constructing CompilationContext means its resolver takes the just-published value rather than performing a second detection pass. Please update the comment so it does not claim an uncached probe or a second warning.

    # Publish first so CompilationContext() -- which resolves through
    # resolve_target_archs() itself -- validates exactly this list rather than
    # repeating the work. rocminfo_gpu_agents() is not cached, so simply
    # validating before publishing would re-run detection and, on a GPU-less
    # host, emit the "no supported AMD GPU detected" warning a second time.

flashinfer/hip_utils.py:505

• The new @functools.cache makes this probe process-wide, but the docstring below still says Not cached: the caller decides and describes callers paying for a fresh subprocess. That is now false and can lead callers to reason incorrectly about cache invalidation; update the stale sentence to document the shared cache and how it is cleared.

    Cached for the process. ``gen_jit_spec`` calls ``check_rocm_arch()`` for
    every operation it builds, and that reaches here through
    ``resolve_target_archs()``, so without this a multi-op process pays one
    ``rocminfo`` subprocess -- and its 10-second timeout -- per module, and a
    GPU-less host repeats the fallback warning each time. The prior

tests/rocm_tests/test_aot_hip.py:196

• This rationale is stale after rocminfo_gpu_agents became cached in the same change: CompilationContext will not perform a second rocminfo subprocess merely because it resolves again. Keep the assertion, but describe the actual invariant—that validation must receive the exact resolved list rather than re-reading an unset environment variable—so the test does not document behavior that no longer exists.

    # Validation must see the resolved list, not an unset variable. Otherwise
    # CompilationContext re-resolves from scratch -- rocminfo_gpu_agents() is
    # not cached, so that is a second detection pass and, on a GPU-less host, a
    # second "no supported AMD GPU detected" warning.

[demandal25]

Review 4985855053 — 8 suppressed; 7 fixed in b3fd023, 1 deferred.

Four were defects the previous commit introduced. Adding @functools.cache to rocminfo_gpu_agents falsified three of my own comments (the leftover "Not cached: the caller decides" paragraph, and the "avoids a second detection pass" rationale in both aot_hip and its test) — all rewritten to the reason that survives caching. The fourth: my new test_environment_gets_the_validated_list_not_the_resolved_one repeated the exact hermeticity bug flagged earlier on a different test, hard-coding gfx950 against the real PyTorch. It now stubs _get_rocm_arch_flags; verified with a plugin simulating a gfx942-only wheel that it and the other two arch-sensitive tests pass.

Three were pre-existing and surfaced because this PR changes what they describe: jit/core.py's "defaults to gfx942" comment, amd-flashinfer-jit-cache/README.md, and rocminfo_gpu_agents catching only FileNotFoundError/TimeoutExpired so an unexecutable rocminfo (PermissionError) aborted resolution instead of reaching the GPU-less fallback.

Deferred: the module-global JIT context (aot_hip.py:260) — same finding as the thread above, unchanged in scope.

164 passed; pre-commit clean.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

flashinfer/hip_utils.py:76

• The visibility resolver only reads HIP_VISIBLE_DEVICES and CUDA_VISIBLE_DEVICES, but ROCr also applies ROCR_VISIBLE_DEVICES to HIP processes; this repository records that variable alongside the other GPU selectors in rocm_profiler.py:594-596. With only ROCR_VISIBLE_DEVICES set on a mixed-architecture host, this code returns every physical agent and can compile for GPUs hidden from the process. Include the ROCr selector with a defined precedence and the same index/UUID handling.

    raw = os.environ.get("HIP_VISIBLE_DEVICES")
    if raw is None:
        raw = os.environ.get("CUDA_VISIBLE_DEVICES")
    if raw is None:
        return agents

flashinfer/hip_utils.py:533

• The new partial-PyTorch filtering branch is still untested. The AOT test uses gfx900,gfx950 on ROCm 7.1, so step 1 removes gfx900 before step 3, and the fake PyTorch flags include the remaining gfx950; missing_in_pytorch is therefore empty. Use two FlashInfer-supported targets (for example gfx942,gfx950) with _get_rocm_arch_flags() returning only gfx950, then assert the resulting gfx950 list so this branch cannot regress unnoticed.

        if missing_in_pytorch:
            # Filter, then raise only if nothing is left -- matching steps 1 and
            # 2, which already drop what they cannot serve and continue. Step 3
            # raising outright made the GPU-less fallback unusable: it returns
            # every supported architecture, so an arch-specific PyTorch wheel

flashinfer/hip_utils.py:80

• -1 is the CUDA/HIP no-device sentinel, but it fails isdigit() here and therefore takes the unmappable-device branch that returns all agents. A build launched with HIP_VISIBLE_DEVICES=-1 or CUDA_VISIBLE_DEVICES=-1 will consequently target hidden GPUs instead of taking the no-device fallback. Handle this sentinel as an empty visible set.

    tokens = [t.strip() for t in raw.split(",") if t.strip()]

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (3)

flashinfer/hip_utils.py:75

• ROCR_VISIBLE_DEVICES is another ROCr visibility control used by HIP applications. If a launcher sets it (for example, to select only device 1) without setting HIP_VISIBLE_DEVICES or CUDA_VISIBLE_DEVICES, this path treats visibility as unset and returns every rocminfo agent, so a mixed gfx942/gfx950 host can resolve an architecture that the process cannot access. Include ROCR_VISIBLE_DEVICES in the precedence chain before the CUDA fallback, preserving an explicitly empty value.

    raw = os.environ.get("HIP_VISIBLE_DEVICES")
    if raw is None:
        raw = os.environ.get("CUDA_VISIBLE_DEVICES")
    if raw is None:

flashinfer/hip_utils.py:81

• -1 is the conventional HIP/CUDA sentinel for hiding all GPUs, but it is non-numeric here, so the not all(...isdigit()) branch returns every physical agent. A GPU-less build launched with HIP_VISIBLE_DEVICES=-1 therefore skips the no-device fallback and can target hardware that the runtime exposes as unavailable. Treat the exact sentinel as an empty visibility selection and cover it with a regression test.

    tokens = [t.strip() for t in raw.split(",") if t.strip()]
    if not all(t.isdigit() for t in tokens):

flashinfer/hip_utils.py:537

• When visible_gpu_agents() cannot map a UUID, it intentionally returns every physical architecture. This new filtering can then choose the wrong one: a process pinned to a gfx942 UUID on a mixed gfx942/gfx950 host with a gfx950-only PyTorch reports gfx950 as usable and silently builds for a GPU the process cannot see. Preserve the ambiguous-visibility state and fail (or require a fat PyTorch build) instead of applying this partial-support fallback to it.

            usable = [f for f in arch_flags if f in pytorch_arch_flags]

[demandal25]

Review 4987723853 — 3 suppressed, all already fixed in e512b84 (which landed after that review ran).

ROCR_VISIBLE_DEVICES and the -1 sentinel are handled directly. The third — UUID visibility returning every agent, then the PyTorch filter picking the wrong one — is closed by the provenance change: that path resolves as detected, not fallback, and only fallback may be narrowed, so it now raises a clear "PyTorch does not support gfx942" instead of silently building the wrong half.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

flashinfer/hip_utils.py:100

• The or chain does not actually give HIP visibility precedence when HIP_VISIBLE_DEVICES is set to an empty/whitespace value (or -1): it falls through to CUDA_VISIBLE_DEVICES. For example, HIP="", CUDA="0" resolves GPU 0 instead of no devices, so the resolver can target hardware hidden from the HIP process. Select the variable by key presence so an explicitly empty HIP value remains authoritative.

    raw = (
        present.get("HIP_VISIBLE_DEVICES")
        or present.get("ROCR_VISIBLE_DEVICES")
        or present.get("CUDA_VISIBLE_DEVICES")
        or ""

flashinfer/hip_utils.py:94

• The ROCR_VISIBLE_DEVICES composition early return runs before the no-device handling below. Consequently, HIP_VISIBLE_DEVICES=-1 or an explicitly empty HIP value combined with any ROCR value returns all physical agents, despite the documented no-device semantics; the resolver then treats the result as detected rather than as the GPU-less fallback and can reject an arch-specific PyTorch wheel for an invisible architecture. Handle deterministic no-device sentinels before declining to scope the composition.

    if "ROCR_VISIBLE_DEVICES" in present and len(present) > 1:
        logger.debug(
            "Not scoping architecture detection: ROCR_VISIBLE_DEVICES composes "
            "with %s and the result cannot be mapped to rocminfo's enumeration "
            "order.",
            ", ".join(n for n in present if n != "ROCR_VISIBLE_DEVICES"),
        )
        return agents

[demandal25]

Review 4987846618 — 2 suppressed, both fixed in a421230. Both were mine, and both from using or over values that are legitimately falsy: HIP_VISIBLE_DEVICES="" fell through to CUDA_VISIBLE_DEVICES and resolved GPUs the operator had hidden, and HIP_VISIBLE_DEVICES=-1 combined with any ROCR value hit the composition early-return and got every agent. Selection is now by key in strict precedence order, and "no devices" is decided before the composition check.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (2)

flashinfer/hip_utils.py:110

• The no-device check only examines the precedence-selected raw value. If a composing layer is the no-device sentinel—for example HIP_VISIBLE_DEVICES=0 with ROCR_VISIBLE_DEVICES=-1 or an empty ROCR value—this branch returns all physical agents instead of (), despite the documented empty/-1 semantics. On a mixed-architecture host that can select invisible architectures and then fail PyTorch validation; handle no-device sentinels in the composing visibility variables before treating the composition as unknowable.

    if "ROCR_VISIBLE_DEVICES" in present and len(present) > 1:
        logger.debug(
            "Not scoping architecture detection: ROCR_VISIBLE_DEVICES composes "
            "with %s and the result cannot be mapped to rocminfo's enumeration "
            "order.",

flashinfer/hip_utils.py:99

• A malformed signed token such as HIP_VISIBLE_DEVICES=--1 reaches this predicate: lstrip("+-").isdigit() is true, but int(t) is applied to the original --1 and raises ValueError. Visibility input should degrade to the existing unmappable-device fallback rather than abort architecture resolution; only convert tokens after validating a single optional sign (or catch the conversion).

    if not tokens or any(t.lstrip("+-").isdigit() and int(t) < 0 for t in tokens):

[demandal25]

Review 4987970293 — 2 suppressed, both fixed in the push above. Both mine.

HIP_VISIBLE_DEVICES=--1 actually crashed: lstrip("+-").isdigit() passes, then int() raises. Parsing is now total and unparseable input takes the unmappable path. And the no-device check only looked at the precedence-selected variable, so ROCR=-1 with HIP=0 returned every agent — it now checks ROCR too, since ROCr composes beneath HIP rather than being overridden.

5 tests, A/B-verified against a421230. One of them was vacuous on first write and only distinguishes the two behaviours with a single-agent list.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (2)

flashinfer/hip_utils.py:75

• rocminfo is an HSA/ROCr client, so unlike HIP_VISIBLE_DEVICES it is affected by ROCR_VISIBLE_DEVICES. With ROCR_VISIBLE_DEVICES=1, rocminfo_gpu_agents() can already return a one-element list for physical GPU 1, but this code then indexes that filtered list with 1, drops the only agent, and falls back to building for all architectures. Collect the rocminfo inventory with the ROCR visibility variable removed (or otherwise distinguish an already-filtered inventory) before applying the requested indices.

    agents = rocminfo_gpu_agents()
    # ROCR_VISIBLE_DEVICES is applied by ROCr *beneath* HIP, so when it is set
    # alongside one of the others the two compose and the composition cannot be
    # reconstructed from indices alone. Refuse to scope in that case rather than
    # apply the wrong one -- over-building is recoverable.

flashinfer/hip_utils.py:67

• Returning the full agent list for an unscopable UUID (and likewise a composed ROCR/HIP visibility setting) is not recoverable in the build path as written. resolve_target_archs_with_origin() labels this result as detected, and the PyTorch check in validate_flashinfer_rocm_arch() only narrows origin == "fallback"; on a mixed gfx942/gfx950 host with a UUID selecting gfx950 and a gfx950-only PyTorch wheel, this therefore resolves both architectures and raises for missing gfx942 instead of building the visible target. Either map the UUID/composition, or carry an explicit “unscoped detection” provenance through validation so the conservative result can be narrowed rather than hard-failing.

    from unset. A value naming devices by UUID (``GPU-...``) cannot be mapped to
    an enumeration index here, so the full list is returned rather than guessing
    -- over-building is recoverable, silently targeting the wrong card is not.

[demandal25]

Review 4988146121 — 2 suppressed, both fixed. The first was a conceptual error on my part, not a detail.

rocminfo is an HSA client, so ROCr has already filtered its output — re-applying ROCR_VISIBLE_DEVICES indices filtered twice, and the composition ambiguity I invented to handle it does not exist. Measured here: ROCR_VISIBLE_DEVICES=-1 rocminfo → 0 GPU agents, HIP_VISIBLE_DEVICES=-1 rocminfo → 1. HIP therefore composes with ROCr for free, and the special-casing is gone.

Unmappable (UUID) visibility now reports "nothing known to be visible" instead of the full list: returning everything labelled it detected, which the provenance rule forbids narrowing, so an arch-specific wheel became a hard failure. It now reaches the narrowable fallback and warns.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

flashinfer/aot_hip.py:249

• Validation success is not build success: gen_all_modules() and build_jit_specs() below can still raise after this assignment. In that case the new value remains in the process-global environment (and can replace a caller's pre-existing value with a filtered list), contradicting the guarantee in lines 233-234 and influencing later JIT/AITER builds. Restore the prior environment value on downstream failure, or narrow that guarantee to validation failures.

    rocm_arch_list = ",".join(
        flag.removeprefix("--offload-arch=") for flag in context.arch_flags
    )
    os.environ["FLASHINFER_ROCM_ARCH_LIST"] = rocm_arch_list

flashinfer/hip_utils.py:618

• Because this probe is now a zero-argument process-wide cache, its result no longer tracks ROCR_VISIBLE_DEVICES. visible_gpu_agents() deliberately leaves that variable to the rocminfo subprocess, so if the first probe runs before that variable is set or after it changes, later resolution can reuse GPUs that ROCr has hidden. Key the cache by the ROCr visibility value (and any other environment that affects rocminfo), or keep the raw probe uncached and cache only a visibility-aware result.

@functools.cache