Skip to content

docs(rocm): generate the per-architecture support matrix from the table - #288

Merged
demandal25 merged 3 commits into
amd-integrationfrom
docs-arch-support-matrix
Aug 20, 2026
Merged

docs(rocm): generate the per-architecture support matrix from the table#288
demandal25 merged 3 commits into
amd-integrationfrom
docs-arch-support-matrix

Conversation

@demandal25

@demandal25 demandal25 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

The README described architecture support as the literal string gfx942/gfx950, repeated at nine call sites. That phrasing cannot express either of the two things the capability table from #285 knows: that an op can be supported on one architecture and broken on the other, and that some rows are measured while others are only declared.

It was already wrong when I started. AITER causal batch prefill is miscompiled on gfx950 under ROCm 7.2.x — silently, wrong numbers rather than an error — and no reader of the README could have learned that from it.

This generates the matrix from flashinfer/arch_caps.py, which is the same data backend="auto" consults at runtime, so the docs cannot disagree with the routing.

What it renders

Four states rather than a checkmark:

meaning
✅ validated a suite was run on that board, and the evidence string is printed below the table
◻️ declared supported, but no per-op measurement has been recorded on that architecture
⚠️ broken on some toolchains with the half-open version window, the reason, and the override env var
❌ not available

The first two are the honest part. Every HIP row currently renders as declared, because those per-op measurements genuinely have not been recorded — the suites cover them, but no per-op HIP evidence was ever attributed. A hand-written table would have given all of them a ✅.

Drift protection

A pre-commit hook runs the generator with --check, so an arch_caps.py edit that does not regenerate the README fails before review rather than after someone trusts a stale row.

It rides the existing pre-commit job rather than needing a new workflow, and deliberately imports nothing from the flashinfer package — __init__.py raises on a CPU-only torch build, so the script loads arch_caps directly by path, the same technique the conformance suite uses. Stdlib only.

Test plan

  • --check exits 0 on a clean tree.
  • --check exits 1 with an actionable message when a row is perturbed (flipped rope/aiter on gfx950 to UNSUPPORTED), and 0 again once restored — so the hook is not vacuously passing.
  • Full pre-commit run green on all three files, including the new hook.
  • Rendered output reviewed: the batch_prefill/aiter gfx950 cell carries the ⚠️ and the footnote reproduces the ROCm [7.2, 7.3) window, the 97.6% figure, the backend='fa2' workaround, and the upstream link.

Columns are derived from whatever architectures the table declares, not hard-coded, so adding a third architecture adds a column without anyone remembering to.

The hook caught a real drift before I did

Not a contrived test. #286 merged while this branch was in flight, renaming the activation row to silu_and_mul so it matches the string the call sites actually pass. This branch was based on 00e93c2c, so its generated README still said activation — and CI failed on my own hook:

--- README.md (committed)
+++ README.md (generated)
-| `activation` | `aiter` | ✅ | ✅ |
+| `silu_and_mul` | `aiter` | ✅ | ✅ |
-| `activation` | `hip` | ◻️ | ◻️ |
+| `silu_and_mul` | `hip` | ◻️ | ◻️ |

That is exactly the failure mode this PR exists to prevent, and it happened on the PR that introduces the prevention. A hand-maintained table would have shipped the stale op name silently. Rebased onto 4ccc4b33 and regenerated.

The second commit adds the diff output above. The first version printed only "out of date", which told me nothing — I could not reproduce the failure on either local interpreter, because the cause was the base branch moving rather than anything about the environment.

Note for reviewers

One interaction worth knowing about, since it is the kind of thing that breaks a week later: markdownlint rewrites - list bullets to * in place, which would leave the generated block failing its own --check on the very next run — two hooks fighting each other over the same file. The generator emits markdownlint-clean output (no inline HTML in headers, * bullets) so they agree. That is why the column headers are plain gfx942 (CDNA3) rather than carrying the SKU list; the SKUs are already spelled out in the Supported GPUs line directly above.

This does not touch the existing per-kernel Feature Support Matrix, which documents routing conditions rather than architecture validation. The two answer different questions.

Copilot AI lite review requested due to automatic review settings August 20, 2026 13:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a generator that renders the ROCm per-architecture support matrix in README.md directly from flashinfer/arch_caps.py, and wires it into pre-commit so the docs cannot drift from the runtime routing/capability table.

Changes:

  • Add scripts/gen_arch_support_matrix.py to render/splice the generated matrix block (with --check for CI/pre-commit enforcement).
  • Update README.md to include the generated “Per-architecture support matrix” section (including known-bad footnotes and evidence lines).
  • Add a local pre-commit hook to fail if the README matrix is out of date with flashinfer/arch_caps.py.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
scripts/gen_arch_support_matrix.py New generator + --check mode to keep README’s matrix synchronized with arch_caps.py.
README.md Adds the generated support-matrix section and rendered output/footnotes.
.pre-commit-config.yaml Adds a local hook to enforce that the generated README block is current.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/gen_arch_support_matrix.py
Comment thread scripts/gen_arch_support_matrix.py Outdated
Copilot AI review requested due to automatic review settings August 20, 2026 13:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

scripts/gen_arch_support_matrix.py:210

  • The generated README block contains non-ASCII characters (emoji). Path.read_text() uses the platform default encoding, which can break on systems where UTF-8 is not the default (e.g., Windows). Read with an explicit UTF-8 encoding to make the hook deterministic across environments.
    current = README.read_text()
    updated = splice(current, render(_load_arch_caps()))

scripts/gen_arch_support_matrix.py:238

  • For the same reason as the UTF-8 read, write the updated README using an explicit UTF-8 encoding so emoji and other non-ASCII characters are preserved consistently across platforms.
    if current != updated:
        README.write_text(updated)
        print(f"updated {README.relative_to(REPO_ROOT)}")

README.md:130

  • The generated evidence line says gfx950 was measured on MI350X, but the earlier "Supported GPUs" line in the same section says gfx950 corresponds to MI355X. This looks like a doc inconsistency that could confuse readers; consider reconciling the SKU naming (either broaden the supported list or adjust the evidence source).
* `gfx942` — MI300X / rocm 7.2.0 / aiter 0.1.10 / torch 2.9.1 / 2026-08-19
* `gfx950` — MI350X / rocm 7.2.0 / aiter 0.1.10 / torch 2.9.1 / 2026-08-19

Comment thread scripts/gen_arch_support_matrix.py
demandal25 and others added 2 commits August 20, 2026 09:34
The README described architecture support as the literal string
"gfx942/gfx950", repeated at nine call sites. That phrasing cannot express
either of the two things the capability table added in #285 knows: that an op
can be supported on one architecture and broken on the other, and that some
rows are measured while others are only declared.

It was already wrong. AITER causal batch prefill is miscompiled on gfx950 under
ROCm 7.2.x -- silently, wrong numbers rather than an error -- and a reader of
the README could not have learned that from it.

Generate the matrix from flashinfer/arch_caps.py instead, which is the same
data backend="auto" consults at runtime, so the docs cannot disagree with the
routing. The generated block renders four states rather than a checkmark:
validated (evidence recorded), declared (supported, unmeasured), broken on some
toolchains (with the version window and the override), and unavailable. The
distinction between the first two is the honest part -- every HIP row currently
renders as declared, because the per-op measurements genuinely have not been
recorded.

A pre-commit hook runs the generator with --check, so an arch_caps.py edit that
does not regenerate the README fails before review. It rides the existing
pre-commit job rather than needing a new workflow, and imports nothing from the
flashinfer package -- __init__.py raises on a CPU-only torch build, so the
script loads arch_caps directly, the same way the conformance suite does.

One interaction worth knowing about: markdownlint rewrites "-" list bullets to
"*" in place, which would leave the generated block failing its own --check on
the very next run. The generator emits markdownlint-clean output (no inline
HTML in headers, "*" bullets) so the two hooks agree.

Verified: --check passes on a clean tree, exits 1 with an actionable message
when a row is perturbed, and passes again once restored. Full `pre-commit run`
green on all three files.

Co-Authored-By: Claude <noreply@anthropic.com>
"Out of date" alone leaves whoever hit this guessing, and the cause is not
always an arch_caps.py edit -- another formatter rewriting the generated block
looks identical from the outside. Needed right now to diagnose a CI-only
failure that does not reproduce on either local interpreter.

Co-Authored-By: Claude <noreply@anthropic.com>
@demandal25
demandal25 force-pushed the docs-arch-support-matrix branch from ccfedb5 to 35d8369 Compare August 20, 2026 13:40
Copilot AI review requested due to automatic review settings August 20, 2026 13:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (4)

scripts/gen_arch_support_matrix.py:113

  • entry.support.SUPPORTED accesses an Enum member via the Enum instance, which will raise AttributeError at runtime. Compare against the Enum class (or use type(entry.support)) so the generator can run.
def _cell(entry, footnote_ids: dict[int, int]) -> str:
    if entry is None or entry.support is not entry.support.SUPPORTED:
        return UNSUPPORTED

scripts/gen_arch_support_matrix.py:78

  • Loading arch_caps.py assumes spec_from_file_location() always returns a spec with a loader. If it returns None (or has loader=None), this will fail with a confusing exception. Add an explicit check and exit with a clear message.
    spec = importlib.util.spec_from_file_location(qualified, PKG_DIR / "arch_caps.py")
    module = importlib.util.module_from_spec(spec)
    sys.modules[qualified] = module
    spec.loader.exec_module(module)

scripts/gen_arch_support_matrix.py:238

  • Write README.md explicitly as UTF-8; otherwise this can fail to encode the generated Unicode status symbols on non-UTF-8 default locales.
    if current != updated:
        README.write_text(updated)
        print(f"updated {README.relative_to(REPO_ROOT)}")

scripts/gen_arch_support_matrix.py:210

  • This script generates Unicode symbols (✅/⚠️/◻️/❌). Using the platform default encoding can raise UnicodeEncodeError on systems where UTF-8 is not the default (notably some Windows setups). Read README.md explicitly as UTF-8.
    current = README.read_text()
    updated = splice(current, render(_load_arch_caps()))

Addresses the three inline comments and three suppressed comments on #288.

- Pin UTF-8 on both README reads and writes. The generated block is emoji, so
  leaving the encoding to the locale means a pre-commit run under LC_ALL=C
  raises UnicodeDecodeError on read and UnicodeEncodeError on write. Verified
  the --check path now passes under LC_ALL=C LANG=C PYTHONUTF8=0.
  (Covers the inline comment and both suppressed comments on the same issue.)

- Give _load_arch_caps a diagnosable failure. The review asked about
  spec.loader being None; the likelier failure is a moved arch_caps.py, and
  spec_from_file_location returns a perfectly good spec for a path that does
  not exist, so that case has to be checked separately rather than inferred
  from the spec. Both now exit with a message naming the file instead of a
  traceback that does not.

- Reconcile the gfx950 SKUs. The evidence line reports MI350X while the
  supported-GPU lines said MI355X. Both are gfx950/CDNA4 and the measurement
  is the fact, so the SKU lists were the incomplete half; MI350X added to
  both. The published-image table further down is left alone -- it records
  what that image was tested on, which is not mine to widen.

- Compare against the Support enum via type(entry.support) rather than
  entry.support.SUPPORTED.

  Note the review's stated reason for that last one is wrong: it claimed the
  member-from-member lookup "raises AttributeError and the generator will
  crash on the first supported entry". It does not. On CPython 3.12.3 the
  lookup resolves, emits no DeprecationWarning even under -W error, and
  `--check` exits 0 against the committed README -- which it could not do if
  the generator crashed on the first row. Adopted anyway on style grounds: the
  lookup has been on and off the deprecation list, and it reads as though
  SUPPORTED were an attribute of the value rather than a sibling member.
Copilot AI review requested due to automatic review settings August 20, 2026 13:51
@demandal25

demandal25 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator Author

Suppressed comments in review 4983274571 — all three accepted, fixed in e256f0d.

  • UTF-8 pinned on both the README read and write; verified --check exits 0 under LC_ALL=C LANG=C PYTHONUTF8=0.
  • gfx950 SKUs reconciled toward the measurement: MI350X and MI355X are both gfx950, so the SKU lists were the incomplete half rather than the evidence being wrong. The published-image table further down is left alone — it records what that image was tested on.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@demandal25
demandal25 merged commit f7eaeab into amd-integration Aug 20, 2026
3 checks passed
@demandal25
demandal25 deleted the docs-arch-support-matrix branch August 20, 2026 13:56
demandal25 added a commit that referenced this pull request Aug 20, 2026
## Summary

`CLAUDE.md` told contributors to build AITER from source master and said
nothing about a version. `README.md` says the opposite — *"Pin `0.1.10`:
it is the version this repo is built and tested against"* — and
`prefill_rocm.py` records the same value as `_AITER_LAST_VALIDATED`.
Following the project instructions therefore put you on an untested
build.

### What changed

- **`CLAUDE.md`** — the AITER install block now leads with the pinned
wheel, copied verbatim from the README; documents the source build as
legitimate-but-untested rather than deleting it; and spells out the
read-the-installed-tree rule for shim work. Also adds MI350X to the
arch/codename line.

### Why it matters

Master is many releases ahead of the pin **with a different C ABI** —
symbols the shim expects are renamed, hidden rather than `extern "C"`,
or absent outright. Concretely, between the installed `amd-aiter 0.1.10`
and a recent source checkout: `mla_reduce_v1` does not exist at all, the
MLA asm entry points are `torch::Tensor&` and C++-mangled rather than
`aiter_tensor_t*` and `extern "C"`, and `reshape_and_cache_flash`
changes signature.

This is not hypothetical. A C++ shim design was drafted against a source
checkout and turned out to be unbuildable on the installed wheel —
caught only when `nm -D` on the installed `.so` disagreed with the
checkout's header. That is why the two `nm -D` / header commands are now
in the file: they are what settles the question in seconds.

The wheel command is copied byte-for-byte from the README so the two
cannot drift, and links to the section explaining why it must be
`--extra-index-url` rather than `--index-url` (AITER's own dependencies
still need to resolve from PyPI).

### MI350X

gfx950 covers both MI350X and MI355X. `README.md` already says so as of
#288, and the `evidence` strings in the capability table were measured
on an MI350X — so the narrower `MI355X = gfx950` line read as a
contradiction against the repo's own recorded measurements.

## Test plan

Docs only — no Python touched, so there are no tests to run.

- [x] `pre-commit run --files CLAUDE.md` — clean (markdownlint
included).
- [x] The README anchor resolves: `### Install AITER wheel package`
exists at `README.md:428`.
- [x] The `pip install` line is byte-identical to `README.md:435`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants