From d03191ecbc52a542492e1b93cb9e4535949cb9ed Mon Sep 17 00:00:00 2001 From: "serhii.n" Date: Sat, 15 Aug 2026 19:05:19 +0300 Subject: [PATCH 1/3] Update ROCm to 7.14.0 for Docker builds 1. Updated ROCm to 7.14.0 with base image to Ubuntu 26.04. 2. Added list of new supported GPUs based on available packages in base container. 3. Added workaround that caused llama.cpp to show "no usable GPU found" error. Added new architectures: gfx950 (MI350) gfx1010 (RX5700XT, RX5700, RX5600XT, RX5600) gfx1011 (V540, V520, PRO5600M) gfx1012 (RX5500XT, RX5500, RX5500M, RX5300) gfx1031 (RX6750XT, RX6700XT, RX6700, RX6800M, RX6700M) gfx1032 (RX6650XT, RX6600XT, RX6600, RX6600M, RX6600S) gfx1033 (Steam Deck GPU, Van Gogh, Aerith) gfx1034 (RX6500XT, RX6400, RX6500M, RX6300M) gfx1035 (Radeon 680M, Radeon 660M, Rembrandt APU) gfx1036 (Radeon 710M, Radeon 610M, Raphael Desktop iGPU, Mendocino APU) gfx1103 (Radeon 780M, Radeon 760M, Radeon 740M, Phoenix/Hawk Point APU) gfx1151 (Ryzen AI Max 300 series iGPU, Strix Halo) gfx1152 (Ryzen AI 300 series iGPU, Krackan Point) gfx1153 (Medusa Point iGPU, upcoming Zen 6 APUs) Ignored new architectures because of error: gfx1250 (Instinct MI400 series, CDNA server AI accelerators) - error: instruction not supported on this GPU (gfx1250): v_dot2_f32_f16 Tested on RX7900XTX and RX5500M (newly added GFX1012) RX5500M tested with Qwen3.5-4B-Q4_K_S, cache-type-k=q8;v=q4_0, gen=30t/s, proc=250t/s PS: No need to use `HSA_OVERRIDE_GFX_VERSION=gfx` seems AMD provided packages with optimized ROCm code for each architecture (see `dpkg -l | grep amdrocm` inside base image) --- .devops/rocm.Dockerfile | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/.devops/rocm.Dockerfile b/.devops/rocm.Dockerfile index 20f6ad636080..6153e14b431d 100644 --- a/.devops/rocm.Dockerfile +++ b/.devops/rocm.Dockerfile @@ -1,11 +1,11 @@ -ARG UBUNTU_VERSION=24.04 +ARG UBUNTU_VERSION=26.04 # This needs to generally match the container host's environment. -ARG ROCM_VERSION=7.2.1 -ARG AMDGPU_VERSION=7.2.1 +ARG ROCM_VERSION=7.14.0 +ARG AMDGPU_VERSION=7.14.0 # Target the ROCm build image -ARG BASE_ROCM_DEV_CONTAINER=docker.io/rocm/dev-ubuntu-${UBUNTU_VERSION}:${ROCM_VERSION}-complete +ARG BASE_ROCM_DEV_CONTAINER=docker.io/rocm/dev-ubuntu-${UBUNTU_VERSION}:${ROCM_VERSION}-full ARG BUILD_DATE=N/A ARG APP_VERSION=N/A @@ -30,11 +30,11 @@ FROM ${BASE_ROCM_DEV_CONTAINER} AS build # Unless otherwise specified, we make a fat build. # This is mostly tied to rocBLAS supported archs. -# check https://rocm.docs.amd.com/projects/install-on-linux/en/docs-7.2.1/reference/system-requirements.html -# check https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/compatibility/compatibilityrad/native_linux/native_linux_compatibility.html -# check https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/compatibility/compatibilityryz/native_linux/native_linux_compatibility.html +# check https://rocm.docs.amd.com/en/docs-7.14.0/install/rocm.html +# check https://rocm.docs.amd.com/en/docs-7.14.0/reference/gpu-specs.html -ARG ROCM_DOCKER_ARCH='gfx908;gfx90a;gfx942;gfx1030;gfx1100;gfx1101;gfx1102;gfx1151;gfx1150;gfx1200;gfx1201' +# Here used all architectures that have amdrocm-blas7.14-gfx*** related package inside container +ARG ROCM_DOCKER_ARCH='gfx908;gfx90a;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201' # Set ROCm architectures ENV AMDGPU_TARGETS=${ROCM_DOCKER_ARCH} @@ -90,8 +90,14 @@ LABEL org.opencontainers.image.created=$BUILD_DATE \ org.opencontainers.image.url=$IMAGE_URL \ org.opencontainers.image.source=$IMAGE_SOURCE +# ROCm's amdrocm-* packaging installs the libraries under /opt/rocm/core-/lib +# behind /etc/alternatives and no longer registers them with the dynamic loader. +# Without this, libggml-hip.so cannot resolve libamdhip64/libhipblas/librocblas +# at runtime and ggml silently skips the HIP backend -> "no usable GPU found". RUN apt-get update \ && apt-get install -y libgomp1 curl ffmpeg \ + && printf '/opt/rocm/lib\n/opt/rocm/lib/llvm/lib\n' > /etc/ld.so.conf.d/rocm.conf \ + && ldconfig \ && apt autoremove -y \ && apt clean -y \ && rm -rf /tmp/* /var/tmp/* \ From 30255fcb4dcfa90440528b123ec4a0459e5e3482 Mon Sep 17 00:00:00 2001 From: "serhii.n" Date: Sun, 16 Aug 2026 21:06:54 +0300 Subject: [PATCH 2/3] Update ROCm documentation 1. Added list of supported GPU by Docker image 2. Added Docker requirements for ROCm 3. Explained (non)official GPUs support by AMD --- docs/build.md | 4 ++ docs/docker.md | 103 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 107 insertions(+) diff --git a/docs/build.md b/docs/build.md index ca086a0be145..7355b42e7cd1 100644 --- a/docs/build.md +++ b/docs/build.md @@ -389,6 +389,10 @@ You can download it from your Linux distro's package manager or from here: [ROCm The environment variable [`HIP_VISIBLE_DEVICES`](https://rocm.docs.amd.com/en/latest/understand/gpu_isolation.html#hip-visible-devices) can be used to specify which GPU(s) will be used. If your GPU is not officially supported you can use the environment variable [`HSA_OVERRIDE_GFX_VERSION`] set to a similar GPU, for example 10.3.0 on RDNA2 (e.g. gfx1030, gfx1031, or gfx1035) or 11.0.0 on RDNA3. Note that [`HSA_OVERRIDE_GFX_VERSION`] is [not supported on Windows](https://github.com/ROCm/ROCm/issues/2654) +As of ROCm 7.14.0, AMD ships a separate optimized library package for each supported architecture, covering the CDNA targets and everything from RDNA 1 (`gfx1010`) through RDNA 4 (`gfx1201`). For those architectures `HSA_OVERRIDE_GFX_VERSION` is no longer needed and should be left unset — build for your real target instead, since spoofing a different architecture now only costs performance. In particular the whole RDNA 2 generation is covered, not just `gfx1030`, so `gfx1031` through `gfx1036` no longer need the `10.3.0` override shown above. Note that this is wider than AMD's official support matrix: RDNA 1 is not officially supported, and of RDNA 2 only `gfx1030` is — the rest work, but are not tuned to the same degree as RDNA 3 and newer. The full list is in [Docker With ROCm](docker.md#docker-with-rocm), which also applies to local builds. + +Note that Docker image only covers user space: the `amdgpu` kernel module and the GPU firmware come from the host and have to be recent enough for the ROCm release you build against, whether that is AMD's `amdgpu-dkms` package or an upstream kernel with a current `linux-firmware`. + ### Unified Memory On Linux it is possible to use unified memory architecture (UMA) to share main memory between the CPU and integrated GPU by setting environment variable `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1`. However, this hurts performance for non-integrated GPUs (but enables working with integrated GPUs). diff --git a/docs/docker.md b/docs/docker.md index b1c6c1f6f9f8..32a1a0eaa01b 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -107,6 +107,109 @@ docker run --gpus all -v /path/to/models:/models local/llama.cpp:light-cuda -m / docker run --gpus all -v /path/to/models:/models local/llama.cpp:server-cuda -m /models/7B/ggml-model-q4_0.gguf --port 8080 --host 0.0.0.0 -n 512 --n-gpu-layers 1 ``` +## Docker With ROCm + +The ROCm images are built from `rocm/dev-ubuntu-26.04:7.14.0-full` and contain the ROCm 7.14.0 user-space runtime. + +The `amdgpu` kernel module and the GPU firmware are **not** part of the image — they always come from the host, and they have to be recent enough for the ROCm version running inside the container. Either: + +- install AMD's packages on the host (`amdgpu-install --usecase=dkms`, or the `amdgpu-dkms` package), matching the same ROCm release, or +- if you would rather not use AMD's repositories, run an upstream kernel new enough to support your GPU together with a current [`linux-firmware`](https://gitlab.com/kernel-firmware/linux-firmware). + +A host driver or firmware that is too old typically shows up as the GPU not being listed at all, rather than as an explicit error. Check the host side before involving llama.cpp: + +```bash +docker run --rm \ + --device /dev/kfd --device /dev/dri \ + --group-add $(getent group video | cut -d: -f3) \ + --group-add $(getent group render | cut -d: -f3) \ + --entrypoint rocminfo ghcr.io/ggml-org/llama.cpp:server-rocm +``` + +Note that the group IDs have to be passed numerically: the images do not define `video` and `render` groups, so `--group-add render` fails to resolve. + +### Supported GPU architectures + +The images are a fat build covering every architecture for which ROCm 7.14.0 ships an optimized library package, so the same image runs on all of the following without a rebuild. + +Parentheses group SKU variants that share a target, so `RX 6700(XT)` covers both the RX 6700 and the RX 6700 XT. + +| Family | Target | Products | +| --- | --- | --- | +| CDNA (server & data center accelerators) | `gfx908` | Instinct MI100 | +| | `gfx90a` | Instinct MI210, MI250(X) | +| | `gfx942` | Instinct MI300A, MI300X(-HF), MI308X, MI325X | +| | `gfx950` | Instinct MI350(X/P), MI355X | +| RDNA 1 (Navi 1X — consumer & workstation) | `gfx1010` | Radeon RX 5700(XT/M), RX 5600(XT/OEM/M), Pro W5700(X), Pro 5700(XT) | +| | `gfx1011` | Radeon Pro 5600M, Pro V520, Pro V540 | +| | `gfx1012` | Radeon RX 5500(XT/OEM/M), RX 5300(XT OEM/M), Pro W5500(X/M), Pro 5500(XT/M), Pro 5300(M) | +| RDNA 2 (Navi 2X — consumer, workstation & APUs) | `gfx1030` | Radeon RX 6950 XT, RX 6900 XT, RX 6800(XT), Pro W6800(X), Pro W6800X Duo, Pro W6900X, Pro V620 | +| | `gfx1031` | Radeon RX 6750(XT/GRE), RX 6700(XT/M), RX 6850M XT, RX 6800M, Pro W6700 | +| | `gfx1032` | Radeon RX 6650(XT/M/M XT), RX 6600(XT/M/S), RX 6800S, RX 6700S, Pro W6600(X/M) | +| | `gfx1033` | Steam Deck LCD/OLED APU — "Van Gogh" ("Aerith", "Sephiroth") | +| | `gfx1034` | Radeon RX 6500(XT/M), RX 6400, RX 6300(M), RX 6550M, RX 6450M, Pro W6400, Pro W6300(M), Pro W6500M | +| | `gfx1035` | Radeon 680M, 660M, 610M — Ryzen 6000 "Rembrandt"/"Rembrandt-R" mobile APUs | +| | `gfx1036` | Radeon Graphics 2CU — Ryzen 7000/9000 desktop, Threadripper 7000/9000, EPYC 4004/4005; Radeon 610M — "Dragon Range" mobile | +| RDNA 3 (Navi 3X — consumer, workstation & APUs) | `gfx1100` | Radeon RX 7900(XT/XTX/GRE/M), Pro W7900, Pro W7800 | +| | `gfx1101` | Radeon RX 7800 XT, RX 7700 XT, Pro W7700, Pro V710 | +| | `gfx1102` | Radeon RX 7600(XT/S/M/M XT), RX 7650 GRE, RX 7700S, RX 7550M, Pro W7600, Pro W7500 | +| | `gfx1103` | Radeon 780M, 760M, 740M, 610M — Ryzen 7040/8040 "Phoenix/Phoenix 2/Hawk Point" mobile APUs; Ryzen Z1(Extreme), Ryzen Z2 Go | +| RDNA 3.5 (Ryzen AI 300 / Max — APUs) | `gfx1150` | Radeon 890M, 880M — Ryzen AI 300 "Strix Point" APUs; Ryzen Z2 Extreme | +| | `gfx1151` | Radeon 8060S, 8050S — Ryzen AI Max "Strix Halo" APUs | +| | `gfx1152` | Radeon 860M, 820M — Ryzen AI "Krackan Point" APUs | +| | `gfx1153` | Radeon 840M — Ryzen AI "Krackan Point 2" APUs | +| RDNA 4 (Navi 4X — consumer & pro) | `gfx1200` | Radeon RX 9060(XT/M/S), RX 9050 | +| | `gfx1201` | Radeon RX 9070(XT/GRE/S/M XT), RX 9080M, Radeon AI PRO R9700 | + +Because ROCm 7.14.0 provides an optimized library package per architecture, `HSA_OVERRIDE_GFX_VERSION` should **not** be set for any of the targets above — spoofing a different architecture only costs performance now, and can produce incorrect results. It remains relevant only for architectures that are absent from this list. + +This is a wider set than AMD's official support matrix, which is worth keeping in mind: + +- RDNA 1 (`gfx1010`, `gfx1011`, `gfx1012`) is not officially supported at all. ROCm still ships packages for it and it does work in practice — an RX 5500M (`gfx1012`) handles Qwen 3.5 4B fine, for instance — but expect it to be validated rather than tuned. +- For RDNA 2, only `gfx1030` is officially supported. The notable change in 7.14.0 is that the rest of the generation (`gfx1031` through `gfx1036`) now has its own packages too, so those cards no longer need `HSA_OVERRIDE_GFX_VERSION=10.3.0` to masquerade as `gfx1030` — they run on their own target. + +Treat everything outside the official matrix as working but less optimized than RDNA 3 and newer, where AMD puts most of the tuning effort. + +## Building Docker locally + +```bash +docker build -t local/llama.cpp:full-rocm --target full -f .devops/rocm.Dockerfile . +docker build -t local/llama.cpp:light-rocm --target light -f .devops/rocm.Dockerfile . +docker build -t local/llama.cpp:server-rocm --target server -f .devops/rocm.Dockerfile . +``` + +Building for every architecture takes a long time and produces a large image. To build only for your own GPU, pass its target through `ROCM_DOCKER_ARCH`: + +```bash +docker build -t local/llama.cpp:server-rocm --target server \ + --build-arg ROCM_DOCKER_ARCH=gfx1100 -f .devops/rocm.Dockerfile . +``` + +Refer to [.devops/rocm.Dockerfile](../.devops/rocm.Dockerfile) for the remaining `ARGS` and their defaults. + +## Usage + +After building locally, usage is similar to the non-ROCm examples, but you'll need to expose the GPU devices to the container and use the `--n-gpu-layers` flag. + +```bash +docker run --device /dev/kfd --device /dev/dri \ + --group-add $(getent group video | cut -d: -f3) \ + --group-add $(getent group render | cut -d: -f3) \ + -v /path/to/models:/models local/llama.cpp:full-rocm --run -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 99 +docker run --device /dev/kfd --device /dev/dri \ + --group-add $(getent group video | cut -d: -f3) \ + --group-add $(getent group render | cut -d: -f3) \ + -v /path/to/models:/models local/llama.cpp:light-rocm -m /models/7B/ggml-model-q4_0.gguf -p "Building a website can be done in 10 simple steps:" -n 512 --n-gpu-layers 99 +docker run --device /dev/kfd --device /dev/dri \ + --group-add $(getent group video | cut -d: -f3) \ + --group-add $(getent group render | cut -d: -f3) \ + -p 8080:8080 -v /path/to/models:/models local/llama.cpp:server-rocm -m /models/7B/ggml-model-q4_0.gguf --port 8080 --host 0.0.0.0 -n 512 --n-gpu-layers 99 +``` + +*Notes:* +- `--device /dev/dri` exposes every render node on the host. To restrict the container to one GPU, pass the individual nodes instead, for example `--device /dev/dri/renderD128 --device /dev/dri/card0`. +- If llama.cpp reports `no usable GPU found` while `rocminfo` inside the same container does list the GPU, the HIP backend failed to load rather than the GPU being unavailable. `--list-devices` shows the loader error. + ## Docker With MUSA Assuming one has the [mt-container-toolkit](https://developer.mthreads.com/musa/native) properly installed on Linux, `muBLAS` should be accessible inside the container. From 53323a86d2d8dcbbb1683327ce1b60020d68e36b Mon Sep 17 00:00:00 2001 From: "serhii.n" Date: Sun, 16 Aug 2026 21:34:21 +0300 Subject: [PATCH 3/3] Restricted list of GPUs for ROCm inside CI because of time execution Now list of added GPUs: gfx950, gfx1010, gfx1011, gfx1012, gfx1031, gfx1032, gfx1034 Basically RDNA1/2 dGPUs fully covered + MI350 --- .devops/rocm.Dockerfile | 9 ++++- docs/build.md | 2 +- docs/docker.md | 81 +++++++++++++++++++++++------------------ 3 files changed, 54 insertions(+), 38 deletions(-) diff --git a/.devops/rocm.Dockerfile b/.devops/rocm.Dockerfile index 6153e14b431d..2786a4fd1bdb 100644 --- a/.devops/rocm.Dockerfile +++ b/.devops/rocm.Dockerfile @@ -33,8 +33,13 @@ FROM ${BASE_ROCM_DEV_CONTAINER} AS build # check https://rocm.docs.amd.com/en/docs-7.14.0/install/rocm.html # check https://rocm.docs.amd.com/en/docs-7.14.0/reference/gpu-specs.html -# Here used all architectures that have amdrocm-blas7.14-gfx*** related package inside container -ARG ROCM_DOCKER_ARCH='gfx908;gfx90a;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1033;gfx1034;gfx1035;gfx1036;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201' +# Discrete GPUs plus the fastest APUs, out of the architectures that have an +# amdrocm-blas7.14-gfx*** package inside the container. The remaining APU targets +# (gfx1033, gfx1035, gfx1036, gfx1103, gfx1152, gfx1153) are left out to keep the +# build within the CI time budget - they are several times slower than gfx1151 +# anyway. ROCm supports them, so build locally with e.g. +# --build-arg ROCM_DOCKER_ARCH=gfx1103 if you need one of them. +ARG ROCM_DOCKER_ARCH='gfx908;gfx90a;gfx942;gfx950;gfx1010;gfx1011;gfx1012;gfx1030;gfx1031;gfx1032;gfx1034;gfx1100;gfx1101;gfx1102;gfx1150;gfx1151;gfx1200;gfx1201' # Set ROCm architectures ENV AMDGPU_TARGETS=${ROCM_DOCKER_ARCH} diff --git a/docs/build.md b/docs/build.md index 7355b42e7cd1..5e3e885bba63 100644 --- a/docs/build.md +++ b/docs/build.md @@ -389,7 +389,7 @@ You can download it from your Linux distro's package manager or from here: [ROCm The environment variable [`HIP_VISIBLE_DEVICES`](https://rocm.docs.amd.com/en/latest/understand/gpu_isolation.html#hip-visible-devices) can be used to specify which GPU(s) will be used. If your GPU is not officially supported you can use the environment variable [`HSA_OVERRIDE_GFX_VERSION`] set to a similar GPU, for example 10.3.0 on RDNA2 (e.g. gfx1030, gfx1031, or gfx1035) or 11.0.0 on RDNA3. Note that [`HSA_OVERRIDE_GFX_VERSION`] is [not supported on Windows](https://github.com/ROCm/ROCm/issues/2654) -As of ROCm 7.14.0, AMD ships a separate optimized library package for each supported architecture, covering the CDNA targets and everything from RDNA 1 (`gfx1010`) through RDNA 4 (`gfx1201`). For those architectures `HSA_OVERRIDE_GFX_VERSION` is no longer needed and should be left unset — build for your real target instead, since spoofing a different architecture now only costs performance. In particular the whole RDNA 2 generation is covered, not just `gfx1030`, so `gfx1031` through `gfx1036` no longer need the `10.3.0` override shown above. Note that this is wider than AMD's official support matrix: RDNA 1 is not officially supported, and of RDNA 2 only `gfx1030` is — the rest work, but are not tuned to the same degree as RDNA 3 and newer. The full list is in [Docker With ROCm](docker.md#docker-with-rocm), which also applies to local builds. +As of ROCm 7.14.0, AMD ships a separate optimized library package for each supported architecture, covering the CDNA targets and everything from RDNA 1 (`gfx1010`) through RDNA 4 (`gfx1201`). For those architectures `HSA_OVERRIDE_GFX_VERSION` is no longer needed and should be left unset, as long as the binary was compiled for the target — build for your real target instead, since spoofing a different architecture now only costs performance. The override is still required to run a binary that has no code for your GPU, such as a prebuilt package covering a different set of architectures. In particular the whole RDNA 2 generation is covered, not just `gfx1030`, so `gfx1031` through `gfx1036` no longer need the `10.3.0` override shown above. Note that this is wider than AMD's official support matrix: RDNA 1 is not officially supported, and of RDNA 2 only `gfx1030` is — the rest work, but are not tuned to the same degree as RDNA 3 and newer. The full list is in [Docker With ROCm](docker.md#docker-with-rocm), which also applies to local builds — note that the prebuilt Docker images cover only part of it (discrete GPUs plus Strix Halo), while a local build such as the one above can target any of these architectures. Note that Docker image only covers user space: the `amdgpu` kernel module and the GPU firmware come from the host and have to be recent enough for the ROCm release you build against, whether that is AMD's `amdgpu-dkms` package or an upstream kernel with a current `linux-firmware`. diff --git a/docs/docker.md b/docs/docker.md index 32a1a0eaa01b..a9fecc84add3 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -130,43 +130,49 @@ Note that the group IDs have to be passed numerically: the images do not define ### Supported GPU architectures -The images are a fat build covering every architecture for which ROCm 7.14.0 ships an optimized library package, so the same image runs on all of the following without a rebuild. - -Parentheses group SKU variants that share a target, so `RX 6700(XT)` covers both the RX 6700 and the RX 6700 XT. - -| Family | Target | Products | -| --- | --- | --- | -| CDNA (server & data center accelerators) | `gfx908` | Instinct MI100 | -| | `gfx90a` | Instinct MI210, MI250(X) | -| | `gfx942` | Instinct MI300A, MI300X(-HF), MI308X, MI325X | -| | `gfx950` | Instinct MI350(X/P), MI355X | -| RDNA 1 (Navi 1X — consumer & workstation) | `gfx1010` | Radeon RX 5700(XT/M), RX 5600(XT/OEM/M), Pro W5700(X), Pro 5700(XT) | -| | `gfx1011` | Radeon Pro 5600M, Pro V520, Pro V540 | -| | `gfx1012` | Radeon RX 5500(XT/OEM/M), RX 5300(XT OEM/M), Pro W5500(X/M), Pro 5500(XT/M), Pro 5300(M) | -| RDNA 2 (Navi 2X — consumer, workstation & APUs) | `gfx1030` | Radeon RX 6950 XT, RX 6900 XT, RX 6800(XT), Pro W6800(X), Pro W6800X Duo, Pro W6900X, Pro V620 | -| | `gfx1031` | Radeon RX 6750(XT/GRE), RX 6700(XT/M), RX 6850M XT, RX 6800M, Pro W6700 | -| | `gfx1032` | Radeon RX 6650(XT/M/M XT), RX 6600(XT/M/S), RX 6800S, RX 6700S, Pro W6600(X/M) | -| | `gfx1033` | Steam Deck LCD/OLED APU — "Van Gogh" ("Aerith", "Sephiroth") | -| | `gfx1034` | Radeon RX 6500(XT/M), RX 6400, RX 6300(M), RX 6550M, RX 6450M, Pro W6400, Pro W6300(M), Pro W6500M | -| | `gfx1035` | Radeon 680M, 660M, 610M — Ryzen 6000 "Rembrandt"/"Rembrandt-R" mobile APUs | -| | `gfx1036` | Radeon Graphics 2CU — Ryzen 7000/9000 desktop, Threadripper 7000/9000, EPYC 4004/4005; Radeon 610M — "Dragon Range" mobile | -| RDNA 3 (Navi 3X — consumer, workstation & APUs) | `gfx1100` | Radeon RX 7900(XT/XTX/GRE/M), Pro W7900, Pro W7800 | -| | `gfx1101` | Radeon RX 7800 XT, RX 7700 XT, Pro W7700, Pro V710 | -| | `gfx1102` | Radeon RX 7600(XT/S/M/M XT), RX 7650 GRE, RX 7700S, RX 7550M, Pro W7600, Pro W7500 | -| | `gfx1103` | Radeon 780M, 760M, 740M, 610M — Ryzen 7040/8040 "Phoenix/Phoenix 2/Hawk Point" mobile APUs; Ryzen Z1(Extreme), Ryzen Z2 Go | -| RDNA 3.5 (Ryzen AI 300 / Max — APUs) | `gfx1150` | Radeon 890M, 880M — Ryzen AI 300 "Strix Point" APUs; Ryzen Z2 Extreme | -| | `gfx1151` | Radeon 8060S, 8050S — Ryzen AI Max "Strix Halo" APUs | -| | `gfx1152` | Radeon 860M, 820M — Ryzen AI "Krackan Point" APUs | -| | `gfx1153` | Radeon 840M — Ryzen AI "Krackan Point 2" APUs | -| RDNA 4 (Navi 4X — consumer & pro) | `gfx1200` | Radeon RX 9060(XT/M/S), RX 9050 | -| | `gfx1201` | Radeon RX 9070(XT/GRE/S/M XT), RX 9080M, Radeon AI PRO R9700 | - -Because ROCm 7.14.0 provides an optimized library package per architecture, `HSA_OVERRIDE_GFX_VERSION` should **not** be set for any of the targets above — spoofing a different architecture only costs performance now, and can produce incorrect results. It remains relevant only for architectures that are absent from this list. +ROCm 7.14.0 ships an optimized library package for every architecture below, but the prebuilt images do not cover all of them: building a fat binary for the full set exceeds the CI time budget, so the images are limited to the discrete GPUs plus the fastest APUs, Strix Point (`gfx1150`) and Strix Halo (`gfx1151`). The smaller APUs are several times slower, and remain supported targets that you build locally — see [Building Docker locally](#building-docker-locally-1). + +The "Prebuilt" column says whether a target is included in the published images. Parentheses group SKU variants that share a target, so `RX 6700(XT)` covers both the RX 6700 and the RX 6700 XT. + +| Family | Target | Prebuilt | Products | +| --- | --- | --- | --- | +| CDNA (server & data center accelerators) | `gfx908` | yes | Instinct MI100 | +| | `gfx90a` | yes | Instinct MI210, MI250(X) | +| | `gfx942` | yes | Instinct MI300A, MI300X(-HF), MI308X, MI325X | +| | `gfx950` | yes | Instinct MI350(X/P), MI355X | +| RDNA 1 (Navi 1X — consumer & workstation) | `gfx1010` | yes | Radeon RX 5700(XT/M), RX 5600(XT/OEM/M), Pro W5700(X), Pro 5700(XT) | +| | `gfx1011` | yes | Radeon Pro 5600M, Pro V520, Pro V540 | +| | `gfx1012` | yes | Radeon RX 5500(XT/OEM/M), RX 5300(XT OEM/M), Pro W5500(X/M), Pro 5500(XT/M), Pro 5300(M) | +| RDNA 2 (Navi 2X — consumer, workstation & APUs) | `gfx1030` | yes | Radeon RX 6950 XT, RX 6900 XT, RX 6800(XT), Pro W6800(X), Pro W6800X Duo, Pro W6900X, Pro V620 | +| | `gfx1031` | yes | Radeon RX 6750(XT/GRE), RX 6700(XT/M), RX 6850M XT, RX 6800M, Pro W6700 | +| | `gfx1032` | yes | Radeon RX 6650(XT/M/M XT), RX 6600(XT/M/S), RX 6800S, RX 6700S, Pro W6600(X/M) | +| | `gfx1033` | no | Steam Deck LCD/OLED APU — "Van Gogh" ("Aerith", "Sephiroth") | +| | `gfx1034` | yes | Radeon RX 6500(XT/M), RX 6400, RX 6300(M), RX 6550M, RX 6450M, Pro W6400, Pro W6300(M), Pro W6500M | +| | `gfx1035` | no | Radeon 680M, 660M, 610M — Ryzen 6000 "Rembrandt"/"Rembrandt-R" mobile APUs | +| | `gfx1036` | no | Radeon Graphics 2CU — Ryzen 7000/9000 desktop, Threadripper 7000/9000, EPYC 4004/4005; Radeon 610M — "Dragon Range" mobile | +| RDNA 3 (Navi 3X — consumer, workstation & APUs) | `gfx1100` | yes | Radeon RX 7900(XT/XTX/GRE/M), Pro W7900, Pro W7800 | +| | `gfx1101` | yes | Radeon RX 7800 XT, RX 7700 XT, Pro W7700, Pro V710 | +| | `gfx1102` | yes | Radeon RX 7600(XT/S/M/M XT), RX 7650 GRE, RX 7700S, RX 7550M, Pro W7600, Pro W7500 | +| | `gfx1103` | no | Radeon 780M, 760M, 740M, 610M — Ryzen 7040/8040 "Phoenix/Phoenix 2/Hawk Point" mobile APUs; Ryzen Z1(Extreme), Ryzen Z2 Go | +| RDNA 3.5 (Ryzen AI 300 / Max — APUs) | `gfx1150` | yes | Radeon 890M, 880M — Ryzen AI 300 "Strix Point" APUs; Ryzen Z2 Extreme | +| | `gfx1151` | yes | Radeon 8060S, 8050S — Ryzen AI Max "Strix Halo" APUs | +| | `gfx1152` | no | Radeon 860M, 820M — Ryzen AI "Krackan Point" APUs | +| | `gfx1153` | no | Radeon 840M — Ryzen AI "Krackan Point 2" APUs | +| RDNA 4 (Navi 4X — consumer & pro) | `gfx1200` | yes | Radeon RX 9060(XT/M/S), RX 9050 | +| | `gfx1201` | yes | Radeon RX 9070(XT/GRE/S/M XT), RX 9080M, Radeon AI PRO R9700 | + +ROCm 7.14.0 has optimized libraries for every target above, so `HSA_OVERRIDE_GFX_VERSION` is only needed when the image itself has no code for your GPU: + +- prebuilt image, `yes` row — leave the override unset; +- prebuilt image, `no` row — set the override, or better, build locally for your target; +- local build — leave it unset and pass your target to `ROCM_DOCKER_ARCH`. + +Spoofing another architecture costs performance and can give incorrect results, so use it only as a fallback. This is a wider set than AMD's official support matrix, which is worth keeping in mind: - RDNA 1 (`gfx1010`, `gfx1011`, `gfx1012`) is not officially supported at all. ROCm still ships packages for it and it does work in practice — an RX 5500M (`gfx1012`) handles Qwen 3.5 4B fine, for instance — but expect it to be validated rather than tuned. -- For RDNA 2, only `gfx1030` is officially supported. The notable change in 7.14.0 is that the rest of the generation (`gfx1031` through `gfx1036`) now has its own packages too, so those cards no longer need `HSA_OVERRIDE_GFX_VERSION=10.3.0` to masquerade as `gfx1030` — they run on their own target. +- For RDNA 2, only `gfx1030` is officially supported. The notable change in 7.14.0 is that the rest of the generation (`gfx1031` through `gfx1036`) now has its own packages too, so those cards no longer need `HSA_OVERRIDE_GFX_VERSION=10.3.0` to masquerade as `gfx1030` — provided llama.cpp was built for the target. `gfx1031`, `gfx1032` and `gfx1034` are in the prebuilt images and work without the override; `gfx1035` and `gfx1036` are not, so on a prebuilt image they still need it, and a local build for the real target is the better option. Treat everything outside the official matrix as working but less optimized than RDNA 3 and newer, where AMD puts most of the tuning effort. @@ -178,13 +184,18 @@ docker build -t local/llama.cpp:light-rocm --target light -f .devops/rocm.Docker docker build -t local/llama.cpp:server-rocm --target server -f .devops/rocm.Dockerfile . ``` -Building for every architecture takes a long time and produces a large image. To build only for your own GPU, pass its target through `ROCM_DOCKER_ARCH`: +`ROCM_DOCKER_ARCH` selects the architectures to compile for. It defaults to the same set as the prebuilt images, so pass it explicitly in either of these cases: + +- your GPU is one of the `no` rows in the table above — the APU targets `gfx1033`, `gfx1035`, `gfx1036`, `gfx1103`, `gfx1152` and `gfx1153`; +- you only care about your own GPU and want a much shorter build and a smaller image. ```bash docker build -t local/llama.cpp:server-rocm --target server \ - --build-arg ROCM_DOCKER_ARCH=gfx1100 -f .devops/rocm.Dockerfile . + --build-arg ROCM_DOCKER_ARCH=gfx1103 -f .devops/rocm.Dockerfile . ``` +Multiple targets are separated by semicolons, for example `--build-arg ROCM_DOCKER_ARCH='gfx1100;gfx1103'`. Build time and image size scale with the number of architectures, which is why the published images stop at the discrete GPUs and Strix Halo. + Refer to [.devops/rocm.Dockerfile](../.devops/rocm.Dockerfile) for the remaining `ARGS` and their defaults. ## Usage