Support for AMD GPUs (Vulkan or ROCm/HIP) for Edge0 120B / 145B #197

Area

New backend (GPU / NPU / other accelerator)

What problem are you trying to solve?

Hi, thanks for releasing Edge0 as open weights!

The v0.2.0 release notes list Edge0 145B / 120B as NVIDIA-only (CUDA 13, compute capability 7.5+). Edge0 35B already runs on Windows via Vulkan.

Are there plans to support the 120B / 145B models on AMD GPUs, either through the existing Vulkan backend or via ROCm/HIP?

My setup:

• GPU: AMD Radeon RX 9070 XT (RDNA4, gfx1201, 16 GB VRAM)
• CPU: Ryzen 5 9600X, 32 GB RAM
• Windows 11

This would meet the stated 16 GiB GPU / 20 GiB system memory minimums apart from the vendor requirement. Is the edge0-infer source available, so the community could help with a port? Happy to test a preview build.

Thanks!

Proposed solution

No response

Alternatives considered

No response