ROCm vLLM Wheels

Linux, uv

I got tired of fighting AMD and vLLM's bad documentation, so I built the latest wheels from source.

Those are built for linux, using uv with python 3.11, 3.12, 3.13 on the latest ROCm 7.14.0 (July 2026), for a R9700 (gfx1201) card.

How to use them:

• Download the one you need

uvx hf buckets cp hf://buckets/dobriak/ROCm-vLLM-wheels/vllm-0.25.1+rocm714-cp313-cp313-linux_x86_64.whl /path/to/

• Prepare your python environment

uv venv --python 3.13 --seed --managed-python
uv pip install --index-url https://repo.amd.com/rocm/whl-multi-arch/ "torch[device-gfx1201]==2.11.0+rocm7.14.0" "torchvision[device-gfx1201]==0.26.0+rocm7.14.0" "torchaudio==2.11.0+rocm7.14.0"
uv pip install https://rocm.frameworks.amd.com/whl-multi-arch/vllm-rdna/flash-attn/flash_attn-2.8.3-py3-none-any.whl

uv pip install /path/to/vllm-0.25.1+rocm714-cp313-cp313-linux_x86_64.whl

• Make sure to always tell vLLM where the relevant binaries are. For example, to serve Gemma 4:

PYTHONPATH=/path/to/your/.venv/lib/python3.13/site-packages/_rocm_sdk_core/share/amd_smi/ FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE uv run vllm serve /path/to/cyankiwi/gemma-4-26B-A4B-it-AWQ-4bit --max-model-len 131072 --gpu-memory-utilization 0.90 --enable-auto-tool-choice --reasoning-parser gemma4 --tool-call-parser gemma4 --limit-mm-per-prompt '{"image": 4, "audio": 1}' --async-scheduling --served-model-name cyankiwi/gemma-4-26B-A4B --host 0.0.0.0 --port 8361 --trust-remote-code

References

• https://github.com/vllm-project/vllm
• https://rocm.docs.amd.com/projects/ai-ecosystem/en/latest/frameworks/pytorch/install.html?fam=radeon&os=linux&pytorch-ver=2.12.0&i=pip&w=compute&gpu=ai-r9700&gfx=gfx1201
• https://rocm.docs.amd.com/projects/ai-ecosystem/en/latest/inference/vllm.html?fam=radeon&vllm-ver=0.23&i=pip&w=compute&gpu=ai-r9700&gfx=gfx1201