Qwen3-VL: Optimized for AMD ROCm
Qwen3-VL-4B-Instruct is a dense vision-language model supporting text and image input, served via vLLM's OpenAI-compatible HTTP API. This repository packages inference for vision-language text generation using vLLM, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs.
This is based on the implementation of Qwen3-VL found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the qwen3_vl AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.
Task Overview
Task: Vision-language text generation (text + image)
Dataset: MMMU (vision, 30 subjects), MMLU (text, 57 subjects, 5-shot)
Output metrics: MMMU accuracy, MMLU accuracy (5-shot), per-modality latency/throughput
vLLM note: GPU-only (vLLM server). No CPU or NPU inference path is available.
--enforce-eageris always on — CUDA graph capture produces degenerate output on gfx1150/1151 for vision models.
AMD ROCm Optimization
This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm. Key points:
- Validated backend: vLLM (ROCm-enabled, built from source, OpenAI-compatible HTTP API).
- No code changes required versus the upstream Qwen3-VL implementation — only environment/runtime configuration differs.
- No AMD NPU (VitisAI) backend for vLLM and no practical CPU path for a 4B-parameter generative VLM.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| GPU | FP16 (default) / BF16 | vLLM | AMD Ryzen AI MAX+ 395 / Radeon 8060S (gfx1150/1151) or compatible AMD GPU | --enforce-eager always on |
Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the qwen3_vl on GitHub.
Model Details
Model Type: Vision-language causal language model (dense)
Base Model: Qwen/Qwen3-VL-4B-Instruct
Model Stats:
- Context ceiling: 32768 tokens
- Precision tested: FP16 (default), BF16
- Weights: ~8 GB (GPU-resident)
Accuracy Pipeline
Higher accuracy means the model's answers agree more closely with the ground-truth choice — 100% would be perfect, and random chance depends on the number of answer options per question (typically 25% for 4-way MCQ). MMMU evaluates vision + multiple-choice reasoning across 30 subjects; MMLU evaluates text-only knowledge across 57 subjects via lm-evaluation-harness in loglikelihood mode.
Metrics Explained
| Metric | Description |
|---|---|
| MMMU accuracy | Multiple-choice accuracy on the MMMU validation split — each question pairs an image with a text question across 30 academic subjects, measuring joint vision-language reasoning. |
| MMLU accuracy | 5-shot multiple-choice accuracy across 57 text-only subjects, measured in loglikelihood mode via lm-evaluation-harness. |
Accuracy Results
Published results:
| Benchmark | Metric | Value | Notes |
|---|---|---|---|
| MMLU | accuracy | 75.61% ± 1.20% | 5-shot, 57 subjects, LIMIT=20 (1,140 questions) |
| MMMU | accuracy | 56.35% | Validation split, 30 subjects, 197/870 samples evaluated |
Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 View the full project on GitHub
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for building vLLM from source and server management
- Per-modality (text/image) latency and throughput benchmarking
- MMMU and MMLU accuracy evaluation and reproduction instructions