Instructions to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4")
model = AutoModelForMultimodalLM.from_pretrained("Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4

SGLang

How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 with Docker Model Runner:
```
docker model run hf.co/Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4
```

Qwen3.8-27B — MXFP4 (AMD Quark) for RDNA4

MXFP4 weight quantisation of Qwen/Qwen3.8-27B, built with AMD Quark 0.12.post1 for RDNA4 (gfx1200/gfx1201: Radeon AI PRO R9700, RX 9070 XT). > ⚡ 2026-08-20 — major performance upgrade (rc10): the serving port gained tuned per-shape GEMM

configs for the R9700, regenerated with vLLM's own tuner on the exact serving stack. This model's throughput rose +12–28% per cell (short c32 579→713 aggregate); the FP8 arm rose up to +34% (c32 754→1,014). All tables below are current rc10 measurements — pull latest and you get them. Credit where due: this port was always geared to concurrent serving (32 sequences, speculative decoding, accuracy-gated) — the single-stream tuning insight came from the community. andysalerno's and prcoe1's benchmarks surfaced the untuned-GEMM gap; folding that lever into the concurrency stack is what closed the loop.

Serve it with our vLLM 0.26.1 RDNA4 port: GitHub · Docker image (capicua25x/vllm-rocm-rdna4:latest, rc10) · benchmark harness: Capicua25x/modelbench.

What's 4-bit: MLP/expert projections only (12.05B packed U8 = 22.7B params). Attention (q/k/v/o + norms), embeddings, lm_head, routers and the entire vision path stay bf16 — ~27.8B logical params, architecture unchanged. (HF's sidebar "8-bit"/"16B" auto-tags read the U8 container, not the contents; every MXFP4 repo on the Hub gets them.) Calibrated fp8 KV-cache scales ship as a side-file; they only activate under --kv-cache-dtype fp8.

Quick start (2× R9700, TP2, full native 262k window, MTP-3)

docker run --rm --name vllm-qwen --network=host \
  --device=/dev/kfd --device=/dev/dri/renderD128 --device=/dev/dri/renderD129 \
  --group-add=video --group-add=render --ipc=host \
  -e NCCL_PROTO=Simple \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint /usr/local/bin/vllm capicua25x/vllm-rocm-rdna4:0.26.1-rdna4-rc10 \
  serve Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4 --served-model-name qwen --port 8011 --trust-remote-code \
  --tensor-parallel-size 2 --gpu-memory-utilization 0.95 --max-model-len 262144 \
  --attention-backend TRITON_ATTN --enable-prefix-caching \
  --max-num-seqs 32 --max-num-batched-tokens 8000 --max-cudagraph-capture-size 128 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"attention_backend":"TRITON_ATTN"}'

We A/B this configuration (C · MXFP4 @ bf16 KV) in production against B · FP8 @ fp8 KV (Qwen/Qwen3.8-27B-FP8 + --kv-cache-dtype fp8 --mamba-ssm-cache-dtype bfloat16). Both pass the same gates; so far Qwen behaves correctly on both.

Measured performance (per-user / aggregate tok/s)

Paired back-to-back on the same box (2× R9700, TP2, thinking ON, 256-token completions). C = this model @ bf16 KV · B = FP8 checkpoint @ fp8 KV.

Short prompts (~30 tokens — interactive chat)

users	C (this model)	B
1	62 / 62	66 / 66
4	53 / 208	60 / 232
8	42 / 335	54 / 421
16	32 / 505	45 / 687
32	24 / 713	33 / 1,014
64	17 / 714	24 / 1,016

B column: in-tree tuned R9700 GEMM configs; C column: measured on rc10 (both 2026-08-20).

6,000-token prompts (RAG / long-system-prompt workloads)

users	C (this model)	B
1	54 / 54	61 / 61
4	43 / 163	43 / 170
8	30 / 231	31 / 241
16	20 / 312	20 / 311
32	12 / 368	11 / 356

At long context the two are a statistical tie; on short prompts B leads from 8 users up. KV pool: C ≈ 415k tokens · B ≈ 539k (2.06× the window). NCCL_PROTO=Simple matters on this PCIe pair (RCCL's LL protocol is 2.8× slower for the ~640 KB decode all-reduces).

Accuracy (AA class-A, paired items, seed 1234, on-spec sampling)

ref = the same checkpoint served in bf16 by a cloud provider. Same judge for all judged rows. Cells show the most recent run at the stated n on the shipping config; ±2 items is the noise band.

benchmark (n)	ref (bf16)	C (this model)	B
GSM8K think, flex·strict (50)	0.96·0.82	0.98·0.96	0.98·0.86
GSM8K nothink (50)	0.98·0.98	0.98·0.98	0.98·0.98
IFEval inst·prompt (80)	0.97·0.95	0.95·0.91	0.98·0.98
AA-LCR judged (100)	0.78	0.78	0.77 · 0.81 (s1234·s99)
GPQA-Diamond (60)	0.78	0.92	0.85
AIME'25 (30)	0.93	0.93	0.97
τ²-telecom agentic (114)	0.939	0.904	⏳
HLE, judged (120)	0.30	0.25	0.275

Reproducing the quantisation

Data-free, CPU-only, file-to-file — ~3 minutes; no calibration set. (The KV-cache scalars are the one calibrated artefact and came from a separate capture pass on the served model.)

from quark.torch.export.api import direct_quantize_checkpoint

EXCLUDE = [
    "lm_head", "*embed_tokens*",
    "*.self_attn.q_proj", "*.self_attn.k_proj", "*.self_attn.v_proj", "*.self_attn.o_proj",
    "*.self_attn.q_norm", "*.self_attn.k_norm", "*norm*",
    "*.linear_attn.conv1d", "*.linear_attn.norm",
    "*.mlp.gate", "*.mlp.shared_expert_gate",
    "mtp*", "*visual*", "*vision*",
]

Full recipe, engine patches and methodology: RDNA4-PORT.md · every number above is reproducible with Capicua25x/modelbench.

Licence and attribution

Apache-2.0, following the base model. Quantised and served by Capicua25x; base model by the Qwen team; quantisation toolkit by AMD (Quark).

Downloads last month: 374

Safetensors

Model size

16B params

Tensor type

BF16

Model tree for Capicua25x/Qwen3.8-27B-MXFP4-Quark-RDNA4

Base model

Qwen/Qwen3.8-27B

Quantized

(691)

this model