---
base_model: Qwen/Qwen3.6-27B
license: other
license_name: qwen
pipeline_tag: text-generation
tags:
- rfp458
- 4-bit
- quantized
- vllm
- rocm
- rdna4
- qwen3.6
---

# Qwen3.6-27B-RFP458 (4.5 bpw)

A 4.5-bit-per-weight RFP458 quantization of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B), a hybrid vision-language model (Gated-DeltaNet linear-attention + full-attention layers, with a vision tower and MTP head).

## Summary

- **Format:** RFP458 (`rfp458-pack-quantized`): iq4_nl non-uniform 4-bit codebook, group size 16, signed-int8 block mantissa + per-channel int8 exponent, with hadamard16 weight rotation.
- **Size:** ~20.5 GB (vs ~27 GB for the FP8 build); ~9.4 GiB per card on a 2x 32 GB setup.
- **What is quantized:** the MLP linears, the GDN `in_proj_qkv` / `in_proj_z` / `out_proj`, and the full self-attention q/k/v/o projections. Embeddings, lm_head, the GDN gating projections (`in_proj_a` / `in_proj_b`), conv1d, norms, and the entire vision tower are kept in bf16.

## Quality

WikiText-2 perplexity (llama.cpp-compatible, n_ctx 2048, full test set):

| Build | Size | PPL |
|---|---|---|
| **RFP458 (this model)** | 20.5 GB | **6.936** |
| FP8 (RedHatAI/Qwen3.6-27B-FP8) | ~27 GB | 7.071 |

RFP458 matches or slightly beats the FP8 build at roughly 25 percent smaller weight footprint.

## Serving

Built for and validated on a vLLM build with native RFP458 dequant kernels on AMD RDNA4 (gfx1201, Radeon AI PRO R9700), tensor-parallel 2. The smaller weights free enough VRAM to serve the full 262K context with a large KV pool. Note that 4-bit-class formats carry a dequant cost, so single-stream decode runs roughly half the speed of the FP8 build on the same hardware; this is a capacity, footprint, and quality choice rather than a speed one.

## License

Inherits the license of the base model, Qwen/Qwen3.6-27B. See the base model card for terms.

This is a community quantization and is not affiliated with the original model authors.