README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.6-27B
tags:
  - safetensors
  - qwen3_5
  - 4-bit
  - auto-round
  - quantized
pipeline_tag: image-text-to-text
---

# Qwen3.6-27B-int4-AutoRound

This is an Int4 [AutoRound](https://github.com/intel/auto-round) quantization of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B), produced using [spark-auto-round](https://github.com/whpthomas/spark-auto-round).

## Quantization Details

| Parameter | Value |
|---|---|
| **Original Model** | [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) |
| **Quantization Method** | AutoRound (W4A16, symmetric) |
| **Bits** | 4 |
| **Group Size** | 128 |
| **Calibration Dataset** | [opencode-instruct](https://huggingface.co/nvidia/OpenCodeInstruct) |
| **Calibration Samples** | 512 |
| **Calibration Sequence Length** | 2048 |
| **Tuning Iterations** | 1000 |
| **Batch Size** | 8 |
| **Packing Format** | `auto_round:auto_gptq` |
| **AutoRound Version** | 0.14.2 |
| **Model Size** | ~19 GB |

### Layers Kept in FP16

The `linear_attn.in_proj_a` and `linear_attn.in_proj_b` projections across all DeltaNet layers, as well as `mtp.fc`, are kept at FP16 precision for quality preservation.

## Quantization Report

All 64 transformer blocks passed sensitivity analysis (63 PASS, 1 WARN at layer 58).

| Layer Range | Cosine Similarity | PSNR (dB) |
|---|---|---|
| Layers 0-10 | 0.9999 - 1.0000 | 80.7 - 84.0 |
| Layers 11-20 | 0.9995 - 0.9999 | 74.9 - 81.5 |
| Layers 21-30 | 0.9988 - 0.9995 | 73.6 - 78.7 |
| Layers 31-40 | 0.9976 - 0.9986 | 69.4 - 73.2 |
| Layers 41-50 | 0.9943 - 0.9976 | 60.2 - 69.2 |
| Layers 51-63 | 0.9883 - 0.9934 | 53.4 - 66.5 |

Full per-layer reports are available in the repository: [`quantization-report.txt`](quantization-report.txt) and [`quantization-report.csv`](quantization-report.csv).

## How to Use

### With vLLM

```bash
vllm serve coder3101/Qwen3.6-27B-int4-AutoRound
```

### With Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "coder3101/Qwen3.6-27B-int4-AutoRound"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

prompt = "Explain the theory of relativity in simple terms."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Acknowledgments

- Quantization performed using [spark-auto-round](https://github.com/whpthomas/spark-auto-round) by [@whpthomas](https://github.com/whpthomas)
- Based on [AutoRound](https://github.com/intel/auto-round) by Intel

---

# Original Model Card -- Qwen3.6-27B

> Below is the original model card from [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B).

## Qwen3.6-27B

### Highlights

Qwen3.6-27B follows the Qwen3.5 series with key upgrades:

- **Agentic Coding**: Handles frontend workflows and repo-level reasoning with greater fluency.
- **Thinking Preservation**: New option to retain reasoning context from historical messages, reducing overhead in iterative development.

### Model Architecture

| Property | Value |
|---|---|
| Type | Causal Language Model with Vision Encoder |
| Parameters | 27B |
| Hidden Dimension | 5120 |
| Token Embedding | 248320 (Padded) |
| Number of Layers | 64 |
| Hidden Layout | `16 x (3 x (Gated DeltaNet -> FFN) -> 1 x (Gated Attention -> FFN))` |
| FFN Intermediate Dimension | 17408 |
| Context Length | 262,144 (natively), up to 1,010,000 with YaRN |

**Gated DeltaNet**: 48 linear attention heads for V, 16 for QK (head dim: 128)
**Gated Attention**: 24 heads for Q, 4 for KV (head dim: 256, RoPE dim: 64)

### Benchmark Results -- Language

| Benchmark | Qwen3.5-27B | Qwen3.5-397B-A17B | Gemma4-31B | Claude 4.5 Opus | Qwen3.6-35B-A3B | **Qwen3.6-27B** |
|---|---|---|---|---|---|---|
| SWE-bench Verified | 75.0 | 76.2 | 52.0 | 80.9 | 73.4 | **77.2** |
| SWE-bench Pro | 51.2 | 50.9 | 35.7 | 57.1 | 49.5 | **53.5** |
| SWE-bench Multilingual | 69.3 | 69.3 | 51.7 | 77.5 | 67.2 | **71.3** |
| Terminal-Bench 2.0 | 41.6 | 52.5 | 42.9 | 59.3 | 51.5 | **59.3** |
| SkillsBench Avg5 | 27.2 | 30.0 | 23.6 | 45.3 | 28.7 | **48.2** |
| QwenWebBench | 1068 | 1186 | 1197 | 1536 | 1397 | **1487** |
| NL2Repo | 27.3 | 32.2 | 15.5 | 43.2 | 29.4 | **36.2** |
| Claw-Eval Avg | 64.3 | 70.7 | 48.5 | 76.6 | 68.7 | **72.4** |
| Claw-Eval Pass^3 | 46.2 | 48.1 | 25.0 | 59.6 | 50.0 | **60.6** |
| QwenClawBench | 52.2 | 51.8 | 41.7 | 52.3 | 52.6 | **53.4** |
| MMLU-Pro | 86.1 | 87.8 | 85.2 | 89.5 | 85.2 | **86.2** |
| MMLU-Redux | 93.2 | 94.9 | 93.7 | 95.6 | 93.3 | **93.5** |
| SuperGPQA | 65.6 | 70.4 | 65.7 | 70.6 | 64.7 | **66.0** |
| C-Eval | 90.5 | 93.0 | 82.6 | 92.2 | 90.0 | **91.4** |
| GPQA Diamond | 85.5 | 88.4 | 84.3 | 87.0 | 86.0 | **87.8** |
| HLE | 24.3 | 28.7 | 19.5 | 30.8 | 21.4 | **24.0** |
| LiveCodeBench v6 | 80.7 | 83.6 | 80.0 | 84.8 | 80.4 | **83.9** |
| HMMT Feb 25 | 92.0 | 94.8 | 88.7 | 92.9 | 90.7 | **93.8** |
| HMMT Nov 25 | 89.8 | 92.7 | 87.5 | 93.3 | 89.1 | **90.7** |
| HMMT Feb 26 | 84.3 | 87.9 | 77.2 | 85.3 | 83.6 | **84.3** |
| IMOAnswerBench | 79.9 | 80.9 | 74.5 | 84.0 | 78.9 | **80.8** |
| AIME26 | 92.6 | 93.3 | 89.2 | 95.1 | 92.7 | **94.1** |

### Benchmark Results -- Vision Language

| Benchmark | Qwen3.5-27B | Qwen3.5-397B-A17B | Gemma4-31B | Claude 4.5 Opus | Qwen3.6-35B-A3B | **Qwen3.6-27B** |
|---|---|---|---|---|---|---|
| MMMU | 82.3 | 85.0 | 80.4 | 80.7 | 81.7 | **82.9** |
| MMMU-Pro | 75.0 | 79.0 | 76.9 | 70.6 | 75.3 | **75.8** |
| MathVista mini | 87.8 | -- | 79.3 | -- | 86.4 | **87.4** |
| DynaMath | 87.7 | 86.3 | 79.5 | 79.7 | 82.8 | **85.6** |
| VlmsAreBlind | 96.9 | -- | 87.2 | -- | 96.6 | **97.0** |
| RealWorldQA | 83.7 | 83.9 | 72.3 | 77.0 | 85.3 | **84.1** |
| MMStar | 81.0 | 83.8 | 77.3 | 73.2 | 80.7 | **81.4** |
| MMBenchEN-DEV-v1.1 | 92.6 | -- | 90.9 | -- | 92.8 | **92.3** |
| SimpleVQA | 56.0 | 67.1 | 52.9 | 65.7 | 58.9 | **56.1** |
| CharXiv RQ | 79.5 | 80.8 | 67.9 | 68.5 | 78.0 | **78.4** |
| CC-OCR | 81.0 | 82.0 | 75.7 | 76.9 | 81.9 | **81.2** |
| OCRBench | 89.4 | -- | 86.1 | -- | 90.0 | **89.4** |
| ERQA | 60.5 | 67.5 | 57.5 | 46.8 | 61.8 | **62.5** |
| CountBench | 97.8 | 97.2 | 96.1 | 90.6 | 96.1 | **97.8** |
| RefCOCO avg | 90.9 | 92.3 | -- | -- | 92.0 | **92.5** |
| EmbSpatialBench | 84.5 | -- | -- | -- | 84.3 | **84.6** |
| RefSpatialBench | 67.7 | -- | 4.7 | -- | 64.3 | **70.0** |
| VideoMME (w sub.) | 87.0 | 87.5 | -- | 77.7 | 86.6 | **87.7** |
| VideoMMMU | 82.3 | 84.7 | 81.6 | 84.4 | 83.7 | **84.4** |
| MLVU | 85.9 | 86.7 | -- | 81.7 | 86.2 | **86.6** |
| MVBench | 74.6 | 77.6 | -- | 67.2 | 74.6 | **75.5** |
| V* | 93.7 | 95.8 | -- | 67.0 | 90.1 | **94.7** |
| AndroidWorld | 64.2 | -- | -- | -- | -- | **70.3** |

### Serving Frameworks

- **SGLang** (>=0.5.10)
- **vLLM** (>=0.19.0)
- **KTransformers**
- **HuggingFace Transformers**

### Sampling Parameters

| Mode | Temperature | top_p | top_k | min_p | presence_penalty |
|---|---|---|---|---|---|
| Thinking (general) | 1.0 | 0.95 | 20 | 0.0 | 0.0 |
| Thinking (precise coding/WebDev) | 0.6 | 0.95 | 20 | -- | -- |
| Non-thinking / Instruct | 0.7 | 0.80 | 20 | -- | 1.5 |

### Key Features

- **Thinking mode** is on by default; can be disabled via `enable_thinking: False`.
- Does **not** support soft switch (`/think` and `/nothink` from Qwen3).
- **Preserve Thinking**: `preserve_thinking: True` retains reasoning traces from history.
- Supports text, image, and video inputs.
- Multi-Token Prediction (MTP) supported.
- Native context length: 262,144 tokens; extensible to 1,010,000 tokens with YaRN RoPE scaling.

### Citation

```bibtex
@misc{qwen3.6-27b,
    title  = {{Qwen3.6-27B}: Flagship-Level Coding in a {27B} Dense Model},
    author = {{Qwen Team}},
    month  = {April},
    year   = {2026},
    url    = {https://qwen.ai/blog?id=qwen3.6-27b}
}
```