---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- qwen
- qwen3.5
- multimodal
- autoround
- quantization
- int4
---

# Qwen3.5-9B Mixed-INT4 AutoRound

This repository contains a Mixed-INT4 quantized version of
[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), produced using
[Intel AutoRound](https://github.com/intel/auto-round).

The model weights were quantized to reduce memory requirements while preserving
the original Qwen3.5 architecture, tokenizer, configuration, and multimodal
capabilities.

## Model Information

* **Base model:** [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)
* **Model type:** Multimodal causal language model with vision encoder
* **Quantization:** Mixed INT4
* **Quantization framework:** Intel AutoRound
* **Weight format:** Safetensors
* **Native context length:** 262,144 tokens
* **License:** Apache 2.0

## Important Notice

This repository is an unofficial community quantization of Qwen3.5-9B.

The model architecture and original model behavior are provided by the Qwen
team. Quantization may cause small differences in output quality, numerical
precision, generation consistency, and benchmark performance compared with the
original model.

No independent benchmark results are currently provided for this quantized
version.

## evaluation
Evaluation was performed using AutoRound’s evaluation CLI, powered by LM Evaluation Harness.

| Benchmark     |   Metric | Qwen3.5-9B | Qwen3.5-9B-MixedInt4-AutoRound |  Difference | Recovery Rate |
| ------------- | -------: | ---------: | -----------------------------: | ----------: | ------------: |
| MMLU          |      acc |     78.66% |                         77.62% |     -1.04%p |        98.68% |
| ARC-Challenge | acc_norm |     55.80% |                         55.03% |     -0.77%p |        98.62% |
| BoolQ         |      acc |     89.17% |                         86.91% |     -2.26%p |        97.47% |
| HellaSwag     | acc_norm |     78.15% |                         77.45% |     -0.70%p |        99.10% |
| PIQA          | acc_norm |     80.03% |                         80.20% |     +0.17%p |       100.21% |
| WinoGrande    |      acc |     73.01% |                         71.82% |     -1.19%p |        98.37% |
| **Average**   |        — | **75.80%** |                     **74.84%** | **-0.97%p** |    **98.73%** |

| MMLU Category   | Qwen3.5-9B | Qwen3.5-9B-MixedInt4-AutoRound | Difference | Recovery Rate |
| --------------- | ---------: | -----------------------------: | ---------: | ------------: |
| Humanities      |     70.48% |                         68.93% |    -1.55%p |        97.80% |
| Other           |     83.20% |                         82.52% |    -0.68%p |        99.18% |
| Social Sciences |     86.90% |                         86.55% |    -0.35%p |        99.60% |
| STEM            |     78.34% |                         77.07% |    -1.27%p |        98.38% |



## Serving with vLLM

If the installed vLLM version supports this model architecture and AutoRound
quantization format, the model can be served using:

```bash
vllm serve YOUR_USERNAME/Qwen3.5-9B-MixedInt4-AutoRound \
  --trust-remote-code
```

Support for newly released model architectures and quantization formats may
require a recent development build of vLLM.

## Quantization Details

* **Method:** Mixed-INT4 AutoRound quantization
* **Source weights:** Qwen/Qwen3.5-9B
* **Fine-tuning:** None
* **Architecture modifications:** None intended

Exact quantization settings, calibration dataset, group size, and AutoRound
version should be documented here when available.

## Limitations

This model inherits the limitations of the original Qwen3.5-9B model.

Additional limitations may result from quantization:

* Reduced numerical precision
* Small changes in generated responses
* Possible degradation on sensitive reasoning or vision-language tasks
* Runtime compatibility differences between inference frameworks
* Potential differences in long-context behavior

Users should evaluate the model on their own workloads before production use.

## Original Model

For complete information about the architecture, supported languages, context
length, multimodal usage, benchmarks, intended uses, and limitations, refer to
the original model card:

* [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)

## License

The original Qwen3.5-9B model is distributed under the Apache License 2.0.

This quantized repository follows the license and usage requirements of the
original model. Users are responsible for reviewing and complying with the
original license terms.

## Credits

* Original model: [Qwen](https://huggingface.co/Qwen)
* Quantization framework: [Intel AutoRound](https://github.com/intel/auto-round)