---
language: en
pipeline_tag: text-generation
library_name: mlx
tags:
- mlx
- qwen
- int4
- quantized
base_model: Qwen/Qwen3.5-4B
---

# Qwen3.5-4B MLX INT4

This is an **MLX INT4 quantized** version of [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), optimized for efficient inference on Apple Silicon.

## Overview

| Property | Value |
|----------|-------|
| Base Model | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) |
| Quantization | INT4 (group_size=64, affine) |
| Framework | [MLX](https://github.com/ml-explore/mlx) |
| Target Hardware | Apple Silicon (M1/M2/M3/M4) |
| Original Size | ~9 GB |
| Quantized Size | ~4.2 GB |

## Quantization Details

- **Method**: Post-training INT4 quantization via `mlx_lm`
- **Group Size**: 64
- **Mode**: Affine
- **Protected Layers**: `embed_tokens` and `lm_head` kept at float16 for output quality
- **Compression**: 53% reduction

## Usage

### Requirements

```bash
pip install mlx mlx-lm
```

### Quick Start

```python
from mlx_lm import load, generate

model, tokenizer = load("pyromind/qwen3.5-4b-debug-mlx-int4")

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain quantum computing in simple terms."},
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

output = generate(
    model, tokenizer,
    prompt=prompt,
    max_tokens=2048,
    temp=0.6,
)
print(output)
```

### Chat via CLI

```bash
mlx_lm.chat --model pyromind/qwen3.5-4b-debug-mlx-int4
```

## License

This model follows the license of the base model [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B).