README.md

---
library_name: mlx
tags:
- mlx
- oq
- quantized
- benchmark
- performance
---

# Qwopus3.5-9B-Coder-oQ4e-mtp

This model was quantized using [oQ](https://github.com/jundot/omlx) (oMLX v0.5.0) mixed-precision quantization.

## Quantization details

- **Model type**: qwen3_5
- **Bits**: 4
- **Group size**: 64
- **Format**: MLX safetensors

## Performance Benchmarks

> **Note**: Results are for reference only and may vary depending on hardware, software configuration, and workload.

### Environment
- **Hardware**: M5 MacBook Air 32GB
- **Inference Framework**: oMLX v0.5.0
- **Settings**:
  - Thinking: Disabled
  - Chat template parameter: enable_thinking=false (forced)
  - TurboQuant KV Cache: Disabled (if enable: reduces intelligence)
  - Native MTP: Enabled (key speed improvement)

### Speed Comparison (pp1024/tg128 single request)

| Model | tg TPS | TTFT (ms) | E2E (s) | Peak Mem | Test Date |
|-------|--------|-----------|---------|----------|-----------|
| Qwopus3.5-4B-Coder-oQ4e-mtp | 67.0 | 868.8 | 2.795 | 3.51 GB | 07-11 |
| Qwopus3.5-4B-Coder-oQ5e-mtp | 61.5 | 895.4 | 2.996 | 3.93 GB | 07-11 |
| Qwopus3.5-4B-Coder-oQ4-mtp | 63.4 | 1053.1 | 3.072 | 3.51 GB | 07-01 |
| Qwen3.5-9B-oQ4e-mtp | 49.7 | 1515.7 | 4.111 | 5.96 GB | 07-11 |
| **Qwopus3.5-9B-Coder-oQ4e-mtp** | **48.0** | **1475.0** | **4.164** | **5.96 GB** | **07-11** |
| Qwopus3.5-9B-Coder-oQ4-mtp | 38.1 | 1735.2 | 5.092 | 5.96 GB | 07-08 |
| Qwopus3.5-9B-Coder-oQ5e-mtp | 28.9 | 2390.5 | 6.850 | 7.00 GB | 07-11 |

### Intelligence Comparison (Accuracy %)

> **Note**: Each benchmark round tests only 30 questions. Results are for reference only.

| Model | MMLU | TRUTHFULQA | GSM8K | MATHQA | HUMANEVAL | Average |
|-------|------|------------|-------|--------|-----------|---------|
| Qwopus3.5-9B-Coder-oQ5e-mtp | 80.0 | 90.0 | 93.3 | 36.7 | 80.0 | 76.0 |
| Qwopus3.5-9B-Coder-oQ4-mtp | 80.0 | 86.7 | 83.3 | 40.0 | 80.0 | 74.0 |
| **Qwopus3.5-9B-Coder-oQ4e-mtp** | **76.7** | **76.7** | **96.7** | **36.7** | **90.0** | **75.3** |
| Qwen3.5-9B-oQ4e-mtp | 76.7 | 86.7 | 90.0 | 26.7 | 86.7 | 73.3 |
| Qwopus3.5-4B-Coder-oQ5e-mtp | 70.0 | 66.7 | 93.3 | 53.3 | 76.7 | 72.0 |
| Qwopus3.5-4B-Coder-oQ4-mtp | 63.3 | 66.7 | 96.7 | 43.3 | 83.3 | 70.7 |
| Qwopus3.5-4B-Coder-oQ4e-mtp | 63.3 | 63.3 | 96.7 | 53.3 | 76.7 | 70.7 |

## Key Findings

### oQ4 vs oQ4e vs oQ5 Differences

| Version | Speed Characteristics | Intelligence Characteristics |
|---------|----------------------|------------------------------|
| oQ4 | Medium speed | Better MATHQA/HUMANEVAL |
| oQ4e | Fastest speed | Highest MATHQA (53.3% for 4B series), but slightly lower MMLU/TRUTHFULQA |
| oQ5 | Slowest speed | Highest MMLU/TRUTHFULQA (more conservative quantization) |

### 4B vs 9B Series

| Dimension | 4B Coder | 9B Coder |
|-----------|----------|----------|
| Speed | 40%+ faster (61-67 vs 29-48 tok/s) | Slower |
| MMLU | 63-70% | 76-80% |
| GSM8K | 93-97% | 83-97% |
| MATHQA | 43-53% | 27-40% |
| HUMANEVAL | 77-83% | 80-90% |
| Memory | 3.5-4.0 GB | 6.0-7.0 GB |

### Specialized Capabilities

| Task | Best Model | Accuracy |
|------|------------|----------|
| Knowledge Q&A (MMLU) | 9B-Coder-oQ4/oQ5e | 80.0% |
| Truthfulness (TRUTHFULQA) | 9B-Coder-oQ5e | 90.0% |
| Math Reasoning (GSM8K) | 4B-Coder-oQ4/oQ4e | 96.7% |
| Math Calculation (MATHQA) | 4B-Coder-oQ4e/oQ5e | 53.3% |
| Code Generation (HUMANEVAL) | 9B-Coder-oQ4e | 90.0% |

## Recommendations

| Scenario | Recommended Model | Reason |
|----------|-------------------|--------|
| Ultra-fast Interaction | Qwopus3.5-4B-Coder-oQ4e-mtp | 67 tok/s + 3.5GB memory |
| Balanced Intelligence | Qwopus3.5-9B-Coder-oQ5e-mtp | 76% average, strongest knowledge+truthfulness |
| Code Generation | Qwopus3.5-9B-Coder-oQ4e-mtp | HUMANEVAL 90% |
| Math Reasoning | Qwopus3.5-4B-Coder-oQ4e-mtp | GSM8K 96.7% + MATHQA 53.3% |
| Low Memory Devices | Qwopus3.5-4B-Coder-oQ4e-mtp | Only 3.51 GB |
| Balanced Choice | Qwen3.5-9B-oQ4e-mtp | Balanced speed+intelligence, 73.3% average |

## Usage

```python
from mlx_lm import load, generate

# Load model
model, tokenizer = load("mlx-works/Qwopus3.5-9B-Coder-oQ4e-mtp")

# Generate text
prompt = "Hello, how are you?"
response = generate(model, tokenizer, prompt=prompt, max_tokens=100)
print(response)
```

## Notes

- This model uses Native MTP (Multi-Token Prediction) for improved speed
- Thinking mode is disabled for faster inference
- For detailed benchmark methodology, see the [oQ repository](https://github.com/jundot/omlx)