README.md

---
base_model: Qwen/Qwen3-Coder-Next
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- mlx
- omlx
- quantized
- qwen3-next
- coding
- agentic
- apple-silicon
---

# Qwen3-Coder-Next-oQ8e

An 8-bit-equivalent quantization of [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next), produced with oMLX and intended for local inference on Apple Silicon.

## Why This Quant Exists

Qwen3-Coder-Next is not the newest coding model on the block anymore, but it remains a reliable, well-behaved workhorse for agentic coding tasks: strong tool use, long-context stability, and a favorable 3B-active / 80B-total parameter ratio that keeps it fast on consumer hardware. Rather than let a still-useful model sit on outdated quantization, it was re-quantized here with a current oMLX quantization pipeline (oQ8e) to keep it fully usable with up-to-date MLX tooling and to serve as a high-fidelity reference point against lower-bit quants.

## Model Details

- **Base model:** Qwen/Qwen3-Coder-Next (qwen3_next architecture, 80B total / 3B active parameters, 256k native context)
- **Quantization method:** oMLX, 8-bit-equivalent (oQ8e)
- **License:** Apache 2.0 (inherited from base model)
- **Format:** MLX

## Usage

```python
from mlx_lm import load, generate

model, tokenizer = load("programmer-666/Qwen3-Coder-Next-oQ8e")

prompt = "Write a quick sort algorithm."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=text, max_tokens=2048)
print(response)
```

The model can also be served through oMLX's OpenAI-compatible API endpoint for use with agentic coding tools.

## Benchmarks

All benchmarks were run with oMLX. Four variants were tested: this model (**oQ8e**) and an oQ2.7e quant, each in a default and an "Adjusted" configuration. Tests were performed on a MacBook Pro M4 Max ARM processor and 128 GB of memory.

> The "Adjusted" columns were run with tuned oMLX serving parameters (context window, sampling, and related runtime settings), rather than the server's default configuration.

### Prompt Processing Speed (tokens/s)

| Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted |
|---|---|---|---|---|
| 1,024 | 831.65 | 1232.10 | 125.90 | 1262.70 |
| 4,096 | 583.70 | 1313.25 | 1088.10 | 1334.30 |
| 8,192 | 562.50 | 1179.90 | 1109.80 | 1178.20 |
| 16,384 | 599.20 | 975.95 | 991.40 | 1021.10 |
| 32,768 | 534.90 | 847.60 | 838.70 | 801.20 |
| 65,536 | 446.70 | 594.85 | 537.50 | 476.80 |
| 131,072 | 252.20 | 342.30 | 290.70 | 350.90 |
| 200,000 | 188.90 | 259.40 | 219.40 | 222.30 |

### Generation Speed (tokens/s)

| Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted |
|---|---|---|---|---|
| 1,024 | 51.80 | 66.25 | 17.90 | 71.20 |
| 4,096 | 37.90 | 64.45 | 67.40 | 66.80 |
| 8,192 | 36.10 | 62.65 | 67.30 | 64.60 |
| 16,384 | 52.20 | 59.35 | 62.20 | 62.00 |
| 32,768 | 36.90 | 55.60 | 59.90 | 56.90 |
| 65,536 | 32.20 | 35.25 | 30.00 | 31.30 |
| 131,072 | 23.60 | 30.40 | 28.30 | 30.90 |
| 200,000 | 20.60 | 24.35 | 23.20 | 20.40 |

### Peak Memory (GB)

| Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted |
|---|---|---|---|---|
| 1,024 | 80.16 | 80.16 | 70.05 | 31.45 |
| 4,096 | 80.92 | 80.92 | 66.92 | 32.21 |
| 8,192 | 81.17 | 81.17 | 66.02 | 32.45 |
| 16,384 | 81.44 | 81.50 | 32.73 | 32.61 |
| 32,768 | 81.94 | 82.06 | 33.34 | 33.22 |
| 65,536 | 82.69 | 82.72 | 33.85 | 34.33 |
| 131,072 | 83.98 | 83.98 | 35.15 | 35.86 |
| 200,000 | 85.78 | 85.72 | 36.94 | 36.83 |

All runs used `tg128` (128 generated tokens) at each listed prompt length.

## Inference Parameters

Benchmarks and general usage were run with the following oMLX serving configuration:

| Parameter | Value |
|---|---|
| Reasoning Parser | qwen_3_coder |
| Context Window | 262,144 |
| Max Tokens | 65,536 |
| Temperature | 1 |
| Top P | 0.95 |
| Top K | 40 |
| Min P | 0 |
| Repetition Penalty | 1 |
| Presence Penalty | 0 |
| TTL | 3,600s (global default) |

## Notes

- The oQ8e quant trades memory footprint (roughly 80 to 86 GB peak) for accuracy closer to the original weights, while oQ2.7e trims memory substantially at the cost of some quality.
- Throughput at very long context (131k+) drops sharply for all variants, which is expected given the attention cost of long-context prefill.

## Acknowledgments

Thanks to the Qwen team for the base model and to the MLX / oMLX community for the tooling used to produce this quant.