| --- |
| base_model: Qwen/Qwen3-Coder-Next |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - mlx |
| - omlx |
| - quantized |
| - qwen3-next |
| - coding |
| - agentic |
| - apple-silicon |
| --- |
| |
| # Qwen3-Coder-Next-oQ8e |
|
|
| An 8-bit-equivalent quantization of [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next), produced with oMLX and intended for local inference on Apple Silicon. |
|
|
| ## Why This Quant Exists |
|
|
| Qwen3-Coder-Next is not the newest coding model on the block anymore, but it remains a reliable, well-behaved workhorse for agentic coding tasks: strong tool use, long-context stability, and a favorable 3B-active / 80B-total parameter ratio that keeps it fast on consumer hardware. Rather than let a still-useful model sit on outdated quantization, it was re-quantized here with a current oMLX quantization pipeline (oQ8e) to keep it fully usable with up-to-date MLX tooling and to serve as a high-fidelity reference point against lower-bit quants. |
|
|
| ## Model Details |
|
|
| - **Base model:** Qwen/Qwen3-Coder-Next (qwen3_next architecture, 80B total / 3B active parameters, 256k native context) |
| - **Quantization method:** oMLX, 8-bit-equivalent (oQ8e) |
| - **License:** Apache 2.0 (inherited from base model) |
| - **Format:** MLX |
| |
| ## Usage |
| |
| ```python |
| from mlx_lm import load, generate |
|
|
| model, tokenizer = load("programmer-666/Qwen3-Coder-Next-oQ8e") |
|
|
| prompt = "Write a quick sort algorithm." |
| messages = [{"role": "user", "content": prompt}] |
| text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
|
|
| response = generate(model, tokenizer, prompt=text, max_tokens=2048) |
| print(response) |
| ``` |
| |
| The model can also be served through oMLX's OpenAI-compatible API endpoint for use with agentic coding tools. |
| |
| ## Benchmarks |
| |
| All benchmarks were run with oMLX. Four variants were tested: this model (**oQ8e**) and an oQ2.7e quant, each in a default and an "Adjusted" configuration. Tests were performed on a MacBook Pro M4 Max ARM processor and 128 GB of memory. |
| |
| > The "Adjusted" columns were run with tuned oMLX serving parameters (context window, sampling, and related runtime settings), rather than the server's default configuration. |
| |
| ### Prompt Processing Speed (tokens/s) |
| |
| | Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted | |
| |---|---|---|---|---| |
| | 1,024 | 831.65 | 1232.10 | 125.90 | 1262.70 | |
| | 4,096 | 583.70 | 1313.25 | 1088.10 | 1334.30 | |
| | 8,192 | 562.50 | 1179.90 | 1109.80 | 1178.20 | |
| | 16,384 | 599.20 | 975.95 | 991.40 | 1021.10 | |
| | 32,768 | 534.90 | 847.60 | 838.70 | 801.20 | |
| | 65,536 | 446.70 | 594.85 | 537.50 | 476.80 | |
| | 131,072 | 252.20 | 342.30 | 290.70 | 350.90 | |
| | 200,000 | 188.90 | 259.40 | 219.40 | 222.30 | |
| |
| ### Generation Speed (tokens/s) |
| |
| | Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted | |
| |---|---|---|---|---| |
| | 1,024 | 51.80 | 66.25 | 17.90 | 71.20 | |
| | 4,096 | 37.90 | 64.45 | 67.40 | 66.80 | |
| | 8,192 | 36.10 | 62.65 | 67.30 | 64.60 | |
| | 16,384 | 52.20 | 59.35 | 62.20 | 62.00 | |
| | 32,768 | 36.90 | 55.60 | 59.90 | 56.90 | |
| | 65,536 | 32.20 | 35.25 | 30.00 | 31.30 | |
| | 131,072 | 23.60 | 30.40 | 28.30 | 30.90 | |
| | 200,000 | 20.60 | 24.35 | 23.20 | 20.40 | |
| |
| ### Peak Memory (GB) |
| |
| | Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted | |
| |---|---|---|---|---| |
| | 1,024 | 80.16 | 80.16 | 70.05 | 31.45 | |
| | 4,096 | 80.92 | 80.92 | 66.92 | 32.21 | |
| | 8,192 | 81.17 | 81.17 | 66.02 | 32.45 | |
| | 16,384 | 81.44 | 81.50 | 32.73 | 32.61 | |
| | 32,768 | 81.94 | 82.06 | 33.34 | 33.22 | |
| | 65,536 | 82.69 | 82.72 | 33.85 | 34.33 | |
| | 131,072 | 83.98 | 83.98 | 35.15 | 35.86 | |
| | 200,000 | 85.78 | 85.72 | 36.94 | 36.83 | |
| |
| All runs used `tg128` (128 generated tokens) at each listed prompt length. |
| |
| ## Inference Parameters |
| |
| Benchmarks and general usage were run with the following oMLX serving configuration: |
| |
| | Parameter | Value | |
| |---|---| |
| | Reasoning Parser | qwen_3_coder | |
| | Context Window | 262,144 | |
| | Max Tokens | 65,536 | |
| | Temperature | 1 | |
| | Top P | 0.95 | |
| | Top K | 40 | |
| | Min P | 0 | |
| | Repetition Penalty | 1 | |
| | Presence Penalty | 0 | |
| | TTL | 3,600s (global default) | |
| |
| ## Notes |
| |
| - The oQ8e quant trades memory footprint (roughly 80 to 86 GB peak) for accuracy closer to the original weights, while oQ2.7e trims memory substantially at the cost of some quality. |
| - Throughput at very long context (131k+) drops sharply for all variants, which is expected given the attention cost of long-context prefill. |
| |
| ## Acknowledgments |
| |
| Thanks to the Qwen team for the base model and to the MLX / oMLX community for the tooling used to produce this quant. |
