Create README.md

README.md

@@ -0,0 +1,104 @@
+---
+base_model: Qwen/Qwen3-Coder-Next
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- mlx
+- omlx
+- quantized
+- qwen3-next
+- coding
+- agentic
+- apple-silicon
+---
+
+# Qwen3-Coder-Next-oQ8e
+
+An 8-bit-equivalent quantization of [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next), produced with oMLX and intended for local inference on Apple Silicon.
+
+## Why This Quant Exists
+
+Qwen3-Coder-Next is not the newest coding model on the block anymore, but it remains a reliable, well-behaved workhorse for agentic coding tasks: strong tool use, long-context stability, and a favorable 3B-active / 80B-total parameter ratio that keeps it fast on consumer hardware. Rather than let a still-useful model sit on outdated quantization, it was re-quantized here with a current oMLX quantization pipeline (oQ8e) to keep it fully usable with up-to-date MLX tooling and to serve as a high-fidelity reference point against lower-bit quants.
+
+## Model Details
+
+- **Base model:** Qwen/Qwen3-Coder-Next (qwen3_next architecture, 80B total / 3B active parameters, 256k native context)
+- **Quantized by:** [programmer-666](https://huggingface.co/programmer-666)
+- **Quantization method:** oMLX, 8-bit-equivalent (oQ8e)
+- **License:** Apache 2.0 (inherited from base model)
+- **Format:** MLX
+
+## Usage
+
+```python
+from mlx_lm import load, generate
+
+model, tokenizer = load("programmer-666/Qwen3-Coder-Next-oQ8e")
+
+prompt = "Write a quick sort algorithm."
+messages = [{"role": "user", "content": prompt}]
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+response = generate(model, tokenizer, prompt=text, max_tokens=2048)
+print(response)
+```
+
+The model can also be served through oMLX's OpenAI-compatible API endpoint for use with agentic coding tools.
+
+## Benchmarks
+
+All benchmarks were run with oMLX. Four variants were tested: this model (**oQ8e**) and an oQ2.7e quant, each in a default and an "Adjusted" configuration.
+
+> The "Adjusted" columns were run with tuned oMLX serving parameters (context window, sampling, and related runtime settings), rather than the server's default configuration.
+
+### Prompt Processing Speed (tokens/s)
+
+| Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted |
+|---|---|---|---|---|
+| 1,024 | 831.65 | 1232.10 | 125.90 | 1262.70 |
+| 4,096 | 583.70 | 1313.25 | 1088.10 | 1334.30 |
+| 8,192 | 562.50 | 1179.90 | 1109.80 | 1178.20 |
+| 16,384 | 599.20 | 975.95 | 991.40 | 1021.10 |
+| 32,768 | 534.90 | 847.60 | 838.70 | 801.20 |
+| 65,536 | 446.70 | 594.85 | 537.50 | 476.80 |
+| 131,072 | 252.20 | 342.30 | 290.70 | 350.90 |
+| 200,000 | 188.90 | 259.40 | 219.40 | 222.30 |
+
+### Generation Speed (tokens/s)
+
+| Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted |
+|---|---|---|---|---|
+| 1,024 | 51.80 | 66.25 | 17.90 | 71.20 |
+| 4,096 | 37.90 | 64.45 | 67.40 | 66.80 |
+| 8,192 | 36.10 | 62.65 | 67.30 | 64.60 |
+| 16,384 | 52.20 | 59.35 | 62.20 | 62.00 |
+| 32,768 | 36.90 | 55.60 | 59.90 | 56.90 |
+| 65,536 | 32.20 | 35.25 | 30.00 | 31.30 |
+| 131,072 | 23.60 | 30.40 | 28.30 | 30.90 |
+| 200,000 | 20.60 | 24.35 | 23.20 | 20.40 |
+
+### Peak Memory (GB)
+
+| Context | oQ8e | oQ8e-Adjusted | oQ2.7e | oQ2.7e-Adjusted |
+|---|---|---|---|---|
+| 1,024 | 80.16 | 80.16 | 70.05 | 31.45 |
+| 4,096 | 80.92 | 80.92 | 66.92 | 32.21 |
+| 8,192 | 81.17 | 81.17 | 66.02 | 32.45 |
+| 16,384 | 81.44 | 81.50 | 32.73 | 32.61 |
+| 32,768 | 81.94 | 82.06 | 33.34 | 33.22 |
+| 65,536 | 82.69 | 82.72 | 33.85 | 34.33 |
+| 131,072 | 83.98 | 83.98 | 35.15 | 35.86 |
+| 200,000 | 85.78 | 85.72 | 36.94 | 36.83 |
+
+All runs used `tg128` (128 generated tokens) at each listed prompt length.
+
+## Notes
+
+- The oQ8e quant trades memory footprint (roughly 80 to 86 GB peak) for accuracy closer to the original weights, while oQ2.7e trims memory substantially at the cost of some quality.
+- Throughput at very long context (131k+) drops sharply for all variants, which is expected given the attention cost of long-context prefill.
+
+## Acknowledgments
+
+Thanks to the Qwen team for the base model and to the MLX / oMLX community for the tooling used to produce this quant.