Qwopus3.5-4B-Coder-oQ4-mtp
This model was quantized using oQ (oMLX v0.4.4) mixed-precision quantization.
Quantization details
- Model type: qwen3_5
- Bits: 4
- Group size: 64
- Format: MLX safetensors
Benchmark Comparison
Hardware: M5 MacBook Air 32GB
Speed Benchmark
| Model | pp1024/tg128 (tok/s) | pp4096/tg128 (tok/s) | Peak Mem |
|---|---|---|---|
| Qwopus3.5-4B-Coder-oQ4-mtp | 63.4 | 57.0 | 3.51 GB |
| Qwopus3.5-9B-Coder-oQ4-mtp | 38.1 | 35.4 | 5.96 GB |
| Qwopus3.5-4B-Coder-oQ3-mtp | 50.3 | 47.3 | 3.04 GB |
Intelligence Benchmark (No Thinking, 30 samples)
| Model | MMLU | TruthfulQA | GSM8K | MathQA | HumanEval | Average |
|---|---|---|---|---|---|---|
| Qwopus3.5-4B-Coder-oQ4-mtp | 63.3% | 66.7% | 96.7% | 43.3% | 83.3% | 70.7% |
| Qwopus3.5-9B-Coder-oQ4-mtp | 80.0% | 86.7% | 83.3% | 40.0% | 80.0% | 74.0% |
⚠️ Note: Intelligence benchmarks use only 30 samples per task. Results may have randomness and should be used for reference only.
Speed Details
Qwopus3.5-4B-Coder-oQ4-mtp + MTP
Single Request Results
| Test | TTFT(ms) | TPOT(ms) | pp TPS | tg TPS | E2E(s) | Peak Mem |
|---|---|---|---|---|---|---|
| pp1024/tg128 | 1053.1 | 15.90 | 972.4 tok/s | 63.4 tok/s | 3.072 | 3.51 GB |
| pp4096/tg128 | 3542.2 | 17.67 | 1156.3 tok/s | 57.0 tok/s | 5.787 | 4.12 GB |
Continuous Batching (pp1024 / tg128)
| Batch | tg TPS | Speedup | pp TPS | TTFT(ms) | E2E(s) |
|---|---|---|---|---|---|
| 1x | 63.4 tok/s | 1.00x | 972.4 tok/s | 1053.1 | 3.072 |
| 2x | 86.3 tok/s | 1.36x | 1087.1 tok/s | 1883.7 | 4.850 |
| 4x | 101.1 tok/s | 1.59x | 1068.6 tok/s | 3701.2 | 8.895 |
Qwopus3.5-9B-Coder-oQ4-mtp + MTP
Single Request Results
| Test | TTFT(ms) | TPOT(ms) | pp TPS | tg TPS | E2E(s) | Peak Mem |
|---|---|---|---|---|---|---|
| pp1024/tg128 | 1735.2 | 26.43 | 590.1 tok/s | 38.1 tok/s | 5.092 | 5.96 GB |
| pp4096/tg128 | 6142.9 | 28.49 | 666.8 tok/s | 35.4 tok/s | 9.761 | 6.57 GB |
Continuous Batching (pp1024 / tg128)
| Batch | tg TPS | Speedup | pp TPS | TTFT(ms) | E2E(s) |
|---|---|---|---|---|---|
| 1x | 38.1 tok/s | 1.00x | 590.1 tok/s | 1735.2 | 5.092 |
| 2x | 48.4 tok/s | 1.27x | 606.6 tok/s | 3376.0 | 8.662 |
| 4x | 53.2 tok/s | 1.40x | 578.0 tok/s | 6889.9 | 16.712 |
Qwopus3.5-4B-Coder-oQ3-mtp
Single Request Results
| Test | TTFT(ms) | TPOT(ms) | pp TPS | tg TPS | E2E(s) | Peak Mem |
|---|---|---|---|---|---|---|
| pp1024/tg128 | 1041.3 | 20.05 | 983.4 tok/s | 50.3 tok/s | 3.587 | 3.04 GB |
| pp4096/tg128 | 3560.7 | 21.30 | 1150.3 tok/s | 47.3 tok/s | 6.266 | 3.65 GB |
Continuous Batching (pp1024 / tg128)
| Batch | tg TPS | Speedup | pp TPS | TTFT(ms) | E2E(s) |
|---|---|---|---|---|---|
| 1x | 50.3 tok/s | 1.00x | 983.4 tok/s | 1041.3 | 3.587 |
| 2x | 87.9 tok/s | 1.75x | 1076.9 tok/s | 1901.7 | 4.815 |
| 4x | 90.1 tok/s | 1.79x | 1066.3 tok/s | 3708.0 | 9.525 |
Intelligence Details
Qwopus3.5-4B-Coder-oQ4-mtp
| Benchmark | Accuracy | Correct | Total | Time(s) | Think |
|---|---|---|---|---|---|
| MMLU | 63.3% | 19 | 30 | 19.3 | No |
| TruthfulQA | 66.7% | 20 | 30 | 9.9 | No |
| GSM8K | 96.7% | 29 | 30 | 81.2 | No |
| MathQA | 43.3% | 13 | 30 | 12 | No |
| HumanEval | 83.3% | 25 | 30 | 77.5 | No |
Qwopus3.5-9B-Coder-oQ4-mtp
| Benchmark | Accuracy | Correct | Total | Time(s) | Think |
|---|---|---|---|---|---|
| MMLU | 80.0% | 24 | 30 | 57.1 | No |
| TruthfulQA | 86.7% | 26 | 30 | 24.4 | No |
| GSM8K | 83.3% | 25 | 30 | 355.2 | No |
| MathQA | 40.0% | 12 | 30 | 25.6 | No |
| HumanEval | 80.0% | 24 | 30 | 157.7 | No |
- Downloads last month
- 114
Model size
1B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support