Benchmarks
| Mean retention, 12 benchmarks | |
|---|---|
| Mach-1 Small | 95.0% |
| Ternary Bonsai 27B (PrismML) | 93.6% |
| Gemma 4 Q2_K_XL (Unsloth) | 85.6% |
Per-benchmark Retention
| Benchmark | Mach-1 Small | Ternary Bonsai 27B | Gemma 4 Q2_K_XL |
|---|---|---|---|
| AIME26 | 99.5% | 92.7% | 67.7% |
| MATH-500 | 99.4% | 98.2% | 95.6% |
| AIME25 | 99.1% | 91.7% | 67.2% |
| GSM8K | 98.4% | 100.2% | 97.3% |
| MBPP+ | 98.3% | 98.4% | 92.2% |
| HumanEval+ | 97.4% | 98.7% | 94.1% |
| MMLU-Redux | 96.2% | 94.0% | 96.9% |
| IFEval | 94.0% | 89.8% | 95.5% |
| MuSR | 92.7% | 91.6% | 91.1% |
| BFCL-v3 | 92.0% | 98.9% | 95.7% |
| τ²-bench | 90.0% | 91.2% | 73.1% |
| IFBench | 83.2% | 77.7% | 61.3% |
| Mean | 95.0% | 93.6% | 85.6% |
Mach-1 Small's own scores and teacher scores:
| Benchmark | Score | Teacher (Qwen3.6-35B-A3B BF16) | Retention |
|---|---|---|---|
| AIME26 | 89.58 | 90.00 | 99.5% |
| MATH-500 | 98.00 | 98.60 | 99.4% |
| AIME25 | 87.50 | 88.33 | 99.1% |
| GSM8K | 94.69 | 96.21 | 98.4% |
| MBPP+ | 94.44 | 96.03 | 98.3% |
| HumanEval+ | 92.68 | 95.12 | 97.4% |
| MMLU-Redux | 89.18 | 92.68 | 96.2% |
| IFEval | 83.75 | 89.05 | 94.0% |
| MuSR | 61.77 | 66.66 | 92.7% |
| BFCL-v3 | 68.97 | 74.98 | 92.0% |
| τ²-bench | 71.58 | 79.51 | 90.0% |
| IFBench | 54.08 | 64.97 | 83.2% |
Speed
