Ornith-1.0-9B-oQ4e-mtp

This model was quantized using oQ (oMLX v0.5.1) mixed-precision quantization.

Quantization details

• Model type: qwen3_5
• Bits: 4
• Group size: 64
• Format: MLX safetensors

Environment

• Hardware: M5 MacBook Air 32GB
• Inference Framework: oMLX v0.5.1
• Settings:
  • Thinking: Disabled
  • Chat template parameter: enable_thinking=false (forced)
  • TurboQuant KV Cache: Disabled (if enable: reduces speed)
  • Native MTP: Enabled (key speed improvement)

Performance Benchmarks

Note: Results are for reference only and may vary depending on hardware, software configuration, and workload.

Single Request Results

Test	TTFT(ms)	TPOT(ms)	pp TPS	tg TPS	E2E(s)	Throughput	Peak Mem
pp1024/tg128	1886.5	25.76	542.8 tok/s	39.1 tok/s	5.183	222.3 tok/s	6.10 GB
pp4096/tg128	7142.9	27.08	573.4 tok/s	37.2 tok/s	10.637	397.1 tok/s	6.72 GB

Continuous Batching (pp1024 / tg128)

Batch	tg TPS	Speedup	pp TPS	pp TPS/req	TTFT(ms)	E2E(s)
1x	39.1 tok/s	1.00x	542.8 tok/s	542.8 tok/s	1886.5	5.183
2x	43.1 tok/s	1.10x	517.8 tok/s	258.9 tok/s	3955.0	9.893
4x	50.1 tok/s	1.28x	479.9 tok/s	120.0 tok/s	8377.5	18.754

Intelligence Benchmark

Note: Each benchmark round tests only 30 questions. Results are for reference only.

Benchmark	Accuracy	Correct	Total	Time(s)	Think
MMLU	80.0%	24	30	56.0	No
TRUTHFULQA	80.0%	24	30	33.7	No
GSM8K	93.3%	28	30	554.0	No
MATHQA	46.7%	14	30	38.4	No
HUMANEVAL	80.0%	24	30	575.7	No

Summary

Benchmark	Accuracy
MMLU	80.0%
TRUTHFULQA	80.0%
GSM8K	93.3%
MATHQA	46.7%
HUMANEVAL	80.0%
Average	76.0%