No longer available on HF due to storage restrictions: archived here
See MiniMax-M2.7 MLX in action: demonstration video
Tested on a M3 Ultra 512GB RAM using Inferencer app
- Single inference ~30 tokens/s @ 1000 tokens (debug build)
- Batched inference ~ total tokens/s across two inferences
- Memory usage: ~240 GiB
Q9 achieves near lossless accuracy in our coding test
| Quantization (bpw) | Perplexity | Token Accuracy | Missed Divergence |
|---|---|---|---|
| Q4.5 | 1.27343 | 92.40% | 24.73% |
| Q6-INF | 1.20312 | 97.40% | 13.92% |
| Q6.5 | 1.21093 | 96.85% | 11.74% |
| Q9 | 1.20312 | 97.50% | 9.95% |
| Base | 1.20312 | 100.0% | 0.000% |
- Perplexity: Measures the confidence for predicting base tokens (lower is better)
- Token Accuracy: The percentage of correctly generated base tokens
- Missed Divergence: Measures severity of misses; how much the token was missed by
Quantized with a modified version of MLX
For more details see demonstration video or visit MiniMaxAI/MiniMax-M2.7.
Model tree for inferencerlabs/MiniMax-M2.7-MLX-Q9
Base model
MiniMaxAI/MiniMax-M2.7