docs: Tier 2 polish — variant matrix + quant trade-off
Browse files
README.md
CHANGED
|
@@ -107,3 +107,31 @@ MERaLiON-2 supports speech recognition in Southeast Asian languages including En
|
|
| 107 |
- [majentik/MERaLiON-2-10B-RotorQuant-MLX-8bit](https://huggingface.co/majentik/MERaLiON-2-10B-RotorQuant-MLX-8bit) -- RotorQuant 8-bit (higher quality)
|
| 108 |
- [majentik/MERaLiON-2-10B-RotorQuant-MLX-2bit](https://huggingface.co/majentik/MERaLiON-2-10B-RotorQuant-MLX-2bit) -- RotorQuant 2-bit (smaller)
|
| 109 |
- [aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B](https://huggingface.co/aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B) -- Original base model
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
- [majentik/MERaLiON-2-10B-RotorQuant-MLX-8bit](https://huggingface.co/majentik/MERaLiON-2-10B-RotorQuant-MLX-8bit) -- RotorQuant 8-bit (higher quality)
|
| 108 |
- [majentik/MERaLiON-2-10B-RotorQuant-MLX-2bit](https://huggingface.co/majentik/MERaLiON-2-10B-RotorQuant-MLX-2bit) -- RotorQuant 2-bit (smaller)
|
| 109 |
- [aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B](https://huggingface.co/aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B) -- Original base model
|
| 110 |
+
|
| 111 |
+
## Quant trade-off (MLX lane)
|
| 112 |
+
|
| 113 |
+
| Bits | Approx size | Use case | Recommendation |
|
| 114 |
+
|---|---|---|---|
|
| 115 |
+
| 2-bit | ~2.6 GB | Aggressive quantization | Very low-RAM Macs |
|
| 116 |
+
| 3-bit | ~3.6 GB | Lossy but small | Low-RAM Macs |
|
| 117 |
+
| **4-bit** | ~4.2 GB | Balanced default | **Recommended for most Macs** |
|
| 118 |
+
| 5-bit | ~5.0 GB | Higher fidelity | Quality-sensitive |
|
| 119 |
+
| 6-bit | ~6.0 GB | Approaching FP16 quality | High-fidelity |
|
| 120 |
+
| 8-bit | ~7.6 GB | Near-lossless reference | Fidelity-critical work |
|
| 121 |
+
|
| 122 |
+
(Current variant — **4bit** — is bolded.)
|
| 123 |
+
|
| 124 |
+
## Variants in this family
|
| 125 |
+
|
| 126 |
+
(Showing 8 sibling variants under `majentik/meralion2-10b-*`. The current variant — `RotorQuant-MLX-4bit` — is **bolded**.)
|
| 127 |
+
|
| 128 |
+
| Variant | Runtime | Approx size | Use case |
|
| 129 |
+
|---|---|---|---|
|
| 130 |
+
| [RotorQuant](https://huggingface.co/majentik/meralion2-10b-rotorquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 131 |
+
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/meralion2-10b-rotorquant-mlx-2bit) | mlx-lm | ~3.2 GB | Apple Silicon, smallest |
|
| 132 |
+
| **RotorQuant-MLX-4bit** | mlx-lm | ~6.2 GB | Apple Silicon balanced |
|
| 133 |
+
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/meralion2-10b-rotorquant-mlx-8bit) | mlx-lm | ~12 GB | Apple Silicon reference |
|
| 134 |
+
| [TurboQuant](https://huggingface.co/majentik/meralion2-10b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 135 |
+
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/meralion2-10b-turboquant-mlx-2bit) | mlx-lm | ~3.2 GB | Apple Silicon, smallest |
|
| 136 |
+
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/meralion2-10b-turboquant-mlx-4bit) | mlx-lm | ~6.2 GB | Apple Silicon balanced |
|
| 137 |
+
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/meralion2-10b-turboquant-mlx-8bit) | mlx-lm | ~12 GB | Apple Silicon reference |
|