docs: Tier 2 polish — variant matrix + quant trade-off
Browse files
README.md
CHANGED
|
@@ -128,3 +128,18 @@ If you want combined weight + KV cache compression, majentik hosts pre-quantized
|
|
| 128 |
- [TurboQuant paper (arXiv 2504.19874)](https://arxiv.org/abs/2504.19874)
|
| 129 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 130 |
- [Base model: aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B](https://huggingface.co/aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
- [TurboQuant paper (arXiv 2504.19874)](https://arxiv.org/abs/2504.19874)
|
| 129 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 130 |
- [Base model: aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B](https://huggingface.co/aisingapore/MERaLiON-AudioLLM-Whisper-SEA-LION-V3-10B)
|
| 131 |
+
|
| 132 |
+
## Variants in this family
|
| 133 |
+
|
| 134 |
+
(Showing 8 sibling variants under `majentik/meralion2-10b-*`. The current variant — `RotorQuant` — is **bolded**.)
|
| 135 |
+
|
| 136 |
+
| Variant | Runtime | Approx size | Use case |
|
| 137 |
+
|---|---|---|---|
|
| 138 |
+
| **RotorQuant** | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 139 |
+
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/meralion2-10b-rotorquant-mlx-2bit) | mlx-lm | ~3.2 GB | Apple Silicon, smallest |
|
| 140 |
+
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/meralion2-10b-rotorquant-mlx-4bit) | mlx-lm | ~6.2 GB | Apple Silicon balanced |
|
| 141 |
+
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/meralion2-10b-rotorquant-mlx-8bit) | mlx-lm | ~12 GB | Apple Silicon reference |
|
| 142 |
+
| [TurboQuant](https://huggingface.co/majentik/meralion2-10b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 143 |
+
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/meralion2-10b-turboquant-mlx-2bit) | mlx-lm | ~3.2 GB | Apple Silicon, smallest |
|
| 144 |
+
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/meralion2-10b-turboquant-mlx-4bit) | mlx-lm | ~6.2 GB | Apple Silicon balanced |
|
| 145 |
+
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/meralion2-10b-turboquant-mlx-8bit) | mlx-lm | ~12 GB | Apple Silicon reference |
|