docs: Tier 2 polish — variant matrix + quant trade-off
Browse files
README.md
CHANGED
|
@@ -142,3 +142,18 @@ If you want combined weight + KV cache compression, majentik hosts pre-quantized
|
|
| 142 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 143 |
- [Base model: mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603)
|
| 144 |
- [Leanstral announcement](https://mistral.ai/news/leanstral)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 143 |
- [Base model: mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603)
|
| 144 |
- [Leanstral announcement](https://mistral.ai/news/leanstral)
|
| 145 |
+
|
| 146 |
+
## Variants in this family
|
| 147 |
+
|
| 148 |
+
(Showing 8 sibling variants under `majentik/leanstral-*`. The current variant — `RotorQuant` — is **bolded**.)
|
| 149 |
+
|
| 150 |
+
| Variant | Runtime | Approx size | Use case |
|
| 151 |
+
|---|---|---|---|
|
| 152 |
+
| **RotorQuant** | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 153 |
+
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
| 154 |
+
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-4bit) | mlx-lm | card-only | Apple Silicon balanced |
|
| 155 |
+
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|
| 156 |
+
| [TurboQuant](https://huggingface.co/majentik/leanstral-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 157 |
+
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
| 158 |
+
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-4bit) | mlx-lm | card-only | Apple Silicon balanced |
|
| 159 |
+
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|