docs: Tier 2 polish — variant matrix + quant trade-off
Browse files
README.md
CHANGED
|
@@ -140,3 +140,26 @@ If you want combined weight + KV cache compression, majentik hosts pre-quantized
|
|
| 140 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 141 |
- [Base model: Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B)
|
| 142 |
- [Qwen3.5-27B announcement](https://qwen.ai/blog?id=qwen3.5)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 141 |
- [Base model: Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B)
|
| 142 |
- [Qwen3.5-27B announcement](https://qwen.ai/blog?id=qwen3.5)
|
| 143 |
+
|
| 144 |
+
## Variants in this family
|
| 145 |
+
|
| 146 |
+
(Showing 16 sibling variants under `majentik/qwen3.5-27b-*`. The current variant — `RotorQuant` — is **bolded**.)
|
| 147 |
+
|
| 148 |
+
| Variant | Runtime | Approx size | Use case |
|
| 149 |
+
|---|---|---|---|
|
| 150 |
+
| **RotorQuant** | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 151 |
+
| [RotorQuant-2bit](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-2bit) | transformers | n/a | Standalone 2-bit weights |
|
| 152 |
+
| [RotorQuant-GGUF-IQ4_XS](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-gguf-IQ4_XS) | llama.cpp | ~23 GB | Lossy 4-bit, low-RAM CPU/edge |
|
| 153 |
+
| [RotorQuant-GGUF-Q2_K](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-gguf-Q2_K) | llama.cpp | ~16 GB | Lossy, low-RAM CPU/edge |
|
| 154 |
+
| [RotorQuant-GGUF-Q3_K_M](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-gguf-Q3_K_M) | llama.cpp | ~21 GB | Smaller 3-bit, CPU-friendly |
|
| 155 |
+
| [RotorQuant-GGUF-Q4_K_M](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-gguf-Q4_K_M) | llama.cpp | ~30 GB | Balanced default |
|
| 156 |
+
| [RotorQuant-GGUF-Q5_K_M](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-gguf-Q5_K_M) | llama.cpp | ~36 GB | Higher fidelity, more RAM |
|
| 157 |
+
| [RotorQuant-GGUF-Q8_0](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-gguf-Q8_0) | llama.cpp | ~57 GB | Near-lossless reference |
|
| 158 |
+
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-mlx-2bit) | mlx-lm | ~8.6 GB | Apple Silicon, smallest |
|
| 159 |
+
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-mlx-4bit) | mlx-lm | ~17 GB | Apple Silicon balanced |
|
| 160 |
+
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/qwen3.5-27b-rotorquant-mlx-8bit) | mlx-lm | ~32 GB | Apple Silicon reference |
|
| 161 |
+
| [TurboQuant](https://huggingface.co/majentik/qwen3.5-27b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 162 |
+
| [TurboQuant-2bit](https://huggingface.co/majentik/qwen3.5-27b-turboquant-2bit) | transformers | n/a | Standalone 2-bit weights |
|
| 163 |
+
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/qwen3.5-27b-turboquant-mlx-2bit) | mlx-lm | ~8.6 GB | Apple Silicon, smallest |
|
| 164 |
+
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/qwen3.5-27b-turboquant-mlx-4bit) | mlx-lm | ~17 GB | Apple Silicon balanced |
|
| 165 |
+
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/qwen3.5-27b-turboquant-mlx-8bit) | mlx-lm | ~32 GB | Apple Silicon reference |
|