docs: Tier 2 polish — variant matrix + quant trade-off
Browse files
README.md
CHANGED
|
@@ -141,3 +141,24 @@ If you want combined weight + KV cache compression, majentik hosts pre-quantized
|
|
| 141 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 142 |
- [Base model: openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)
|
| 143 |
- [gpt-oss-20b announcement](https://openai.com/blog/gpt-oss)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
- [llama-cpp-turboquant fork](https://github.com/johndpope/llama-cpp-turboquant/tree/feature/planarquant-kv-cache)
|
| 142 |
- [Base model: openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)
|
| 143 |
- [gpt-oss-20b announcement](https://openai.com/blog/gpt-oss)
|
| 144 |
+
|
| 145 |
+
## Variants in this family
|
| 146 |
+
|
| 147 |
+
(Showing 14 sibling variants under `majentik/gpt-oss-20b-*`. The current variant — `RotorQuant` — is **bolded**.)
|
| 148 |
+
|
| 149 |
+
| Variant | Runtime | Approx size | Use case |
|
| 150 |
+
|---|---|---|---|
|
| 151 |
+
| **RotorQuant** | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 152 |
+
| [RotorQuant-GGUF-IQ4_XS](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-gguf-IQ4_XS) | llama.cpp | ~17 GB | Lossy 4-bit, low-RAM CPU/edge |
|
| 153 |
+
| [RotorQuant-GGUF-Q2_K](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-gguf-Q2_K) | llama.cpp | ~12 GB | Lossy, low-RAM CPU/edge |
|
| 154 |
+
| [RotorQuant-GGUF-Q3_K_M](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-gguf-Q3_K_M) | llama.cpp | ~16 GB | Smaller 3-bit, CPU-friendly |
|
| 155 |
+
| [RotorQuant-GGUF-Q4_K_M](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-gguf-Q4_K_M) | llama.cpp | ~22 GB | Balanced default |
|
| 156 |
+
| [RotorQuant-GGUF-Q5_K_M](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-gguf-Q5_K_M) | llama.cpp | ~26 GB | Higher fidelity, more RAM |
|
| 157 |
+
| [RotorQuant-GGUF-Q8_0](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-gguf-Q8_0) | llama.cpp | ~42 GB | Near-lossless reference |
|
| 158 |
+
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-mlx-2bit) | mlx-lm | ~6.4 GB | Apple Silicon, smallest |
|
| 159 |
+
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-mlx-4bit) | mlx-lm | ~12 GB | Apple Silicon balanced |
|
| 160 |
+
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/gpt-oss-20b-rotorquant-mlx-8bit) | mlx-lm | ~24 GB | Apple Silicon reference |
|
| 161 |
+
| [TurboQuant](https://huggingface.co/majentik/gpt-oss-20b-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 162 |
+
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/gpt-oss-20b-turboquant-mlx-2bit) | mlx-lm | ~6.4 GB | Apple Silicon, smallest |
|
| 163 |
+
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/gpt-oss-20b-turboquant-mlx-4bit) | mlx-lm | ~12 GB | Apple Silicon balanced |
|
| 164 |
+
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/gpt-oss-20b-turboquant-mlx-8bit) | mlx-lm | ~24 GB | Apple Silicon reference |
|