MLX
Safetensors
mistral3
rotorquant
kv-cache-quantization
2-bit
weight-quantization
leanstral
lean4
formal-proofs
theorem-proving
quantized
apple-silicon
mistral
Mixture of Experts
Instructions to use majentik/Leanstral-RotorQuant-MLX-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/Leanstral-RotorQuant-MLX-2bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Leanstral-RotorQuant-MLX-2bit majentik/Leanstral-RotorQuant-MLX-2bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- LM Studio
docs: Tier 2 polish — variant matrix + quant trade-off
Browse files
README.md
CHANGED
|
@@ -2,23 +2,20 @@
|
|
| 2 |
base_model: mistralai/Leanstral-2603
|
| 3 |
library_name: mlx
|
| 4 |
tags:
|
| 5 |
-
- rotorquant
|
| 6 |
-
- kv-cache-quantization
|
| 7 |
-
- mlx
|
| 8 |
-
- 2-bit
|
| 9 |
-
- weight-quantization
|
| 10 |
-
- leanstral
|
| 11 |
-
- lean4
|
| 12 |
-
- formal-proofs
|
| 13 |
-
- theorem-proving
|
| 14 |
-
- quantized
|
| 15 |
-
- apple-silicon
|
| 16 |
-
- mistral
|
| 17 |
-
- moe
|
| 18 |
license: apache-2.0
|
| 19 |
-
pipeline_tag: text-generation
|
| 20 |
-
language:
|
| 21 |
-
- en
|
| 22 |
---
|
| 23 |
|
| 24 |
# Leanstral-RotorQuant-MLX-2bit
|
|
@@ -95,3 +92,31 @@ Leanstral excels at:
|
|
| 95 |
- [majentik/Leanstral-RotorQuant-MLX-1bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-1bit) -- MLX 1-bit + RotorQuant
|
| 96 |
- [majentik/Leanstral-TurboQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-TurboQuant-MLX-2bit) -- MLX 2-bit + TurboQuant
|
| 97 |
- [RotorQuant repository](https://github.com/scrya-com/rotorquant)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
base_model: mistralai/Leanstral-2603
|
| 3 |
library_name: mlx
|
| 4 |
tags:
|
| 5 |
+
- rotorquant
|
| 6 |
+
- kv-cache-quantization
|
| 7 |
+
- mlx
|
| 8 |
+
- 2-bit
|
| 9 |
+
- weight-quantization
|
| 10 |
+
- leanstral
|
| 11 |
+
- lean4
|
| 12 |
+
- formal-proofs
|
| 13 |
+
- theorem-proving
|
| 14 |
+
- quantized
|
| 15 |
+
- apple-silicon
|
| 16 |
+
- mistral
|
| 17 |
+
- moe
|
| 18 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
| 19 |
---
|
| 20 |
|
| 21 |
# Leanstral-RotorQuant-MLX-2bit
|
|
|
|
| 92 |
- [majentik/Leanstral-RotorQuant-MLX-1bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-1bit) -- MLX 1-bit + RotorQuant
|
| 93 |
- [majentik/Leanstral-TurboQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-TurboQuant-MLX-2bit) -- MLX 2-bit + TurboQuant
|
| 94 |
- [RotorQuant repository](https://github.com/scrya-com/rotorquant)
|
| 95 |
+
|
| 96 |
+
## Quant trade-off (MLX lane)
|
| 97 |
+
|
| 98 |
+
| Bits | Approx size | Use case | Recommendation |
|
| 99 |
+
|---|---|---|---|
|
| 100 |
+
| **2-bit** | ~31 GB | Aggressive quantization | **Very low-RAM Macs** |
|
| 101 |
+
| 3-bit | ~43 GB | Lossy but small | Low-RAM Macs |
|
| 102 |
+
| 4-bit | ~50 GB | Balanced default | Recommended for most Macs |
|
| 103 |
+
| 5-bit | ~60 GB | Higher fidelity | Quality-sensitive |
|
| 104 |
+
| 6-bit | ~71 GB | Approaching FP16 quality | High-fidelity |
|
| 105 |
+
| 8-bit | ~90 GB | Near-lossless reference | Fidelity-critical work |
|
| 106 |
+
|
| 107 |
+
(Current variant — **2bit** — is bolded.)
|
| 108 |
+
|
| 109 |
+
## Variants in this family
|
| 110 |
+
|
| 111 |
+
(Showing 8 sibling variants under `majentik/leanstral-*`. The current variant — `RotorQuant-MLX-2bit` — is **bolded**.)
|
| 112 |
+
|
| 113 |
+
| Variant | Runtime | Approx size | Use case |
|
| 114 |
+
|---|---|---|---|
|
| 115 |
+
| [RotorQuant](https://huggingface.co/majentik/leanstral-rotorquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 116 |
+
| **RotorQuant-MLX-2bit** | mlx-lm | card-only | Apple Silicon, smallest |
|
| 117 |
+
| [RotorQuant-MLX-4bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-4bit) | mlx-lm | card-only | Apple Silicon balanced |
|
| 118 |
+
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|
| 119 |
+
| [TurboQuant](https://huggingface.co/majentik/leanstral-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
|
| 120 |
+
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
| 121 |
+
| [TurboQuant-MLX-4bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-4bit) | mlx-lm | card-only | Apple Silicon balanced |
|
| 122 |
+
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|