Text Generation
MLX
Safetensors
mistral3
turboquant
kv-cache-quantization
2-bit
weight-quantization
leanstral
lean4
formal-proofs
theorem-proving
quantized
apple-silicon
mistral
Mixture of Experts
Instructions to use majentik/Leanstral-TurboQuant-MLX-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/Leanstral-TurboQuant-MLX-2bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("majentik/Leanstral-TurboQuant-MLX-2bit") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use majentik/Leanstral-TurboQuant-MLX-2bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "majentik/Leanstral-TurboQuant-MLX-2bit" --prompt "Once upon a time"
- Atomic Chat
Remove links to retired duplicate brand variants
Browse files
README.md
CHANGED
|
@@ -102,7 +102,6 @@ Leanstral excels at:
|
|
| 102 |
## See Also
|
| 103 |
|
| 104 |
- [mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) -- Base model
|
| 105 |
-
- [majentik/Leanstral-RotorQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-2bit) -- MLX 2-bit + RotorQuant
|
| 106 |
- [TurboQuant paper](https://arxiv.org/abs/2504.19874)
|
| 107 |
|
| 108 |
## Quant trade-off (MLX lane)
|
|
@@ -124,7 +123,5 @@ Leanstral excels at:
|
|
| 124 |
|
| 125 |
| Variant | Runtime | Approx size | Use case |
|
| 126 |
|---|---|---|---|
|
| 127 |
-
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
| 128 |
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|
| 129 |
| **TurboQuant-MLX-2bit** | mlx-lm | card-only | Apple Silicon, smallest |
|
| 130 |
-
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|
|
|
|
| 102 |
## See Also
|
| 103 |
|
| 104 |
- [mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) -- Base model
|
|
|
|
| 105 |
- [TurboQuant paper](https://arxiv.org/abs/2504.19874)
|
| 106 |
|
| 107 |
## Quant trade-off (MLX lane)
|
|
|
|
| 123 |
|
| 124 |
| Variant | Runtime | Approx size | Use case |
|
| 125 |
|---|---|---|---|
|
|
|
|
| 126 |
| [RotorQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|
| 127 |
| **TurboQuant-MLX-2bit** | mlx-lm | card-only | Apple Silicon, smallest |
|
|
|