Text Generation
MLX
Safetensors
mistral3
rotorquant
kv-cache-quantization
8bit
weight-quantization
leanstral
lean4
formal-proofs
theorem-proving
quantized
apple-silicon
mistral
Mixture of Experts
8-bit precision
Instructions to use majentik/Leanstral-RotorQuant-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use majentik/Leanstral-RotorQuant-MLX-8bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("majentik/Leanstral-RotorQuant-MLX-8bit") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use majentik/Leanstral-RotorQuant-MLX-8bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "majentik/Leanstral-RotorQuant-MLX-8bit" --prompt "Once upon a time"
- Atomic Chat
Remove links to retired duplicate brand variants
Browse files
README.md
CHANGED
|
@@ -111,8 +111,6 @@ Leanstral excels at:
|
|
| 111 |
## See Also
|
| 112 |
|
| 113 |
- [mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) -- Base model
|
| 114 |
-
- [majentik/Leanstral-RotorQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-2bit) -- MLX 2-bit + RotorQuant
|
| 115 |
-
- [majentik/Leanstral-TurboQuant-MLX-8bit](https://huggingface.co/majentik/Leanstral-TurboQuant-MLX-8bit) -- MLX 8-bit + TurboQuant
|
| 116 |
- [RotorQuant GitHub](https://github.com/scrya-com/rotorquant)
|
| 117 |
- [MLX Framework](https://github.com/ml-explore/mlx)
|
| 118 |
|
|
@@ -135,7 +133,5 @@ Leanstral excels at:
|
|
| 135 |
|
| 136 |
| Variant | Runtime | Approx size | Use case |
|
| 137 |
|---|---|---|---|
|
| 138 |
-
| [RotorQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
| 139 |
| **RotorQuant-MLX-8bit** | mlx-lm | card-only | Apple Silicon reference |
|
| 140 |
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
| 141 |
-
| [TurboQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
|
|
|
|
| 111 |
## See Also
|
| 112 |
|
| 113 |
- [mistralai/Leanstral-2603](https://huggingface.co/mistralai/Leanstral-2603) -- Base model
|
|
|
|
|
|
|
| 114 |
- [RotorQuant GitHub](https://github.com/scrya-com/rotorquant)
|
| 115 |
- [MLX Framework](https://github.com/ml-explore/mlx)
|
| 116 |
|
|
|
|
| 133 |
|
| 134 |
| Variant | Runtime | Approx size | Use case |
|
| 135 |
|---|---|---|---|
|
|
|
|
| 136 |
| **RotorQuant-MLX-8bit** | mlx-lm | card-only | Apple Silicon reference |
|
| 137 |
| [TurboQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
|
|
|