majentik commited on
Commit
9bd62b3
·
verified ·
1 Parent(s): cd06d72

docs: Tier 2 polish — variant matrix + quant trade-off

Browse files
Files changed (1) hide show
  1. README.md +41 -16
README.md CHANGED
@@ -2,23 +2,20 @@
2
  base_model: mistralai/Leanstral-2603
3
  library_name: mlx
4
  tags:
5
- - rotorquant
6
- - kv-cache-quantization
7
- - mlx
8
- - 2-bit
9
- - weight-quantization
10
- - leanstral
11
- - lean4
12
- - formal-proofs
13
- - theorem-proving
14
- - quantized
15
- - apple-silicon
16
- - mistral
17
- - moe
18
  license: apache-2.0
19
- pipeline_tag: text-generation
20
- language:
21
- - en
22
  ---
23
 
24
  # Leanstral-RotorQuant-MLX-2bit
@@ -95,3 +92,31 @@ Leanstral excels at:
95
  - [majentik/Leanstral-RotorQuant-MLX-1bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-1bit) -- MLX 1-bit + RotorQuant
96
  - [majentik/Leanstral-TurboQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-TurboQuant-MLX-2bit) -- MLX 2-bit + TurboQuant
97
  - [RotorQuant repository](https://github.com/scrya-com/rotorquant)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  base_model: mistralai/Leanstral-2603
3
  library_name: mlx
4
  tags:
5
+ - rotorquant
6
+ - kv-cache-quantization
7
+ - mlx
8
+ - 2-bit
9
+ - weight-quantization
10
+ - leanstral
11
+ - lean4
12
+ - formal-proofs
13
+ - theorem-proving
14
+ - quantized
15
+ - apple-silicon
16
+ - mistral
17
+ - moe
18
  license: apache-2.0
 
 
 
19
  ---
20
 
21
  # Leanstral-RotorQuant-MLX-2bit
 
92
  - [majentik/Leanstral-RotorQuant-MLX-1bit](https://huggingface.co/majentik/Leanstral-RotorQuant-MLX-1bit) -- MLX 1-bit + RotorQuant
93
  - [majentik/Leanstral-TurboQuant-MLX-2bit](https://huggingface.co/majentik/Leanstral-TurboQuant-MLX-2bit) -- MLX 2-bit + TurboQuant
94
  - [RotorQuant repository](https://github.com/scrya-com/rotorquant)
95
+
96
+ ## Quant trade-off (MLX lane)
97
+
98
+ | Bits | Approx size | Use case | Recommendation |
99
+ |---|---|---|---|
100
+ | **2-bit** | ~31 GB | Aggressive quantization | **Very low-RAM Macs** |
101
+ | 3-bit | ~43 GB | Lossy but small | Low-RAM Macs |
102
+ | 4-bit | ~50 GB | Balanced default | Recommended for most Macs |
103
+ | 5-bit | ~60 GB | Higher fidelity | Quality-sensitive |
104
+ | 6-bit | ~71 GB | Approaching FP16 quality | High-fidelity |
105
+ | 8-bit | ~90 GB | Near-lossless reference | Fidelity-critical work |
106
+
107
+ (Current variant — **2bit** — is bolded.)
108
+
109
+ ## Variants in this family
110
+
111
+ (Showing 8 sibling variants under `majentik/leanstral-*`. The current variant — `RotorQuant-MLX-2bit` — is **bolded**.)
112
+
113
+ | Variant | Runtime | Approx size | Use case |
114
+ |---|---|---|---|
115
+ | [RotorQuant](https://huggingface.co/majentik/leanstral-rotorquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
116
+ | **RotorQuant-MLX-2bit** | mlx-lm | card-only | Apple Silicon, smallest |
117
+ | [RotorQuant-MLX-4bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-4bit) | mlx-lm | card-only | Apple Silicon balanced |
118
+ | [RotorQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-rotorquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |
119
+ | [TurboQuant](https://huggingface.co/majentik/leanstral-turboquant) | runtime modifier | n/a | KV-cache root (weight-agnostic) |
120
+ | [TurboQuant-MLX-2bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-2bit) | mlx-lm | card-only | Apple Silicon, smallest |
121
+ | [TurboQuant-MLX-4bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-4bit) | mlx-lm | card-only | Apple Silicon balanced |
122
+ | [TurboQuant-MLX-8bit](https://huggingface.co/majentik/leanstral-turboquant-mlx-8bit) | mlx-lm | card-only | Apple Silicon reference |