Update README.md
Browse files
README.md
CHANGED
|
@@ -23,17 +23,13 @@ Default arguments used except when the bits per weight is above 6.0, at that poi
|
|
| 23 |
|
| 24 |
Original model: https://huggingface.co/rombodawg/Everyone-Coder-33b-v2-Base
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
<a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0">3.0 bits per weight</a>
|
| 34 |
-
|
| 35 |
-
<a href="https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/2_4">2.4 bits per weight</a>
|
| 36 |
-
|
| 37 |
|
| 38 |
## Download instructions
|
| 39 |
|
|
|
|
| 23 |
|
| 24 |
Original model: https://huggingface.co/rombodawg/Everyone-Coder-33b-v2-Base
|
| 25 |
|
| 26 |
+
| Branch | Bits | lm_head bits | VRAM (4k) | VRAM (16k) | VRAM (32k) | Description |
|
| 27 |
+
| ------ | ---- | ------------ | ---- | ---- | ---- | ----------- |
|
| 28 |
+
| [6_5](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/6_5) | 6.5 | 8.0 | 28.9 GB | 31.6 GB | 35.6 GB | Near unquantized performance at vastly reduced size, **recommended**. |
|
| 29 |
+
| [4_25](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/4_25) | 4.25 | 6.0 | 19.5 GB | 22.2 GB | 26.2 GB | GPTQ equivalent bits per weight, slightly higher quality. |
|
| 30 |
+
| [3_5](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_5) | 3.5 | 6.0 | 16.5 GB | 19.2 GB | 23.2 GB | Lower quality, only use if you have to. |
|
| 31 |
+
| [3_0](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0) | 3.0 | 6.0 | 14.3 GB | 17.0 GB | 21.0 GB | Very low quality, usable with 16gb of VRAM. |
|
| 32 |
+
| [2_4](https://huggingface.co/bartowski/Everyone-Coder-33b-v2-Base-exl2/tree/3_0) | 2.4 | 6.0 | 12.0 GB | 14.7 GB | 18.7 GB | Extremely low quality, only recommended if trying to fit into 12GB with under 4k context. |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
## Download instructions
|
| 35 |
|