Update README.md
Browse files
README.md
CHANGED
|
@@ -11,17 +11,17 @@ pipeline_tag: text-generation
|
|
| 11 |
|
| 12 |
project original [source](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) (base model)
|
| 13 |
|
| 14 |
-
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
|
| 27 |
|
|
|
|
| 11 |
|
| 12 |
project original [source](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) (base model)
|
| 13 |
|
| 14 |
+
Q_2 (not nice)
|
| 15 |
|
| 16 |
+
Q_3 (acceptable)
|
| 17 |
|
| 18 |
+
Q_4 family is recommanded (good for running with CPU as well)
|
| 19 |
|
| 20 |
+
Q_5 (good in general)
|
| 21 |
|
| 22 |
+
Q_6 is good also; if you want a better result; take this one instead of Q_5
|
| 23 |
|
| 24 |
+
Q_8 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
|
| 25 |
|
| 26 |
16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
|
| 27 |
|