Update README.md
Browse files
README.md
CHANGED
|
@@ -6,11 +6,14 @@ Quantized for the older GPTQ before it broke all the models.
|
|
| 6 |
Use with
|
| 7 |
|
| 8 |
https://github.com/Ph0rk0z/text-generation-webui-testing
|
|
|
|
| 9 |
https://github.com/Ph0rk0z/GPTQ-Merged
|
|
|
|
| 10 |
https://github.com/Curlypla/peft-GPTQ
|
| 11 |
|
| 12 |
|
| 13 |
-
|
|
|
|
| 14 |
python cuda_setup.py install inside GPTQ-Merged to compile nvidia kernel.
|
| 15 |
|
| 16 |
|
|
@@ -18,5 +21,6 @@ python server.py --cai-chat --gptq-bits 4 --model opt-6b --autograd
|
|
| 18 |
|
| 19 |
|
| 20 |
Don't forget to get configs from: https://huggingface.co/facebook/opt-6.7b/tree/main
|
| 21 |
-
|
|
|
|
| 22 |
|
|
|
|
| 6 |
Use with
|
| 7 |
|
| 8 |
https://github.com/Ph0rk0z/text-generation-webui-testing
|
| 9 |
+
|
| 10 |
https://github.com/Ph0rk0z/GPTQ-Merged
|
| 11 |
+
|
| 12 |
https://github.com/Curlypla/peft-GPTQ
|
| 13 |
|
| 14 |
|
| 15 |
+
Clone the 2 repos into text-generation-webui-testing/repositories
|
| 16 |
+
|
| 17 |
python cuda_setup.py install inside GPTQ-Merged to compile nvidia kernel.
|
| 18 |
|
| 19 |
|
|
|
|
| 21 |
|
| 22 |
|
| 23 |
Don't forget to get configs from: https://huggingface.co/facebook/opt-6.7b/tree/main
|
| 24 |
+
|
| 25 |
+
You only need the json files, don't forget merges.txt
|
| 26 |
|