Update README.md
Browse files
README.md
CHANGED
|
@@ -49,7 +49,9 @@ AWQ is an efficient, accurate and blazing-fast low-bit weight quantization metho
|
|
| 49 |
|
| 50 |
These are experimental first AWQs for the brand-new model format, Mistral.
|
| 51 |
|
| 52 |
-
|
|
|
|
|
|
|
| 53 |
|
| 54 |
<!-- description end -->
|
| 55 |
<!-- repositories-available start -->
|
|
@@ -66,7 +68,6 @@ They will not work from vLLM or TGI. They can only be used from AutoAWQ, and the
|
|
| 66 |
|
| 67 |
```
|
| 68 |
<s>[INST] {prompt} [/INST]
|
| 69 |
-
|
| 70 |
```
|
| 71 |
|
| 72 |
<!-- prompt-template end -->
|
|
|
|
| 49 |
|
| 50 |
These are experimental first AWQs for the brand-new model format, Mistral.
|
| 51 |
|
| 52 |
+
As of September 29th 2023, they are supported by AutoAWQ, and vLLM (version 0.2).
|
| 53 |
+
|
| 54 |
+
To use from AutoAW£Q requires installing both AutoAWQ and Transformers from Github. More details are below.
|
| 55 |
|
| 56 |
<!-- description end -->
|
| 57 |
<!-- repositories-available start -->
|
|
|
|
| 68 |
|
| 69 |
```
|
| 70 |
<s>[INST] {prompt} [/INST]
|
|
|
|
| 71 |
```
|
| 72 |
|
| 73 |
<!-- prompt-template end -->
|