Update README.md
Browse files
README.md
CHANGED
|
@@ -6,5 +6,5 @@ This repo includes a Version of Phi-3 that was quantized to AWQ using AutoAWQ.
|
|
| 6 |
Currently hosting via the TGI docker image fails due to its fallback on AutoModel and that not being compatible with AWQ.
|
| 7 |
Hosting on vLLM is recommended.
|
| 8 |
|
| 9 |
-
To run the model you
|
| 10 |
While the remote code comes from microsoft (see LICENSE information in the file) you should validate the code yourself before deployment.
|
|
|
|
| 6 |
Currently hosting via the TGI docker image fails due to its fallback on AutoModel and that not being compatible with AWQ.
|
| 7 |
Hosting on vLLM is recommended.
|
| 8 |
|
| 9 |
+
To run the model you need to set the trust-remote-code (or similar) flag.
|
| 10 |
While the remote code comes from microsoft (see LICENSE information in the file) you should validate the code yourself before deployment.
|