Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,7 @@ This model is a mixed int4 model with group_size 128 and asymmetric quantization
|
|
| 10 |
|
| 11 |
**The model is quantized with pure RTN mode**
|
| 12 |
|
| 13 |
-
##
|
| 14 |
|
| 15 |
**Setup**
|
| 16 |
~~~bash
|
|
|
|
| 10 |
|
| 11 |
**The model is quantized with pure RTN mode**
|
| 12 |
|
| 13 |
+
## vllm inference
|
| 14 |
|
| 15 |
**Setup**
|
| 16 |
~~~bash
|