[Minor] Add command
Browse files
README.md
CHANGED
|
@@ -1,4 +1,16 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
---
|
| 4 |
-
Here, we provide AWQ-quantized versions of the most popular NVILA models. These files help you seamlessly deploy TinyChat to unlock the full potential of NVILA and your hardware.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
---
|
| 4 |
+
Here, we provide AWQ-quantized versions of the most popular NVILA models. These files help you seamlessly deploy TinyChat to unlock the full potential of NVILA and your hardware.
|
| 5 |
+
|
| 6 |
+
One-command demo to chat with quantized NVILA models via [llm-awq](ttps://github.com/mit-han-lab/llm-awq/tree/main/tinychat#:~:text=TinyChat%20support%20NVILA) (NVILA-8B as an example):
|
| 7 |
+
|
| 8 |
+
```bash
|
| 9 |
+
'''
|
| 10 |
+
cd llm-awq/tinychat
|
| 11 |
+
python nvila_demo.py --model-path PATH/TO/NVILA \
|
| 12 |
+
--quant_path NVILA-AWQ/NVILA-8B-w4-g128-awq-v2.pt \
|
| 13 |
+
--media PATH/TO/ANY/IMAGES/VIDEOS \
|
| 14 |
+
--act_scale_path NVILA-AWQ/NVILA-8B-smooth-scale.pt \
|
| 15 |
+
--all --chunk --model_type nvila
|
| 16 |
+
'''
|