Louym commited on
Commit
676ccfe
·
verified ·
1 Parent(s): 747bacf

[Minor] Add command

Browse files
Files changed (1) hide show
  1. README.md +16 -4
README.md CHANGED
@@ -1,4 +1,16 @@
1
- ---
2
- license: apache-2.0
3
- ---
4
- Here, we provide AWQ-quantized versions of the most popular NVILA models. These files help you seamlessly deploy TinyChat to unlock the full potential of NVILA and your hardware.
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+ Here, we provide AWQ-quantized versions of the most popular NVILA models. These files help you seamlessly deploy TinyChat to unlock the full potential of NVILA and your hardware.
5
+
6
+ One-command demo to chat with quantized NVILA models via [llm-awq](ttps://github.com/mit-han-lab/llm-awq/tree/main/tinychat#:~:text=TinyChat%20support%20NVILA) (NVILA-8B as an example):
7
+
8
+ ```bash
9
+ '''
10
+ cd llm-awq/tinychat
11
+ python nvila_demo.py --model-path PATH/TO/NVILA \
12
+ --quant_path NVILA-AWQ/NVILA-8B-w4-g128-awq-v2.pt \
13
+ --media PATH/TO/ANY/IMAGES/VIDEOS \
14
+ --act_scale_path NVILA-AWQ/NVILA-8B-smooth-scale.pt \
15
+ --all --chunk --model_type nvila
16
+ '''