Update README.md

README.md

@@ -13,9 +13,9 @@ license: apache-2.0
 
 
 
-This is a 4-bit GPTQ version of the [Tigerbot 70b chat v2](https://huggingface.co/TigerResearch/tigerbot-70b-chat).
+This is a 4-bit EXL2 version of the [tigerbot-70b-chat-v2](https://huggingface.co/TigerResearch/tigerbot-70b-chat-v2).
 
-It was quantized to 4bit using: https://github.com/PanQiWei/AutoGPTQ
+It was quantized to 4bit using: https://github.com/turboderp/exllamav2
 
 ## How to download and use this model in github: https://github.com/TigerResearch/TigerBot
 
@@ -33,20 +33,13 @@ pip install -r requirements.txt
 
 Inference with command line interface
 
-infer with exllama
+infer with exllamav2
 ```
-# 安装exllama_lib
-pip install exllama_lib@git+https://github.com/taprosoft/exllama.git
-
-# 启动推理
-CUDA_VISIBLE_DEVICES=0 python other_infer/exllama_infer.py --model_path TigerResearch/tigerbot-70b-chat-4bit
-```
-
-infer with auto-gptq 
-```
-# 安装auto-gptq
-pip install auto-gptq
+# install exllamav2
+git clone https://github.com/turboderp/exllamav2
+cd exllamav2
+pip install -r requirements.txt
 
-# 启动推理
-CUDA_VISIBLE_DEVICES=0 python other_infer/gptq_infer.py --model_path TigerResearch/tigerbot-70b-chat-4bit
+# infer command
+CUDA_VISIBLE_DEVICES=0 python other_infer/exllamav2_hf_infer.py --model_path TigerResearch/tigerbot-70b-chat-v2-4bit-exl2
 ```