kvaishnavi commited on
Commit
de94d9a
·
verified ·
1 Parent(s): 73058b2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -9
README.md CHANGED
@@ -23,9 +23,10 @@ huggingface-cli download onnxruntime/DeepSeek-R1-Distill-ONNX --include deepseek
23
  # Install the CPU package of ONNX Runtime GenAI
24
  pip install onnxruntime-genai
25
 
26
- # Please adjust the model directory (-m) accordingly
27
- curl -o https://raw.githubusercontent.com/microsoft/onnxruntime-genai/refs/heads/main/examples/python/model-chat.py
28
- python model-chat.py -m /path/to/cpu-int4-rtn-block-32-acc-level-4/ -e cpu --chat_template "<|begin▁of▁sentence|><|User|>{input}<|Assistant|>"
 
29
  ```
30
 
31
  For CUDA:
@@ -37,9 +38,10 @@ huggingface-cli download onnxruntime/DeepSeek-R1-Distill-ONNX --include deepseek
37
  # Install the CUDA package of ONNX Runtime GenAI
38
  pip install onnxruntime-genai-cuda
39
 
40
- # Please adjust the model directory (-m) accordingly
41
- curl -o https://raw.githubusercontent.com/microsoft/onnxruntime-genai/refs/heads/main/examples/python/model-chat.py
42
- python model-chat.py -m /path/to/gpu-int4-rtn-block-32/ -e cuda --chat_template "<|begin▁of▁sentence|><|User|>{input}<|Assistant|>"
 
43
  ```
44
 
45
  For DirectML:
@@ -51,9 +53,10 @@ huggingface-cli download onnxruntime/DeepSeek-R1-Distill-ONNX --include deepseek
51
  # Install the DirectML package of ONNX Runtime GenAI
52
  pip install onnxruntime-genai-directml
53
 
54
- # Please adjust the model directory (-m) accordingly
55
- curl -o https://raw.githubusercontent.com/microsoft/onnxruntime-genai/refs/heads/main/examples/python/model-chat.py
56
- python model-chat.py -m /path/to/gpu-int4-rtn-block-32/ -e dml --chat_template "<|begin▁of▁sentence|><|User|>{input}<|Assistant|>"
 
57
  ```
58
 
59
 
 
23
  # Install the CPU package of ONNX Runtime GenAI
24
  pip install onnxruntime-genai
25
 
26
+ # Please adjust the model directory (-m) accordingly
27
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py
28
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-chat.py -o model-chat.py
29
+ python model-chat.py -m /path/to/cpu-int4-rtn-block-32-acc-level-4/ -e cpu
30
  ```
31
 
32
  For CUDA:
 
38
  # Install the CUDA package of ONNX Runtime GenAI
39
  pip install onnxruntime-genai-cuda
40
 
41
+ # Please adjust the model directory (-m) accordingly
42
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py
43
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-chat.py -o model-chat.py
44
+ python model-chat.py -m /path/to/gpu-int4-rtn-block-32/ -e cuda
45
  ```
46
 
47
  For DirectML:
 
53
  # Install the DirectML package of ONNX Runtime GenAI
54
  pip install onnxruntime-genai-directml
55
 
56
+ # Please adjust the model directory (-m) accordingly
57
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py
58
+ curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-chat.py -o model-chat.py
59
+ python model-chat.py -m /path/to/gpu-int4-rtn-block-32/ -e dml
60
  ```
61
 
62