yuccaaa commited on
Commit
df44ac9
·
verified ·
1 Parent(s): 97f8eb9

Upload ms-swift/examples/deploy/server/demo.sh with huggingface_hub

Browse files
ms-swift/examples/deploy/server/demo.sh ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ CUDA_VISIBLE_DEVICES=0 swift deploy \
2
+ --model Qwen/Qwen2.5-7B-Instruct \
3
+ --infer_backend vllm \
4
+ --served_model_name Qwen2.5-7B-Instruct
5
+
6
+ # After the server-side deployment above is successful, use the command below to perform a client call test.
7
+
8
+ # curl http://localhost:8000/v1/chat/completions \
9
+ # -H "Content-Type: application/json" \
10
+ # -d '{
11
+ # "model": "Qwen2.5-7B-Instruct",
12
+ # "messages": [{"role": "user", "content": "What is your name?"}],
13
+ # "temperature": 0
14
+ # }'