yuccaaa's picture
Upload ms-swift/examples/deploy/server/README.md with huggingface_hub
97f8eb9 verified

Please refer to the examples in examples/infer and change swift infer to swift deploy to start the service. (You need to additionally remove --val_dataset)

e.g.

CUDA_VISIBLE_DEVICES=0 \
swift deploy \
    --model Qwen/Qwen2.5-7B-Instruct \
    --infer_backend vllm