怎么在vllm serve上使用？
#6
by
a463724055
- opened
This comment has been hidden (marked as Resolved)
Refer to discussion here: https://github.com/ByteDance-Seed/Seed-X-7B/issues/1
环境: vllm == 0.8.0, transformers=4.51.3
代码:
启动 server: python -m vllm.entrypoints.openai.api_server --model ByteDance-Seed/Seed-X-PPO-7B
发送 query: curl http://localhost:8000/v1/completions -H "Content-Type: application/json" -d '{"model": "ByteDance-Seed/Seed-X-PPO-7B", "prompt": "Translate the following English sentence into Chinese:\nMay the force be with you ", "max_tokens": 512, "temperature": 0.0}'