How to use it? Can I have a conversation?
#10
by viaho - opened
General conversation or specific questions？？？
You can use vLLM or SGLang to serve it and then use the OpenAI chat completions API to poll the model!
General conversation or specific questions？？？
You can use vLLM or SGLang to serve it and then use the OpenAI chat completions API to poll the model!