Document SenseNova-Loop-9B AWQ serving with vLLM

#15
by JYILY - opened

Document how to serve the published SenseNova-Loop-9B-AWQ checkpoint with vLLM in both the English and Chinese model cards. The examples use the vLLM revision that adds AWQ support.

The AWQ checkpoint was served on one NVIDIA H200 with a 4,096-token context limit, --max-num-seqs 4, and a 2 GiB KV cache budget. A text request to /v1/chat/completions asking for 13 × 7 returned HTTP 200 and the expected answer 91.

ppcc1135 changed pull request status to merged

Sign up or log in to comment