zylin12 commited on
Commit
7f2c1bd
·
verified ·
1 Parent(s): 8fa0577

Update vLLM install instructions in README

Browse files
Files changed (1) hide show
  1. README.md +15 -2
README.md CHANGED
@@ -251,10 +251,23 @@ More prompt recipes are available in the GitHub repository: <https://github.com/
251
 
252
  ### Serve with vLLM and SGLang
253
 
254
- MOSS-Transcribe-Diarize 0.9B supports vLLM serving through the OpenAI-compatible transcription API:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
255
 
256
  ```bash
257
- pip install vllm
258
  vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code
259
  ```
260
 
 
251
 
252
  ### Serve with vLLM and SGLang
253
 
254
+ MOSS-Transcribe-Diarize supports vLLM serving through the OpenAI-compatible transcription API. Use a pinned vLLM nightly build that includes the MOSS-Transcribe-Diarize model registration. Choose one of the following commands: for CUDA 12 environments, use `cu129`; for CUDA 13 environments, use `cu130`.
255
+
256
+ ```bash
257
+ uv pip install -U vllm \
258
+ --torch-backend=auto \
259
+ --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
260
+ ```
261
+
262
+ or:
263
+
264
+ ```bash
265
+ uv pip install -U vllm \
266
+ --torch-backend=auto \
267
+ --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu130
268
+ ```
269
 
270
  ```bash
 
271
  vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize --trust-remote-code
272
  ```
273