docs: add verified vLLM Docker image to Deploy section

#2
by zhiyucheng - opened
Files changed (1) hide show
  1. README.md +4 -8
README.md CHANGED
@@ -106,15 +106,11 @@ This model was obtained by quantizing the weights and activations of DeepSeek-V4
106
 
107
  ### Deploy with vLLM
108
 
109
- To deploy the quantized NVFP4 checkpoint with [vLLM](https://github.com/vllm-project/vllm), use the following command (you need 8xB200 GPUs):
110
 
111
- ```bash
112
- python -m vllm.entrypoints.cli.main serve \
113
- nvidia/DeepSeek-V4-Flash-NVFP4 \
114
- --tensor-parallel-size 8 \
115
- --trust-remote-code \
116
- --kv-cache-dtype fp8 \
117
- --served-model-name nvfp4
118
  ```
119
 
120
  ### Deploy with SGLang
 
106
 
107
  ### Deploy with vLLM
108
 
109
+ Verified serving this NVFP4 checkpoint on **GB300** with the `vllm/vllm-openai:nightly-aarch64` Docker image (vLLM 0.22.1rc1.dev504), tensor-parallel 4:
110
 
111
+ ```sh
112
+ vllm serve nvidia/DeepSeek-V4-Flash-NVFP4 \
113
+ --tensor-parallel-size 4 --trust-remote-code --kv-cache-dtype fp8
 
 
 
 
114
  ```
115
 
116
  ### Deploy with SGLang