docs: add verified vLLM Docker image to Deploy section
#2
by zhiyucheng - opened
README.md
CHANGED
|
@@ -106,15 +106,11 @@ This model was obtained by quantizing the weights and activations of DeepSeek-V4
|
|
| 106 |
|
| 107 |
### Deploy with vLLM
|
| 108 |
|
| 109 |
-
|
| 110 |
|
| 111 |
-
```
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
--tensor-parallel-size 8 \
|
| 115 |
-
--trust-remote-code \
|
| 116 |
-
--kv-cache-dtype fp8 \
|
| 117 |
-
--served-model-name nvfp4
|
| 118 |
```
|
| 119 |
|
| 120 |
### Deploy with SGLang
|
|
|
|
| 106 |
|
| 107 |
### Deploy with vLLM
|
| 108 |
|
| 109 |
+
Verified serving this NVFP4 checkpoint on **GB300** with the `vllm/vllm-openai:nightly-aarch64` Docker image (vLLM 0.22.1rc1.dev504), tensor-parallel 4:
|
| 110 |
|
| 111 |
+
```sh
|
| 112 |
+
vllm serve nvidia/DeepSeek-V4-Flash-NVFP4 \
|
| 113 |
+
--tensor-parallel-size 4 --trust-remote-code --kv-cache-dtype fp8
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
```
|
| 115 |
|
| 116 |
### Deploy with SGLang
|