zhiyucheng commited on
Commit
2600f6c
·
verified ·
1 Parent(s): 586d3b4

docs: note vLLM nightly + PR #46380 required for MiniMax-M3 NVFP4 serving

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -116,7 +116,7 @@ This model is NVFP4 quantized with nvidia-modelopt **v0.44.0**
116
  This model was obtained by quantizing the weights and activations of Minimax-M3 to NVFP4 data type. This optimization reduces the number of bits per parameter from 8 to 4, reducing disk size and GPU memory requirements by approximately 2x.
117
 
118
  ## Usage
119
- To serve this checkpoint with [vLLM](https://github.com/vllm-project/vllm), you can launch the docker image `vllm/vllm-openai:latest` and run the sample command below:
120
 
121
  ```
122
  vllm serve nvidia/MiniMax-M3-NVFP4 \
 
116
  This model was obtained by quantizing the weights and activations of Minimax-M3 to NVFP4 data type. This optimization reduces the number of bits per parameter from 8 to 4, reducing disk size and GPU memory requirements by approximately 2x.
117
 
118
  ## Usage
119
+ To serve this checkpoint with [vLLM](https://github.com/vllm-project/vllm), you currently need the nightly docker image that includes MiniMax-M3 NVFP4 support from [vllm-project/vllm#46380](https://github.com/vllm-project/vllm/pull/46380) (not yet in a stable release). Launch the nightly image and run the sample command below:
120
 
121
  ```
122
  vllm serve nvidia/MiniMax-M3-NVFP4 \