Update vLLM recipe

#2
by jeeejeee - opened
Files changed (2) hide show
  1. README.md +1 -25
  2. README_CN.md +1 -27
README.md CHANGED
@@ -169,31 +169,7 @@ For production serving, we recommend using [vLLM](https://github.com/vllm-projec
169
 
170
  ### vLLM
171
 
172
- Build vLLM from source:
173
- ```bash
174
- uv venv --python 3.12 --seed --managed-python
175
- source .venv/bin/activate
176
- git clone https://github.com/vllm-project/vllm.git
177
- cd vllm
178
- uv pip install --editable . --torch-backend=auto
179
- ```
180
-
181
- Start the vLLM server with MTP enabled:
182
-
183
- ```bash
184
- vllm serve tencent/Hy4-preview-FP8 \
185
- --tensor-parallel-size 8 \
186
- --speculative-config.method mtp \
187
- --speculative-config.num_speculative_tokens 3 \
188
- --attention-backend FLASHMLA_SPARSE \
189
- --tool-call-parser hy_v4 \
190
- --reasoning-parser hy_v4 \
191
- --enable-auto-tool-choice \
192
- --port 8000 \
193
- --served-model-name hy4-preview
194
- ```
195
-
196
- Or use official prebuilt image `vllm/vllm-openai:hy4-preview`:
197
 
198
  ```bash
199
  docker run --gpus all \
 
169
 
170
  ### vLLM
171
 
172
+ Use official prebuilt image `vllm/vllm-openai:hy4-preview`:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
173
 
174
  ```bash
175
  docker run --gpus all \
README_CN.md CHANGED
@@ -158,33 +158,7 @@ print(response.choices[0].message.content)
158
 
159
  ### vLLM
160
 
161
- 从源码构建 vLLM
162
-
163
- ```bash
164
- uv venv --python 3.12 --seed --managed-python
165
- source .venv/bin/activate
166
- git clone https://github.com/vllm-project/vllm.git
167
- cd vllm
168
- uv pip install --editable . --torch-backend=auto
169
- ```
170
-
171
- 启动 vLLM 服务,开启 MTP:
172
-
173
- ```bash
174
- vllm serve tencent/Hy4-preview-FP8 \
175
- --tensor-parallel-size 8 \
176
- --speculative-config.method mtp \
177
- --speculative-config.num_speculative_tokens 3 \
178
- --attention-backend FLASHMLA_SPARSE \
179
- --tool-call-parser hy_v4 \
180
- --reasoning-parser hy_v4 \
181
- --enable-auto-tool-choice \
182
- --port 8000 \
183
- --served-model-name hy4-preview
184
- ```
185
-
186
-
187
- 或者使用社区官方镜像部署:`vllm/vllm-openai:hy4-preview`:
188
 
189
  ```bash
190
  docker run --gpus all \
 
158
 
159
  ### vLLM
160
 
161
+ 使用社区官方镜像部署`vllm/vllm-openai:hy4-preview`:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
162
 
163
  ```bash
164
  docker run --gpus all \