Text Generation
Transformers
Safetensors
step3p5
conversational
custom_code
Eval Results
Tingdan commited on
Commit
8847e96
·
verified ·
1 Parent(s): 6c124fa

Update README.md

Browse files

update sglang deploy cmd

Files changed (1) hide show
  1. README.md +3 -7
README.md CHANGED
@@ -221,12 +221,10 @@ pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git"
221
  ```
222
 
223
  2. Launch the server.
 
224
 
225
  ```bash
226
- - For bf16 model
227
- SGLANG_ENABLE_SPEC_V2=1
228
- python3 -m sglang.launch_server \
229
- --model-path <MODEL_PATH_OR_HF_ID> \
230
  --served-model-name step3p5-flash \
231
  --tp-size 8 \
232
  --tool-call-parser step3p5 \
@@ -241,9 +239,7 @@ python3 -m sglang.launch_server \
241
  ```
242
  - For fp8 model
243
  ```bash
244
- SGLANG_ENABLE_SPEC_V2=1
245
- python3 -m sglang.launch_server \
246
- --model-path <MODEL_PATH_OR_HF_ID> \
247
  --served-model-name step3p5-flash \
248
  --tp-size 8 \
249
  --ep-size 8 \
 
221
  ```
222
 
223
  2. Launch the server.
224
+ - For bf16 model
225
 
226
  ```bash
227
+ sglang serve --model-path <MODEL_PATH_OR_HF_ID> \
 
 
 
228
  --served-model-name step3p5-flash \
229
  --tp-size 8 \
230
  --tool-call-parser step3p5 \
 
239
  ```
240
  - For fp8 model
241
  ```bash
242
+ sglang serve --model-path <MODEL_PATH_OR_HF_ID> \
 
 
243
  --served-model-name step3p5-flash \
244
  --tp-size 8 \
245
  --ep-size 8 \