zhanghanxiao commited on
Commit
3dab585
·
verified ·
1 Parent(s): f8425ce

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -7
README.md CHANGED
@@ -142,27 +142,25 @@ pip3 install -U sglang sgl-kernel
142
 
143
  Both BF16 and FP8 models are supported by SGLang now. It depends on the dtype of the model in ${MODEL_PATH}.
144
 
145
- Here is the example to run [Ring-1T](https://zenmux.ai/inclusionai/ring-1t?utm_source=hf_inclusionAI) with multiple GPU nodes, where the master node IP is ${MASTER_IP} and port is ${PORT}:
146
 
147
  - Start server:
148
  ```bash
149
  # Node 0:
150
- python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:$PORT --nnodes 4 --node-rank 0
151
 
152
  # Node 1:
153
- python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:$PORT --nnodes 4 --node-rank 1
154
 
155
  # Node 2:
156
- python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:$PORT --nnodes 4 --node-rank 2
157
 
158
  # Node 3:
159
- python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:$PORT --nnodes 4 --node-rank 3
160
 
161
  # This is only an example. Please adjust arguments according to your actual environment.
162
  ```
163
 
164
- MTP is supported for the base model, but not yet for the chat model. You can add parameter `--speculative-algorithm NEXTN` to the start command.
165
-
166
  - Client:
167
 
168
  ```shell
 
142
 
143
  Both BF16 and FP8 models are supported by SGLang now. It depends on the dtype of the model in ${MODEL_PATH}.
144
 
145
+ Here is the example to run [Ring-1T](https://zenmux.ai/inclusionai/ring-1t?utm_source=hf_inclusionAI) with multiple GPU nodes, where the master node IP is ${MASTER_IP} and server port is ${PORT}:
146
 
147
  - Start server:
148
  ```bash
149
  # Node 0:
150
+ python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:2345 --port $PORT --nnodes 4 --node-rank 0
151
 
152
  # Node 1:
153
+ python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:2345 --port $PORT --nnodes 4 --node-rank 1
154
 
155
  # Node 2:
156
+ python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:2345 --port $PORT --nnodes 4 --node-rank 2
157
 
158
  # Node 3:
159
+ python -m sglang.launch_server --model-path $MODEL_PATH --tp-size 8 --pp-size 4 --dp-size 1 --trust-remote-code --dist-init-addr $MASTER_IP:2345 --port $PORT --nnodes 4 --node-rank 3
160
 
161
  # This is only an example. Please adjust arguments according to your actual environment.
162
  ```
163
 
 
 
164
  - Client:
165
 
166
  ```shell