Update SGLang guide
#14
by
JustinTong
- opened
README.md
CHANGED
|
@@ -59,7 +59,12 @@ pip install git+https://github.com/huggingface/transformers.git
|
|
| 59 |
|
| 60 |
### SGLang
|
| 61 |
|
| 62 |
-
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
|
| 64 |
### transformers
|
| 65 |
|
|
@@ -126,6 +131,7 @@ python3 -m sglang.launch_server \
|
|
| 126 |
--host 0.0.0.0 \
|
| 127 |
--port 8000
|
| 128 |
```
|
|
|
|
| 129 |
|
| 130 |
## Citation
|
| 131 |
|
|
|
|
| 59 |
|
| 60 |
### SGLang
|
| 61 |
|
| 62 |
+
+ Install the supported versions of SGLang and Transformers (using `uv` is recommended):
|
| 63 |
+
|
| 64 |
+
```shell
|
| 65 |
+
uv pip install sglang==0.3.2.dev9039+pr-17247.g90c446848 --extra-index-url https://sgl-project.github.io/whl/pr/
|
| 66 |
+
uv pip install git+https://github.com/huggingface/transformers.git@76732b4e7120808ff989edbd16401f61fa6a0afa
|
| 67 |
+
```
|
| 68 |
|
| 69 |
### transformers
|
| 70 |
|
|
|
|
| 131 |
--host 0.0.0.0 \
|
| 132 |
--port 8000
|
| 133 |
```
|
| 134 |
+
+ For Blackwell GPUs, include `--attention-backend triton --speculative-draft-attention-backend triton` in your SGLang launch command.
|
| 135 |
|
| 136 |
## Citation
|
| 137 |
|