Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -104,8 +104,22 @@ vllm serve \
104
 
105
  #### SGLang
106
 
107
- Laguna M.1 is supported in SGLang via [sgl-project/sglang#28400](https://github.com/sgl-project/sglang/pull/28400). Quantization is detected automatically from `quantization_config`, so no extra flags are required. A full serving recipe will be added to the main [Laguna M.1 model card](https://huggingface.co/poolside/Laguna-M.1).
 
 
 
 
108
 
 
 
 
 
 
 
 
 
 
 
109
  #### TRT-LLM
110
 
111
  Laguna is supported in TensorRT-LLM thanks to the team at NVIDIA ([NVIDIA/TensorRT-LLM#13559](https://github.com/NVIDIA/TensorRT-LLM/pull/13559), with partial-RoPE fusion in [#15110](https://github.com/NVIDIA/TensorRT-LLM/pull/15110)). The full recipe is on the main [Laguna M.1 model card](https://huggingface.co/poolside/Laguna-M.1). Quantization is detected automatically from `quantization_config` in this checkpoint, so no extra flags are required.
 
104
 
105
  #### SGLang
106
 
107
+ The full SGLang recipe is on the [SGLang Cookbook](https://docs.sglang.io/cookbook/autoregressive/Poolside/Laguna-M.1). Quantization is detected automatically, so no extra flags are required.
108
+ ```shell
109
+ git clone https://github.com/sgl-project/sglang.git
110
+ cd sglang
111
+ pip install -e "python[all]"
112
 
113
+ sglang serve \
114
+ --model-path poolside/Laguna-M.1-FP8 \
115
+ --trust-remote-code \
116
+ --reasoning-parser poolside_v1 \
117
+ --tool-call-parser poolside_v1 \
118
+ --tp 8 \
119
+ --host 0.0.0.0 \
120
+ --port 30000 \
121
+ --fp8-gemm-backend triton # Needed for SM100
122
+ ```
123
  #### TRT-LLM
124
 
125
  Laguna is supported in TensorRT-LLM thanks to the team at NVIDIA ([NVIDIA/TensorRT-LLM#13559](https://github.com/NVIDIA/TensorRT-LLM/pull/13559), with partial-RoPE fusion in [#15110](https://github.com/NVIDIA/TensorRT-LLM/pull/15110)). The full recipe is on the main [Laguna M.1 model card](https://huggingface.co/poolside/Laguna-M.1). Quantization is detected automatically from `quantization_config` in this checkpoint, so no extra flags are required.