Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -88,11 +88,13 @@ Laguna M.1 has upstream support in vLLM, SGLang, and TRT-LLM thanks to the suppo
88
 
89
  #### vLLM
90
 
91
- The full vLLM recipe is on the main [Laguna M.1 model card](https://huggingface.co/poolside/Laguna-M.1). Quantization is detected automatically from `quantization_config` in this checkpoint, so the same command works with `poolside/Laguna-M.1-FP8` substituted for the model ID. No extra flags required.
92
 
93
  ```shell
94
  pip install 'vllm>=0.21.0'
95
 
 
 
96
  vllm serve \
97
  --model poolside/Laguna-M.1-FP8 \
98
  --tool-call-parser poolside_v1 \
 
88
 
89
  #### vLLM
90
 
91
+ The full vLLM recipe is on the main [Laguna M.1 model card](https://huggingface.co/poolside/Laguna-M.1). Quantization is detected automatically from `quantization_config` in this checkpoint, so the same command works with `poolside/Laguna-M.1-FP8` substituted for the model ID. Set `VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER=0` when serving with vLLM.
92
 
93
  ```shell
94
  pip install 'vllm>=0.21.0'
95
 
96
+ export VLLM_BLOCKSCALE_FP8_GEMM_FLASHINFER=0
97
+
98
  vllm serve \
99
  --model poolside/Laguna-M.1-FP8 \
100
  --tool-call-parser poolside_v1 \