Update docs/deploy_guidance.md (#3)
Browse files- Update docs/deploy_guidance.md (1068600b548f97cd88375a8be5c09389e0f2dde2)
- docs/deploy_guidance.md +1 -0
docs/deploy_guidance.md
CHANGED
|
@@ -18,6 +18,7 @@ docker pull jdopensource/joyai-llm-vllm:v0.13.0-joyai_llm_flash
|
|
| 18 |
vllm serve ${MODEL_PATH} --tp 8 --trust-remote-code \
|
| 19 |
--tool-call-parser qwen3_coder --enable-auto-tool-choice \
|
| 20 |
--speculative-config $'{"method": "mtp", "num_speculative_tokens": 3}'
|
|
|
|
| 21 |
**Key notes**
|
| 22 |
- `--tool-call-parser qwen3_coder`: Required for enabling tool calling
|
| 23 |
|
|
|
|
| 18 |
vllm serve ${MODEL_PATH} --tp 8 --trust-remote-code \
|
| 19 |
--tool-call-parser qwen3_coder --enable-auto-tool-choice \
|
| 20 |
--speculative-config $'{"method": "mtp", "num_speculative_tokens": 3}'
|
| 21 |
+
```
|
| 22 |
**Key notes**
|
| 23 |
- `--tool-call-parser qwen3_coder`: Required for enabling tool calling
|
| 24 |
|