Update README.md
Browse files
README.md
CHANGED
|
@@ -72,7 +72,7 @@ curl http://localhost:8000/v1/chat/completions \
|
|
| 72 |
## How to use advanced vllm options
|
| 73 |
Description of each option
|
| 74 |
```--compilation_config '{"full_cuda_graph": true}'``` : Activates cuda [full graph capture](https://docs.vllm.ai/en/stable/design/cuda_graphs/#cudagraphmodes)
|
| 75 |
-
```--rope-scaling '{"rope_type":"yarn","factor":2.0,"original_max_position_embeddings":65536}'```: Apply [yarn](https://arxiv.org/abs/2309.00071)
|
| 76 |
```--enable-auto-tool-choice --tool-call-parser hermes``` : Enables [tool calling](https://docs.vllm.ai/en/latest/features/tool_calling/)
|
| 77 |
```--logits-processors logit_:WrappedPerReqLogitsProcessor``` : Enables VLLM_THINK_BUDGET_RATIO env variable and repetition-based auto-stop for thinking autostop
|
| 78 |
```--reasoning-parser deepseek_r1``` : Parses [reasoning outputs](https://docs.vllm.ai/en/latest/features/reasoning_outputs/)
|
|
|
|
| 72 |
## How to use advanced vllm options
|
| 73 |
Description of each option
|
| 74 |
```--compilation_config '{"full_cuda_graph": true}'``` : Activates cuda [full graph capture](https://docs.vllm.ai/en/stable/design/cuda_graphs/#cudagraphmodes)
|
| 75 |
+
```--rope-scaling '{"rope_type":"yarn","factor":2.0,"original_max_position_embeddings":65536}'```: Apply [yarn](https://arxiv.org/abs/2309.00071) to support 128K context length
|
| 76 |
```--enable-auto-tool-choice --tool-call-parser hermes``` : Enables [tool calling](https://docs.vllm.ai/en/latest/features/tool_calling/)
|
| 77 |
```--logits-processors logit_:WrappedPerReqLogitsProcessor``` : Enables VLLM_THINK_BUDGET_RATIO env variable and repetition-based auto-stop for thinking autostop
|
| 78 |
```--reasoning-parser deepseek_r1``` : Parses [reasoning outputs](https://docs.vllm.ai/en/latest/features/reasoning_outputs/)
|