Files changed (1) hide show
  1. README.md +14 -4
README.md CHANGED
@@ -74,13 +74,23 @@ Benchmarks were run using [SpecForge](https://github.com/sgl-project/SpecForge/b
74
  ### Requirements
75
 
76
  - NVIDIA GPU with CUDA 12.0+
 
77
  - [SGLang](https://github.com/sgl-project/sglang) ≥ 0.5.8
78
 
79
- ### Launch Server
 
 
 
 
 
 
 
 
 
80
 
81
  ```bash
82
  python -m sglang.launch_server \
83
- --model-path /path/to/Kimi-K2.5 \
84
  --tp 8 \
85
  --trust-remote-code \
86
  --speculative-algorithm EAGLE3 \
@@ -96,11 +106,11 @@ python -m sglang.launch_server \
96
 
97
  ```bash
98
  python bench_eagle3.py \
99
- --model-path /path/to/Kimi-K2.5 \
100
  --port 30000 \
101
  --config-list 1,3,1,4 \
102
  --benchmark-list <benchmark_name> \
103
  --skip-launch-server
104
  ```
105
 
106
- `--config-list` format: `topk,num_steps,topk,num_draft_tokens`.
 
74
  ### Requirements
75
 
76
  - NVIDIA GPU with CUDA 12.0+
77
+ - [vLLM](https://github.com/vllm-project/vllm) >= (0.18.0) or you can install the [nightly wheel/docker image](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/#install-the-latest-code).
78
  - [SGLang](https://github.com/sgl-project/sglang) ≥ 0.5.8
79
 
80
+ ### Launch Server (vLLM)
81
+ ```bash
82
+ vllm serve moonshotai/Kimi-K2.5 \
83
+ --tensor-parallel-size 8 \
84
+ --speculative-config '{"model": "lightseekorg/kimi-k2.5-eagle3", "method": "eagle3", "num_speculative_tokens": 3}' \
85
+ --trust-remote-code
86
+ ```
87
+ For deployment configuration, refer to [official vLLM recipes](https://docs.vllm.ai/projects/recipes/en/latest/moonshotai/Kimi-K2.5.html#running-kimi-k25-with-vllm).
88
+
89
+ ### Launch Server (SGLang)
90
 
91
  ```bash
92
  python -m sglang.launch_server \
93
+ --model-path moonshotai/Kimi-K2.5 \
94
  --tp 8 \
95
  --trust-remote-code \
96
  --speculative-algorithm EAGLE3 \
 
106
 
107
  ```bash
108
  python bench_eagle3.py \
109
+ --model-path moonshotai/Kimi-K2.5 \
110
  --port 30000 \
111
  --config-list 1,3,1,4 \
112
  --benchmark-list <benchmark_name> \
113
  --skip-launch-server
114
  ```
115
 
116
+ `--config-list` format: `topk,num_steps,topk,num_draft_tokens`.