Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DeepSeek-R1-NextN
|
| 2 |
+
NextN layer for DeepSeek-R1 (https://huggingface.co/deepseek-ai/DeepSeek-R1) model. Used for speculative decoding in SGLang (https://github.com/sgl-project/sglang).
|
| 3 |
+
|
| 4 |
+
## Usage
|
| 5 |
+
```
|
| 6 |
+
python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-R1 --speculative-algo EAGLE --speculative-draft lmsys/DeepSeek-R1-NextN --speculative-num-steps 2 --speculat
|
| 7 |
+
ive-eagle-topk 4 --speculative-num-draft-tokens 4 --trust-remote --tp 8
|
| 8 |
+
```
|