Update README.md
Browse files
README.md
CHANGED
|
@@ -12,4 +12,6 @@ This repository provides the low-rank version of Deepseek-V3, the route expert w
|
|
| 12 |
|
| 13 |
- [`gh-efforts/DeepSeek-V3`](https://github.com/gh-efforts/DeepSeek-V3/commit/86ce41fd5628ac1656ad560d35740ce76fcb73c7)
|
| 14 |
- [`gh-efforts/sglang`](https://github.com/gh-efforts/sglang/tree/deepseek-lowrank)
|
|
|
|
|
|
|
| 15 |
- [`xutianyi1999/mistral.rs`](https://github.com/xutianyi1999/mistral.rs/commit/c360b28f16504cfa94c6978e8307cf054d07cc42)
|
|
|
|
| 12 |
|
| 13 |
- [`gh-efforts/DeepSeek-V3`](https://github.com/gh-efforts/DeepSeek-V3/commit/86ce41fd5628ac1656ad560d35740ce76fcb73c7)
|
| 14 |
- [`gh-efforts/sglang`](https://github.com/gh-efforts/sglang/tree/deepseek-lowrank)
|
| 15 |
+
|
| 16 |
+
SGLang launch command: `DEEPSEEK_RANK=1280 DEEPSEEK_SCALE_RANK=10 python3 -m sglang.launch_server --model-path <MODEL PATH> --host 0.0.0.0 --port 40000 --tp-size 8 --enable-ep-moe --trust-remote-code --mem-fraction-static 0.9 --disable-cuda-graph`
|
| 17 |
- [`xutianyi1999/mistral.rs`](https://github.com/xutianyi1999/mistral.rs/commit/c360b28f16504cfa94c6978e8307cf054d07cc42)
|