Instructions to use moonshotai/Kimi-K2-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moonshotai/Kimi-K2-Instruct with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("moonshotai/Kimi-K2-Instruct", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update docs/deploy_guidance.md
Browse files- docs/deploy_guidance.md +1 -1
docs/deploy_guidance.md
CHANGED
|
@@ -65,7 +65,7 @@ python -m sglang.launch_server --model-path $MODEL_PATH --tp 16 --dist-init-addr
|
|
| 65 |
|
| 66 |
### Data Parallelism + Expert Parallelism
|
| 67 |
|
| 68 |
-
Here is an example for large scale Prefill-Decode
|
| 69 |
|
| 70 |
``` bash
|
| 71 |
# for prefill node
|
|
|
|
| 65 |
|
| 66 |
### Data Parallelism + Expert Parallelism
|
| 67 |
|
| 68 |
+
Here is an example for large scale Prefill-Decode Disaggregation (4P12D H200) with DP+EP in SGLang:
|
| 69 |
|
| 70 |
``` bash
|
| 71 |
# for prefill node
|