Instructions to use tiny-random/kimi-k3.1-pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tiny-random/kimi-k3.1-pro with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="tiny-random/kimi-k3.1-pro", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("tiny-random/kimi-k3.1-pro", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tiny-random/kimi-k3.1-pro with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tiny-random/kimi-k3.1-pro"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/kimi-k3.1-pro",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/tiny-random/kimi-k3.1-pro

SGLang

How to use tiny-random/kimi-k3.1-pro with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tiny-random/kimi-k3.1-pro" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/kimi-k3.1-pro",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tiny-random/kimi-k3.1-pro" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/kimi-k3.1-pro",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use tiny-random/kimi-k3.1-pro with Docker Model Runner:
```
docker model run hf.co/tiny-random/kimi-k3.1-pro
```

kimi-k3.1-pro / README.md

yujiepan

Upload folder using huggingface_hub

f67fbd2 verified about 12 hours ago

preview code

Raw History Blame Contribute Delete

5.26 kB

---
library_name: transformers
pipeline_tag: image-text-to-text
base_model:
- moonshotai/Kimi-K3.1-Pro
---
This is a randomly initialized tiny model based on [moonshotai/Kimi-K3.1-Pro](https://huggingface.co/moonshotai/Kimi-K3.1-Pro) for testing and debugging.

🎉 Thanks to [@akopytko](https://huggingface.co/akopytko) for contributing to this repo!

| Feature | Source | This repo |
|---|---|---|
| Checkpoint tensor elements | 57.36B BF16, 11.12M F32, 1.45T U8 | 13.98M BF16, 163.07K F32, 6.68M U8 |
| Safetensors weight files | 1.56 TB (96 files) | 35.58 MB (model.safetensors) |
| Total / DSA-gated layers | 93 / 24 (plus 69 KDA, 1 dense) | 17 / 5 (plus 11 KDA, 1 dense) |
| Attention composition | 69 KDA + 24 gated DSA | 11 KDA + 5 DSA cadence |
| Hidden width / MLP width | 7,168 / 33,792 | 8 / 32 |
| Full-attn Q heads / head dim | 96 / 128 (nope 128, rope 64, v 128) | 8 / 128 (same per-head dims) |
| MLA q / kv LoRA rank | 1,536 / 512 | 128 / 512 |
| Routed / shared experts, top-k | 896 / 2, 16 | 16 / 2, 8 |
| LSA index heads / head dim / topk | 32 / 128 / 2,048 | 32 / 128 / 2,048 (unchanged) |
| Physical DSA indexer layers | 24 (dsa_layers - 1) | [3, 7, 11, 15, 16] |
| Routed expert precision | MXFP4 packed (group 32, E8M0 scales) | Same |
| Vision depth / hidden / heads | 27 / 1,024 / 12 | 2 / 64 / 2 |
| Patch size | 14 | Same |
| Vocabulary rows / tying | 163,840 / untied | Same |
| Maximum context length | 1,048,576 | Same |
K3.1-Pro ships no modeling code; the included pinned K3 `modeling_kimi_k3.py`/`modeling_kimi_linear.py` backs the tiny model. The LSA indexer tensors are included forward-compatibly — released Transformers/vLLM/SGLang have no KimiK31 gated-DSA kernel yet, so the DSA layers run their MLA core.
| File path | Size |
|------|------|
| model.safetensors | 35.6MB |
### Example usage
#### Transformers (✅ verified)
Loads with the pinned K3 reference modeling on GPU (KDA delta attention uses Triton via fla-core). The K3.1 gated-DSA layers run their MLA attention core; the LSA indexer tensors are present but not consumed because released Transformers lacks the gated-DSA operator.
```python
def transformers_example(model_id: str):
    import numpy as np
    import torch
    from PIL import Image
    from transformers import AutoModel, AutoProcessor
    model_id = str(model_id)
    processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModel.from_pretrained(
        model_id,
        dtype=torch.bfloat16,
        device_map="cuda",
        trust_remote_code=True,
        attn_implementation="eager",
    ).eval()
    image = Image.fromarray(np.random.default_rng(42).integers(0, 255, (56, 56, 3), dtype=np.uint8))
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": "Hello, world!"},
            ],
        }
    ]
    inputs = processor(messages=messages, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=16)
    generated_ids = outputs.sequences if hasattr(outputs, "sequences") else outputs
    print(processor.decode(generated_ids[0].detach().cpu().tolist()))

if __name__ == "__main__":
    transformers_example(model_id="tiny-random/kimi-k3.1-pro")
```
#### vLLM serving (unverified)

Requires a vLLM build with KimiK31 gated-DSA support (newer than the released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro.

```bash
# K3.1 gated-DSA serving requires a vLLM build with KimiK31 DSA support (newer than the
# released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro
vllm serve tiny-random/kimi-k3.1-pro \
    --trust-remote-code \
    --tensor-parallel-size 8 \
    --load-format safetensors \
    --max-model-len 131072 \
    --gpu-memory-utilization 0.90
```

#### SGLang serving (unverified)

Requires an SGLang build newer than 0.5.23 with the KimiK31 DSA indexer (the gated-DSA kernels had not landed in the released wheel at tiny-model creation time).

```bash
# K3.1 gated-DSA serving requires an SGLang build newer than 0.5.23 (the KimiK31 DSA indexer
# kernels had not landed in the released wheel at tiny-model creation time).
python3 -m sglang.launch_server \
    --model-path tiny-random/kimi-k3.1-pro \
    --trust-remote-code \
    --tp-size 8 \
    --chunked-prefill-size 16384 \
    --cuda-graph-backend-prefill breakable \
    --dtype bfloat16 \
    --kv-cache-dtype fp8_e4m3 \
    --reasoning-parser kimi_k3 \
    --tool-call-parser kimi_k3 \
    --mem-fraction-static 0.85
```

### Creating this repo

- Code: [.meta/create_model.py](.meta/create_model.py)
- Model structure: [.meta/model.log](.meta/model.log)
- Environment: [.meta/env.json](.meta/env.json)

### Change log

- 2026-11-05: Initial version: 17-layer K3.1-Pro (12 KDA + 5 gated-DSA cadence) with the source LSA indexer contract and packed MXFP4 experts, reusing the pinned K3 remote modeling.
- 2026-11-06: Follow source revision d9e99b0 (updated encoding_k3.py); architectures now `KimiK31ForConditionalGeneration` to match the source config.
*This repo is generated by GPT-6.1-Sol.*

	---
	library_name: transformers
	pipeline_tag: image-text-to-text
	base_model:
	- moonshotai/Kimi-K3.1-Pro
	---

	This is a randomly initialized tiny model based on [moonshotai/Kimi-K3.1-Pro](https://huggingface.co/moonshotai/Kimi-K3.1-Pro) for testing and debugging.

	🎉 Thanks to [@akopytko](https://huggingface.co/akopytko) for contributing to this repo!

	\| Feature \| Source \| This repo \|
	\|---\|---\|---\|
	\| Checkpoint tensor elements \| 57.36B BF16, 11.12M F32, 1.45T U8 \| 13.98M BF16, 163.07K F32, 6.68M U8 \|
	\| Safetensors weight files \| 1.56 TB (96 files) \| 35.58 MB (model.safetensors) \|
	\| Total / DSA-gated layers \| 93 / 24 (plus 69 KDA, 1 dense) \| 17 / 5 (plus 11 KDA, 1 dense) \|
	\| Attention composition \| 69 KDA + 24 gated DSA \| 11 KDA + 5 DSA cadence \|
	\| Hidden width / MLP width \| 7,168 / 33,792 \| 8 / 32 \|
	\| Full-attn Q heads / head dim \| 96 / 128 (nope 128, rope 64, v 128) \| 8 / 128 (same per-head dims) \|
	\| MLA q / kv LoRA rank \| 1,536 / 512 \| 128 / 512 \|
	\| Routed / shared experts, top-k \| 896 / 2, 16 \| 16 / 2, 8 \|
	\| LSA index heads / head dim / topk \| 32 / 128 / 2,048 \| 32 / 128 / 2,048 (unchanged) \|
	\| Physical DSA indexer layers \| 24 (dsa_layers - 1) \| [3, 7, 11, 15, 16] \|
	\| Routed expert precision \| MXFP4 packed (group 32, E8M0 scales) \| Same \|
	\| Vision depth / hidden / heads \| 27 / 1,024 / 12 \| 2 / 64 / 2 \|
	\| Patch size \| 14 \| Same \|
	\| Vocabulary rows / tying \| 163,840 / untied \| Same \|
	\| Maximum context length \| 1,048,576 \| Same \|

	K3.1-Pro ships no modeling code; the included pinned K3 `modeling_kimi_k3.py`/`modeling_kimi_linear.py` backs the tiny model. The LSA indexer tensors are included forward-compatibly — released Transformers/vLLM/SGLang have no KimiK31 gated-DSA kernel yet, so the DSA layers run their MLA core.

	\| File path \| Size \|
	\|------\|------\|
	\| model.safetensors \| 35.6MB \|

	### Example usage

	#### Transformers (✅ verified)

	Loads with the pinned K3 reference modeling on GPU (KDA delta attention uses Triton via fla-core). The K3.1 gated-DSA layers run their MLA attention core; the LSA indexer tensors are present but not consumed because released Transformers lacks the gated-DSA operator.

	```python
	def transformers_example(model_id: str):
	import numpy as np
	import torch
	from PIL import Image
	from transformers import AutoModel, AutoProcessor

	model_id = str(model_id)
	processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
	model = AutoModel.from_pretrained(
	model_id,
	dtype=torch.bfloat16,
	device_map="cuda",
	trust_remote_code=True,
	attn_implementation="eager",
	).eval()
	image = Image.fromarray(np.random.default_rng(42).integers(0, 255, (56, 56, 3), dtype=np.uint8))
	messages = [
	{
	"role": "user",
	"content": [
	{"type": "image", "image": image},
	{"type": "text", "text": "Hello, world!"},
	],
	}
	]
	inputs = processor(messages=messages, return_tensors="pt").to(model.device)
	with torch.no_grad():
	outputs = model.generate(**inputs, max_new_tokens=16)
	generated_ids = outputs.sequences if hasattr(outputs, "sequences") else outputs
	print(processor.decode(generated_ids[0].detach().cpu().tolist()))


	if __name__ == "__main__":
	transformers_example(model_id="tiny-random/kimi-k3.1-pro")
	```

	#### vLLM serving (unverified)

	Requires a vLLM build with KimiK31 gated-DSA support (newer than the released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro.

	```bash
	# K3.1 gated-DSA serving requires a vLLM build with KimiK31 DSA support (newer than the
	# released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro
	vllm serve tiny-random/kimi-k3.1-pro \
	--trust-remote-code \
	--tensor-parallel-size 8 \
	--load-format safetensors \
	--max-model-len 131072 \
	--gpu-memory-utilization 0.90
	```

	#### SGLang serving (unverified)

	Requires an SGLang build newer than 0.5.23 with the KimiK31 DSA indexer (the gated-DSA kernels had not landed in the released wheel at tiny-model creation time).

	```bash
	# K3.1 gated-DSA serving requires an SGLang build newer than 0.5.23 (the KimiK31 DSA indexer
	# kernels had not landed in the released wheel at tiny-model creation time).
	python3 -m sglang.launch_server \
	--model-path tiny-random/kimi-k3.1-pro \
	--trust-remote-code \
	--tp-size 8 \
	--chunked-prefill-size 16384 \
	--cuda-graph-backend-prefill breakable \
	--dtype bfloat16 \
	--kv-cache-dtype fp8_e4m3 \
	--reasoning-parser kimi_k3 \
	--tool-call-parser kimi_k3 \
	--mem-fraction-static 0.85
	```

	### Creating this repo

	- Code: [.meta/create_model.py](.meta/create_model.py)
	- Model structure: [.meta/model.log](.meta/model.log)
	- Environment: [.meta/env.json](.meta/env.json)

	### Change log

	- 2026-11-05: Initial version: 17-layer K3.1-Pro (12 KDA + 5 gated-DSA cadence) with the source LSA indexer contract and packed MXFP4 experts, reusing the pinned K3 remote modeling.
	- 2026-11-06: Follow source revision d9e99b0 (updated encoding_k3.py); architectures now `KimiK31ForConditionalGeneration` to match the source config.

	This repo is generated by GPT-6.1-Sol.