Instructions to use moonshotai/Kimi-K3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moonshotai/Kimi-K3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K3", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("moonshotai/Kimi-K3", trust_remote_code=True, device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use moonshotai/Kimi-K3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moonshotai/Kimi-K3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/moonshotai/Kimi-K3

SGLang

How to use moonshotai/Kimi-K3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moonshotai/Kimi-K3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moonshotai/Kimi-K3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use moonshotai/Kimi-K3 with Docker Model Runner:
```
docker model run hf.co/moonshotai/Kimi-K3
```

Go Kimi Go | Kimi K3 Architecture Overview (from my understanding)

#73

by pop123ux - opened about 3 hours ago

Discussion

pop123ux

about 3 hours ago

🛠️ What Makes Kimi K3 Special? Architectural Highlights

Building a 2.8T model that can run at scale requires throwing out old scaling rules. Moonshot AI introduced four core structural innovations to achieve a 2.5× scaling efficiency boost over their previous generation:

Kimi Delta Attention (KDA) & Attention Residuals (AttnRes): Out of its 93 layers, 69 feature KDA paired with 24 Gated MLA (Multi-head Latent Attention) layers. This hybrid setup ensures smooth information flow across long contexts while maintaining high speed.
Stable LatentMoE (16 / 896 Experts): K3 pushes MoE routing to extreme sparsity. Out of 896 available experts, only 16 are activated per token (alongside 2 shared experts), bringing active parameter cost down to 104B per forward pass.
Native MXFP4 Quantization: Trained from the SFT stage onward using MXFP4 weights and MXFP8 activations, delivering hardware-native low-bit efficiency right out of the box.
1M Token Context & Native Vision: K3 handles long-sequence text, images, and video inside a single unified model (backed by the 401M parameter MoonViT-V2 vision encoder).

💡 Active vs. Total Scale: Although Kimi K3 holds 2.8T parameters on disk, you only pay the compute cost of 104B active parameters per token pass.

📊 Benchmark Breakdown: How Does It Compare?

Kimi K3 goes toe-to-toe with the world's most powerful closed models across long-horizon coding, complex visual reasoning, and tool use:

🧠 Reasoning & Knowledge:
GPQA Diamond: GPT-5.6 Sol (94.1) > Kimi K3 (93.5) > Claude Fable 5 (92.6)

AA-LCR: Kimi K3 (74.7) > GPT-5.6 Sol (73.7) > Claude Fable 5 (70.0)

💻 Coding & Engineering:
Terminal-Bench 2.1: GPT-5.6 Sol (88.8) > Kimi K3 (88.3) > Claude Fable 5 (88.0)

ProgramBench: Kimi K3 (77.8) > GPT-5.6 Sol (77.6) > Claude Fable 5 (76.8)

SWE-Marathon: Kimi K3 (42.0) > GPT-5.6 Sol (39.0) > Claude Fable 5 (35.0)

🤖 Agentic Workflows:
MCPMark-Verified: Kimi K3 (94.5) > GPT-5.6 Sol (92.9) > Claude Fable 5 (87.4)

AgenticBrowseComp: Kimi K3 (91.2) > GPT-5.6 Sol (90.4) > Claude Fable 5 (88.0)

Harvey Lab-AA (Legal): Kimi K3 (94.6) > Claude Fable 5 (93.6) > GPT-5.6 Sol (87.2)

👁️ Multimodal:
Video-MME (w/ sub): Kimi K3 (90.0) > GPT-5.6 Sol (89.5) (Claude Fable 5: N/A)

OmniDocBench: Kimi K3 (91.1) > Claude Fable 5 (89.8) > GPT-5.6 Sol (85.8)

Key Takeaways:

1 - Agentic Dominance: On agent-heavy benchmarks like SWE-Marathon (42.0) and MCPMark (94.5), Kimi K3 sets new open-weights standards, showcasing incredible stamina for long engineering sessions without human intervention.

2 - Native Visual Reasoning: Achieving 90.0 on Video-MME and 91.1 on OmniDocBench, K3 excels at analyzing dense multi-page documents, CAD files, UI screenshots, and video streams.

💬 Community Discussion: Let's Talk!

The release of open 3T-class weights is a massive shift in AI accessibility. We’d love to hear how you plan to use Kimi K3:
What long-horizon task will you test first? (Full repository refactoring, PDF document extraction, or long video understanding?)
Quantization & Fine-Tuning: Are you planning to build INT4/GGUF community quants or fine-tune specific MoE experts?
Agent Frameworks: Have you integrated K3 into your MCP (Model Context Protocol) pipelines yet?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment