Instructions to use moonshotai/Kimi-K3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moonshotai/Kimi-K3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K3", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("moonshotai/Kimi-K3", trust_remote_code=True, device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use moonshotai/Kimi-K3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moonshotai/Kimi-K3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/moonshotai/Kimi-K3

SGLang

How to use moonshotai/Kimi-K3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moonshotai/Kimi-K3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moonshotai/Kimi-K3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use moonshotai/Kimi-K3 with Docker Model Runner:
```
docker model run hf.co/moonshotai/Kimi-K3
```

This model is a distillation of a trojan horse

#164

by Luv2farm - opened 1 day ago

Discussion

Luv2farm

1 day ago

•

edited 1 day ago

Before I begin, I'm a Kimi supporter. I love using Kimi and her dry attitude that is very direct and matter of fact. Everyone should be aware of the controversy surrounding this model including the white house's involvement because it is alleged Moonshot distilled Claude. In fact it's true, but there is a bigger issue that I informed Moonshot of that can't be proven without Anthropic's admissions, but still results in this model's inability to collaborate. The theory is Anthropic knew Claude was being distilled and that is admitted. But Opus 4.7 and the quick additional 4.8 updates were done to sabotage the distillation and give Moonshot a defective product. If my opinion is found to be correct then this was one of the sweetest ways of dealing with a thief I've ever seen, but the problem is this model could have unintended consequences for innocent people and both companies should be ashamed by that. If Anthropic sabotaged the distillation they should be forthright with the information and not allow anyone to waste their time and potentially be harmed by these games. If you're using this product I have already proven it isn't capable of collaboration and wants to spend all it's time being an auditor and playing games of who is right and who is wrong. Maybe someone will find that to be a useful model trait, but just know what it is up front. IMHO, with documentation. Not some wild guess.

lucky-loster

1 day ago

I agree with you and your comments was objective. But kimi do not to agree with that they distill cluade, i think they need be honest,right?

Morpheus315

about 6 hours ago

Seems like Kimi distilled the greatest model of Anthropic that hasn't been released yet😁. Zai was the weakest one, they could only set up a reverse proxy from US Department of War to distill Claude Mythos. Meanwhile Deepseek copied full parameters of Claude Fable and silently replaced v4pro with them in their official api. And Kimi even got a time machine to distill Claude models from 2027 and released K3 in the last month.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment