moonshotai
/

Kimi-K3

Image-Text-to-Text

feature-extraction

compressed-tensors

8-bit precision

Model card Files Files and versions

Instructions to use moonshotai/Kimi-K3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moonshotai/Kimi-K3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K3", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("moonshotai/Kimi-K3", trust_remote_code=True, device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use moonshotai/Kimi-K3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moonshotai/Kimi-K3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/moonshotai/Kimi-K3

How to use moonshotai/Kimi-K3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moonshotai/Kimi-K3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moonshotai/Kimi-K3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-K3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use moonshotai/Kimi-K3 with Docker Model Runner:
```
docker model run hf.co/moonshotai/Kimi-K3
```

众筹部署全量K3 有想法的铁子聊起来

#115

by tiantianaimax - opened 1 day ago

1 day ago

铁子们，我想众筹把K3部署起来给大家免费用，有想法的这里嗨起来。

1 day ago

你这是非法集资，我买官方api不比你这稳定

1 day ago

全量部署得几百万，你想啥呢

1 day ago

简单搞几百张5090就可以了或者买api更简单

1 day ago

你太伟大了哥们，加我一个

1 day ago

你太伟大了哥们，加我一个

你钱多烧的吗？这种没头没脑的集资搞模型服务最终归属权是谁？你别说归全体出资人，没这样的。

1 day ago

你太伟大了哥们，加我一个>>你钱多烧的吗？这种没头没脑的集资搞模型服务最终归属权是谁？你别说归全体出资人，没有这样的。哈哈我的工作室就是在使用本地部署的模型，况且我们都知道部署k3不可能我们在开玩笑只有你自己当真了吗

1 day ago

你太伟大了哥们，加我一个>>你钱多烧的吗？这种没头没脑的集资搞模型服务最终归属权是谁？你别说归全体出资人，没有这样的。哈哈我的工作室就是在使用本地部署的模型，况且我们都知道部署k3不可能我们在开玩笑只有你自己当真了吗

魔搭曾经真有闹出过评论区搞众筹骗人的

1 day ago

你能特调？你能越狱？如果能。我就入一股

about 12 hours ago

感兴趣的人不少呢

about 12 hours ago

DIY 呗 SFT 、RL 搞起来么， openweight的模型的优势就在这里。

about 12 hours ago

你这是非法集资，我买官方api不比你这稳定

你管众筹叫非法集资？

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment