Instructions to use Kwaipilot/KAT-Dev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kwaipilot/KAT-Dev with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kwaipilot/KAT-Dev")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Kwaipilot/KAT-Dev")
model = AutoModelForCausalLM.from_pretrained("Kwaipilot/KAT-Dev")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Kwaipilot/KAT-Dev with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kwaipilot/KAT-Dev"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kwaipilot/KAT-Dev",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Kwaipilot/KAT-Dev

SGLang

How to use Kwaipilot/KAT-Dev with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kwaipilot/KAT-Dev" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kwaipilot/KAT-Dev",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kwaipilot/KAT-Dev" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kwaipilot/KAT-Dev",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Kwaipilot/KAT-Dev with Docker Model Runner:
```
docker model run hf.co/Kwaipilot/KAT-Dev
```

KAT-Dev

Commit History

Update README.md

62012dd
verified

shunxing1234 commited on Oct 14, 2025

Update README.md

7ec96f1
verified

shunxing1234 commited on Oct 10, 2025

Update README.md

6c5678c
verified

shunxing1234 commited on Oct 10, 2025

Update README.md

6a84153
verified

shunxing1234 commited on Oct 10, 2025

Update README.md

21fe6f9
verified

shunxing1234 commited on Oct 10, 2025

Delete LICENSE

b157220
verified

shunxing1234 commited on Oct 10, 2025

Update README.md

5aa63e9
verified

shunxing1234 commited on Oct 10, 2025

Update README.md

674efd4
verified

shunxing1234 commited on Oct 9, 2025

Upload LICENSE

d1fd33f
verified

shunxing1234 commited on Oct 3, 2025

Upload qwen3coder_tool_parser.py

5a0c9ac
verified

shunxing1234 commited on Sep 28, 2025

Update README.md

d0c503b
verified

shunxing1234 commited on Sep 26, 2025

Update README.md

0ea5921
verified

shunxing1234 commited on Sep 26, 2025

Update README.md

19928c5
verified

shunxing1234 commited on Sep 25, 2025

Update README.md

cebdde4
verified

shunxing1234 commited on Sep 25, 2025

Update README.md

2560616
verified

shunxing1234 commited on Sep 25, 2025

update

f1bc1f0

root commited on Sep 23, 2025

Update README.md

e8fd46a
verified

shunxing1234 commited on Sep 15, 2025

Update chat_template.jinja

bb77a7c
verified

hxt commited on Sep 15, 2025

Update README.md

d3728cf
verified

shunxing1234 commited on Sep 15, 2025

Update README.md

2a58a3a
verified

shunxing1234 commited on Sep 15, 2025

Update README.md

75c1193
verified

shunxing1234 commited on Sep 15, 2025

Initial commit

3a5b714

root commited on Sep 15, 2025

initial commit

8699bbd
verified

shunxing1234 commited on Sep 15, 2025

Commit History

Update README.md 62012dd verified

Update README.md 7ec96f1 verified

Update README.md 6c5678c verified

Update README.md 6a84153 verified

Update README.md 21fe6f9 verified

Delete LICENSE b157220 verified

Update README.md 5aa63e9 verified

Update README.md 674efd4 verified

Upload LICENSE d1fd33f verified

Upload qwen3coder_tool_parser.py 5a0c9ac verified

Update README.md d0c503b verified

Update README.md 0ea5921 verified

Update README.md 19928c5 verified

Update README.md cebdde4 verified

Update README.md 2560616 verified

update f1bc1f0

Update README.md e8fd46a verified

Update chat_template.jinja bb77a7c verified

Update README.md d3728cf verified

Update README.md 2a58a3a verified

Update README.md 75c1193 verified

Initial commit 3a5b714

initial commit 8699bbd verified

Update README.md

62012dd
verified

Update README.md

7ec96f1
verified

Update README.md

6c5678c
verified

Update README.md

6a84153
verified

Update README.md

21fe6f9
verified

Delete LICENSE

b157220
verified

Update README.md

5aa63e9
verified

Update README.md

674efd4
verified

Upload LICENSE

d1fd33f
verified

Upload qwen3coder_tool_parser.py

5a0c9ac
verified

Update README.md

d0c503b
verified

Update README.md

0ea5921
verified

Update README.md

19928c5
verified

Update README.md

cebdde4
verified

Update README.md

2560616
verified

update

f1bc1f0

Update README.md

e8fd46a
verified

Update chat_template.jinja

bb77a7c
verified

Update README.md

d3728cf
verified

Update README.md

2a58a3a
verified

Update README.md

75c1193
verified

Initial commit

3a5b714

initial commit

8699bbd
verified