Instructions to use Kwaipilot/KAT-Coder-V2.5-Dev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kwaipilot/KAT-Coder-V2.5-Dev with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kwaipilot/KAT-Coder-V2.5-Dev") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Kwaipilot/KAT-Coder-V2.5-Dev") model = AutoModelForMultimodalLM.from_pretrained("Kwaipilot/KAT-Coder-V2.5-Dev", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kwaipilot/KAT-Coder-V2.5-Dev with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kwaipilot/KAT-Coder-V2.5-Dev" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kwaipilot/KAT-Coder-V2.5-Dev", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kwaipilot/KAT-Coder-V2.5-Dev
- SGLang
How to use Kwaipilot/KAT-Coder-V2.5-Dev with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kwaipilot/KAT-Coder-V2.5-Dev" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kwaipilot/KAT-Coder-V2.5-Dev", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kwaipilot/KAT-Coder-V2.5-Dev" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kwaipilot/KAT-Coder-V2.5-Dev", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kwaipilot/KAT-Coder-V2.5-Dev with Docker Model Runner:
docker model run hf.co/Kwaipilot/KAT-Coder-V2.5-Dev
As a non SWE agentic hobbyist I appreciate this model
I’ve got no coding background, I’m just hobbyist who works an electrical trade day to day.
I’ve been using it with hermes, I’m not trying to code applications.
I’m really liking this model, it follows instructions well, works fast, and gives well structured outputs.
I’m using it with llama.cpp on a 5060ti 16gb with n-cpu-MoE 14, 90k q8 context.
Sure when the context gets long it can start to loop or lose track of the goal, but this could be a context compression artifact and it’s something that I see in all other models, as well as the free/cheap cloud models I’ve used like nemotron ultra, deepseek flash v4 Hy3 etc.
It seems to stay focused and not go down rabbit holes better than my second favorite model - ornith 1.0 35b which I find gets confused about what I want from it and it just generally less knowledgeable around tool calling and architecture.
Agents a1 35b on the other hand is great with tool calling but seemed like a liability, I’d ask it to check something and it would start making unauthorised changes.
This is the first feedback I’ve given on huggingface, I just wanted to thank the team and encourage others to try this model.
Keep up the great work.
It's great to hear the model is working well for your workflow with Hermes and llama.cpp. We're particularly happy that you found it follows instructions well, stays focused on the task, and behaves conservatively with tool use. That's exactly the kind of user experience we aimed for during post-training.
Your observation about long-context behavior is fair. We still see cases where performance degrades as context grows very large, and we're actively exploring ways to improve long-horizon reasoning and context management in future versions.