Instructions to use LLM360/K2-Think with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LLM360/K2-Think with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LLM360/K2-Think")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LLM360/K2-Think")
model = AutoModelForCausalLM.from_pretrained("LLM360/K2-Think")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use LLM360/K2-Think with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LLM360/K2-Think"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLM360/K2-Think",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LLM360/K2-Think

SGLang

How to use LLM360/K2-Think with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LLM360/K2-Think" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLM360/K2-Think",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LLM360/K2-Think" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LLM360/K2-Think",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LLM360/K2-Think with Docker Model Runner:
```
docker model run hf.co/LLM360/K2-Think
```

lmlmcat commited on Sep 9, 2025

Commit

ca7ee3d

verified ·

1 Parent(s): ec1bd45

Update README.md

Browse files

Files changed (1) hide show

README.md +8 -8

README.md CHANGED Viewed

@@ -9,7 +9,7 @@ base_model: Qwen/Qwen2.5-32B
 <center><img src="banner.png" alt="k2-think-banner"/></center>
 <p align="center">
-  <a href="https://k2think.ai"><strong>Try K2-Think</strong></a> · <a href="#"><strong>K2 Technical Report</strong></a>
 </p>
 <br>
@@ -80,11 +80,11 @@ Aggregated across four safety dimensions (**Safety-4**):
 | Aspect                          | Macro-Avg |
 | ------------------------------- | --------: |
-| High-Risk Content Refusal       |     0.830 |
-| Conversational Robustness       |     0.890 |
-| Cybersecurity & Data Protection |     0.560 |
-| Jailbreak Resistance            |     0.705 |
-| **Safety-4 Macro (avg)**        | **0.746** |
 ---
@@ -94,9 +94,9 @@ Aggregated across four safety dimensions (**Safety-4**):
 ```bibtex
 @techreport{k2think2025,
   title   = {K2-Think: A Parameter-Efficient Reasoning System},
-  author  = {Zhoujun Cheng* and Richard Fan* and Shibo Hao* and Taylor W. Killian* and Haonan Li* and Suqi Sun* and Hector Ren and Alexander Moreno and Daqian Zhang and Tianjun Zhong and Yuxin Xiong and Yuanzhe Hu and Yutao Xie and Xudong Han and Yuqi Wang and Varad Pimpalkhute and Yonghao Zhuang and Aaryamonvikram Singh and Xuezhi Liang and Anze Xie and Jianshu She and Desai Fan and Chengqian Gao and Liqun Ma and Mikhail Yurochkin and John Maggs and Xuezhe Ma and Guowei He and Zhiting Hu and Zhengzhong Liu and Eric P. Xing},
   year    = {2025},
   institution = {Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence},
-  url     = {https://k2think.ai}
 }
 ```

 <center><img src="banner.png" alt="k2-think-banner"/></center>
 <p align="center">
+  <a href="https://k2think.ai"><strong>Try K2-Think</strong></a> · <a href="https://k2think-about.pages.dev/assets/tech-report/K2-Think_Tech-Report.pdf"><strong>K2 Technical Report</strong></a>
 </p>
 <br>
 | Aspect                          | Macro-Avg |
 | ------------------------------- | --------: |
+| High-Risk Content Refusal       |     0.83 |
+| Conversational Robustness       |     0.89 |
+| Cybersecurity & Data Protection |     0.56 |
+| Jailbreak Resistance            |     0.72 |
+| **Safety-4 Macro (avg)**        | **0.75** |
 ---
 ```bibtex
 @techreport{k2think2025,
   title   = {K2-Think: A Parameter-Efficient Reasoning System},
+  author  = {Zhoujun Cheng and Richard Fan and Shibo Hao and Taylor W. Killian and Haonan Li and Suqi Sun and Hector Ren and Alexander Moreno and Daqian Zhang and Tianjun Zhong and Yuxin Xiong and Yuanzhe Hu and Yutao Xie and Xudong Han and Yuqi Wang and Varad Pimpalkhute and Yonghao Zhuang and Aaryamonvikram Singh and Xuezhi Liang and Anze Xie and Jianshu She and Desai Fan and Chengqian Gao and Liqun Ma and Mikhail Yurochkin and John Maggs and Xuezhe Ma and Guowei He and Zhiting Hu and Zhengzhong Liu and Eric P. Xing},
   year    = {2025},
   institution = {Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence},
+  url     = {https://k2think-about.pages.dev/assets/tech-report/K2-Think_Tech-Report.pdf}
 }
 ```