Instructions to use IFM/K2-Think-V2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use IFM/K2-Think-V2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="IFM/K2-Think-V2") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("IFM/K2-Think-V2") model = AutoModelForCausalLM.from_pretrained("IFM/K2-Think-V2", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use IFM/K2-Think-V2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "IFM/K2-Think-V2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IFM/K2-Think-V2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/IFM/K2-Think-V2
- SGLang
How to use IFM/K2-Think-V2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "IFM/K2-Think-V2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IFM/K2-Think-V2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "IFM/K2-Think-V2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "IFM/K2-Think-V2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use IFM/K2-Think-V2 with Docker Model Runner:
docker model run hf.co/IFM/K2-Think-V2
Adding chat_template information
Browse filesAdded text to provide information about how we set-up the `reasoning_effort` parameter from K2-V2.
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
base_model:
|
| 3 |
-
- LLM360/K2-V2
|
| 4 |
language:
|
| 5 |
- en
|
| 6 |
library_name: transformers
|
|
@@ -16,13 +16,15 @@ pipeline_tag: text-generation
|
|
| 16 |
|
| 17 |
<br>
|
| 18 |
|
| 19 |
-
K2-Think (70B) is a 70 billion parameter open-weights general reasoning model with strong performance in competitive mathematical problem solving built on-top of [K2-V2](huggingface.co/LLM360/K2-V2), comprising a fully sovereign reasoning system.
|
| 20 |
|
| 21 |
# Quickstart
|
| 22 |
|
| 23 |
### Transformers
|
| 24 |
You can use `K2-Think (70B)` with Transformers. If you use `transformers.pipeline`, it will apply the chat template automatically. If you use `model.generate` directly, you need to apply the chat template mannually.
|
| 25 |
|
|
|
|
|
|
|
| 26 |
```python
|
| 27 |
from transformers import pipeline
|
| 28 |
import torch
|
|
@@ -47,6 +49,28 @@ outputs = pipe(
|
|
| 47 |
print(outputs[0]["generated_text"][-1])
|
| 48 |
```
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
---
|
| 51 |
|
| 52 |
# Evaluation & Performance
|
|
@@ -110,7 +134,7 @@ If you use K2-Think (70B) in your research, please use the following citation:
|
|
| 110 |
```bibtex
|
| 111 |
@misc{k2thinkteam2026k2think70B,
|
| 112 |
title={K2-{T}hink 70{B}: A Fully-Sovereign Reasoning System},
|
| 113 |
-
author={K2-Think Team and Taylor W. Killian and Varad Pimpalkhute and Richard Fan and Haonan Li and Chengqian Gao and Ming Shan Hee and John Maggs and Guowei He and Zhengzhong Liu and Eric P. Xing},
|
| 114 |
year={2026},
|
| 115 |
url={https://tbd.org},
|
| 116 |
}
|
|
|
|
| 1 |
---
|
| 2 |
base_model:
|
| 3 |
+
- LLM360/K2-V2-Instruct
|
| 4 |
language:
|
| 5 |
- en
|
| 6 |
library_name: transformers
|
|
|
|
| 16 |
|
| 17 |
<br>
|
| 18 |
|
| 19 |
+
K2-Think (70B) is a 70 billion parameter open-weights general reasoning model with strong performance in competitive mathematical problem solving built on-top of [K2-V2-Instruct](huggingface.co/LLM360/K2-V2-Instruct), comprising a fully sovereign reasoning system.
|
| 20 |
|
| 21 |
# Quickstart
|
| 22 |
|
| 23 |
### Transformers
|
| 24 |
You can use `K2-Think (70B)` with Transformers. If you use `transformers.pipeline`, it will apply the chat template automatically. If you use `model.generate` directly, you need to apply the chat template mannually.
|
| 25 |
|
| 26 |
+
The chat template is directly inherited from K2-V2-Instruct, with the default `reasoning_effort` set to `"high"`. The other levels of reasoning effort (`"low"` and `"medium"`) are still available but have not been tested or evaluated. As such, the model's behavior under such settings is not assured to maintain reported performance.
|
| 27 |
+
|
| 28 |
```python
|
| 29 |
from transformers import pipeline
|
| 30 |
import torch
|
|
|
|
| 49 |
print(outputs[0]["generated_text"][-1])
|
| 50 |
```
|
| 51 |
|
| 52 |
+
If you cannot use `tokenizer.apply_chat_template`, you may also pass in these arguments using `extra_body` and `chat_template_kwargs`:
|
| 53 |
+
|
| 54 |
+
```
|
| 55 |
+
from openai import OpenAI
|
| 56 |
+
|
| 57 |
+
client = OpenAI(
|
| 58 |
+
base_url="http://localhost:8000/v1",
|
| 59 |
+
api_key="key"
|
| 60 |
+
)
|
| 61 |
+
|
| 62 |
+
completion = client.chat.completions.create(
|
| 63 |
+
model="LLM360/K2-Think-60B",
|
| 64 |
+
messages = [
|
| 65 |
+
{"role": "system", "content": "You are K2-Think, a helpful assistant created by Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) Institute of Foundation Models (IFM)."},
|
| 66 |
+
{"role": "user", "content": "Solve the 24 game [2, 3, 5, 6]"}
|
| 67 |
+
],
|
| 68 |
+
extra_body={
|
| 69 |
+
"chat_template_kwargs": {"reasoning_effort": "high"},
|
| 70 |
+
},
|
| 71 |
+
)
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
---
|
| 75 |
|
| 76 |
# Evaluation & Performance
|
|
|
|
| 134 |
```bibtex
|
| 135 |
@misc{k2thinkteam2026k2think70B,
|
| 136 |
title={K2-{T}hink 70{B}: A Fully-Sovereign Reasoning System},
|
| 137 |
+
author={K2-Think Team and Taylor W. Killian and Varad Pimpalkhute and Richard Fan and Haonan Li and Chengqian Gao and Ming Shan Hee and Xudong Han and John Maggs and Guowei He and Zhengzhong Liu and Eric P. Xing},
|
| 138 |
year={2026},
|
| 139 |
url={https://tbd.org},
|
| 140 |
}
|