File size: 1,709 Bytes
3c0a600 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | ---
language:
- en
library_name: transformers
tags:
- text-generation
pipeline_tag: text-generation
---
# Koishi 1.5
Koishi 1.5 is an updated version of our Koishi model, fine-tuned specifically to augment conversational data by generating Chain of Thought (CoT) reasoning. It is built upon Qwen 2.5 3B Instruct.
Given an input/output pair, Koishi generates a CoT trace.
## Use Cases
- Updating older datasets with reasoning traces.
- Adding Chain of Thought to instruct model responses for training reasoning models.
- Generating CoT for model responses where the true reasoning process is unavailable.
### Chat Template
The model expects the following structure. Note that Koishi is trained to always begin its generation with `Sure, here's the chain of thought:`.
**Example:**
```
<|im_start|>system
Generate a Chain of Thought chain.<|im_end|>
<|im_start|>user
Input: Where is Paris?
Response: France<|im_end|>
<|im_start|>assistant
```
### Example Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "LucidityAI/Koishi-1.5"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
input_text = "What is the capital of France?"
response_text = "Paris"
messages = [
{"role": "system", "content": "Generate a Chain of Thought chain."},
{"role": "user", "content": f"Input: Where is Paris?\nResponse: France"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
``` |