---
license: apache-2.0
language:
- en
tags:
- reasoning
- chain-of-thought
- qwen
- tiny
- whirlwindai
pipeline_tag: text-generation
datasets:
- WhirlwindAI/Soft-CoT-1K
library_name: transformers
base_model:
- Qwen/Qwen2.5-0.5B-Instruct
---
---
# ๐ก The Idea
> **Good answers come from good thinking.**
Qwen-R1-0.5B is a fine-tuned version of Qwen2.5-0.5B-Instruct trained to **reason before it answers** using explicit `` tags.
Instead of jumping straight to the answer, this model generates its reasoning first โ making it more transparent, more reliable, and easier to debug.
---
# ๐ง How It Works
Every response is structured as:
```
User: {question}
Assistant: {reasoning}
{answer}
```
The model learns to:
1. **Think** โ generate step-by-step reasoning
2. **Answer** โ provide the final response
---
# ๐ Training Details
| Property | Value |
|----------|-------|
| Base Model | Qwen2.5-0.5B-Instruct |
| Dataset | WhirlwindAI/Soft-CoT-1K |
| Examples | 1,355 |
| Method | QLoRA (4-bit) |
| Epochs | 3 |
| Learning Rate | 2e-4 |
| LoRA Rank | 16 |
| LoRA Alpha | 32 |
---
# ๐ Quick Start
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "WhirlwindAI/Qwen-R1-0.5B"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)
prompt = "User: What is 2+2?\nAssistant: "
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---
# ๐ Sample Output
```
User: What is the capital of France?
Assistant: Paris is the capital of France.
Paris
```
---
# ๐ Performance
The model was evaluated on 10 out-of-distribution questions:
| Category | Performance |
|----------|-------------|
| Format (thinking tags) | โ
Excellent |
| General Knowledge | โ
Good |
| Creative Reasoning | โ
Good |
| Math/Logic | โ ๏ธ Needs improvement |
| Physics/Science | โ ๏ธ Needs improvement |
---
# ๐ฌ What It Learned
| Strength | Weakness |
|----------|----------|
| โ
Consistent `` format | โ Sometimes hallucinates facts |
| โ
Generates reasoning before answering | โ Struggles with multi-step math |
| โ
Retains general knowledge | โ Physics reasoning needs more data |
---
# ๐งช Test It Yourself
```python
questions = [
"What is the capital of France?",
"Explain entropy like I'm 5.",
"Write a short poem about a robot.",
]
for q in questions:
prompt = f"User: {q}\nAssistant: "
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=80)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---
# ๐ Citation
```bibtex
@model{qwenr1_2026,
title={Qwen-R1-0.5B},
author={WhirlwindAI},
year={2026},
publisher={Hugging Face}
}
```
---
### ๐ช๏ธ WhirlwindAI
**Efficient Models โข Practical Research โข Open AI**
