|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- amd/ReasonLite-Dataset |
|
|
--- |
|
|
|
|
|
|
|
|
|
|
|
<p align="center"> |
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/66a056d0229269a861ac1245/9xXXms4ub9dzbcsN1IGqq.png" alt="ReasonLite" width="200"> |
|
|
</p> |
|
|
|
|
|
<p align="center"> |
|
|
<a href="https://github.com/AMD-AGI/ReasonLite"><b>GitHub</b></a> | |
|
|
<a href="https://huggingface.co/datasets/amd/ReasonLite-Dataset"><b>Dataset</b></a> | |
|
|
<b>Blog</b></a> |
|
|
|
|
|
</p> |
|
|
|
|
|
|
|
|
**ReasonLite is an ultra-lightweight math reasoning model.** With only 0.6B parameters, it leverages **high-quality data distillation** to achieve performance comparable to models over 10× its size, such as Qwen3-8B, **reaching 75.2 on AIME24 and extending the scaling law of small models.** |
|
|
|
|
|
* 🔥 **Best-performing 0.6B math reasoning model** |
|
|
* 🔓 Fully open-source — weights, scripts, datasets, synthesis pipeline |
|
|
* ⚙️ Distilled in two stages to balance **efficiency** and **high performance**, using **6.1M** high-quality samples. |
|
|
|
|
|
|
|
|
<p align="center"> |
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/66a056d0229269a861ac1245/2VZPy7mlgpq9vFvwDc00Q.png"" alt="ReasonLite" height="500"> |
|
|
</p> |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
# 🚀 Model |
|
|
|
|
|
The model is trained in **two progressive distillation stages**. |
|
|
First, short-CoT data is used to distill **Qwen3-0.6B** into **AMD-0.6B-Turbo**, improving **AIME24 accuracy from 11.0 → 57.1**. |
|
|
Then, long-CoT data is used to obtain **AMD-0.6B**, further boosting accuracy to **75.2**. |
|
|
|
|
|
| Model | Description | AIME24 | Link | |
|
|
| ------------------------- | ----------------------------------------------| ------ | ---- | |
|
|
| **amd/ReasonLite-0.6B-Turbo** | Short CoT balancing performance and efficiency | 57.1 | [🤗 HuggingFace](https://huggingface.co/amd/ReasonLite-0.6B-Turbo) | |
|
|
| **amd/ReasonLite-0.6B** | Long CoT for high performance | 75.2 | [🤗 HuggingFace](https://huggingface.co/amd/ReasonLite-0.6B) | |
|
|
|
|
|
--- |
|
|
|
|
|
# 📊 Evaluation Results |
|
|
|
|
|
**Metrics** |
|
|
|
|
|
* **avg@16** — average accuracy from 16 sampled answers |
|
|
* **pass@8** — probability at least one correct answer appears among 8 samples |
|
|
|
|
|
| Model | Parameters | AMC23 avg@16 | AMC23 pass@8 | AIME25 avg@16 | AIME25 pass@8 | AIME24 avg@16 | AIME24 pass@8 | |
|
|
|---------------------------|------------|-------------|-------------|---------------|---------------|---------------|---------------| |
|
|
| Qwen2.5-14B | 14B | 58.3 | 82.3 | 12.3 | 32.3 | 12.7 | 32.4 | |
|
|
| Deepseek-qwen-14B | 14B | 93.9 | 98.7 | 50.2 | 71.0 | 65.0 | 83.0 | |
|
|
| Qwen3-0.6B | 0.6B | 52.7 | 85.0 | 16.0 | 33.0 | 11.0 | 31.5 | |
|
|
| Qwen3-1.7B | 1.7B | 83.4 | 96.3 | 36.0 | 55.1 | 47.3 | 73.9 | |
|
|
| Qwen3-4B | 4B | 96.1 | 100 | 63.5 | 85.4 | 72.7 | 85.1 | |
|
|
| Qwen3-8B | 8B | 94.8 | 100 | 68.3 | 84.2 | 74.6 | 85.0 | |
|
|
| Qwen3-14B | 14B | 98.6 | 98.7 | 71.5 | 84.1 | 78.3 | 88.4 | |
|
|
| DeepscaleR-1.5B | 1.5B | 83.8 | 95.0 | 29.0 | 48.9 | 40.4 | 69.0 | |
|
|
| POLARIS-1.7B-Preview | 1.7B | 92.2 | 97.4 | 52.3 | 80.2 | 65.0 | 76.7 | |
|
|
| OpenMath-Nemotron-1.5B | 1.5B | 88.8 | 96.7 | 39.8 | 65.8 | 61.5 | 81.3 | |
|
|
| ReasonLite-0.6B-Turbo | 0.6B | 81.6 | 99.3 | 42.7 | 69.2 | 57.1 | 79.6 | |
|
|
| **ReasonLite-0.6B** | **0.6B** | **95.2** | **100** | **62.9** | **84.1** | **75.2** | **90.2** | |
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
# 📚 Dataset |
|
|
|
|
|
We collected 343K math problems from Polaris and OpenMathReasoning. Using GPT-OSS as the teacher, we generated 9.1M raw answers under medium and high reasoning modes. We then produced pseudo-labels via majority voting, and finally retained 6.1M samples. |
|
|
|
|
|
| Dataset | Description | Size | Link | |
|
|
| ---------------------- | ------ |---- | ---- | |
|
|
| **amd/ReasonLite-Dataset** | Short CoT | 4.3M | [🤗 HuggingFace](https://huggingface.co/datasets/amd/ReasonLite-Dataset/viewer/default/medium) | |
|
|
| **amd/ReasonLite-Dataset** | Long Cot | 1.8M | [🤗 HuggingFace](https://huggingface.co/datasets/amd/ReasonLite-Dataset/viewer/default/high) | |
|
|
|
|
|
--- |
|
|
|
|
|
# 📌 Citation |
|
|
|
|
|
```bibtex |
|
|
@misc{reasonlite2025, |
|
|
title = {ReasonLite: An Ultra-Lightweight 0.6B Reasoning Model}, |
|
|
author = {An, Zihao and Chen, Chushi and Liu, Ziqiong and Li, Dong and Barsoum, Emad}, |
|
|
year = {2025}, |
|
|
url = {https://github.com/AMD-AGI/ReasonLite}, |
|
|
note = {Open-source project} |
|
|
} |
|
|
``` |
