README.md

---
license: llama3.2
datasets:
- LLM360/MegaMath
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- math
- code
- cot
- pal
---

# MegaMath-Llama-3.2-3B

[Arxiv](https://arxiv.org/abs/2504.02807) | [Datasets](https://huggingface.co/datasets/LLM360/MegaMath)

A proof-of-concept model train on [MegaMath](https://huggingface.co/datasets/LLM360/MegaMath) dataset, capable of both Chain-of-Thought and Program-Aided-Language problem solving.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/628f6e5ab90dde28ef57d293/Sw4P-clZhFMxBSNmVAaww.png)

## Performance


![image/png](https://cdn-uploads.huggingface.co/production/uploads/628f6e5ab90dde28ef57d293/nZYsgAj1vhuoKhpJb4ZU7.png)

## Citation
If you find our work useful, please cite
```bibtex
@article{zhou2025megamath,
  title     = {MegaMath: Pushing the Limits of Open Math Corpora},
  author    = {Zhou, Fan and Wang, Zengzhi and Ranjan, Nikhil and Cheng, Zhoujun and Tang, Liping and He, Guowei and Liu, Zhengzhong and Xing, Eric P.},
  journal   = {arXiv preprint arXiv:2504.02807},
  year      = {2025},
  note      = {Preprint}
}
```