SmolMath-135M

SmolMath is a full finetuned version of SmolLM2-135M parameter, trained to obtain the highest math accuracy, with least drop in other text benchmarks.

Important: All training codes are present in the Github Important: Please refer to the Blog for methodology and Training details.

Usage

model_path = "Ashed00/SmolMath-135M"                                                                                                                                                                                                                                                                                                # Path where your fine-tuned model is saved
from transformers import pipeline

pipe = pipeline("text-generation", model=model_path)

question = "What is 2+2?"

prompt = "Question: " + question + "\nAnswer:"

output = pipe(
    prompt,
    max_length=100,
    do_sample=False,  # disable sampling for greedy decoding
)[0]["generated_text"]

Evaluation and Performance

Comparision with Base Model

Metrics	SmolLM2-135M-8k	SmolMath-135M	Δ (Change)
HellaSwag	42.1	41.15	−0.95
PIQA	68.4	63.55	−4.85
CommonsenseQA	33.9	33.42	−0.48
TriviaQA	4.1	0.0	−4.10
Winogrande	51.3	51.78	+0.48
OpenBookQA	34.6	30.80	−3.80
GSM8K (0-shot)*	0.0	6.9	+6.90

*This was evaluated using the lighteval script, which is favoured by the SmolLM2 creators in their evaluation and varies from the SmolMath prompt structure.

Math Benchmarks

Model	AddSub* (%)	MAWPS** (%)	GSM8K* (%)
apple/OpenELM-270M-Instruct	2.14	2.83	2.05
HuggingFaceTB/SmolLM2-135M-Instruct	1.52	4.04	0.45
SmolMath-no GRPO (ours)	9.64	7.47	6.22
SmolMath (ours)	12.05	8.31	7.51

*Evaluated only on the test set, not included in the training **Evaluated on complete dataset, not included in the training

Citation

Incase you want to use this model in your work, you can site us.

@misc{SmolMath,
    title = {Building SmolMath: A Math Reasoning SLM Under 150M Parameters},
    url = {https://hackmd.io/@ashu-00/SmolMath},
    author = {ashu-00},
    month = {July},
    year = {2025}
}