Math Tutor - Qwen (LoRA)

Model Description

Math Tutor - Qwen is a QLoRA fine-tuned version of Qwen2.5-1.5B-Instruct trained on the GSM8K dataset. The objective is to improve multi-step mathematical reasoning for grade-school arithmetic word problems.

This repository contains only the LoRA adapter weights and must be loaded together with the original Qwen2.5-1.5B-Instruct base model.

Model Details

Developed by: Moiz Baloch
Model type: LoRA Adapter (PEFT)
Base model: Qwen/Qwen2.5-1.5B-Instruct
Language: English
License: MIT
Fine-tuning method: QLoRA (4-bit)

Intended Use

This model is intended for:

Mathematical reasoning
Step-by-step arithmetic solutions
Educational tutoring
Grade-school math assistance
Learning demonstrations for LLM fine-tuning

Out-of-Scope Use

This model is not intended for:

Medical advice
Legal advice
Financial advice
High-stakes decision making
General knowledge evaluation

Training Dataset

Dataset:

GSM8K (Grade School Math 8K)

7,473 training samples
1,319 evaluation samples

Dataset: https://huggingface.co/datasets/openai/gsm8k

Training Procedure

Fine-tuning Method

QLoRA
4-bit Quantization
PEFT

Hyperparameters

Epochs: 2
Learning Rate: 2e-4
Batch Size: 2
Gradient Accumulation: 4
Optimizer: paged_adamw_8bit

Hardware

Training performed on:

Kaggle Notebook
NVIDIA Tesla T4 GPU (16 GB)

Evaluation

The model was evaluated on the GSM8K evaluation split during training.

Future work includes comparison against the base Qwen model on unseen mathematical reasoning tasks.

Limitations

The model has only been fine-tuned on GSM8K.

It may perform poorly on:

Advanced mathematics
Symbolic algebra
Geometry proofs
Scientific reasoning
Non-English problems

Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct"
)

tokenizer = AutoTokenizer.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct"
)

model = PeftModel.from_pretrained(
    base_model,
    "igmoiiz/math-tutor-qwen-lora"
)

Example

Input

John has 12 apples. He gives 5 to his friend and buys 8 more. How many apples does he have now?

Output

John starts with 12 apples.

After giving away 5:
12 − 5 = 7

After buying 8 more:
7 + 8 = 15

Final Answer: 15 apples.

Citation

If you use this model, please cite the original GSM8K paper:

@article{cobbe2021gsm8k,
  title={Training Verifiers to Solve Math Word Problems},
  author={Cobbe, Karl and others},
  journal={arXiv preprint arXiv:2110.14168},
  year={2021}
}