---
base_model: unsloth/Qwen2.5-0.5B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/Qwen2.5-0.5B-Instruct
- grpo
- lora
- math
- gsm8k
- transformers
- trl
- unsloth
---

# Qwen2.5-0.5B-Instruct — GSM8K Math Reasoning (SFT → GRPO)

LoRA adapter fine-tuned on Qwen2.5-0.5B-Instruct for mathematical reasoning
using a two-stage SFT → GRPO pipeline. Trained on Kaggle T4 GPU.

**IBA Karachi · NLP with Deep Learning · Assignment 04 · Option C**  
**Authors:** Immaduddin Durrani, Raahin Tajuddin, Ibad Khan

## Results

| Stage | Judge Score | vs Baseline |
|-------|------------|-------------|
| Baseline (no fine-tuning) | 5.93/10 | — |
| Best SFT (T1) | 6.40/10 | +7.9% |
| **Best GRPO (G1) ← this model** | **7.03/10** | **+18.6%** |

Evaluated on 30 held-out GSM8K test prompts using LLM-as-Judge
(Llama 3.3-70B via NVIDIA NIM), 4-axis rubric
(Correctness 0-4, Reasoning 0-3, Format 0-2, Conciseness 0-1).

## How to Use

```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("ZalzalaKhan/Qwen2.5-0.5B-GSM8K-GRPO")
tokenizer = AutoTokenizer.from_pretrained("ZalzalaKhan/Qwen2.5-0.5B-GSM8K-GRPO")

messages = [
    {"role": "system", "content": "Reason step by step and clearly provide the final numerical answer."},
    {"role": "user", "content": "Janet has 10 apples. She gives 3 to her friend. How many does she have left?"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
output = model.generate(input_ids, max_new_tokens=256, temperature=0.6, top_p=0.95)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Training Details

### Pipeline Overview

**Stage 1 — SFT (NB-1):** Fine-tuned on MetaMathQA hard split (1,500 samples,
LoRA rank-32, 2 epochs, 8 min on T4). Best trial T1 reached 6.40/10.

**Stage 2 — GRPO (NB-2):** Reinforcement learning from correctness reward on
GSM8K train split (500 samples, group size 8, KL=0.1, 162 min on T4).
Best trial G1 reached 7.03/10.

### GRPO Hyperparameters (G1)

| Parameter | Value |
|-----------|-------|
| Base model | SFT T1 checkpoint |
| LoRA Rank | 32 |
| Target Modules | q_proj, v_proj |
| KL Coefficient | 0.1 |
| Learning Rate | 1e-5 |
| Group Size | 8 |
| Generation Temp | 0.6 |
| Train Samples | 500 (GSM8K) |
| Training Time | 162 min (Kaggle T4) |

### Data

- **SFT:** MetaMathQA hard split (no overlap with test set)
- **GRPO:** GSM8K train split, stratified 400 medium + 100 easy
- **Eval:** 30 held-out GSM8K test prompts (10 easy / 10 medium / 10 hard)

## Framework Versions

- PEFT 0.18.1
- Unsloth 2026.5.8
- TRL 0.24.0
- Transformers 5.5.0
- PyTorch 2.10.0+cu118