🚀 Qwen2.5 1.5B Python Coder
Supervised Fine-Tuning (SFT) + VERL Reinforcement Learning
🧠 Training Overview
🔹 Supervised Fine-Tuning (SFT)
- Hardware: 2× T4 GPUs (Kaggle)
- Dataset: https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca
🔹 Reinforcement Learning (VERL)
- Platform: L4 GPU (Google Colab)
- Samples: 2,000
- Dataset: https://huggingface.co/datasets/KodCode/KodCode-V1-SFT-4o
- Reward Function:
- Based on the proportion of unit tests passed
📊 Evaluation
| Model Variant | Score |
|---|---|
| Baseline (Plain) | 0.000 |
| After SFT | 0.165 |
| After SFT + VERL | 0.287 |
✨ Summary
- SFT provides a strong initial boost in coding capability
- VERL further improves performance by reinforcing test-passing behavior
- Combined approach yields a ~74% improvement over SFT alone