| ## 🚀 Qwen2.5 1.5B Python Coder |
| **Supervised Fine-Tuning (SFT) + VERL Reinforcement Learning** |
|
|
| --- |
|
|
| ### 🧠 Training Overview |
|
|
| #### 🔹 Supervised Fine-Tuning (SFT) |
| - **Hardware**: 2× T4 GPUs (Kaggle) |
| - **Dataset**: https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca |
|
|
| #### 🔹 Reinforcement Learning (VERL) |
| - **Platform**: L4 GPU (Google Colab) |
| - **Samples**: 2,000 |
| - **Dataset**: https://huggingface.co/datasets/KodCode/KodCode-V1-SFT-4o |
| - **Reward Function**: |
| - Based on the **proportion of unit tests passed** |
|
|
| --- |
|
|
| ### 📊 Evaluation |
|
|
| - **Benchmark**: https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard |
|
|
| | Model Variant | Score | |
| |---------------------|-------| |
| | Baseline (Plain) | 0.000 | |
| | After SFT | 0.165 | |
| | After SFT + VERL | 0.287 | |
|
|
| --- |
|
|
| ### ✨ Summary |
|
|
| - SFT provides a strong initial boost in coding capability |
| - VERL further improves performance by reinforcing test-passing behavior |
| - Combined approach yields a **~74% improvement over SFT alone** |
