| license: apache-2.0 | |
| tags: | |
| - openrlhf | |
| - simpo | |
| - sft | |
| - gsm8k | |
| - mathematical-reasoning | |
| - preference-optimization | |
| # ASSN2 SimPO+SFT Model | |
| This repository contains the SimPO+SFT model checkpoint for Assignment 2. | |
| ## Pipeline | |
| - Base model: `meta-llama/Llama-3.2-1B-Instruct` | |
| - SFT merged checkpoint: `/workspace_data/checkpoints/assn2-sft-merged-hf` | |
| - SimPO initialization: SFT merged checkpoint | |
| - SimPO checkpoint: `/workspace_data/checkpoints/assn2-simpo-sft` | |
| ## Evaluation | |
| - SimPO+SFT accuracy: `0.62` | |
| ## Notes | |
| This model was produced by applying the reference-free SimPO objective starting from the SFT-merged checkpoint. | |