---
license: apache-2.0
tags:
- openrlhf
- simpo
- sft
- gsm8k
- mathematical-reasoning
- preference-optimization
---

# ASSN2 SimPO+SFT Model

This repository contains the SimPO+SFT model checkpoint for Assignment 2.

## Pipeline

- Base model: `meta-llama/Llama-3.2-1B-Instruct`
- SFT merged checkpoint: `/workspace_data/checkpoints/assn2-sft-merged-hf`
- SimPO initialization: SFT merged checkpoint
- SimPO checkpoint: `/workspace_data/checkpoints/assn2-simpo-sft`

## Evaluation

- SimPO+SFT accuracy: `0.62`

## Notes

This model was produced by applying the reference-free SimPO objective starting from the SFT-merged checkpoint.