mamoeed/Mellum2-12B-post-post-train

Mellum2-12B-SFT-DPO-Experiment

An experimental SFT and DPO fine-tune of JetBrains/Mellum2-12B-A2.5B-Thinking using a multi-GPU setup. This model was trained to test distributed training pipelines and data filtering techniques on small datasets. We release 2 model checkpoints. A SFT only trained version and a DPO (with SFT) version.

Pipeline Details

• SFT Data: 3,000 decontaminated medium/hard samples from nvidia/OpenCodeReasoning.
• DPO Data: 2,439 sandbox-validated pairs from allenai/Dolci-Think-RL-7B-Completions-DPO.
• Hardware: 4x NVIDIA A100 (80GB) utilizing PyTorch FSDP full sharding.
• Precision: BF16

Results & Limitations

When tested against LiveCodeBench v6, the model's Pass@1 score dropped from 0.30 (base) to 0.19. The model appears to have overfitted to the specific reasoning trace formatting of the training data, which introduced syntax styling changes that impacted the evaluation suite.

This model checkpoint is uploaded for infrastructure validation and reproducibility analysis.