| --- |
| license: gemma |
| base_model: google/gemma-4-12b-it |
| tags: [math, aime, competition-math, lora, sft, 1111AI] |
| pipeline_tag: text-generation |
| --- |
| |
| # 1111AI_MathR1-SFT |
| |
| Gemma 4 12B supervised-fine-tuned on contamination-filtered olympiad CoT |
| for AIME 2026, by **mvbalaji / 1111AI_MathR1**. |
| |
| ## Results (AIME 2026, majority vote @ 4) |
| |
| | Model | Score | |
| |-------|-------| |
| | Base Gemma 4 12B-IT | 80.0% | |
| | **+ SFT (this model)** | **20.0%** | |
| | Lift | **-60.0%** | |
| |
| ## Training |
| - LoRA r=128, alpha=256, 1 epochs |
| - Datasets: NuminaMath-CoT, DeepMath-103K, AIME 1983-2024, MATH |
| - 8-gram contamination filter vs AIME 2026 (threshold 0.15) |
| - Hardware: A100/H100 80GB |
| |
| ## Usage |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| import torch |
| m = AutoModelForCausalLM.from_pretrained("mvbalaji/1111AI_MathR1-SFT", torch_dtype=torch.bfloat16, device_map="auto") |
| t = AutoTokenizer.from_pretrained("mvbalaji/1111AI_MathR1-SFT") |
| ``` |
| |
