metadata
license: gemma
base_model: google/gemma-4-12b-it
tags:
- math
- aime
- competition-math
- lora
- sft
- 1111AI
pipeline_tag: text-generation
1111AI_MathR1-SFT
Gemma 4 12B supervised-fine-tuned on contamination-filtered olympiad CoT for AIME 2026, by mvbalaji / 1111AI_MathR1.
Results (AIME 2026, majority vote @ 4)
| Model | Score |
|---|---|
| Base Gemma 4 12B-IT | 80.0% |
| + SFT (this model) | 20.0% |
| Lift | -60.0% |
Training
- LoRA r=128, alpha=256, 1 epochs
- Datasets: NuminaMath-CoT, DeepMath-103K, AIME 1983-2024, MATH
- 8-gram contamination filter vs AIME 2026 (threshold 0.15)
- Hardware: A100/H100 80GB
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
m = AutoModelForCausalLM.from_pretrained("mvbalaji/1111AI_MathR1-SFT", torch_dtype=torch.bfloat16, device_map="auto")
t = AutoTokenizer.from_pretrained("mvbalaji/1111AI_MathR1-SFT")