metadata
license: mit
language:
- en
base_model: Qwen/Qwen3-1.7B
pipeline_tag: text-generation
library_name: transformers
tags:
- qwen3
- math
- reasoning
- olympiad-math
- supervised-fine-tuning
- lora
- cs-552
datasets:
- EleutherAI/hendrycks_math
- open-r1/OpenR1-Math-220k
- AI-MO/NuminaMath-CoT
metrics:
- accuracy
Math Model — Olympiad-Focused SFT Checkpoint
This model is a fine-tuned version of Qwen/Qwen3-1.7B for mathematical reasoning, developed for the CS-552 standard project math track.
The model was trained to solve competition-style mathematics problems and produce final answers in boxed LaTeX format.
Base Model
- Base model:
Qwen/Qwen3-1.7B - Fine-tuning method: LoRA supervised fine-tuning
- Target task: mathematical reasoning and short-answer competition problems
Training Data
The final submitted checkpoint was trained on approximately 25,165 examples from hard mathematical reasoning datasets:
| Dataset / Source | Examples |
|---|---|
| Hendrycks MATH | 4,759 |
| OpenR1-Math-220k | 7,999 |
| NuminaMath-CoT | 12,407 |
| Total | 25,165 |
The NuminaMath subset was filtered to focus on harder mathematical sources:
- Olympiads
- AoPS Forum
- AMC/AIME
- MATH
The OpenR1 subset was filtered to competition-relevant categories:
- Algebra
- Geometry
- Number Theory
- Combinatorics
- Inequalities
Training Details
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen3-1.7B |
| Fine-tuning method | LoRA |
| Epochs | 1 |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| LoRA dropout | 0.05 |
| Learning rate | 1e-4 |
| Batch size | 1 |
| Gradient accumulation steps | 8 |
| Precision | bfloat16 |
| Hardware | 1 × NVIDIA A100 40GB |
| Training steps | 3,146 |
| Runtime | 6,761 seconds |
| Tokens processed | 14.7M |
| Final training loss | 0.6114 |
| Mean token accuracy | 0.8302 |
Generation Configuration
The submitted generation configuration uses sampling:
{
"do_sample": true,
"temperature": 0.25,
"top_p": 0.85,
"top_k": 30
}