arnavbajpai commited on
Commit
e00aab0
·
verified ·
1 Parent(s): 59e54b1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +90 -0
README.md CHANGED
@@ -1,3 +1,93 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language:
4
+ - en
5
+ base_model: Qwen/Qwen3-1.7B
6
+ pipeline_tag: text-generation
7
+ library_name: transformers
8
+ tags:
9
+ - qwen3
10
+ - math
11
+ - reasoning
12
+ - olympiad-math
13
+ - supervised-fine-tuning
14
+ - lora
15
+ - cs-552
16
+ datasets:
17
+ - EleutherAI/hendrycks_math
18
+ - open-r1/OpenR1-Math-220k
19
+ - AI-MO/NuminaMath-CoT
20
+ metrics:
21
+ - accuracy
22
  ---
23
+
24
+ # Math Model — Olympiad-Focused SFT Checkpoint
25
+
26
+ This model is a fine-tuned version of **Qwen/Qwen3-1.7B** for mathematical reasoning, developed for the CS-552 standard project math track.
27
+
28
+ The model was trained to solve competition-style mathematics problems and produce final answers in boxed LaTeX format.
29
+
30
+ ## Base Model
31
+
32
+ - Base model: `Qwen/Qwen3-1.7B`
33
+ - Fine-tuning method: LoRA supervised fine-tuning
34
+ - Target task: mathematical reasoning and short-answer competition problems
35
+
36
+ ## Training Data
37
+
38
+ The final submitted checkpoint was trained on approximately **25,165** examples from hard mathematical reasoning datasets:
39
+
40
+ | Dataset / Source | Examples |
41
+ |---|---:|
42
+ | Hendrycks MATH | 4,759 |
43
+ | OpenR1-Math-220k | 7,999 |
44
+ | NuminaMath-CoT | 12,407 |
45
+ | **Total** | **25,165** |
46
+
47
+ The NuminaMath subset was filtered to focus on harder mathematical sources:
48
+
49
+ - Olympiads
50
+ - AoPS Forum
51
+ - AMC/AIME
52
+ - MATH
53
+
54
+ The OpenR1 subset was filtered to competition-relevant categories:
55
+
56
+ - Algebra
57
+ - Geometry
58
+ - Number Theory
59
+ - Combinatorics
60
+ - Inequalities
61
+
62
+ ## Training Details
63
+
64
+ | Setting | Value |
65
+ |---|---|
66
+ | Base model | Qwen/Qwen3-1.7B |
67
+ | Fine-tuning method | LoRA |
68
+ | Epochs | 1 |
69
+ | LoRA rank | 32 |
70
+ | LoRA alpha | 64 |
71
+ | LoRA dropout | 0.05 |
72
+ | Learning rate | 1e-4 |
73
+ | Batch size | 1 |
74
+ | Gradient accumulation steps | 8 |
75
+ | Precision | bfloat16 |
76
+ | Hardware | 1 × NVIDIA A100 40GB |
77
+ | Training steps | 3,146 |
78
+ | Runtime | 6,761 seconds |
79
+ | Tokens processed | 14.7M |
80
+ | Final training loss | 0.6114 |
81
+ | Mean token accuracy | 0.8302 |
82
+
83
+ ## Generation Configuration
84
+
85
+ The submitted generation configuration uses sampling:
86
+
87
+ ```json
88
+ {
89
+ "do_sample": true,
90
+ "temperature": 0.25,
91
+ "top_p": 0.85,
92
+ "top_k": 30
93
+ }