train_winogrande_789_1760637957

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 2.8871
• Num Input Tokens Seen: 38393344

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.234	1.0	9090	0.2330	1919360
0.2308	2.0	18180	0.2316	3838064
0.234	3.0	27270	0.2314	5755984
0.2274	4.0	36360	0.2316	7675760
0.2309	5.0	45450	0.2313	9596528
0.2313	6.0	54540	0.2316	11515248
0.2329	7.0	63630	0.2302	13435888
0.2362	8.0	72720	0.2305	15356016
0.2318	9.0	81810	0.2298	17274448
0.2329	10.0	90900	0.2278	19194672
0.2008	11.0	99990	0.2264	21115984
0.2045	12.0	109080	0.2224	23036144
0.2482	13.0	118170	0.2209	24955120
0.2147	14.0	127260	0.2191	26874400
0.2175	15.0	136350	0.2177	28793728
0.2517	16.0	145440	0.2184	30713760
0.1614	17.0	154530	0.2186	32634016
0.182	18.0	163620	0.2185	34554208
0.1544	19.0	172710	0.2176	36474880
0.2015	20.0	181800	0.2182	38393344

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4