train_winogrande_789_1760637955

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.2312
• Num Input Tokens Seen: 38393344

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2321	1.0	9090	0.2316	1919360
0.2263	2.0	18180	0.2314	3838064
0.2319	3.0	27270	0.2314	5755984
0.2288	4.0	36360	0.2313	7675760
0.2303	5.0	45450	0.2314	9596528
0.2299	6.0	54540	0.2316	11515248
0.234	7.0	63630	0.2314	13435888
0.2278	8.0	72720	0.2314	15356016
0.2308	9.0	81810	0.2316	17274448
0.2314	10.0	90900	0.2313	19194672
0.2298	11.0	99990	0.2313	21115984
0.2319	12.0	109080	0.2314	23036144
0.2303	13.0	118170	0.2313	24955120
0.2308	14.0	127260	0.2314	26874400
0.2298	15.0	136350	0.2312	28793728
0.2329	16.0	145440	0.2313	30713760
0.2313	17.0	154530	0.2314	32634016
0.2309	18.0	163620	0.2315	34554208
0.2309	19.0	172710	0.2313	36474880
0.2329	20.0	181800	0.2313	38393344

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4