train_winogrande_1754507497

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.0618
• Num Input Tokens Seen: 30830624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1766	0.5	4545	0.1348	1541600
0.1051	1.0	9090	0.0875	3081600
0.086	1.5	13635	0.0729	4623680
0.0054	2.0	18180	0.0678	6165104
0.096	2.5	22725	0.0674	7706064
0.1931	3.0	27270	0.0618	9248016
0.0674	3.5	31815	0.0738	10789584
0.0167	4.0	36360	0.0667	12330800
0.002	4.5	40905	0.0705	13871920
0.0024	5.0	45450	0.0733	15413776
0.0254	5.5	49995	0.0855	16954320
0.0323	6.0	54540	0.0835	18496992
0.0678	6.5	59085	0.0960	20039264
0.0986	7.0	63630	0.0982	21579792
0.0024	7.5	68175	0.1056	23122160
0.0009	8.0	72720	0.1139	24664400
0.0006	8.5	77265	0.1227	26207280
0.0002	9.0	81810	0.1261	27747856
0.0003	9.5	86355	0.1236	29287888
0.0002	10.0	90900	0.1245	30830624

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1