train_winogrande_1754507496

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.0448
• Num Input Tokens Seen: 30830624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0674	0.5	4545	0.0677	1541600
0.0179	1.0	9090	0.0493	3081600
0.0014	1.5	13635	0.0663	4623680
0.0031	2.0	18180	0.0448	6165104
0.0041	2.5	22725	0.0593	7706064
0.1068	3.0	27270	0.0530	9248016
0.0001	3.5	31815	0.0757	10789584
0.0549	4.0	36360	0.0706	12330800
0.0001	4.5	40905	0.0856	13871920
0.0	5.0	45450	0.0821	15413776
0.0001	5.5	49995	0.0772	16954320
0.0	6.0	54540	0.0772	18496992
0.0	6.5	59085	0.0867	20039264
0.0	7.0	63630	0.1004	21579792
0.0	7.5	68175	0.1003	23122160
0.0	8.0	72720	0.1206	24664400
0.0	8.5	77265	0.1284	26207280
0.0	9.0	81810	0.1297	27747856
0.0	9.5	86355	0.1332	29287888
0.0	10.0	90900	0.1332	30830624

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1