train_winogrande_1754507495

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 0.0651
• Num Input Tokens Seen: 30830624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2623	0.5	4545	0.1974	1541600
0.1042	1.0	9090	0.1176	3081600
0.1129	1.5	13635	0.0953	4623680
0.0374	2.0	18180	0.0856	6165104
0.1131	2.5	22725	0.0791	7706064
0.2129	3.0	27270	0.0749	9248016
0.0589	3.5	31815	0.0731	10789584
0.0501	4.0	36360	0.0695	12330800
0.019	4.5	40905	0.0684	13871920
0.0519	5.0	45450	0.0669	15413776
0.0412	5.5	49995	0.0668	16954320
0.0962	6.0	54540	0.0654	18496992
0.0645	6.5	59085	0.0653	20039264
0.1485	7.0	63630	0.0652	21579792
0.0751	7.5	68175	0.0652	23122160
0.0255	8.0	72720	0.0651	24664400
0.0227	8.5	77265	0.0656	26207280
0.0451	9.0	81810	0.0654	27747856
0.1337	9.5	86355	0.0651	29287888
0.0109	10.0	90900	0.0653	30830624

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1