train_winogrande_456_1760637845

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the winogrande dataset. It achieves the following results on the evaluation set:

• Loss: 6.6260
• Num Input Tokens Seen: 38395408

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
7.2981	1.0	9090	7.1402	1919808
6.6428	2.0	18180	6.7236	3839104
6.5911	3.0	27270	6.6469	5758016
6.6569	4.0	36360	6.6260	7678560
6.6821	5.0	45450	6.6466	9598912
6.6463	6.0	54540	6.6417	11518656
6.9183	7.0	63630	6.6400	13438320
6.5224	8.0	72720	6.6449	15358064
6.4944	9.0	81810	6.6428	17278064
6.6539	10.0	90900	6.6399	19196144
6.4319	11.0	99990	6.6478	21117200
6.6206	12.0	109080	6.6492	23037584
6.7071	13.0	118170	6.6319	24956720
6.5288	14.0	127260	6.6319	26875344
6.5083	15.0	136350	6.6319	28793344
6.6317	16.0	145440	6.6319	30713568
6.5688	17.0	154530	6.6319	32635088
6.5285	18.0	163620	6.6319	34555376
6.7285	19.0	172710	6.6319	36474544
6.745	20.0	181800	6.6319	38395408

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4