train_copa_789_1760637877

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0584
• Num Input Tokens Seen: 563744

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6696	1.0	90	0.5326	28192
0.1462	2.0	180	0.0699	56192
0.126	3.0	270	0.0657	84192
0.0396	4.0	360	0.0668	112544
0.1156	5.0	450	0.0635	140960
0.0155	6.0	540	0.0645	169216
0.0091	7.0	630	0.0612	197248
0.1638	8.0	720	0.0614	225440
0.0195	9.0	810	0.0608	253632
0.0328	10.0	900	0.0622	281984
0.0279	11.0	990	0.0616	310176
0.0731	12.0	1080	0.0589	338400
0.0441	13.0	1170	0.0590	366688
0.0081	14.0	1260	0.0601	394752
0.0287	15.0	1350	0.0598	422912
0.0147	16.0	1440	0.0593	451008
0.0935	17.0	1530	0.0584	479104
0.0122	18.0	1620	0.0601	507392
0.0685	19.0	1710	0.0614	535584
0.0292	20.0	1800	0.0607	563744

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4