train_copa_456_1760637763

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.3642
• Num Input Tokens Seen: 562720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5368	1.0	90	0.6683	28096
0.5056	2.0	180	0.6310	56352
0.7204	3.0	270	0.5381	84544
0.4489	4.0	360	0.4787	112800
0.4704	5.0	450	0.4330	140800
0.2983	6.0	540	0.4173	168736
0.2401	7.0	630	0.3922	196896
0.4421	8.0	720	0.3907	225056
0.3726	9.0	810	0.3852	253312
0.4097	10.0	900	0.3728	281408
0.3804	11.0	990	0.3765	309440
0.302	12.0	1080	0.3738	337536
0.3272	13.0	1170	0.3697	365760
0.2044	14.0	1260	0.3764	393856
0.3384	15.0	1350	0.3772	421952
0.4509	16.0	1440	0.3693	450016
0.2987	17.0	1530	0.3788	478240
0.2864	18.0	1620	0.3862	506368
0.2756	19.0	1710	0.3642	534528
0.2657	20.0	1800	0.3741	562720

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4