train_copa_456_1760637764

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1052
• Num Input Tokens Seen: 562720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5191	1.0	90	0.6678	28096
0.0507	2.0	180	0.1193	56352
0.2477	3.0	270	0.1124	84544
0.0426	4.0	360	0.1091	112800
0.0768	5.0	450	0.1081	140800
0.0279	6.0	540	0.1082	168736
0.0106	7.0	630	0.1064	196896
0.0887	8.0	720	0.1052	225056
0.0372	9.0	810	0.1058	253312
0.0663	10.0	900	0.1073	281408
0.0592	11.0	990	0.1070	309440
0.0125	12.0	1080	0.1067	337536
0.0365	13.0	1170	0.1074	365760
0.0131	14.0	1260	0.1112	393856
0.0314	15.0	1350	0.1107	421952
0.0777	16.0	1440	0.1097	450016
0.0145	17.0	1530	0.1097	478240
0.0132	18.0	1620	0.1102	506368
0.0173	19.0	1710	0.1116	534528
0.022	20.0	1800	0.1131	562720

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4