train_copa_1757340231

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1181
• Num Input Tokens Seen: 281408

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2041	0.5	45	0.1785	14016
0.0848	1.0	90	0.1181	28096
0.069	1.5	135	0.1267	42144
0.0283	2.0	180	0.1196	56352
0.1055	2.5	225	0.1192	70432
0.2017	3.0	270	0.1242	84544
0.001	3.5	315	0.1257	98688
0.0121	4.0	360	0.1298	112800
0.0068	4.5	405	0.1304	126816
0.0379	5.0	450	0.1350	140800
0.0014	5.5	495	0.1375	154848
0.0027	6.0	540	0.1372	168736
0.0014	6.5	585	0.1412	182688
0.0013	7.0	630	0.1433	196896
0.0014	7.5	675	0.1439	211072
0.0444	8.0	720	0.1430	225056
0.0927	8.5	765	0.1423	239168
0.0111	9.0	810	0.1446	253312
0.001	9.5	855	0.1462	267392
0.0256	10.0	900	0.1399	281408

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1