train_copa_1757340228

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1133
• Num Input Tokens Seen: 281408

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1925	0.5	45	0.1451	14016
0.2215	1.0	90	0.1133	28096
0.1174	1.5	135	0.1259	42144
0.1442	2.0	180	0.1162	56352
0.1186	2.5	225	0.1170	70432
0.2272	3.0	270	0.1436	84544
0.0004	3.5	315	0.1587	98688
0.0823	4.0	360	0.1553	112800
0.0133	4.5	405	0.1595	126816
0.0796	5.0	450	0.1653	140800
0.0007	5.5	495	0.1703	154848
0.002	6.0	540	0.1747	168736
0.0014	6.5	585	0.1790	182688
0.0013	7.0	630	0.1758	196896
0.0012	7.5	675	0.1775	211072
0.0293	8.0	720	0.1754	225056
0.031	8.5	765	0.1780	239168
0.0176	9.0	810	0.1784	253312
0.0006	9.5	855	0.1767	267392
0.1336	10.0	900	0.1792	281408

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1