train_copa_1757340280

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0275
• Num Input Tokens Seen: 281312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1757	0.5	45	0.0844	14144
0.3782	1.0	90	0.0447	28192
0.0451	1.5	135	0.0391	42208
0.0381	2.0	180	0.0326	56256
0.1683	2.5	225	0.0317	70368
0.0061	3.0	270	0.0296	84320
0.0759	3.5	315	0.0295	98400
0.0138	4.0	360	0.0284	112416
0.1811	4.5	405	0.0293	126496
0.0018	5.0	450	0.0299	140544
0.0038	5.5	495	0.0290	154592
0.0004	6.0	540	0.0281	168768
0.0105	6.5	585	0.0275	182848
0.0076	7.0	630	0.0290	196896
0.0572	7.5	675	0.0279	210912
0.0018	8.0	720	0.0305	225024
0.0037	8.5	765	0.0297	239200
0.0221	9.0	810	0.0277	253152
0.0106	9.5	855	0.0290	267040
0.0802	10.0	900	0.0288	281312

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1