train_copa_1757340181

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0219
• Num Input Tokens Seen: 282368

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0078	0.5	45	0.1139	14176
0.0978	1.0	90	0.0465	28256
0.0487	1.5	135	0.0219	42240
0.0552	2.0	180	0.0483	56480
0.0431	2.5	225	0.0480	70688
0.0	3.0	270	0.0731	84736
0.0	3.5	315	0.0869	98784
0.098	4.0	360	0.0774	113024
0.0	4.5	405	0.0356	127264
0.0	5.0	450	0.0372	141440
0.0	5.5	495	0.0358	155488
0.0	6.0	540	0.0389	169600
0.0	6.5	585	0.0409	183648
0.0	7.0	630	0.0423	197792
0.0	7.5	675	0.0429	211936
0.0	8.0	720	0.0450	225984
0.0	8.5	765	0.0371	239936
0.0	9.0	810	0.0430	254112
0.0	9.5	855	0.0421	268256
0.0	10.0	900	0.0430	282368

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1