train_copa_1754652160

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.1383
• Num Input Tokens Seen: 281856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5598	0.5	45	0.3143	14016
1.6129	1.0	90	0.2823	28096
0.8986	1.5	135	0.3364	42144
0.162	2.0	180	0.1252	56128
0.0545	2.5	225	0.1659	70272
0.0493	3.0	270	0.1168	84352
0.0166	3.5	315	0.1661	98464
0.0146	4.0	360	0.1141	112576
0.1392	4.5	405	0.1262	126624
0.0007	5.0	450	0.1610	140832
0.0002	5.5	495	0.2902	154976
0.0003	6.0	540	0.1879	169056
0.0013	6.5	585	0.2377	183200
0.0001	7.0	630	0.2483	197344
0.0002	7.5	675	0.2539	211392
0.0001	8.0	720	0.2521	225536
0.0001	8.5	765	0.2462	239680
0.0001	9.0	810	0.2545	253696
0.0001	9.5	855	0.2486	267840
0.0001	10.0	900	0.2497	281856

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1