train_copa_1754652161

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2702
• Num Input Tokens Seen: 281856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
12.1438	0.5	45	11.9527	14016
8.1873	1.0	90	7.8263	28096
4.4943	1.5	135	4.2840	42144
2.6224	2.0	180	2.3926	56128
1.0627	2.5	225	0.9169	70272
0.4207	3.0	270	0.4399	84352
0.3508	3.5	315	0.3616	98464
0.3926	4.0	360	0.3224	112576
0.3482	4.5	405	0.3299	126624
0.3226	5.0	450	0.2843	140832
0.2539	5.5	495	0.2821	154976
0.2535	6.0	540	0.2891	169056
0.2596	6.5	585	0.2706	183200
0.2504	7.0	630	0.2804	197344
0.2704	7.5	675	0.2717	211392
0.2719	8.0	720	0.2754	225536
0.2558	8.5	765	0.2734	239680
0.261	9.0	810	0.2730	253696
0.2524	9.5	855	0.2721	267840
0.2772	10.0	900	0.2702	281856

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1