train_copa_123_1760637644

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2325
• Num Input Tokens Seen: 563328

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3653	1.0	90	0.2385	28096
0.2361	2.0	180	0.2344	56128
0.244	3.0	270	0.4452	84352
0.2377	4.0	360	0.2366	112576
0.2286	5.0	450	0.2611	140832
0.2333	6.0	540	0.2369	169056
0.208	7.0	630	0.2651	197344
0.2633	8.0	720	0.2547	225536
0.2563	9.0	810	0.2335	253696
0.2474	10.0	900	0.2349	281856
0.2115	11.0	990	0.2423	310080
0.2385	12.0	1080	0.2326	338144
0.2391	13.0	1170	0.2346	366336
0.2267	14.0	1260	0.2341	394464
0.2321	15.0	1350	0.2352	422592
0.23	16.0	1440	0.2346	450624
0.232	17.0	1530	0.2346	478720
0.2373	18.0	1620	0.2352	507008
0.229	19.0	1710	0.2327	535136
0.2311	20.0	1800	0.2325	563328

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4