train_copa_101112_1760637990

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2454
• Num Input Tokens Seen: 562848

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.9372	1.0	90	0.5577	28192
0.4927	2.0	180	0.4810	56256
0.4039	3.0	270	0.4305	84320
0.3324	4.0	360	0.3752	112416
0.3169	5.0	450	0.3170	140544
0.234	6.0	540	0.2961	168768
0.2645	7.0	630	0.2850	196896
0.3519	8.0	720	0.2605	225024
0.4317	9.0	810	0.2576	253152
0.5063	10.0	900	0.2590	281312
0.4017	11.0	990	0.2540	309280
0.3525	12.0	1080	0.2536	337536
0.2926	13.0	1170	0.2568	365632
0.485	14.0	1260	0.2578	393632
0.6499	15.0	1350	0.2454	421696
0.2837	16.0	1440	0.2583	449984
0.3363	17.0	1530	0.2573	478016
0.2509	18.0	1620	0.2586	506272
0.2798	19.0	1710	0.2472	534432
0.4574	20.0	1800	0.2472	562848

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4