train_rte_789_1760637903

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.0799
• Num Input Tokens Seen: 6947288

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1194	1.0	561	0.1887	347936
0.0953	2.0	1122	0.1194	694664
0.113	3.0	1683	0.1061	1039864
0.1167	4.0	2244	0.0975	1384096
0.0817	5.0	2805	0.0923	1732712
0.0809	6.0	3366	0.0877	2080184
0.0758	7.0	3927	0.0884	2425192
0.0417	8.0	4488	0.0839	2772384
0.0766	9.0	5049	0.0832	3119968
0.0119	10.0	5610	0.0812	3466384
0.0392	11.0	6171	0.0822	3817120
0.0495	12.0	6732	0.0833	4163160
0.0462	13.0	7293	0.0812	4511312
0.0216	14.0	7854	0.0808	4861864
0.0168	15.0	8415	0.0813	5210208
0.0874	16.0	8976	0.0814	5555776
0.1341	17.0	9537	0.0799	5902048
0.0646	18.0	10098	0.0805	6252128
0.0512	19.0	10659	0.0804	6598768
0.0133	20.0	11220	0.0801	6947288

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4