train_rte_789_1760637902

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.2694
• Num Input Tokens Seen: 6947288

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1821	1.0	561	0.2963	347936
0.3367	2.0	1122	0.2897	694664
0.3817	3.0	1683	0.2832	1039864
0.4455	4.0	2244	0.2751	1384096
0.4203	5.0	2805	0.2708	1732712
0.416	6.0	3366	0.2694	2080184
0.4403	7.0	3927	0.2723	2425192
0.1521	8.0	4488	0.2719	2772384
0.3981	9.0	5049	0.2707	3119968
0.0784	10.0	5610	0.2720	3466384
0.3488	11.0	6171	0.2716	3817120
0.2286	12.0	6732	0.2709	4163160
0.1447	13.0	7293	0.2722	4511312
0.3894	14.0	7854	0.2721	4861864
0.1804	15.0	8415	0.2707	5210208
0.244	16.0	8976	0.2700	5555776
0.3173	17.0	9537	0.2710	5902048
0.1673	18.0	10098	0.2714	6252128
0.1376	19.0	10659	0.2718	6598768
0.1808	20.0	11220	0.2736	6947288

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4