train_rte_789_1760637901

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1069
• Num Input Tokens Seen: 6947288

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0084	1.0	561	0.1088	347936
0.0807	2.0	1122	0.1069	694664
0.0532	3.0	1683	0.1103	1039864
0.1269	4.0	2244	0.1370	1384096
0.0	5.0	2805	0.1608	1732712
0.0	6.0	3366	0.1882	2080184
0.0	7.0	3927	0.2061	2425192
0.0	8.0	4488	0.2188	2772384
0.0	9.0	5049	0.2278	3119968
0.0	10.0	5610	0.2353	3466384
0.0	11.0	6171	0.2406	3817120
0.0	12.0	6732	0.2506	4163160
0.0	13.0	7293	0.2556	4511312
0.0	14.0	7854	0.2638	4861864
0.0	15.0	8415	0.2639	5210208
0.0	16.0	8976	0.2678	5555776
0.0	17.0	9537	0.2681	5902048
0.0	18.0	10098	0.2709	6252128
0.0	19.0	10659	0.2701	6598768
0.0	20.0	11220	0.2707	6947288

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4