train_rte_1752763926

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.2959
• Num Input Tokens Seen: 4064632

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
9.5359	0.5018	141	9.4152	203648
7.0247	1.0036	282	6.8418	406928
4.2339	1.5053	423	4.0476	610512
3.027	2.0071	564	2.9776	813088
2.0266	2.5089	705	2.1938	1015968
1.6747	3.0107	846	1.6150	1223224
1.1669	3.5125	987	1.1844	1428600
0.9172	4.0142	1128	0.8680	1631448
0.6215	4.5160	1269	0.6575	1837912
0.5119	5.0178	1410	0.5287	2041584
0.4422	5.5196	1551	0.4399	2244528
0.3739	6.0214	1692	0.3888	2446880
0.3502	6.5231	1833	0.3524	2652384
0.3197	7.0249	1974	0.3305	2855064
0.3087	7.5267	2115	0.3155	3060184
0.3024	8.0285	2256	0.3067	3263920
0.3079	8.5302	2397	0.3018	3467440
0.2873	9.0320	2538	0.2982	3670728
0.2892	9.5338	2679	0.2959	3878344

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1