train_rte_789_1760637899

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1376
• Num Input Tokens Seen: 6947288

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1452	1.0	561	0.1747	347936
0.1532	2.0	1122	0.1554	694664
0.156	3.0	1683	0.1555	1039864
0.1543	4.0	2244	0.1552	1384096
0.1582	5.0	2805	0.1558	1732712
0.1527	6.0	3366	0.1549	2080184
0.1544	7.0	3927	0.1545	2425192
0.1498	8.0	4488	0.1556	2772384
0.1369	9.0	5049	0.1555	3119968
0.1412	10.0	5610	0.1529	3466384
0.1395	11.0	6171	0.1451	3817120
0.1415	12.0	6732	0.1489	4163160
0.1445	13.0	7293	0.1376	4511312
0.1311	14.0	7854	0.1419	4861864
0.143	15.0	8415	0.1378	5210208
0.1211	16.0	8976	0.1605	5555776
0.0537	17.0	9537	0.1531	5902048
0.1002	18.0	10098	0.1813	6252128
0.0431	19.0	10659	0.1902	6598768
0.0367	20.0	11220	0.1904	6947288

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4