train_rte_1756729601

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.5380
• Num Input Tokens Seen: 2923240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1555	0.5004	561	0.1690	148000
0.1769	1.0009	1122	0.1560	292608
0.178	1.5013	1683	0.1559	440304
0.1531	2.0018	2244	0.1722	586640
0.1739	2.5022	2805	0.1546	733968
0.1729	3.0027	3366	0.1425	879160
0.1375	3.5031	3927	0.1440	1025720
0.1783	4.0036	4488	0.1415	1171832
0.1669	4.5040	5049	0.1572	1317624
0.1418	5.0045	5610	0.1529	1464496
0.131	5.5049	6171	0.1565	1612464
0.088	6.0054	6732	0.1698	1755968
0.0291	6.5058	7293	0.2970	1901984
0.0449	7.0062	7854	0.2542	2048856
0.0649	7.5067	8415	0.3933	2193608
0.1111	8.0071	8976	0.4099	2340704
0.0078	8.5076	9537	0.4722	2486032
0.0061	9.0080	10098	0.4967	2632408
0.0013	9.5085	10659	0.5344	2780920

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1