train_rte_456_1760637789

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.0871
• Num Input Tokens Seen: 6973272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2115	1.0	561	0.1628	351952
0.0982	2.0	1122	0.1093	702416
0.1085	3.0	1683	0.0984	1052056
0.1084	4.0	2244	0.0960	1400296
0.0963	5.0	2805	0.0905	1748504
0.0802	6.0	3366	0.0906	2097920
0.0906	7.0	3927	0.0877	2447856
0.053	8.0	4488	0.0882	2795952
0.0648	9.0	5049	0.0883	3144128
0.0249	10.0	5610	0.0883	3492600
0.0298	11.0	6171	0.0872	3839488
0.018	12.0	6732	0.0871	4187064
0.0957	13.0	7293	0.0885	4535000
0.0546	14.0	7854	0.0881	4881752
0.0425	15.0	8415	0.0882	5227704
0.0466	16.0	8976	0.0878	5576848
0.0279	17.0	9537	0.0884	5926536
0.0212	18.0	10098	0.0891	6276832
0.0526	19.0	10659	0.0895	6623720
0.0211	20.0	11220	0.0890	6973272

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4