train_rte_456_1760637786

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.2154
• Num Input Tokens Seen: 6973272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0669	1.0	561	0.1022	351952
0.0443	2.0	1122	0.0915	702416
0.0637	3.0	1683	0.0765	1052056
0.0332	4.0	2244	0.0885	1400296
0.0156	5.0	2805	0.0791	1748504
0.0473	6.0	3366	0.0924	2097920
0.0419	7.0	3927	0.1230	2447856
0.0217	8.0	4488	0.1281	2795952
0.0277	9.0	5049	0.1019	3144128
0.007	10.0	5610	0.1385	3492600
0.0024	11.0	6171	0.1674	3839488
0.0076	12.0	6732	0.1528	4187064
0.0003	13.0	7293	0.1853	4535000
0.0002	14.0	7854	0.1941	4881752
0.0002	15.0	8415	0.2112	5227704
0.0001	16.0	8976	0.2252	5576848
0.0001	17.0	9537	0.2305	5926536
0.0003	18.0	10098	0.2325	6276832
0.0001	19.0	10659	0.2354	6623720
0.0001	20.0	11220	0.2354	6973272

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4