train_rte_456_1760637788

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.2604
• Num Input Tokens Seen: 6973272

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3744	1.0	561	0.2863	351952
0.247	2.0	1122	0.2786	702416
0.4228	3.0	1683	0.2694	1052056
0.1704	4.0	2244	0.2675	1400296
0.4997	5.0	2805	0.2692	1748504
0.2328	6.0	3366	0.2645	2097920
0.2438	7.0	3927	0.2611	2447856
0.1773	8.0	4488	0.2618	2795952
0.3218	9.0	5049	0.2637	3144128
0.1316	10.0	5610	0.2604	3492600
0.0924	11.0	6171	0.2607	3839488
0.0762	12.0	6732	0.2629	4187064
0.3812	13.0	7293	0.2617	4535000
0.2453	14.0	7854	0.2627	4881752
0.1328	15.0	8415	0.2641	5227704
0.2867	16.0	8976	0.2611	5576848
0.181	17.0	9537	0.2628	5926536
0.1844	18.0	10098	0.2640	6276832
0.2029	19.0	10659	0.2635	6623720
0.1102	20.0	11220	0.2635	6973272

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4