train_rte_42_1760637557

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.2834
• Num Input Tokens Seen: 6976960

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3462	1.0	561	0.3068	352952
0.2577	2.0	1122	0.3023	701160
0.21	3.0	1683	0.2946	1049376
0.1578	4.0	2244	0.2909	1397896
0.3242	5.0	2805	0.2870	1746728
0.3548	6.0	3366	0.2864	2097448
0.2384	7.0	3927	0.2836	2447040
0.385	8.0	4488	0.2845	2794744
0.2705	9.0	5049	0.2850	3143192
0.0668	10.0	5610	0.2834	3491160
0.4615	11.0	6171	0.2862	3843760
0.2479	12.0	6732	0.2868	4194656
0.4065	13.0	7293	0.2862	4544752
0.1272	14.0	7854	0.2879	4893272
0.0508	15.0	8415	0.2858	5242768
0.3445	16.0	8976	0.2853	5588240
0.2512	17.0	9537	0.2843	5935704
0.1825	18.0	10098	0.2849	6279912
0.3034	19.0	10659	0.2856	6627720
0.4089	20.0	11220	0.2856	6976960

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4