train_rte_42_1760637553

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.9097
• Num Input Tokens Seen: 6163136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.18	2.0	996	0.1796	610656
0.2014	4.0	1992	0.1461	1228672
0.167	6.0	2988	0.1603	1844416
0.1529	8.0	3984	0.1982	2461152
0.0673	10.0	4980	0.3291	3077440
0.0002	12.0	5976	0.5553	3691680
0.0034	14.0	6972	0.8192	4310720
0.0001	16.0	7968	0.8832	4928480
0.0	18.0	8964	0.9032	5548128
0.0	20.0	9960	0.9097	6163136

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4