train_rte_789_1760637898

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.7707
• Num Input Tokens Seen: 6189664

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1478	2.0	996	0.2074	619488
0.1495	4.0	1992	0.1772	1240064
0.1632	6.0	2988	0.1543	1860512
0.0409	8.0	3984	0.2595	2474656
0.0753	10.0	4980	0.2932	3093312
0.0513	12.0	5976	0.5241	3709312
0.1141	14.0	6972	0.6726	4331808
0.0001	16.0	7968	0.7528	4951936
0.0	18.0	8964	0.7663	5572096
0.0	20.0	9960	0.7707	6189664

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4