train_rte_101112_1760638013

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1049
• Num Input Tokens Seen: 6980984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1633	1.0	561	0.1552	350480
0.0922	2.0	1122	0.0744	700992
0.0854	3.0	1683	0.0500	1050848
0.0312	4.0	2244	0.0514	1400856
0.1382	5.0	2805	0.0433	1749544
0.0079	6.0	3366	0.0453	2099368
0.0504	7.0	3927	0.0411	2447504
0.0083	8.0	4488	0.0563	2794592
0.009	9.0	5049	0.0496	3145760
0.0257	10.0	5610	0.0579	3495600
0.0017	11.0	6171	0.0622	3844488
0.0002	12.0	6732	0.0751	4191800
0.0002	13.0	7293	0.0812	4538416
0.0002	14.0	7854	0.0872	4888904
0.0002	15.0	8415	0.0878	5236560
0.0002	16.0	8976	0.1036	5587768
0.0001	17.0	9537	0.1000	5935088
0.0001	18.0	10098	0.1024	6283144
0.0001	19.0	10659	0.1041	6632504
0.0004	20.0	11220	0.1030	6980984

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4