train_rte_101112_1760638012

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1180
• Num Input Tokens Seen: 6980984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1662	1.0	561	0.1547	350480
0.1516	2.0	1122	0.1623	700992
0.1574	3.0	1683	0.1553	1050848
0.1538	4.0	2244	0.1548	1400856
0.1563	5.0	2805	0.1535	1749544
0.1569	6.0	3366	0.1531	2099368
0.149	7.0	3927	0.1562	2447504
0.1617	8.0	4488	0.1577	2794592
0.1572	9.0	5049	0.1547	3145760
0.1643	10.0	5610	0.1578	3495600
0.139	11.0	6171	0.1322	3844488
0.1544	12.0	6732	0.1293	4191800
0.1003	13.0	7293	0.1227	4538416
0.1138	14.0	7854	0.1242	4888904
0.1415	15.0	8415	0.1180	5236560
0.0987	16.0	8976	0.1280	5587768
0.1316	17.0	9537	0.1340	5935088
0.0492	18.0	10098	0.1550	6283144
0.0644	19.0	10659	0.1651	6632504
0.1374	20.0	11220	0.1674	6980984

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4