train_rte_42_1760637555

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the rte dataset. It achieves the following results on the evaluation set:

• Loss: 0.1057
• Num Input Tokens Seen: 6976960

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2133	1.0	561	0.1567	352952
0.1579	2.0	1122	0.1577	701160
0.089	3.0	1683	0.0629	1049376
0.048	4.0	2244	0.0499	1397896
0.0098	5.0	2805	0.0454	1746728
0.0447	6.0	3366	0.0510	2097448
0.0038	7.0	3927	0.0501	2447040
0.0245	8.0	4488	0.0595	2794744
0.0005	9.0	5049	0.0739	3143192
0.0054	10.0	5610	0.0676	3491160
0.0832	11.0	6171	0.0710	3843760
0.0018	12.0	6732	0.0733	4194656
0.0704	13.0	7293	0.0888	4544752
0.0003	14.0	7854	0.1021	4893272
0.0002	15.0	8415	0.1064	5242768
0.0002	16.0	8976	0.1130	5588240
0.0003	17.0	9537	0.1191	5935704
0.0002	18.0	10098	0.1252	6279912
0.0005	19.0	10659	0.1297	6627720
0.0001	20.0	11220	0.1274	6976960

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4