train_stsb_1753094147

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 0.5502
• Num Input Tokens Seen: 4364240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.5253	0.5	647	1.9189	217472
0.798	1.0	1294	0.8439	435488
0.5855	1.5	1941	0.7013	652480
0.4832	2.0	2588	0.6499	871200
0.4486	2.5	3235	0.6241	1089120
0.6037	3.0	3882	0.6089	1307968
0.5	3.5	4529	0.5952	1529024
0.3736	4.0	5176	0.5833	1745568
0.5145	4.5	5823	0.5771	1965984
0.6988	5.0	6470	0.5683	2182352
0.4154	5.5	7117	0.5630	2399760
0.5992	6.0	7764	0.5602	2619888
0.5624	6.5	8411	0.5578	2837808
0.4731	7.0	9058	0.5548	3057216
0.4235	7.5	9705	0.5525	3275904
0.4538	8.0	10352	0.5528	3493600
0.5556	8.5	10999	0.5509	3712320
0.4933	9.0	11646	0.5510	3928704
0.4055	9.5	12293	0.5502	4147200
0.4789	10.0	12940	0.5507	4364240

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1