train_stsb_1754652140

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 2.6397
• Num Input Tokens Seen: 4364240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7201	0.5	647	0.9210	217472
0.6524	1.0	1294	0.8728	435488
0.5352	1.5	1941	0.7179	652480
0.5722	2.0	2588	0.6825	871200
0.486	2.5	3235	0.6683	1089120
0.6085	3.0	3882	0.5942	1307968
0.5084	3.5	4529	0.5508	1529024
0.3727	4.0	5176	0.5626	1745568
0.4597	4.5	5823	0.5266	1965984
0.7069	5.0	6470	0.5204	2182352
0.4146	5.5	7117	0.5107	2399760
0.5929	6.0	7764	0.5058	2619888
0.5626	6.5	8411	0.5047	2837808
0.3735	7.0	9058	0.5093	3057216
0.3429	7.5	9705	0.5020	3275904
0.4286	8.0	10352	0.5120	3493600
0.4575	8.5	10999	0.5040	3712320
0.3782	9.0	11646	0.5048	3928704
0.3722	9.5	12293	0.5039	4147200
0.4486	10.0	12940	0.5031	4364240

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1