train_stsb_789_1760637925

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 2.0362
• Num Input Tokens Seen: 8752512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.5907	1.0	1294	0.7307	437440
0.563	2.0	2588	0.6334	875776
0.6581	3.0	3882	0.5495	1313392
0.3971	4.0	5176	0.5104	1753440
0.4524	5.0	6470	0.5233	2191488
0.4007	6.0	7764	0.5258	2629728
0.3989	7.0	9058	0.4883	3066368
0.7768	8.0	10352	0.4948	3502736
0.3426	9.0	11646	0.4880	3940080
0.4817	10.0	12940	0.4825	4376768
0.4085	11.0	14234	0.4948	4813216
0.3803	12.0	15528	0.4869	5251344
0.4514	13.0	16822	0.5048	5689568
0.2871	14.0	18116	0.5139	6127200
0.2586	15.0	19410	0.5306	6564352
0.2629	16.0	20704	0.5747	7002400
0.2755	17.0	21998	0.6002	7438560
0.267	18.0	23292	0.6279	7876400
0.183	19.0	24586	0.6496	8314032
0.2444	20.0	25880	0.6525	8752512

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4