train_stsb_789_1760637928

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 0.5147
• Num Input Tokens Seen: 8752512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.9829	1.0	1294	1.1232	437440
0.7221	2.0	2588	0.6964	875776
0.6064	3.0	3882	0.6232	1313392
0.4493	4.0	5176	0.5906	1753440
0.4965	5.0	6470	0.5666	2191488
0.44	6.0	7764	0.5572	2629728
0.4313	7.0	9058	0.5460	3066368
0.9524	8.0	10352	0.5379	3502736
0.3738	9.0	11646	0.5304	3940080
0.5899	10.0	12940	0.5268	4376768
0.4901	11.0	14234	0.5220	4813216
0.4776	12.0	15528	0.5217	5251344
0.533	13.0	16822	0.5163	5689568
0.5067	14.0	18116	0.5156	6127200
0.3583	15.0	19410	0.5164	6564352
0.5465	16.0	20704	0.5150	7002400
0.5388	17.0	21998	0.5147	7438560
0.3847	18.0	23292	0.5155	7876400
0.4517	19.0	24586	0.5148	8314032
0.4342	20.0	25880	0.5158	8752512

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4