train_stsb_42_1760637579

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the stsb dataset. It achieves the following results on the evaluation set:

• Loss: 0.7832
• Num Input Tokens Seen: 7766120

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7028	2.0	2300	0.7534	775200
1.2601	4.0	4600	0.6052	1552456
0.5281	6.0	6900	0.5416	2329712
0.4363	8.0	9200	0.5329	3106792
0.6061	10.0	11500	0.5405	3883352
0.4263	12.0	13800	0.5790	4658104
0.3153	14.0	16100	0.6075	5433200
0.2626	16.0	18400	0.7033	6211064
0.313	18.0	20700	0.7560	6988664
0.3153	20.0	23000	0.7832	7766120

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4