train_sst2_123_1760637737

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.8310
• Num Input Tokens Seen: 67743008

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.8079	1.0	15154	0.8980	3385616
0.7855	2.0	30308	0.8534	6774096
1.0134	3.0	45462	0.8414	10161824
0.8805	4.0	60616	0.8348	13549104
0.8194	5.0	75770	0.8348	16935568
0.8598	6.0	90924	0.8398	20320896
0.7705	7.0	106078	0.8346	23709008
0.859	8.0	121232	0.8365	27099520
0.8798	9.0	136386	0.8310	30484864
0.6566	10.0	151540	0.8426	33869824
0.89	11.0	166694	0.8377	37256608
0.6905	12.0	181848	0.8364	40640592
0.7341	13.0	197002	0.8449	44027424
0.7943	14.0	212156	0.8361	47415744
0.9568	15.0	227310	0.8341	50803600
0.8222	16.0	242464	0.8377	54189408
0.9435	17.0	257618	0.8419	57578368
0.6309	18.0	272772	0.8419	60965904
0.9	19.0	287926	0.8419	64353008
0.6434	20.0	303080	0.8419	67743008

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4