train_sst2_42_1763630697

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the sst2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0795
• Num Input Tokens Seen: 30603904

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.31	0.5000	15154	0.1167	1530848
0.0004	1.0000	30308	0.0902	3059440
0.1351	1.5000	45462	0.0868	4588624
0.0202	2.0001	60616	0.0968	6120448
0.2144	2.5001	75770	0.0841	7647648
0.0005	3.0001	90924	0.0885	9179728
0.0016	3.5001	106078	0.0795	10712368
0.1482	4.0001	121232	0.0953	12240816
0.4227	4.5001	136386	0.0884	13771360
0.0007	5.0002	151540	0.0921	15302384
0.0003	5.5002	166694	0.0947	16833680
0.0017	6.0002	181848	0.0958	18362464
0.1466	6.5002	197002	0.1087	19889616
0.0005	7.0002	212156	0.1042	21421568
0.5259	7.5002	227310	0.1066	22953920
0.0002	8.0003	242464	0.1053	24483488
0.0009	8.5003	257618	0.1112	26013248
0.0002	9.0003	272772	0.1089	27544816
0.0001	9.5003	287926	0.1119	29075248

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4