train_cola_1757340234

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1857
• Num Input Tokens Seen: 3663512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1364	0.5	962	0.2231	182656
0.241	1.0	1924	0.1857	365728
0.2613	1.5	2886	0.2296	548992
0.3619	2.0	3848	0.2091	731984
0.0575	2.5	4810	0.2246	915792
0.5262	3.0	5772	0.2300	1098920
0.1518	3.5	6734	0.2180	1281640
0.1225	4.0	7696	0.2018	1465464
0.2075	4.5	8658	0.2135	1649720
0.023	5.0	9620	0.2038	1831920
0.237	5.5	10582	0.2079	2014928
0.3227	6.0	11544	0.2203	2198176
0.1221	6.5	12506	0.2235	2381440
0.171	7.0	13468	0.2170	2564952
0.3842	7.5	14430	0.2183	2748568
0.3918	8.0	15392	0.2206	2931096
0.1045	8.5	16354	0.2195	3113624
0.0791	9.0	17316	0.2215	3296808
0.0553	9.5	18278	0.2204	3480168
0.2145	10.0	19240	0.2197	3663512

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1