train_cola_1757340235

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.4594
• Num Input Tokens Seen: 3663512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1236	0.5	962	0.2745	182656
0.2375	1.0	1924	0.1683	365728
0.3172	1.5	2886	0.2014	548992
0.2088	2.0	3848	0.1443	731984
0.0806	2.5	4810	0.1764	915792
0.3512	3.0	5772	0.1655	1098920
0.0369	3.5	6734	0.1680	1281640
0.0703	4.0	7696	0.1568	1465464
0.0718	4.5	8658	0.1608	1649720
0.1062	5.0	9620	0.1466	1831920
0.2303	5.5	10582	0.1536	2014928
0.2191	6.0	11544	0.1693	2198176
0.1416	6.5	12506	0.1756	2381440
0.1436	7.0	13468	0.1585	2564952
0.0112	7.5	14430	0.1843	2748568
0.15	8.0	15392	0.1909	2931096
0.0999	8.5	16354	0.1853	3113624
0.0045	9.0	17316	0.2035	3296808
0.0655	9.5	18278	0.2026	3480168
0.0811	10.0	19240	0.2036	3663512

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1