train_cola_1757340238

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1522
• Num Input Tokens Seen: 3663512

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0737	0.5	962	0.2573	182656
0.2517	1.0	1924	0.1771	365728
0.2159	1.5	2886	0.1765	548992
0.1765	2.0	3848	0.1651	731984
0.1305	2.5	4810	0.1704	915792
0.33	3.0	5772	0.1675	1098920
0.0959	3.5	6734	0.1576	1281640
0.1044	4.0	7696	0.1552	1465464
0.1593	4.5	8658	0.1579	1649720
0.071	5.0	9620	0.1549	1831920
0.1529	5.5	10582	0.1570	2014928
0.1885	6.0	11544	0.1530	2198176
0.1467	6.5	12506	0.1522	2381440
0.1482	7.0	13468	0.1539	2564952
0.2243	7.5	14430	0.1545	2748568
0.1888	8.0	15392	0.1522	2931096
0.073	8.5	16354	0.1533	3113624
0.0907	9.0	17316	0.1530	3296808
0.0881	9.5	18278	0.1536	3480168
0.1452	10.0	19240	0.1530	3663512

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1