train_cola_1757340162

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 2.5707
• Num Input Tokens Seen: 3668584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3787	0.5	962	0.2067	183584
0.1551	1.0	1924	0.1908	366856
0.178	1.5	2886	0.1650	550664
0.1303	2.0	3848	0.1713	734320
0.0829	2.5	4810	0.1501	918128
0.1613	3.0	5772	0.1431	1100800
0.0503	3.5	6734	0.1495	1284064
0.0731	4.0	7696	0.1464	1467824
0.0135	4.5	8658	0.1622	1650992
0.0492	5.0	9620	0.1628	1834632
0.1138	5.5	10582	0.1768	2018408
0.0079	6.0	11544	0.1696	2202264
0.1319	6.5	12506	0.1881	2386136
0.1117	7.0	13468	0.1781	2568880
0.2818	7.5	14430	0.1997	2751696
0.0057	8.0	15392	0.2113	2935520
0.0057	8.5	16354	0.2260	3119168
0.2071	9.0	17316	0.2144	3302192
0.0056	9.5	18278	0.2165	3485264
0.0051	10.0	19240	0.2195	3668584

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1