train_cola_1757340164

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1750
• Num Input Tokens Seen: 3668584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.315	0.5	962	0.2051	183584
0.1334	1.0	1924	0.1750	366856
0.1928	1.5	2886	0.1785	550664
0.1384	2.0	3848	0.1808	734320
0.039	2.5	4810	0.1766	918128
0.1714	3.0	5772	0.1766	1100800
0.0943	3.5	6734	0.1781	1284064
0.0741	4.0	7696	0.1813	1467824
0.0154	4.5	8658	0.1859	1650992
0.0351	5.0	9620	0.1852	1834632
0.1571	5.5	10582	0.1827	2018408
0.0239	6.0	11544	0.2080	2202264
0.0864	6.5	12506	0.2007	2386136
0.1876	7.0	13468	0.1903	2568880
0.3352	7.5	14430	0.2047	2751696
0.0101	8.0	15392	0.1963	2935520
0.1877	8.5	16354	0.2024	3119168
0.2306	9.0	17316	0.2038	3302192
0.0095	9.5	18278	0.2047	3485264
0.1085	10.0	19240	0.2040	3668584

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1