train_cola_1757340189

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1547
• Num Input Tokens Seen: 3669168

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2618	0.5	962	0.2586	184192
0.2482	1.0	1924	0.1782	367320
0.2209	1.5	2886	0.1775	550840
0.1899	2.0	3848	0.1634	734600
0.2056	2.5	4810	0.1659	918600
0.2991	3.0	5772	0.1640	1101216
0.2078	3.5	6734	0.1624	1284288
0.2235	4.0	7696	0.1623	1468552
0.0837	4.5	8658	0.1574	1651528
0.1162	5.0	9620	0.1660	1834816
0.1533	5.5	10582	0.1557	2018016
0.0846	6.0	11544	0.1547	2201584
0.157	6.5	12506	0.1639	2385200
0.0713	7.0	13468	0.1584	2568288
0.1789	7.5	14430	0.1560	2751584
0.1127	8.0	15392	0.1593	2935056
0.187	8.5	16354	0.1566	3118000
0.1747	9.0	17316	0.1572	3301760
0.1222	9.5	18278	0.1569	3485344
0.2532	10.0	19240	0.1569	3669168

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1