train_cola_1757340259

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1799
• Num Input Tokens Seen: 3663392

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2934	0.5	962	0.2120	183040
0.3732	1.0	1924	0.2309	366136
0.4703	1.5	2886	0.2004	548856
0.2372	2.0	3848	0.1920	732880
0.3041	2.5	4810	0.1919	916368
0.0637	3.0	5772	0.1799	1099816
0.1252	3.5	6734	0.2229	1283208
0.1097	4.0	7696	0.2023	1465464
0.1015	4.5	8658	0.2159	1648696
0.1285	5.0	9620	0.2113	1831728
0.0953	5.5	10582	0.2402	2014288
0.1351	6.0	11544	0.2271	2198176
0.0715	6.5	12506	0.2463	2382016
0.2271	7.0	13468	0.2445	2564208
0.1723	7.5	14430	0.2490	2746960
0.0247	8.0	15392	0.2439	2930240
0.2369	8.5	16354	0.2467	3113696
0.1645	9.0	17316	0.2471	3297136
0.2399	9.5	18278	0.2480	3480592
0.1377	10.0	19240	0.2435	3663392

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1