train_cola_1757340165

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1561
• Num Input Tokens Seen: 3668584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4767	0.5	962	0.2656	183584
0.1401	1.0	1924	0.1793	366856
0.1743	1.5	2886	0.1719	550664
0.1711	2.0	3848	0.1666	734320
0.0791	2.5	4810	0.1645	918128
0.1537	3.0	5772	0.1631	1100800
0.1057	3.5	6734	0.1621	1284064
0.1309	4.0	7696	0.1591	1467824
0.0793	4.5	8658	0.1600	1650992
0.0978	5.0	9620	0.1576	1834632
0.1469	5.5	10582	0.1569	2018408
0.0666	6.0	11544	0.1594	2202264
0.1117	6.5	12506	0.1582	2386136
0.1871	7.0	13468	0.1584	2568880
0.2852	7.5	14430	0.1561	2751696
0.0893	8.0	15392	0.1565	2935520
0.2126	8.5	16354	0.1570	3119168
0.1809	9.0	17316	0.1572	3302192
0.0832	9.5	18278	0.1566	3485264
0.1004	10.0	19240	0.1570	3668584

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1