train_cola_456_1760637820

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 1.1815
• Num Input Tokens Seen: 7334376

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.7972	1.0	1924	1.3528	366712
1.1318	2.0	3848	1.2295	734016
1.1001	3.0	5772	1.2147	1100824
1.1073	4.0	7696	1.2094	1467248
1.3069	5.0	9620	1.2004	1834568
1.0908	6.0	11544	1.2137	2201464
1.051	7.0	13468	1.1887	2568040
1.3433	8.0	15392	1.1878	2934360
1.3921	9.0	17316	1.1815	3301448
1.4202	10.0	19240	1.1946	3668312
1.2241	11.0	21164	1.1932	4034856
0.9093	12.0	23088	1.1956	4401344
1.2531	13.0	25012	1.1913	4767736
0.9851	14.0	26936	1.1841	5134344
1.1953	15.0	28860	1.2019	5501408
1.1277	16.0	30784	1.1983	5867920
1.3087	17.0	32708	1.2029	6234920
1.2584	18.0	34632	1.2029	6601944
0.8433	19.0	36556	1.1922	6968096
1.3133	20.0	38480	1.1922	7334376

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4