train_cola_789_1760637934

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 1.1858
• Num Input Tokens Seen: 7327648

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.5907	1.0	1924	1.3464	365728
1.0188	2.0	3848	1.2216	731984
1.5831	3.0	5772	1.2011	1098920
1.4566	4.0	7696	1.2124	1465464
1.354	5.0	9620	1.1876	1831920
1.4282	6.0	11544	1.1858	2198176
1.2482	7.0	13468	1.1950	2564952
1.0891	8.0	15392	1.2005	2931096
1.2576	9.0	17316	1.1914	3296808
1.4372	10.0	19240	1.2168	3663512
1.3681	11.0	21164	1.2076	4029608
1.2829	12.0	23088	1.2074	4395616
1.2138	13.0	25012	1.2026	4762456
0.865	14.0	26936	1.2204	5128712
1.2082	15.0	28860	1.1977	5495008
1.1529	16.0	30784	1.2001	5861104
1.2101	17.0	32708	1.1962	6228320
1.3946	18.0	34632	1.1962	6595032
1.0783	19.0	36556	1.1962	6961416
0.9839	20.0	38480	1.1962	7327648

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4