train_cola_123_1760637702

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.2477
• Num Input Tokens Seen: 7337920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1966	1.0	1924	0.2676	367320
0.2111	2.0	3848	0.2492	734600
0.2279	3.0	5772	0.2480	1101216
0.2468	4.0	7696	0.2480	1468552
0.245	5.0	9620	0.2533	1834816
0.2622	6.0	11544	0.2490	2201584
0.2493	7.0	13468	0.2502	2568288
0.2361	8.0	15392	0.2529	2935056
0.2459	9.0	17316	0.2483	3301760
0.2579	10.0	19240	0.2480	3669168
0.1968	11.0	21164	0.2486	4036096
0.2383	12.0	23088	0.2501	4403128
0.2646	13.0	25012	0.2478	4769264
0.3323	14.0	26936	0.2511	5136352
0.256	15.0	28860	0.2496	5503048
0.2452	16.0	30784	0.2477	5869824
0.212	17.0	32708	0.2482	6236752
0.303	18.0	34632	0.2486	6603776
0.2732	19.0	36556	0.2485	6970736
0.2678	20.0	38480	0.2491	7337920

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4