train_cola_1757340214

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cola dataset. It achieves the following results on the evaluation set:

• Loss: 0.1502
• Num Input Tokens Seen: 3668312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2758	0.5	962	0.2624	183008
0.2847	1.0	1924	0.1785	366712
0.1576	1.5	2886	0.1687	550360
0.1648	2.0	3848	0.1651	734016
0.2008	2.5	4810	0.1625	917408
0.1271	3.0	5772	0.1577	1100824
0.1204	3.5	6734	0.1567	1283896
0.103	4.0	7696	0.1541	1467248
0.094	4.5	8658	0.1543	1651280
0.1326	5.0	9620	0.1535	1834568
0.129	5.5	10582	0.1527	2017960
0.1704	6.0	11544	0.1532	2201464
0.1212	6.5	12506	0.1558	2384536
0.1028	7.0	13468	0.1502	2568040
0.0721	7.5	14430	0.1502	2750664
0.1329	8.0	15392	0.1504	2934360
0.1154	8.5	16354	0.1502	3118424
0.2044	9.0	17316	0.1505	3301448
0.1361	9.5	18278	0.1505	3485512
0.2229	10.0	19240	0.1504	3668312

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1