train_cb_456_1760637758

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1637
• Num Input Tokens Seen: 721856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.0221	1.0	57	1.1781	36072
0.7632	2.0	114	0.5823	72896
0.1503	3.0	171	0.1779	109080
0.1435	4.0	228	0.1711	145936
0.148	5.0	285	0.1740	182024
0.0421	6.0	342	0.1692	218672
0.1542	7.0	399	0.1716	254232
0.1302	8.0	456	0.1699	290912
0.0934	9.0	513	0.1685	326432
0.0439	10.0	570	0.1668	362240
0.1436	11.0	627	0.1637	397880
0.0766	12.0	684	0.1674	433352
0.0927	13.0	741	0.1653	469568
0.0488	14.0	798	0.1684	505048
0.054	15.0	855	0.1677	541088
0.116	16.0	912	0.1683	577512
0.0392	17.0	969	0.1715	614128
0.1652	18.0	1026	0.1661	649608
0.0601	19.0	1083	0.1682	685200
0.0863	20.0	1140	0.1684	721856

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4