train_cb_456_1757596107

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1779
• Num Input Tokens Seen: 359688

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.3594	0.5088	29	1.2933	18048
1.1167	1.0175	58	1.2933	36328
0.788	1.5263	87	0.6089	56168
0.1122	2.0351	116	0.2240	73792
0.3827	2.5439	145	0.1988	92768
0.1443	3.0526	174	0.1914	110064
0.1009	3.5614	203	0.1858	129808
0.9001	4.0702	232	0.1853	147240
0.0891	4.5789	261	0.1873	164744
0.0949	5.0877	290	0.1844	184440
0.1282	5.5965	319	0.1851	202456
0.0808	6.1053	348	0.1826	220288
0.1746	6.6140	377	0.1816	239200
0.2217	7.1228	406	0.1812	256296
0.06	7.6316	435	0.1779	275688
0.2938	8.1404	464	0.1833	294608
0.2037	8.6491	493	0.1790	312144
0.0767	9.1579	522	0.1816	330152
0.0454	9.6667	551	0.1834	347976

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1