train_cb_456_1760637756

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1905
• Num Input Tokens Seen: 721856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2174	1.0	57	0.2306	36072
0.1809	2.0	114	0.2118	72896
0.0463	3.0	171	0.1905	109080
0.0021	4.0	228	0.2566	145936
0.0053	5.0	285	0.2994	182024
0.0	6.0	342	0.2084	218672
0.0001	7.0	399	0.3513	254232
0.0	8.0	456	0.3168	290912
0.0	9.0	513	0.3357	326432
0.0	10.0	570	0.3217	362240
0.0	11.0	627	0.3119	397880
0.0	12.0	684	0.3194	433352
0.0	13.0	741	0.3077	469568
0.0	14.0	798	0.3258	505048
0.0	15.0	855	0.3141	541088
0.0	16.0	912	0.3169	577512
0.0	17.0	969	0.3124	614128
0.0	18.0	1026	0.3188	649608
0.0	19.0	1083	0.3250	685200
0.0	20.0	1140	0.3199	721856

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4