train_cb_123_1760637638

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.0834
• Num Input Tokens Seen: 742296

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7941	1.0	57	5.3791	37160
0.2777	2.0	114	0.4515	73720
0.1865	3.0	171	0.1773	110296
0.2287	4.0	228	0.2263	147784
0.2082	5.0	285	0.1402	184368
0.2529	6.0	342	0.1531	221536
0.0837	7.0	399	0.1571	258720
0.1676	8.0	456	0.1298	295408
0.205	9.0	513	0.1373	332648
0.2853	10.0	570	0.1247	369976
0.106	11.0	627	0.1205	406840
0.1006	12.0	684	0.0834	444728
0.1631	13.0	741	0.1492	481720
0.1947	14.0	798	0.1688	518664
0.0517	15.0	855	0.0892	555728
0.036	16.0	912	0.1012	593096
0.0262	17.0	969	0.1146	629760
0.0158	18.0	1026	0.1004	667432
0.0446	19.0	1083	0.0991	704816
0.0233	20.0	1140	0.0999	742296

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4