train_cb_789_1760637867

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1673
• Num Input Tokens Seen: 711112

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.353	1.0	57	1.0482	35448
0.4513	2.0	114	0.2206	70496
0.2683	3.0	171	0.1803	106416
0.2404	4.0	228	0.1673	142480
0.2199	5.0	285	0.2525	177224
0.226	6.0	342	0.2050	212000
0.0928	7.0	399	0.2429	248272
0.1848	8.0	456	0.1789	284248
0.271	9.0	513	0.1997	319488
0.1533	10.0	570	0.2440	354472
0.0974	11.0	627	0.2553	389408
0.1862	12.0	684	0.1732	425328
0.0931	13.0	741	0.2376	461216
0.0754	14.0	798	0.2124	496704
0.0246	15.0	855	0.2732	532504
0.0136	16.0	912	0.2968	567952
0.0147	17.0	969	0.2886	603760
0.0111	18.0	1026	0.2870	639784
0.0129	19.0	1083	0.2892	675800
0.0085	20.0	1140	0.2861	711112

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4