train_cb_789_1760637871

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1367
• Num Input Tokens Seen: 711112

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.9866	1.0	57	1.1528	35448
0.836	2.0	114	0.5759	70496
0.1908	3.0	171	0.1696	106416
0.3003	4.0	228	0.1466	142480
0.1807	5.0	285	0.1476	177224
0.0604	6.0	342	0.1496	212000
0.0518	7.0	399	0.1477	248272
0.0906	8.0	456	0.1408	284248
0.1919	9.0	513	0.1367	319488
0.0829	10.0	570	0.1411	354472
0.0778	11.0	627	0.1451	389408
0.1052	12.0	684	0.1408	425328
0.2006	13.0	741	0.1421	461216
0.0984	14.0	798	0.1415	496704
0.0945	15.0	855	0.1410	532504
0.0651	16.0	912	0.1422	567952
0.101	17.0	969	0.1411	603760
0.1725	18.0	1026	0.1413	639784
0.1481	19.0	1083	0.1404	675800
0.1104	20.0	1140	0.1382	711112

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4