train_cb_1757340169

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.0116
• Num Input Tokens Seen: 361992

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3406	0.5088	29	0.1035	18048
0.1048	1.0175	58	0.1365	36928
0.0077	1.5263	87	0.0798	54176
0.2551	2.0351	116	0.0907	73136
0.0269	2.5439	145	0.0793	91216
0.0198	3.0526	174	0.0905	110696
0.0793	3.5614	203	0.1550	129448
0.0592	4.0702	232	0.0793	147176
0.0775	4.5789	261	0.0778	164424
0.0001	5.0877	290	0.0200	183416
0.0003	5.5965	319	0.0146	203256
0.0001	6.1053	348	0.0222	220912
0.0001	6.6140	377	0.0213	240336
0.0001	7.1228	406	0.0148	257848
0.0001	7.6316	435	0.0165	276824
0.0001	8.1404	464	0.0152	294504
0.0001	8.6491	493	0.0116	313576
0.0001	9.1579	522	0.0156	332256
0.0001	9.6667	551	0.0198	350336

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1