train_cb_1757340269

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1639
• Num Input Tokens Seen: 359824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
1.081	0.5088	29	1.2085	19872
1.2816	1.0175	58	1.2085	36432
0.6154	1.5263	87	0.5919	53680
0.1858	2.0351	116	0.2524	72160
0.1156	2.5439	145	0.2084	91904
0.351	3.0526	174	0.1916	108856
0.2955	3.5614	203	0.1786	128056
0.0914	4.0702	232	0.1804	146952
0.1035	4.5789	261	0.1801	165128
0.0952	5.0877	290	0.1761	183224
0.0392	5.5965	319	0.1748	202424
0.1394	6.1053	348	0.1756	220000
0.1559	6.6140	377	0.1660	238272
0.1349	7.1228	406	0.1702	255984
0.0485	7.6316	435	0.1688	275536
0.1528	8.1404	464	0.1659	293296
0.1347	8.6491	493	0.1672	312304
0.0932	9.1579	522	0.1661	329216
0.0989	9.6667	551	0.1639	346944

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1