train_cb_1756128976

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.1483
• Accuracy: 0.92
• Num Input Tokens Seen: 367864

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Input Tokens Seen
1.2393	0.5088	29	1.1732	0.75	20064
1.196	1.0175	58	1.1732	0.75	37832
0.6503	1.5263	87	0.5590	0.81	57288
0.225	2.0351	116	0.1888	0.91	74520
0.568	2.5439	145	0.1758	0.92	93080
0.1509	3.0526	174	0.1615	0.93	111928
0.3084	3.5614	203	0.1592	0.93	131160
0.1682	4.0702	232	0.1610	0.92	150056
0.0562	4.5789	261	0.1624	0.92	167208
0.096	5.0877	290	0.1559	0.92	186160
0.0737	5.5965	319	0.1526	0.92	206000
0.1728	6.1053	348	0.1553	0.92	224064
0.1545	6.6140	377	0.1514	0.93	243840
0.0507	7.1228	406	0.1484	0.93	261504
0.166	7.6316	435	0.1568	0.92	280352
0.0494	8.1404	464	0.1483	0.92	299344
0.3338	8.6491	493	0.1508	0.93	318672
0.0833	9.1579	522	0.1507	0.93	337480
0.1172	9.6667	551	0.1484	0.93	356456

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1