train_cb_1757340267

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.0632
• Num Input Tokens Seen: 359824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1901	0.5088	29	0.2935	19872
0.144	1.0175	58	0.2516	36432
0.0659	1.5263	87	0.1955	53680
0.1028	2.0351	116	0.1596	72160
0.0053	2.5439	145	0.0632	91904
0.2008	3.0526	174	0.1121	108856
0.0239	3.5614	203	0.0735	128056
0.0017	4.0702	232	0.1148	146952
0.0469	4.5789	261	0.0746	165128
0.001	5.0877	290	0.0689	183224
0.0001	5.5965	319	0.0707	202424
0.0003	6.1053	348	0.0893	220000
0.0001	6.6140	377	0.0922	238272
0.0001	7.1228	406	0.0707	255984
0.0001	7.6316	435	0.0760	275536
0.0001	8.1404	464	0.0724	293296
0.0001	8.6491	493	0.0769	312304
0.0001	9.1579	522	0.0682	329216
0.0001	9.6667	551	0.0694	346944

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1