train_cb_101112_1760637982

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.0047
• Num Input Tokens Seen: 723584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3984	1.0	57	0.2891	36112
0.435	2.0	114	0.2766	71552
0.2867	3.0	171	0.2166	108088
0.3724	4.0	228	0.2117	144720
0.1607	5.0	285	0.2170	181120
0.2411	6.0	342	0.0957	217128
0.1948	7.0	399	0.1211	253536
0.0966	8.0	456	0.0722	290112
0.0577	9.0	513	0.0301	325872
0.1367	10.0	570	0.0447	361920
0.0579	11.0	627	0.0327	398432
0.0031	12.0	684	0.0060	435536
0.0162	13.0	741	0.0169	471520
0.0036	14.0	798	0.0151	507256
0.0021	15.0	855	0.0079	543064
0.0015	16.0	912	0.0068	579704
0.0011	17.0	969	0.0065	615960
0.0006	18.0	1026	0.0062	652368
0.0006	19.0	1083	0.0058	687976
0.0019	20.0	1140	0.0062	723584

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4