train_cb_101112_1760637979

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the cb dataset. It achieves the following results on the evaluation set:

• Loss: 0.3804
• Num Input Tokens Seen: 644672

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4609	2.0	100	0.2625	64960
0.2918	4.0	200	0.2345	128832
0.2208	6.0	300	0.2710	193024
0.1982	8.0	400	0.2228	258784
0.0209	10.0	500	0.3576	323264
0.0429	12.0	600	0.4701	388384
0.0001	14.0	700	0.3956	451904
0.0001	16.0	800	0.3769	515456
0.0001	18.0	900	0.3770	580576
0.0001	20.0	1000	0.3804	644672

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4