train_codealpacapy_123_1762532132

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the codealpacapy dataset. It achieves the following results on the evaluation set:

• Loss: 0.6497
• Num Input Tokens Seen: 24941912

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6045	1.0	1908	0.5060	1248304
0.4393	2.0	3816	0.4936	2497016
0.4425	3.0	5724	0.4876	3742552
0.4323	4.0	7632	0.4798	4985200
0.4724	5.0	9540	0.4778	6233920
0.4912	6.0	11448	0.4766	7478504
0.4458	7.0	13356	0.4769	8722744
0.7625	8.0	15264	0.4764	9977520
0.4813	9.0	17172	0.4771	11225416
0.6371	10.0	19080	0.4768	12472912
0.5587	11.0	20988	0.4760	13721824
0.4563	12.0	22896	0.4780	14970528
0.5769	13.0	24804	0.4780	16220808
0.3753	14.0	26712	0.4807	17464792
0.3651	15.0	28620	0.4802	18706976
0.3806	16.0	30528	0.4817	19956544
0.3822	17.0	32436	0.4826	21204416
0.4014	18.0	34344	0.4838	22451928
0.3717	19.0	36252	0.4843	23696296
0.5983	20.0	38160	0.4844	24941912

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1