train_apps_1754507525

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the apps dataset. It achieves the following results on the evaluation set:

• Loss: 0.6625
• Num Input Tokens Seen: 880041568

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7099	0.5000	13189	0.7341	44223136
0.6547	1.0000	26378	0.7023	87957952
0.6593	1.5001	39567	0.6880	131814656
0.5829	2.0001	52756	0.6804	175975840
0.596	2.5001	65945	0.6759	219881664
0.637	3.0001	79134	0.6724	263949472
0.6751	3.5001	92323	0.6702	307925280
0.5969	4.0002	105512	0.6682	352048320
0.5763	4.5002	118701	0.6669	396106880
0.6231	5.0002	131890	0.6656	440014752
0.7088	5.5002	145079	0.6649	484066880
0.6676	6.0002	158268	0.6641	528105600
0.8398	6.5002	171457	0.6636	572089824
0.5797	7.0003	184646	0.6631	616130592
0.7106	7.5003	197835	0.6630	660063168
0.6742	8.0003	211024	0.6627	704033600
0.7597	8.5003	224213	0.6626	747976128
0.6676	9.0003	237402	0.6625	792077152
0.575	9.5004	250591	0.6625	836063392

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1