train_apps_1754507524

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the apps dataset. It achieves the following results on the evaluation set:

• Loss: 0.5797
• Num Input Tokens Seen: 880041568

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6623	0.5000	13189	0.6743	44223136
0.6062	1.0000	26378	0.6460	87957952
0.558	1.5001	39567	0.6230	131814656
0.4957	2.0001	52756	0.6068	175975840
0.5104	2.5001	65945	0.5992	219881664
0.5623	3.0001	79134	0.5885	263949472
0.5484	3.5001	92323	0.5873	307925280
0.4142	4.0002	105512	0.5799	352048320
0.4757	4.5002	118701	0.5836	396106880
0.4429	5.0002	131890	0.5797	440014752
0.5043	5.5002	145079	0.5846	484066880
0.5017	6.0002	158268	0.5801	528105600
0.6066	6.5002	171457	0.5872	572089824
0.3325	7.0003	184646	0.5866	616130592
0.4256	7.5003	197835	0.5956	660063168
0.3976	8.0003	211024	0.5935	704033600
0.381	8.5003	224213	0.6035	747976128
0.4467	9.0003	237402	0.6028	792077152
0.3695	9.5004	250591	0.6070	836063392

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1