train_apps_1754897204

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the apps dataset. It achieves the following results on the evaluation set:

• Loss: 0.7171
• Num Input Tokens Seen: 880041568

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.7667	0.5000	13189	0.7823	44223136
0.7102	1.0000	26378	0.7484	87957952
0.7129	1.5001	39567	0.7365	131814656
0.6309	2.0001	52756	0.7293	175975840
0.6354	2.5001	65945	0.7260	219881664
0.6694	3.0001	79134	0.7247	263949472
0.7233	3.5001	92323	0.7227	307925280
0.6627	4.0002	105512	0.7221	352048320
0.6283	4.5002	118701	0.7200	396106880
0.6722	5.0002	131890	0.7191	440014752
0.768	5.5002	145079	0.7185	484066880
0.7321	6.0002	158268	0.7182	528105600
0.8997	6.5002	171457	0.7176	572089824
0.6457	7.0003	184646	0.7174	616130592
0.7701	7.5003	197835	0.7173	660063168
0.7298	8.0003	211024	0.7171	704033600
0.8252	8.5003	224213	0.7172	747976128
0.7198	9.0003	237402	0.7172	792077152
0.6224	9.5004	250591	0.7172	836063392

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1