train_hellaswag_1754507491

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.0818
• Num Input Tokens Seen: 108930064

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1794	0.5001	4490	0.2092	5450816
0.1404	1.0001	8980	0.1461	10899840
0.0578	1.5002	13470	0.1207	16338976
0.1291	2.0002	17960	0.1079	21789168
0.1142	2.5003	22450	0.1026	27236592
0.0492	3.0003	26940	0.0953	32696128
0.1513	3.5004	31430	0.0918	38137920
0.0191	4.0004	35920	0.0895	43579472
0.0867	4.5005	40410	0.0878	49022960
0.1137	5.0006	44900	0.0847	54468496
0.0294	5.5006	49390	0.0835	59917136
0.1162	6.0007	53880	0.0838	65358976
0.0641	6.5007	58370	0.0824	70806016
0.0495	7.0008	62860	0.0824	76259312
0.0668	7.5008	67350	0.0821	81705616
0.0515	8.0009	71840	0.0821	87153488
0.0292	8.5009	76330	0.0823	92602480
0.0376	9.0010	80820	0.0822	98051504
0.1167	9.5011	85310	0.0818	103491728

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1