train_hellaswag_123_1760637745

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.0771
• Num Input Tokens Seen: 218506144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1942	1.0	8979	0.1737	10932896
0.026	2.0	17958	0.1204	21856400
0.1132	3.0	26937	0.0994	32797696
0.0637	4.0	35916	0.0902	43715520
0.0597	5.0	44895	0.0837	54639040
0.2529	6.0	53874	0.0812	65562352
0.0659	7.0	62853	0.0793	76495264
0.085	8.0	71832	0.0779	87424000
0.0729	9.0	80811	0.0771	98355744
0.0876	10.0	89790	0.0781	109279616
0.0054	11.0	98769	0.0772	120190896
0.1594	12.0	107748	0.0799	131118336
0.008	13.0	116727	0.0797	142033584
0.0087	14.0	125706	0.0799	152960704
0.0414	15.0	134685	0.0806	163884192
0.0072	16.0	143664	0.0813	174816592
0.0756	17.0	152643	0.0817	185740864
0.0044	18.0	161622	0.0822	196657440
0.0566	19.0	170601	0.0824	207581424
0.0165	20.0	179580	0.0824	218506144

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4