train_hellaswag_123_1760637741

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.2248
• Num Input Tokens Seen: 218506144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4683	1.0	8979	0.4638	10932896
0.0219	2.0	17958	0.1065	21856400
0.0579	3.0	26937	0.0787	32797696
0.0284	4.0	35916	0.0647	43715520
0.0572	5.0	44895	0.0613	54639040
0.1779	6.0	53874	0.0566	65562352
0.0859	7.0	62853	0.0563	76495264
0.0914	8.0	71832	0.0558	87424000
0.0538	9.0	80811	0.0591	98355744
0.0217	10.0	89790	0.0617	109279616
0.0041	11.0	98769	0.0675	120190896
0.0665	12.0	107748	0.0817	131118336
0.0026	13.0	116727	0.0881	142033584
0.0002	14.0	125706	0.1030	152960704
0.0071	15.0	134685	0.1084	163884192
0.0001	16.0	143664	0.1274	174816592
0.0002	17.0	152643	0.1297	185740864
0.0001	18.0	161622	0.1373	196657440
0.0	19.0	170601	0.1431	207581424
0.0001	20.0	179580	0.1437	218506144

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4