train_hellaswag_42_1760637626

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 4.2286
• Num Input Tokens Seen: 218263888

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4638	1.0	8979	0.4624	10917120
0.461	2.0	17958	0.4641	21836032
0.4641	3.0	26937	0.4626	32746560
0.4629	4.0	35916	0.4624	43661424
0.4608	5.0	44895	0.4623	54578912
0.4593	6.0	53874	0.4623	65488016
0.4678	7.0	62853	0.4623	76410304
0.4643	8.0	71832	0.4623	87327296
0.4605	9.0	80811	0.4623	98229232
0.4623	10.0	89790	0.4619	109127968
0.4614	11.0	98769	0.4615	120042688
0.452	12.0	107748	0.4617	130954720
0.4556	13.0	116727	0.4623	141874656
0.4748	14.0	125706	0.4620	152783392
0.4576	15.0	134685	0.4640	163694096
0.4613	16.0	143664	0.4625	174604544
0.4611	17.0	152643	0.4637	185523328
0.458	18.0	161622	0.4654	196433472
0.4408	19.0	170601	0.4646	207345200
0.4612	20.0	179580	0.4648	218263888

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4