train_hellaswag_123_1760637739

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the hellaswag dataset. It achieves the following results on the evaluation set:

• Loss: 0.7937
• Num Input Tokens Seen: 194223872

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.458	2.0	15962	0.4626	19416032
0.4588	4.0	31924	0.4626	38836480
0.4638	6.0	47886	0.4632	58267680
0.4649	8.0	63848	0.4644	77691392
0.454	10.0	79810	0.4773	97109312
0.4571	12.0	95772	0.5238	116509824
0.313	14.0	111734	0.5585	135932928
0.3174	16.0	127696	0.6658	155372064
0.2419	18.0	143658	0.7505	174790976
0.2758	20.0	159620	0.7937	194223872

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4