train_piqa_42_1760873744

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0971
• Num Input Tokens Seen: 44231432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.1761	1.0	3626	0.1278	2217296
0.2681	2.0	7252	0.1086	4434064
0.0428	3.0	10878	0.1029	6651920
0.0987	4.0	14504	0.0998	8863368
0.1642	5.0	18130	0.0985	11072168
0.1449	6.0	21756	0.0981	13277800
0.1075	7.0	25382	0.0982	15488632
0.0601	8.0	29008	0.0971	17701536
0.1063	9.0	32634	0.0983	19904744
0.0703	10.0	36260	0.1010	22114288
0.0235	11.0	39886	0.1009	24319944
0.0762	12.0	43512	0.1026	26531960
0.0276	13.0	47138	0.1038	28750368
0.0897	14.0	50764	0.1051	30958840
0.0249	15.0	54390	0.1069	33171360
0.0969	16.0	58016	0.1074	35383136
0.0785	17.0	61642	0.1078	37593824
0.0838	18.0	65268	0.1075	39808768
0.017	19.0	68894	0.1077	42019120
0.0301	20.0	72520	0.1077	44231432

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1