train_piqa_42_1760803315

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2311
• Num Input Tokens Seen: 44231432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2383	1.0	3626	0.2317	2217296
0.2303	2.0	7252	0.2313	4434064
0.2306	3.0	10878	0.2327	6651920
0.2379	4.0	14504	0.2324	8863368
0.2314	5.0	18130	0.2316	11072168
0.2316	6.0	21756	0.2322	13277800
0.2334	7.0	25382	0.2324	15488632
0.2319	8.0	29008	0.2312	17701536
0.232	9.0	32634	0.2312	19904744
0.2329	10.0	36260	0.2314	22114288
0.2319	11.0	39886	0.2313	24319944
0.2313	12.0	43512	0.2314	26531960
0.2324	13.0	47138	0.2311	28750368
0.2319	14.0	50764	0.2314	30958840
0.2329	15.0	54390	0.2313	33171360
0.2303	16.0	58016	0.2313	35383136
0.2329	17.0	61642	0.2315	37593824
0.2314	18.0	65268	0.2313	39808768
0.2298	19.0	68894	0.2314	42019120
0.2319	20.0	72520	0.2316	44231432

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1