train_piqa_789_1767846401

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the piqa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2714
• Num Input Tokens Seen: 44134648

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.224	1.0	3626	0.2342	2200752
0.2368	2.0	7252	0.2320	4414368
0.2315	3.0	10878	0.2316	6619552
0.2355	4.0	14504	0.2327	8822272
0.2324	5.0	18130	0.2318	11026904
0.2342	6.0	21756	0.2314	13231496
0.2387	7.0	25382	0.2316	15445728
0.2326	8.0	29008	0.2320	17655032
0.2303	9.0	32634	0.2309	19867456
0.2313	10.0	36260	0.2313	22063544
0.2297	11.0	39886	0.2311	24265960
0.2287	12.0	43512	0.2311	26475880
0.2246	13.0	47138	0.2314	28688312
0.2282	14.0	50764	0.2310	30896248
0.2309	15.0	54390	0.2314	33103912
0.2428	16.0	58016	0.2311	35300696
0.2271	17.0	61642	0.2315	37507928
0.2311	18.0	65268	0.2314	39713176
0.227	19.0	68894	0.2317	41919472
0.231	20.0	72520	0.2320	44134648

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1