train_math_qa_789_1760637951

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the math_qa dataset. It achieves the following results on the evaluation set:

• Loss: 0.6015
• Num Input Tokens Seen: 77933776

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.6772	1.0	6714	0.6794	3898224
0.6364	2.0	13428	0.6400	7796616
0.6076	3.0	20142	0.6015	11688128
0.2153	4.0	26856	0.6934	15585640
0.4709	5.0	33570	0.8307	19481256
0.194	6.0	40284	1.0368	23379928
0.7205	7.0	46998	1.1229	27274992
0.3048	8.0	53712	1.4383	31169464
0.2298	9.0	60426	1.3880	35061680
0.1602	10.0	67140	1.5225	38957336
0.2557	11.0	73854	1.6556	42854488
0.0919	12.0	80568	1.7859	46754376
0.0001	13.0	87282	2.0551	50647376
0.0	14.0	93996	2.3546	54543272
0.0002	15.0	100710	2.3859	58447368
0.0	16.0	107424	2.5504	62343120
0.0	17.0	114138	2.7032	66240072
0.0	18.0	120852	3.1993	70140040
0.0	19.0	127566	3.3702	74035080
0.0	20.0	134280	3.3980	77933776

Framework versions

• PEFT 0.17.1
• Transformers 4.51.3
• Pytorch 2.9.0+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4