train_record_789_1769460306

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.2761
• Num Input Tokens Seen: 928969632

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3675	1.0	31242	0.2960	46450496
0.245	2.0	62484	0.2761	92891936
0.1738	3.0	93726	0.2914	139349440
0.1252	4.0	124968	0.3047	185796864
0.6167	5.0	156210	0.3308	232235744
0.1784	6.0	187452	0.3655	278704192
0.2005	7.0	218694	0.3915	325156032
0.0853	8.0	249936	0.4406	371599168
0.0985	9.0	281178	0.4697	418050784
0.0858	10.0	312420	0.4797	464504128
0.1518	11.0	343662	0.5315	510961472
0.1531	12.0	374904	0.5858	557400608
0.1324	13.0	406146	0.6130	603828768
0.1545	14.0	437388	0.6696	650269472
0.1309	15.0	468630	0.7676	696703648
0.1588	16.0	499872	0.7628	743153504
0.0826	17.0	531114	0.8930	789592640
0.1123	18.0	562356	1.0192	836057504
0.1253	19.0	593598	1.1423	882513984
0.1248	20.0	624840	1.1739	928969632

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1