train_record_789_1768212478

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 9.6171
• Num Input Tokens Seen: 928969632

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 789
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4212	1.0	31242	0.3264	46450496
0.3017	2.0	62484	0.3063	92891936
0.255	3.0	93726	0.2864	139349440
0.2157	4.0	124968	0.2823	185796864
0.3542	5.0	156210	0.2759	232235744
0.2109	6.0	187452	0.2774	278704192
0.2709	7.0	218694	0.2737	325156032
0.2031	8.0	249936	0.2701	371599168
0.1626	9.0	281178	0.2727	418050784
0.1573	10.0	312420	0.2741	464504128
0.288	11.0	343662	0.2713	510961472
0.2644	12.0	374904	0.2778	557400608
0.2252	13.0	406146	0.2858	603828768
0.1825	14.0	437388	0.2844	650269472
0.1774	15.0	468630	0.2920	696703648
0.215	16.0	499872	0.2929	743153504
0.2033	17.0	531114	0.2980	789592640
0.1136	18.0	562356	0.3022	836057504
0.2077	19.0	593598	0.3040	882513984
0.2002	20.0	624840	0.3055	928969632

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1