train_record_456_1767057226

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the record dataset. It achieves the following results on the evaluation set:

• Loss: 0.3059
• Num Input Tokens Seen: 928892640

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3687	1.0	31242	0.4938	46454656
0.4754	2.0	62484	0.3933	92898208
0.2509	3.0	93726	0.3549	139330944
0.4351	4.0	124968	0.3370	185787424
0.3618	5.0	156210	0.3253	232232736
0.2276	6.0	187452	0.3197	278675168
0.2595	7.0	218694	0.3150	325124320
0.3843	8.0	249936	0.3107	371565312
0.3652	9.0	281178	0.3092	418010016
0.2225	10.0	312420	0.3070	464454880
0.2347	11.0	343662	0.3059	510906784
0.1691	12.0	374904	0.3065	557340128
0.2567	13.0	406146	0.3068	603790528
0.2291	14.0	437388	0.3071	650253184
0.3002	15.0	468630	0.3061	696691296
0.1885	16.0	499872	0.3059	743122464
0.156	17.0	531114	0.3063	789557088
0.398	18.0	562356	0.3063	835994816
0.3884	19.0	593598	0.3062	882444928
0.4039	20.0	624840	0.3063	928892640

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1