wage_prediction

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3221
• Mse: -0.3221

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 32
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Mse
No log	1.0	18	43.7387	-43.7387
No log	2.0	36	22.7879	-22.7879
No log	3.0	54	12.8629	-12.8629
No log	4.0	72	6.6085	-6.6085
No log	5.0	90	2.5628	-2.5628
No log	6.0	108	0.6254	-0.6254
No log	7.0	126	0.4496	-0.4496
No log	8.0	144	0.5790	-0.5790
No log	9.0	162	0.4185	-0.4185
No log	10.0	180	0.5515	-0.5515
No log	11.0	198	0.4093	-0.4093
No log	12.0	216	0.3644	-0.3644
No log	13.0	234	0.4353	-0.4353
No log	14.0	252	0.3436	-0.3436
No log	15.0	270	0.3732	-0.3732
No log	16.0	288	0.3497	-0.3497
No log	17.0	306	0.4740	-0.4740
No log	18.0	324	0.3512	-0.3512
No log	19.0	342	0.3436	-0.3436
No log	20.0	360	0.3857	-0.3857
No log	21.0	378	0.3253	-0.3253
No log	22.0	396	0.3281	-0.3281
No log	23.0	414	0.3349	-0.3349
No log	24.0	432	0.3329	-0.3329
No log	25.0	450	0.3280	-0.3280
No log	26.0	468	0.3268	-0.3268
No log	27.0	486	0.3205	-0.3205
5.0934	28.0	504	0.3351	-0.3351
5.0934	29.0	522	0.3283	-0.3283
5.0934	30.0	540	0.3193	-0.3193
5.0934	31.0	558	0.3204	-0.3204
5.0934	32.0	576	0.3221	-0.3221

Framework versions

• Transformers 4.52.4
• Pytorch 2.6.0+cu124
• Datasets 2.14.4
• Tokenizers 0.21.1