training_with_callbacks

This model is a fine-tuned version of MikeGreen2710/mlm_listing_1.79 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7291
• F1: 0.9208

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 24
• eval_batch_size: 12
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.3171	1.0	450	0.2395	0.9020
0.1889	2.0	900	0.2344	0.9135
0.1356	3.0	1350	0.3104	0.8986
0.0919	4.0	1800	0.3622	0.9101
0.0553	5.0	2250	0.4539	0.9085
0.0347	6.0	2700	0.4787	0.9132
0.0221	7.0	3150	0.5269	0.9084
0.0113	8.0	3600	0.5861	0.9150
0.016	9.0	4050	0.6784	0.9098
0.0079	10.0	4500	0.6497	0.9200
0.005	11.0	4950	0.7084	0.9140
0.0074	12.0	5400	0.7291	0.9208
0.0061	13.0	5850	0.7240	0.9174
0.0053	14.0	6300	0.7493	0.9149
0.004	15.0	6750	0.8256	0.9106
0.0035	16.0	7200	0.7821	0.9158
0.0054	17.0	7650	0.8016	0.9148
0.0005	18.0	8100	0.8021	0.9166
0.0012	19.0	8550	0.8046	0.9182
0.002	20.0	9000	0.8045	0.9191

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1