distilbert-finetuned-ner

This model is a fine-tuned version of distilbert-base-uncased on the conll2003 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0551
• Precision: 0.9169
• Recall: 0.9382
• F1: 0.9275
• Accuracy: 0.9850
• Loc Precision: 0.9422
• Loc Recall: 0.9668
• Loc F1: 0.9543
• Loc Number: 1837
• Misc Precision: 0.8065
• Misc Recall: 0.8590
• Misc F1: 0.8319
• Misc Number: 922
• Org Precision: 0.8885
• Org Recall: 0.8971
• Org F1: 0.8928
• Org Number: 1341
• Per Precision: 0.9704
• Per Recall: 0.9794
• Per F1: 0.9749
• Per Number: 1842

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy	Loc Precision	Loc Recall	Loc F1	Loc Number	Misc Precision	Misc Recall	Misc F1	Misc Number	Org Precision	Org Recall	Org F1	Org Number	Per Precision	Per Recall	Per F1	Per Number
0.0787	1.0	878	0.0596	0.9015	0.9273	0.9142	0.9825	0.9169	0.9668	0.9412	1837	0.7994	0.8037	0.8015	922	0.8573	0.9053	0.8807	1341	0.9711	0.9658	0.9684	1842
0.037	2.0	1756	0.0551	0.9169	0.9382	0.9275	0.9850	0.9422	0.9668	0.9543	1837	0.8065	0.8590	0.8319	922	0.8885	0.8971	0.8928	1341	0.9704	0.9794	0.9749	1842
0.0208	3.0	2634	0.0567	0.9259	0.9406	0.9332	0.9856	0.9522	0.9652	0.9586	1837	0.8377	0.8677	0.8524	922	0.8900	0.9053	0.8976	1341	0.9714	0.9783	0.9748	1842

Framework versions

• Transformers 4.53.2
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.2