joe-xhedi/ModernBERT-NER

ModernBERT Fine-tuned for Named Entity Recognition

This model is a fine-tuned version of answerdotai/ModernBERT-base on the CoNLL-2003 Named Entity Recognition dataset.

Model Details

Model Description

• Developed by: Jayesh Thakare
• Model type: Token Classification (Named Entity Recognition)
• Language(s): English
• License: Apache 2.0
• Finetuned from model: answerdotai/ModernBERT-base

Model Sources

• Repository: joe-xhedi/ModernBERT-NER
• Base Model: answerdotai/ModernBERT-base
• Dataset: CoNLL-2003

Uses

Direct Use

This model can be used for Named Entity Recognition in English text, identifying the following entity types:

• PER: Person
• ORG: Organization
• LOC: Location
• MISC: Miscellaneous

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("joe-xhedi/ModernBERT-NER")
model = AutoModelForTokenClassification.from_pretrained("joe-xhedi/ModernBERT-NER")

# Create NER pipeline
ner_pipeline = pipeline("ner", 
                       model=model, 
                       tokenizer=tokenizer, 
                       aggregation_strategy="simple")

# Example usage
text = "John Doe works at OpenAI in San Francisco."
results = ner_pipeline(text)
print(results)

Training Details

Training Data

The model was fine-tuned on the CoNLL-2003 Named Entity Recognition dataset, which contains:

• Training examples: ~14,000 sentences
• Validation examples: ~3,200 sentences
• Test examples: ~3,400 sentences

Training Procedure

Training Hyperparameters

• Training regime: Mixed precision (fp16)
• Learning rate: 2e-05
• Training epochs: 3
• Batch size: 16
• Weight decay: 0.01
• Warmup steps: 500

Evaluation Strategy

• Evaluation steps: 500
• Save steps: 500
• Logging steps: 100

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model was evaluated on the CoNLL-2003 test dataset.

Metrics

The model was evaluated using seqeval metrics:

• Accuracy: Token-level accuracy
• F1 Score: Entity-level F1 score (macro-averaged)
• Precision: Entity-level precision
• Recall: Entity-level recall

Results

Metric	Value
Accuracy	0.9892527549550251
F1 Score	0.9363408521303258
Precision	N/A
Recall	N/A

Entity Labels

The model recognizes the following entity types:

• B-PER: B-PER
• I-PER: I-PER
• B-ORG: B-ORG
• I-ORG: I-ORG
• B-LOC: B-LOC
• I-LOC: I-LOC
• B-MISC: B-MISC
• I-MISC: I-MISC

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Technical Specifications

Model Architecture and Objective

• Base Architecture: ModernBERT (BERT-like transformer)
• Objective: Token Classification with Cross-Entropy Loss
• Number of Parameters: ~110M (inherited from ModernBERT-base)
• Number of Labels: 9

Compute Infrastructure

Hardware

• GPU: T4

Software

• Framework: PyTorch + Transformers
• Training Library: Hugging Face Transformers
• Evaluation Library: seqeval

Citation

If you use this model, please cite:

@misc{ModernBERT-NER,
  title={ModernBERT Fine-tuned for Named Entity Recognition},
  author={Jayesh Thakare},
  year={2025},
  url={https://huggingface.co/joe-xhedi/ModernBERT-NER}
}