yiwenX/bert-finetuned-ner

BERT Fine-tuned for Named Entity Recognition

Model Description

This model is a fine-tuned version of bert-base-cased on the WikiANN (WikiNER) English dataset for Named Entity Recognition (NER). The model can identify and classify named entities in text into predefined categories.

Performance Metrics

The model achieves strong performance on the WikiANN English validation set:

• F1 Score: 82.47%
• Precision: 80.92%
• Recall: 84.07%
• Accuracy: 92.50%

Supported Entity Types

The model recognizes 3 main entity types with BIO tagging:

• PER (Person): Names of people
• ORG (Organization): Names of organizations, companies, institutions
• LOC (Location): Names of locations, cities, countries

Label Set

• O: Outside of any entity
• B-PER: Beginning of a person entity
• I-PER: Inside a person entity
• B-ORG: Beginning of an organization entity
• I-ORG: Inside an organization entity
• B-LOC: Beginning of a location entity
• I-LOC: Inside a location entity

Intended Uses & Limitations

Intended Uses

• Extract named entities from English text
• Information extraction tasks
• Text preprocessing for downstream NLP applications
• Content analysis and categorization

Limitations

• The model is trained on WikiANN dataset, which may not generalize well to domain-specific texts
• Performance may vary on informal text (social media, chat messages)
• Limited to English language only
• Only recognizes PER, ORG, and LOC entity types

Training Data

The model was trained on the WikiANN (WikiNER) English dataset, which contains:

• Training samples: 20,000
• Validation samples: 10,000
• Test samples: 10,000

WikiANN is a multilingual named entity recognition dataset derived from Wikipedia, providing annotated text with named entity labels.

How to Use

Using with Transformers Pipeline

from transformers import pipeline

# Load the NER pipeline
nlp = pipeline("ner", model="yiwenX/bert-finetuned-ner-accelerate")

# Example text
text = "Apple Inc. was founded by Steve Jobs in Cupertino, California."

# Get predictions
results = nlp(text)
print(results)

Using with AutoModel

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("yiwenX/bert-finetuned-ner-accelerate")
model = AutoModelForTokenClassification.from_pretrained("yiwenX/bert-finetuned-ner-accelerate")

# Example text
text = "Apple Inc. was founded by Steve Jobs in Cupertino, California."

# Tokenize and predict
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

# Get predictions
predictions = torch.argmax(outputs.logits, dim=-1)
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])

# Map predictions to labels
label_list = model.config.id2label
entities = []
for token, pred in zip(tokens, predictions[0]):
    if pred != 0:  # 0 is 'O' (outside)
        entities.append((token, label_list[pred.item()]))

print(entities)

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• Learning rate: 2e-05
• Batch size: 16 (both training and evaluation)
• Number of epochs: 3
• Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
• LR scheduler: Linear
• Seed: 42

Training Results

Epoch	Training Loss	Validation Loss	Precision	Recall	F1	Accuracy
1.0	0.3066	0.2636	78.23%	81.86%	80.00%	91.89%
2.0	0.2059	0.2566	79.60%	83.37%	81.44%	92.42%
3.0	0.1455	0.2777	80.92%	84.07%	82.47%	92.50%

Framework Versions

• Transformers: 4.56.0
• PyTorch: 2.8.0+cu128
• Datasets: 4.0.0
• Tokenizers: 0.22.0

Citation

If you use this model in your research, please cite:

@misc{bert-finetuned-ner-accelerate,
  author = {yiwenX},
  title = {BERT Fine-tuned for Named Entity Recognition},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/yiwenX/bert-finetuned-ner-accelerate}
}

License

This model is licensed under the Apache 2.0 License.