bert-finetuned-ner

This model is a fine-tuned version of bert-base-cased on unimelb-nlp/wikiann dataset English Language . It achieves the following results on the evaluation set:

• Loss: 0.2904
• Precision: 0.8249
• Recall: 0.8498
• F1: 0.8372
• Accuracy: 0.9311

Model description

This model is a BERT-based Named Entity Recognition (NER) system fine-tuned from bert-base-cased for English token classification.

It identifies and classifies named entities using the BIO tagging scheme across the following entity types:

PER (Person)

ORG (Organization)

LOC (Location)

O (Outside)

The model processes tokenized text and outputs entity spans using contextualized embeddings learned through transformer self-attention mechanisms.

Intended uses & limitations

Intended Uses

Information extraction from English text

Named entity recognition in NLP pipelines

Academic research and educational projects

Preprocessing step for downstream tasks (e.g., relation extraction, QA)

Limitations

Trained only on English data

Performance may degrade on domain-specific text (medical, legal, informal)

Limited to PER, ORG, LOC entity types

Sensitive to tokenization artifacts in noisy or misspelled text

Training and evaluation data

The model was trained and evaluated using the WikiAnn (PAN-X) dataset for English.

Dataset Details

Multilingual, Wikipedia-based NER corpus

Automatically annotated

BIO labeling scheme

Final Data Split

Training: 30,000 sentences

Validation: 5,000 sentences

Test: 5,000 sentences

Entity Labels

O, B-PER, I-PER, B-ORG, I-ORG, B-LOC, I-LOC

Training procedure

Base Model: bert-base-cased

Framework: Hugging Face Transformers

Task: Token Classification (NER)

Epochs: 3

Learning Rate: 2e-5

Optimizer: AdamW

Weight Decay: 0.01

Evaluation Metric: SeqEval (Precision, Recall, F1, Accuracy)

Label Alignment: Subword-aware BIO label propagation

Trainer API: Hugging Face Trainer

The model was evaluated after each epoch and achieved strong overall performance on the held-out test set.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.2821	1.0	3750	0.2421	0.7914	0.8387	0.8143	0.9259
0.1919	2.0	7500	0.2524	0.8163	0.8433	0.8296	0.9289
0.1307	3.0	11250	0.2904	0.8249	0.8498	0.8372	0.9311

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.1