learnrr/roberta-base-ontonotes5-ner

RoBERTa-base fine-tuned on OntoNotes 5.0

This model is a fine-tuned version of FacebookAI/roberta-base on the English subset of the OntoNotes 5.0 (CoNLL-2012) dataset. RoBERTa optimizes the BERT pretraining approach by using dynamic masking, removing next-sentence prediction, and training on larger batches with byte-level BPE.

📊 Performance

The following results were achieved on the OntoNotes 5.0 (v12) test set:

Entity	Precision	Recall	F1-Score	Support
CARDINAL	0.7813	0.8070	0.7939	1005
DATE	0.8032	0.8729	0.8366	1786
EVENT	0.5566	0.6941	0.6178	85
FAC	0.7059	0.6443	0.6737	149
GPE	0.9283	0.9356	0.9319	2546
LANGUAGE	0.8667	0.5909	0.7027	22
LAW	0.4643	0.5909	0.5200	44
LOC	0.7354	0.7628	0.7489	215
MONEY	0.8414	0.8817	0.8611	355
NORP	0.9004	0.9404	0.9200	990
ORDINAL	0.6944	0.8454	0.7625	207
ORG	0.8653	0.8986	0.8816	2002
PERCENT	0.8605	0.9091	0.8841	407
PERSON	0.9083	0.9236	0.9159	2134
PRODUCT	0.6771	0.7222	0.6989	90
QUANTITY	0.7410	0.6732	0.7055	153
TIME	0.5670	0.6578	0.6091	225
WORK_OF_ART	0.6105	0.6864	0.6462	169
micro avg	0.8471	0.8822	0.8643	12584
macro avg	0.7504	0.7798	0.7617	12584
weighted avg	0.8497	0.8822	0.8652	12584

🛠 Training Details

• Architecture: RobertaForTokenClassification
• Tokenizer: RobertaTokenizerFast (with add_prefix_space=True)
• Epochs: 5
• Learning Rate: 2e-5
• Batch Size: 16 per device (Total 32 on 2x V100 GPUs)
• Max Sequence Length: 128
• Weight Decay: 0.01
• Mixed Precision (FP16): Enabled

📂 Project Assets

• GitHub Repository: Learnrr/ontonotes5_ner_evaluation

Asset	File	Description
Model Weights	model.safetensors	Fine-tuned RoBERTa weights (~496 MB).
Configuration	config.json	Model architecture & id2label mappings.
Vocabulary	vocab.json / merges.txt	Byte-level BPE vocabulary and merge rules.
Tokenizer	tokenizer.json	Full fast tokenizer configuration.
Special Tokens	special_tokens_map.json	Definitions for BOS, EOS, and Padding tokens.
Training Args	training_args.bin	Detailed dump of the training hyperparameters.

🚀 Usage

from transformers import pipeline

model_checkpoint = "learnrr/roberta-base-ontonotes5-ner"
token_classifier = pipeline(
    "token-classification", 
    model=model_checkpoint, 
    aggregation_strategy="simple"
)

text = "Microsoft Corporation was founded by Bill Gates and Paul Allen."
results = token_classifier(text)

for entity in results:
    print(f"Entity: {entity['word']} | Label: {entity['entity_group']} | Score: {entity['score']:.4f}")