---
license: mit
datasets:
- ontonotes/conll2012_ontonotesv5
language:
- en
base_model:
- FacebookAI/roberta-base
pipeline_tag: token-classification
---

# RoBERTa-base fine-tuned on OntoNotes 5.0

This model is a fine-tuned version of [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) on the English subset of the **OntoNotes 5.0** (CoNLL-2012) dataset. RoBERTa optimizes the BERT pretraining approach by using dynamic masking, removing next-sentence prediction, and training on larger batches with byte-level BPE.

## 📊 Performance
The following results were achieved on the OntoNotes 5.0 (v12) test set:

| **Entity** | **Precision** | **Recall** | **F1-Score** | **Support** |
| :--- | :---: | :---: | :---: | :---: |
| CARDINAL | 0.7813 | 0.8070 | 0.7939 | 1005 |
| DATE | 0.8032 | 0.8729 | 0.8366 | 1786 |
| EVENT | 0.5566 | 0.6941 | 0.6178 | 85 |
| FAC | 0.7059 | 0.6443 | 0.6737 | 149 |
| GPE | 0.9283 | 0.9356 | 0.9319 | 2546 |
| LANGUAGE | 0.8667 | 0.5909 | 0.7027 | 22 |
| LAW | 0.4643 | 0.5909 | 0.5200 | 44 |
| LOC | 0.7354 | 0.7628 | 0.7489 | 215 |
| MONEY | 0.8414 | 0.8817 | 0.8611 | 355 |
| NORP | 0.9004 | 0.9404 | 0.9200 | 990 |
| ORDINAL | 0.6944 | 0.8454 | 0.7625 | 207 |
| ORG | 0.8653 | 0.8986 | 0.8816 | 2002 |
| PERCENT | 0.8605 | 0.9091 | 0.8841 | 407 |
| PERSON | 0.9083 | 0.9236 | 0.9159 | 2134 |
| PRODUCT | 0.6771 | 0.7222 | 0.6989 | 90 |
| QUANTITY | 0.7410 | 0.6732 | 0.7055 | 153 |
| TIME | 0.5670 | 0.6578 | 0.6091 | 225 |
| WORK_OF_ART | 0.6105 | 0.6864 | 0.6462 | 169 |
| **micro avg** | **0.8471** | **0.8822** | **0.8643** | **12584** |
| **macro avg** | **0.7504** | **0.7798** | **0.7617** | **12584** |
| **weighted avg** | **0.8497** | **0.8822** | **0.8652** | **12584** |

## 🛠 Training Details
- **Architecture**: `RobertaForTokenClassification`
- **Tokenizer**: `RobertaTokenizerFast` (with `add_prefix_space=True`)
- **Epochs**: 5
- **Learning Rate**: 2e-5
- **Batch Size**: 16 per device (Total 32 on 2x V100 GPUs)
- **Max Sequence Length**: 128
- **Weight Decay**: 0.01
- **Mixed Precision (FP16)**: Enabled

## 📂 Project Assets
- **GitHub Repository**: [Learnrr/ontonotes5_ner_evaluation](https://github.com/Learnrr/ontonotes5_ner_evaluation.git)

| **Asset** | **File** | **Description** |
| :--- | :--- | :--- |
| **Model Weights** | `model.safetensors` | Fine-tuned RoBERTa weights (~496 MB). |
| **Configuration** | `config.json` | Model architecture & `id2label` mappings. |
| **Vocabulary** | `vocab.json` / `merges.txt` | Byte-level BPE vocabulary and merge rules. |
| **Tokenizer** | `tokenizer.json` | Full fast tokenizer configuration. |
| **Special Tokens** | `special_tokens_map.json` | Definitions for BOS, EOS, and Padding tokens. |
| **Training Args** | `training_args.bin` | Detailed dump of the training hyperparameters. |

## 🚀 Usage
```python
from transformers import pipeline

model_checkpoint = "learnrr/roberta-base-ontonotes5-ner"
token_classifier = pipeline(
    "token-classification", 
    model=model_checkpoint, 
    aggregation_strategy="simple"
)

text = "Microsoft Corporation was founded by Bill Gates and Paul Allen."
results = token_classifier(text)

for entity in results:
    print(f"Entity: {entity['word']} | Label: {entity['entity_group']} | Score: {entity['score']:.4f}")
```