README.md

---
license: cc-by-sa-4.0
language:
  - de
tags:
  - ner
  - named-entity-recognition
  - xlm-roberta
  - token-classification
  - sdvm
datasets:
  - SDVM/xtreme-PAN-X.de
metrics:
  - f1
pipeline_tag: token-classification
model-index:
  - name: multilingual-ner-original
    results:
      - task:
          type: token-classification
          name: Named Entity Recognition
        dataset:
          type: SDVM/xtreme-PAN-X.de
          name: PAN-X.de (Original)
          config: default
          split: test
        metrics:
          - type: f1
            name: F1
            value: 0.880
---

# SDVM Multilingual NER — Original

An XLM-RoBERTa-base model fine-tuned for Named Entity Recognition on the **original** (unrefined) PAN-X.de dataset from the XTREME benchmark.

This model is part of a paired experiment by [SDVM](https://huggingface.co/SDVM) to demonstrate the impact of data quality on NER performance. Compare with [SDVM/multilingual-ner-refined](https://huggingface.co/SDVM/multilingual-ner-refined), which was trained on cleaned data.

## Training Details

- **Base model**: [xlm-roberta-base](https://huggingface.co/xlm-roberta-base)
- **Dataset**: [SDVM/xtreme-PAN-X.de](https://huggingface.co/datasets/SDVM/xtreme-PAN-X.de) — `tokens` and `ner_tags` columns (original, uncleaned)
- **Training**: 3 epochs, batch size 8, learning rate 2e-5, weight decay 0.01
- **Task**: Token classification with IOB2 tags

## Labels

| ID | Tag |
|----|-----|
| 0 | O |
| 1 | B-PER |
| 2 | I-PER |
| 3 | B-ORG |
| 4 | I-ORG |
| 5 | B-LOC |
| 6 | I-LOC |

## Usage

```python
from transformers import pipeline

ner = pipeline("token-classification", model="SDVM/multilingual-ner-original")
result = ner("Angela Merkel wurde in Hamburg geboren.")
print(result)
```

## Context

This model was trained on the original PAN-X.de data which contains ~8.5% Wikipedia markup noise tokens (bold markers, quote marks, redirect tags, etc.). These artifacts can confuse the model during both training and inference.

For a cleaner alternative, see [SDVM/multilingual-ner-refined](https://huggingface.co/SDVM/multilingual-ner-refined).

## Reference

- Based on Chapter 4 of [Natural Language Processing with Transformers](https://github.com/nlp-with-transformers/notebooks/blob/main/04_multilingual-ner.ipynb)
- Part of the [SDVM](https://huggingface.co/SDVM) data quality demonstration series