README.md

metadata

license: cc-by-sa-4.0
language:
  - de
tags:
  - ner
  - named-entity-recognition
  - xlm-roberta
  - token-classification
  - sdvm
datasets:
  - SDVM/xtreme-PAN-X.de
metrics:
  - f1
pipeline_tag: token-classification
model-index:
  - name: multilingual-ner-original
    results:
      - task:
          type: token-classification
          name: Named Entity Recognition
        dataset:
          type: SDVM/xtreme-PAN-X.de
          name: PAN-X.de (Original)
          config: default
          split: test
        metrics:
          - type: f1
            name: F1
            value: 0.88

SDVM Multilingual NER — Original

An XLM-RoBERTa-base model fine-tuned for Named Entity Recognition on the original (unrefined) PAN-X.de dataset from the XTREME benchmark.

This model is part of a paired experiment by SDVM to demonstrate the impact of data quality on NER performance. Compare with SDVM/multilingual-ner-refined, which was trained on cleaned data.

Training Details

• Base model: xlm-roberta-base
• Dataset: SDVM/xtreme-PAN-X.de — tokens and ner_tags columns (original, uncleaned)
• Training: 3 epochs, batch size 8, learning rate 2e-5, weight decay 0.01
• Task: Token classification with IOB2 tags

Labels

ID	Tag
0	O
1	B-PER
2	I-PER
3	B-ORG
4	I-ORG
5	B-LOC
6	I-LOC

Usage

from transformers import pipeline

ner = pipeline("token-classification", model="SDVM/multilingual-ner-original")
result = ner("Angela Merkel wurde in Hamburg geboren.")
print(result)

Context

This model was trained on the original PAN-X.de data which contains ~8.5% Wikipedia markup noise tokens (bold markers, quote marks, redirect tags, etc.). These artifacts can confuse the model during both training and inference.

For a cleaner alternative, see SDVM/multilingual-ner-refined.

Reference

• Based on Chapter 4 of Natural Language Processing with Transformers
• Part of the SDVM data quality demonstration series