File size: 2,740 Bytes
707e3ae | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | ---
language:
- en
- nl
- de
library_name: transformers
pipeline_tag: token-classification
base_model: xlm-roberta-base
tags:
- token-classification
- named-entity-recognition
- legal
- multilingual
---
# PedroDKE/multilingual-ner-abb-improved
Multilingual (English, Dutch, German) NER model for legal/administrative
decision documents.
This *improved* version adds native German training data (municipal decisions
from Freiburg and Bamberg) on top of the original Dutch (Ghent) and English
(translated) data, improving detection quality on German text and on text
translated from German.
- **Base model:** xlm-roberta-base
- **Languages:** English (en), Dutch (nl), German (de)
- **Labels (BIO):** DATE, LOCATION, LEGAL_GROUND, ADMINISTRATIVE_BODY, MANDATARY
## Quickstart
```python
from transformers import pipeline
ner = pipeline("token-classification", model="PedroDKE/multilingual-ner-abb-improved", aggregation_strategy="simple")
# German
print(ner("Der Stadtrat der Stadt Bamberg hat am 12. März 2024 beschlossen."))
# Dutch
print(ner("De gemeenteraad van Gent heeft op 12 maart 2024 besloten."))
# English
print(ner("The city council of London decided on March 12, 2024."))
```
## Evaluation
strict (seqeval, entity-level) micro on held-out validation set.
| Language | Samples | Precision | Recall | F1 |
|---|---|---|---|---|
| ALL | 920 | 0.766 | 0.823 | 0.794 |
| EN | 57 | 0.675 | 0.705 | 0.690 |
| NL | 69 | 0.751 | 0.783 | 0.767 |
| DE | 794 | 0.785 | 0.855 | 0.818 |
### Per-label (strict) — ALL
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| ADMINISTRATIVE_BODY | 0.76 | 0.86 | 0.81 | 459 |
| DATE | 0.84 | 0.88 | 0.86 | 773 |
| LEGAL_GROUND | 0.82 | 0.85 | 0.84 | 362 |
| LOCATION | 0.61 | 0.66 | 0.63 | 678 |
| MANDATARY | 0.82 | 0.88 | 0.85 | 580 |
### Per-label (strict) — EN
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| ADMINISTRATIVE_BODY | 0.64 | 0.72 | 0.68 | 64 |
| DATE | 0.67 | 0.66 | 0.67 | 59 |
| LEGAL_GROUND | 0.63 | 0.75 | 0.69 | 32 |
| LOCATION | 0.38 | 0.62 | 0.47 | 32 |
| MANDATARY | 0.85 | 0.72 | 0.78 | 138 |
### Per-label (strict) — NL
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| ADMINISTRATIVE_BODY | 0.60 | 0.67 | 0.63 | 58 |
| DATE | 0.77 | 0.91 | 0.84 | 104 |
| LEGAL_GROUND | 0.67 | 0.74 | 0.71 | 47 |
| LOCATION | 0.70 | 0.61 | 0.65 | 174 |
| MANDATARY | 0.85 | 0.91 | 0.88 | 188 |
### Per-label (strict) — DE
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| ADMINISTRATIVE_BODY | 0.81 | 0.92 | 0.86 | 337 |
| DATE | 0.87 | 0.90 | 0.89 | 610 |
| LEGAL_GROUND | 0.88 | 0.88 | 0.88 | 283 |
| LOCATION | 0.61 | 0.68 | 0.64 | 472 |
| MANDATARY | 0.80 | 0.95 | 0.87 | 254 |
|