README.md

---
license: mit
language:
- pl
base_model: FacebookAI/xlm-roberta-large
library_name: transformers
pipeline_tag: token-classification
tags:
- polish
- pii
- ner
- xlm-roberta
- hybrid
---

# NERGAL 1.0.2

**Named Entity Recognition with Grounded Additive Labels**

SlayerLab hybrid PII cleaner for Polish. Not a chat model. Not a drop-in `pipeline("token-classification")`.

## TL;DR

Python rules do the identifiers they can prove. A transformer NER head adds phone and other PII spans the regex misses. The cleaner **unions** the two on the original text, then replaces hits with `[Telefon]` or `[PII]`.

- **Version:** `1.0.2` (`hybrid.json`, `CHANGELOG.md`)
- **Ground:** `scrub_pii` regex (SHA256 `3016ae5b…`)
- **Additive labels:** XLM-RoBERTa-large token classifier, BIO tags `phone` / `pii`, threshold 0.95
- **This snapshot:** seed `202609160`, **epoch 5** of a seven-epoch schedule

## Versions

841-dev, union at 0.95, 354 gold spans. Same weights and API is a patch; new capability is minor; API, threshold, or weight recipe is major. A release that changes these numbers updates `hybrid.json` `eval` and this table.

| Version | Whole /354 | Residual | Rules FP | Union FP | Char P | Char R | What changed |
|---|---:|---:|---:|---:|---:|---:|---|
| 1.0.0 | 323 | 25 | 133 | 133 | 97.76% | 95.95% | First Hub snapshot |
| 1.0.1 | 323 | 25 | 98 | 123 | 97.93% | 95.95% | Prefix-only glued-email trim |
| **1.0.2** | **324** | **24** | **98** | **123** | **97.93%** | **96.12%** | Labelled country-area phone fix |

## 841-dev

Tables use one development split: 841 passages, 215 with gold PII, **354 spans** (169 phone, 185 other PII). It is the `dev` side of a 4,500-passage labelled tranche (3,655 train / 841 dev). Sources match Dynaword (EUR-Lex, HPLT, Wikipedia, parliamentary and government text, plus smaller news/literary slices). Labels mix unchanged silver with human review.

The files contain real identifiers, so they are not released with the weights.

## Why XLM-R

GLiNER, HerBERT-large, and XLM-R-large were trained on the same split and unioned with the same regex. Plot: diagnostic threshold 0.50; selection used the full threshold grid. GLiNER covers more at 0.50 and then dumps precision. XLM-R is the architecture we kept.

![Primary three-model curves](figures/primary-three-model-curves.png)

## Why epoch 5

Fresh XLM-R, seven epochs. 133 epoch/threshold combinations. Epoch 5 at 0.95 was the only point that both beat the historical GLiNER∪regex incumbent on coverage and introduced zero new false-mask characters. Epoch 7 covers more PII (334/354) but adds 15 new false characters.

![Seven-epoch XLM-R curves](figures/xlmr-seven-epoch-curves.png)

| Epoch | Covered @ 0.95 /354 | Residual passages | False characters | New false vs incumbent |
|---|---:|---:|---:|---:|
| 1 | 272 | 59 | 175 | 42 |
| 2 | 291 | 49 | 141 | 8 |
| 3 | 317 | 30 | 140 | 7 |
| 4 | 320 | 28 | 143 | 10 |
| **5** | **323** | **25** | **133** | **0** |
| 6 | 328 | 20 | 147 | 14 |
| 7 | 334 | 16 | 148 | 15 |

## Compared with other systems

Same 841-dev split, threshold **0.95**. **Naked** is the transformer alone. **∪ regex** is that model unioned with the current rules. Character scores are gold vs masked characters.

| System | Mode | Whole /354 | Residual | False chars | Char P | Char R |
|---|---|---:|---:|---:|---:|---:|
| Regex (`scrub_pii`) | rules | 259 | 70 | 98 | 98.13% | 85.04% |
| GLiNER 2.5-multi zero-shot | naked | 81 | 188 | 970 | 56.98% | 21.22% |
| GLiNER 2.5-multi zero-shot | ∪ regex | 273 | 61 | 1,068 | 83.33% | 88.16% |
| Historical GLiNER email12 | naked | 249 | 70 | 10 | 99.81% | 85.39% |
| Historical GLiNER email12 | ∪ regex | 290 | 50 | 108 | 98.13% | 93.76% |
| XLM-R epoch 5 | naked | 298 | 42 | 57 | 98.96% | 89.38% |
| **NERGAL 1.0.2** | **∪ regex** | 324 | 24 | 123 | 97.93% | 96.12% |

Zero-shot [GLiNER 2.5](https://huggingface.co/fastino/gliner2.5-multi-v1) is not competitive here, especially on non-phone PII (7/185 whole vs 160 naked / 179 union). Fine-tuned historical GLiNER is the precise naked baseline (10 false characters) and still trails XLM-R on coverage. NERGAL is XLM-R epoch 5 plus the regex: 145/169 phone, 179/185 other PII. Exact-span precision 86.34%, recall 89.27%, F1 87.78%.

Trained GLiNER, HerBERT-large, and XLM-R-large were also compared on this split (plot above). GLiNER’s 0.50 coverage lead is the precision collapse in that figure; no GLiNER or HerBERT operating point passed the content-preservation gate.

## Extra seeds

Historical seed-comparison results, before the 1.0.2 parser fix.

| Seed | Whole /354 | False chars | New false vs historical union |
|---|---:|---:|---:|
| 202609160 (selected weights) | 323 | 123 | 0 |
| 202609161 | 322 | 134 | 1 |
| 202609162 | 316 | 151 | 18 |

Seed 160 is the published weights. 161 and 162 were confirmation runs of the same recipe.

## Load

This repo is the PII island: `scrub_pii.py` plus `nergal.py`. `pipeline("token-classification")` will not match. Regex runs on the original text, the model adds spans at 0.95, then the two are unioned and replaced with `[Telefon]` / `[PII]`.

```python
from pathlib import Path
from huggingface_hub import snapshot_download

root = Path(snapshot_download("SlayerLab/NERGAL"))
import sys
sys.path.insert(0, str(root))
from nergal import Nergal

nergal = Nergal.from_pretrained(root, local_files_only=True)
masked, counts = nergal.scrub(text)
```

`hybrid.json` records version `1.0.2`, threshold 0.95, gap ids `250002` / `250003`, and the 841-dev `eval` block. `test_nergal.py` is synthetic (no corpus text). From this snapshot: `python -m unittest test_nergal`.

Base weights: [`FacebookAI/xlm-roberta-large`](https://huggingface.co/FacebookAI/xlm-roberta-large) revision `c23d21b0620b635a76227c604d44e43a9f0ee389` (MIT).