File size: 3,975 Bytes
53cf067 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | ---
language: pl
license: cc-by-4.0
tags:
- token-classification
- ner
- pii
- polish
- onnx
- bert
base_model: clarin-pl/FastPDN
datasets:
- clarin-pl/kpwr-ner
---
# FastPDN NER — Polish PII (ONNX)
Fine-tuned [`clarin-pl/FastPDN`](https://huggingface.co/clarin-pl/FastPDN)
for detecting personal data (PII) and organizations in Polish text.
Detects: **PERSON** (full name), **PERSON_F** (first name), **PERSON_L** (last name),
**STREET** (street/road), **CITY**, **ORG** (company/institution).
House numbers are intentionally excluded from NER — handled downstream by regex in post-processing.
Intended use: Polish web forms — browser-side inference via
[`@xenova/transformers`](https://github.com/xenova/transformers.js) + ONNX Runtime Web (WASM),
no backend required.
## Training data
Merged in `data/merged/conll/` from two sources:
| Split | Synthetic | KPWr | Total | Tokens |
|-------|-----------|------|-------|--------|
| train | 12 750 | 3 935 | 16 685 | 248 437 |
| validation | 1 500 | 437 | 1 937 | 29 384 |
| test | 750 | 750 | 1 500 | 25 547 |
- **KPWr filtered** — [clarin-pl/kpwr-ner](https://huggingface.co/datasets/clarin-pl/kpwr-ner)
Polish press corpus; LOC-only (geographic) and schematic form-label samples removed.
- **LLM-synthetic** — generated with GPT-4o-mini and claude-haiku-4-5,
covering STREET with city suffix, PERSON in email context, and mixed cases.
Fine-tuned for 3 epochs with early stopping (patience=2), best checkpoint selected by **eval F1 STREET**.
## Evaluation (`data/merged/conll/test.conll`, 1 500 sentences, seqeval)
| Entity | fp32 F1 | int8 F1 | Δ fp32→int8 |
|--------|---------|---------|-------------|
| PERSON | 97.7% | 97.1% | −0.6pp |
| PERSON_F | 100.0% | 99.4% | −0.6pp |
| PERSON_L | 100.0% | 99.2% | −0.8pp |
| ORG | 85.1% | 82.7% | −2.5pp |
| STREET | 98.9% | 99.0% | +0.2pp |
| CITY | 96.9% | 96.7% | −0.2pp |
| **overall** | **96.0%** | **95.4%** | **−0.6pp** |
INT8 quantization loss = **−0.6pp overall** (target was <3pp). No entity loses more than 2.5pp.
## Label mapping
The model outputs 13 BIO classes:
| Model label | Meaning |
|-------------|---------|
| `B-PERSON` / `I-PERSON` | full person name |
| `B-PERSON_F` / `I-PERSON_F` | first name only |
| `B-PERSON_L` / `I-PERSON_L` | last name only |
| `B-ORG` / `I-ORG` | company / institution |
| `B-STREET` / `I-STREET` | street / road / avenue |
| `B-CITY` / `I-CITY` | city |
| `O` | not an entity |
## Files
| File | Format | Notes |
|------|--------|-------|
| `model.onnx` | FP32 | highest quality, BERT graph-optimized |
| `model_quantized.onnx` | INT8 | **recommended for browser** |
| `onnx/model_quantized.onnx` | INT8 | alias for Transformers.js `dtype:"q8"` |
| `config.json` | JSON | label mapping, model config |
| `tokenizer.json` | JSON | HerBERT tokenizer |
| `tokenizer_config.json` | JSON | HerBERT tokenizer config |
## Usage
### Python (Transformers)
```python
from transformers import pipeline
ner = pipeline(
"token-classification",
model="ArkadiuszPawlak/fastpdn-ner-polish-pii",
aggregation_strategy="simple",
)
result = ner("Jan Kowalski mieszka przy ul. Marszałkowskiej 1, 00-001 Warszawa.")
# [{"entity_group": "PERSON", "word": "Jan Kowalski", ...},
# {"entity_group": "STREET", "word": "ul. Marszałkowskiej", ...},
# {"entity_group": "CITY", "word": "Warszawa", ...}]
```
### Browser (@xenova/transformers + ONNX Runtime Web)
```js
import { pipeline } from "@xenova/transformers";
const ner = await pipeline(
"token-classification",
"ArkadiuszPawlak/fastpdn-ner-polish-pii",
{ dtype: "q8", aggregation_strategy: "simple" }
);
const raw = await ner("Jan Kowalski mieszka przy ul. Marszałkowskiej 1 w Warszawie.");
console.log(raw);
// [{ entity_group: "PERSON", word: "Jan Kowalski", score: 0.99 },
// { entity_group: "STREET", word: "ul. Marszałkowskiej", score: 0.98 },
// { entity_group: "CITY", word: "Warszawa", score: 0.97 }]
```
|