| --- |
| language: pl |
| license: cc-by-4.0 |
| tags: |
| - token-classification |
| - ner |
| - pii |
| - polish |
| - onnx |
| - bert |
| base_model: clarin-pl/FastPDN |
| datasets: |
| - clarin-pl/kpwr-ner |
| --- |
| |
| # FastPDN NER — Polish PII (ONNX) |
|
|
| Fine-tuned [`clarin-pl/FastPDN`](https://huggingface.co/clarin-pl/FastPDN) |
| for detecting personal data (PII) and organizations in Polish text. |
|
|
| Detects: **PERSON** (full name), **PERSON_F** (first name), **PERSON_L** (last name), |
| **STREET** (street/road), **CITY**, **ORG** (company/institution). |
|
|
| House numbers are intentionally excluded from NER — handled downstream by regex in post-processing. |
|
|
| Intended use: Polish web forms — browser-side inference via |
| [`@xenova/transformers`](https://github.com/xenova/transformers.js) + ONNX Runtime Web (WASM), |
| no backend required. |
|
|
| ## Training data |
|
|
| Merged in `data/merged/conll/` from two sources: |
|
|
| | Split | Synthetic | KPWr | Total | Tokens | |
| |-------|-----------|------|-------|--------| |
| | train | 12 750 | 3 935 | 16 685 | 248 437 | |
| | validation | 1 500 | 437 | 1 937 | 29 384 | |
| | test | 750 | 750 | 1 500 | 25 547 | |
|
|
| - **KPWr filtered** — [clarin-pl/kpwr-ner](https://huggingface.co/datasets/clarin-pl/kpwr-ner) |
| Polish press corpus; LOC-only (geographic) and schematic form-label samples removed. |
| - **LLM-synthetic** — generated with GPT-4o-mini and claude-haiku-4-5, |
| covering STREET with city suffix, PERSON in email context, and mixed cases. |
|
|
| Fine-tuned for 3 epochs with early stopping (patience=2), best checkpoint selected by **eval F1 STREET**. |
|
|
| ## Evaluation (`data/merged/conll/test.conll`, 1 500 sentences, seqeval) |
|
|
| | Entity | fp32 F1 | int8 F1 | Δ fp32→int8 | |
| |--------|---------|---------|-------------| |
| | PERSON | 97.7% | 97.1% | −0.6pp | |
| | PERSON_F | 100.0% | 99.4% | −0.6pp | |
| | PERSON_L | 100.0% | 99.2% | −0.8pp | |
| | ORG | 85.1% | 82.7% | −2.5pp | |
| | STREET | 98.9% | 99.0% | +0.2pp | |
| | CITY | 96.9% | 96.7% | −0.2pp | |
| | **overall** | **96.0%** | **95.4%** | **−0.6pp** | |
|
|
| INT8 quantization loss = **−0.6pp overall** (target was <3pp). No entity loses more than 2.5pp. |
|
|
| ## Label mapping |
|
|
| The model outputs 13 BIO classes: |
|
|
| | Model label | Meaning | |
| |-------------|---------| |
| | `B-PERSON` / `I-PERSON` | full person name | |
| | `B-PERSON_F` / `I-PERSON_F` | first name only | |
| | `B-PERSON_L` / `I-PERSON_L` | last name only | |
| | `B-ORG` / `I-ORG` | company / institution | |
| | `B-STREET` / `I-STREET` | street / road / avenue | |
| | `B-CITY` / `I-CITY` | city | |
| | `O` | not an entity | |
|
|
| ## Files |
|
|
| | File | Format | Notes | |
| |------|--------|-------| |
| | `model.onnx` | FP32 | highest quality, BERT graph-optimized | |
| | `model_quantized.onnx` | INT8 | **recommended for browser** | |
| | `onnx/model_quantized.onnx` | INT8 | alias for Transformers.js `dtype:"q8"` | |
| | `config.json` | JSON | label mapping, model config | |
| | `tokenizer.json` | JSON | HerBERT tokenizer | |
| | `tokenizer_config.json` | JSON | HerBERT tokenizer config | |
|
|
| ## Usage |
|
|
| ### Python (Transformers) |
|
|
| ```python |
| from transformers import pipeline |
| |
| ner = pipeline( |
| "token-classification", |
| model="ArkadiuszPawlak/fastpdn-ner-polish-pii", |
| aggregation_strategy="simple", |
| ) |
| result = ner("Jan Kowalski mieszka przy ul. Marszałkowskiej 1, 00-001 Warszawa.") |
| # [{"entity_group": "PERSON", "word": "Jan Kowalski", ...}, |
| # {"entity_group": "STREET", "word": "ul. Marszałkowskiej", ...}, |
| # {"entity_group": "CITY", "word": "Warszawa", ...}] |
| ``` |
|
|
| ### Browser (@xenova/transformers + ONNX Runtime Web) |
|
|
| ```js |
| import { pipeline } from "@xenova/transformers"; |
| |
| const ner = await pipeline( |
| "token-classification", |
| "ArkadiuszPawlak/fastpdn-ner-polish-pii", |
| { dtype: "q8", aggregation_strategy: "simple" } |
| ); |
| |
| const raw = await ner("Jan Kowalski mieszka przy ul. Marszałkowskiej 1 w Warszawie."); |
| console.log(raw); |
| // [{ entity_group: "PERSON", word: "Jan Kowalski", score: 0.99 }, |
| // { entity_group: "STREET", word: "ul. Marszałkowskiej", score: 0.98 }, |
| // { entity_group: "CITY", word: "Warszawa", score: 0.97 }] |
| ``` |
|
|
