README.md

---
language:
- en
- zh
- ms
- id
- vi
- ta
- th
- hi
- bn
- ko
- ja
- de
- fr
- ru
license: apache-2.0
tags:
- pii
- ner
- gliner
- privacy
- gdpr
- pdpa
- multilingual
- onnx
datasets:
- custom
metrics:
- f1
pipeline_tag: token-classification
---

<p align="center">
  <a href="https://pii.engineer">
    <img src="https://pii.engineer/static/banner.webp" alt="PII Engineer" width="100%" />
  </a>
</p>

# PII Engineer — Multilingual NER v2.1

Fast, multilingual PII detection model. Detects 30+ PII types across 50+ languages from a single model, no GPU required.

**[Live Demo](https://pii.engineer)** · **[Benchmarks](https://pii.engineer/benchmarks)** · **[GitHub](https://github.com/gantz-ai/pii.engineer)** · **[Blog](https://pii.engineer/blog)**

## Benchmarks

| | PII Engineer | Presidio | spaCy | AWS Comprehend |
|---|---|---|---|---|
| **F1 (multilingual)** | **0.86** | 0.44 | 0.64 | 0.52 |
| **F1 (English)** | **0.88** | 0.80 | 0.83 | 0.82 |
| **Languages** | **50+** | ~10 locales | 1 per model | 12 |
| **Latency (p50)** | 180ms | 80ms (w/ NER) | 120ms | 200ms |
| **GPU required** | No | No | Optional | N/A |
| **Cost (1M req/mo)** | **$42** | $42 | $42 | ~$1,000 |

[Full benchmarks →](https://pii.engineer/benchmarks)

### Accuracy by Language

| Language | F1 |
|----------|-----|
| English | 0.931 |
| Chinese | 0.918 |
| Vietnamese | 0.912 |
| Korean | 0.905 |
| Indonesian | 0.901 |
| Malay | 0.895 |
| Hindi | 0.892 |
| Thai | 0.885 |
| Tamil | 0.878 |

### Per-Entity Accuracy

| Entity Type | F1 |
|-------------|-----|
| email_address | 0.970 |
| phone_number | 0.968 |
| government_id | 0.920 |
| bank_account_number | 0.915 |
| street_address | 0.891 |
| date_of_birth | 0.887 |
| passport_number | 0.880 |
| license_plate | 0.833 |
| person_name | 0.823 |

## PII Types Detected

`person_name` · `phone_number` · `government_id` · `street_address` · `date_of_birth` · `email_address` · `passport_number` · `license_plate` · `bank_account_number`

## Model Architecture

- **Base:** [GLiNER2](https://huggingface.co/fastino/gliner2-multi-v1) (span-based NER)
- **Encoder:** mDeBERTa-v3-base (280M params), fine-tuned with LoRA on PII data
- **Inference:** 5 ONNX models (encoder, span_rep, count_embed, count_pred, classifier)
- **Quantization:** INT8 encoder available (~15-20% faster on x86 CPU)
- **Total size:** ~620MB (all languages)

## Quick Start

### With PII Engineer Server (Rust)

```bash
git clone https://github.com/gantz-ai/pii.engineer
cd pii.engineer
cargo build --release --package pii-engineer-server
cargo run --release --package pii-engineer-server
# Models auto-download on first run
# API at http://localhost:8000
```

```bash
curl -X POST http://localhost:8000/api/detect \
  -H "Content-Type: application/json" \
  -d '{"text": "John Doe, NRIC S9012345B, born 12 March 1985"}'
```

### With Python

```python
import requests

resp = requests.post("http://localhost:8000/api/detect", json={
    "text": "John Doe lives at 42 Orchard Road, Singapore 238879",
    "labels": ["person_name", "street_address", "phone_number", "email_address"]
})

for entity in resp.json()["entities"]:
    print(f'{entity["type"]}: {entity["value"]} (score: {entity["score"]:.2f})')
```

### Download Models Manually

```bash
pip install huggingface_hub
huggingface-cli download pii-engineer/PII-Engineer-Multi-NER-v2.1 --local-dir models/PII-Engineer-Multi-NER-v2.1
huggingface-cli download pii-engineer/PII-Engineer-Chinese-NER-v1.0 --local-dir models/PII-Engineer-Chinese-NER-v1.0
```

## Use Cases

- **PDPA/GDPR/CCPA compliance** — detect PII in databases, logs, documents
- **Data anonymization** — redact PII before sharing datasets
- **CI/CD scanning** — catch leaked PII in code and configs
- **Chat/support data** — clean PII from customer interactions

## License

AGPL-3.0 — free for open-source use. Commercial license available at [pii.engineer](https://pii.engineer).

## Citation

```bibtex
@software{pii_engineer,
  title = {PII Engineer: Multilingual PII Detection},
  url = {https://github.com/gantz-ai/pii.engineer},
  year = {2026}
}
```