Duplicate from urchade/gliner_multi_pii-v1

Co-authored-by: Urchade Zaratiana <urchade@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,53 @@
+---
+license: apache-2.0
+language:
+- en
+- fr
+- de
+- es
+- pt
+- it
+library_name: gliner
+pipeline_tag: token-classification
+datasets:
+- urchade/synthetic-pii-ner-mistral-v1
+---
+
+
+# Model Card for GLiNER PII
+
+GLiNER is a Named Entity Recognition (NER) model capable of identifying any entity type using a bidirectional transformer encoder (BERT-like). It provides a practical alternative to traditional NER models, which are limited to predefined entities, and Large Language Models (LLMs) that, despite their flexibility, are costly and large for resource-constrained scenarios.
+
+This model has been trained by fine-tuning `urchade/gliner_multi-v2.1` on the `urchade/synthetic-pii-ner-mistral-v1` dataset.
+
+This model is capable of recognizing various types of *personally identifiable information* (PII), including but not limited to these entity types: `person`, `organization`, `phone number`, `address`, `passport number`, `email`, `credit card number`, `social security number`, `health insurance id number`, `date of birth`, `mobile phone number`, `bank account number`, `medication`, `cpf`, `driver's license number`, `tax identification number`, `medical condition`, `identity card number`, `national id number`, `ip address`, `email address`, `iban`, `credit card expiration date`, `username`, `health insurance number`, `registration number`, `student id number`, `insurance number`, `flight number`, `landline phone number`, `blood type`, `cvv`, `reservation number`, `digital signature`, `social media handle`, `license plate number`, `cnpj`, `postal code`, `passport_number`, `serial number`, `vehicle registration number`, `credit card brand`, `fax number`, `visa number`, `insurance company`, `identity document number`, `transaction number`, `national health insurance number`, `cvc`, `birth certificate number`, `train ticket number`, `passport expiration date`, and `social_security_number`.
+  
+## Links
+
+* Paper: https://arxiv.org/abs/2311.08526
+* Repository: https://github.com/urchade/GLiNER
+
+```python
+from gliner import GLiNER
+
+model = GLiNER.from_pretrained("urchade/gliner_multi_pii-v1")
+
+text = """
+Harilala Rasoanaivo, un homme d'affaires local d'Antananarivo, a enregistré une nouvelle société nommée "Rasoanaivo Enterprises" au Lot II M 92 Antohomadinika. Son numéro est le +261 32 22 345 67, et son adresse électronique est harilala.rasoanaivo@telma.mg. Il a fourni son numéro de sécu 501-02-1234 pour l'enregistrement.
+"""
+
+labels = ["work", "booking number", "personally identifiable information", "driver licence", "person", "book", "full address", "company", "actor", "character", "email", "passport number", "Social Security Number", "phone number"]
+entities = model.predict_entities(text, labels)
+
+for entity in entities:
+    print(entity["text"], "=>", entity["label"])
+```
+
+```
+Harilala Rasoanaivo => person
+Rasoanaivo Enterprises => company
+Lot II M 92 Antohomadinika => full address
++261 32 22 345 67 => phone number
+harilala.rasoanaivo@telma.mg => email
+501-02-1234 => Social Security Number
+```

gliner_config.json

@@ -0,0 +1,22 @@
+{
+  "size_sup": -1,
+  "max_types": 25,
+  "shuffle_types": true,
+  "random_drop": false,
+  "max_neg_type_ratio": 3,
+  "max_len": 384,
+  "lr_encoder": "5e-6",
+  "lr_others": "1e-5",
+  "num_steps": 30000,
+  "warmup_ratio": 5000,
+  "train_batch_size": 8,
+  "eval_every": 5000,
+  "max_width": 12,
+  "model_name": "microsoft/mdeberta-v3-base",
+  "fine_tune": true,
+  "subtoken_pooling": "first",
+  "hidden_size": 512,
+  "span_mode": "markerV0",
+  "dropout": 0.4,
+  "name": "correct"
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3003753fba99e40645cf088c7367a2c6211fc174897dc64f1f9c147c29d18d2d
+size 1155900362