Upload PII-NER v1 model (F1=0.904, 40 entity types)

README.md

@@ -0,0 +1,175 @@
+---
+library_name: transformers
+license: apache-2.0
+language:
+- en
+tags:
+- token-classification
+- ner
+- pii
+- privacy
+- deberta
+- crf
+datasets:
+- ai4privacy/internationalised_pii_dataset
+- gretelai/gretel-pii-masking-en-v1
+pipeline_tag: token-classification
+model-index:
+- name: datafog-pii-ner-v1
+  results:
+  - task:
+      type: token-classification
+      name: Named Entity Recognition
+    metrics:
+    - type: f1
+      value: 0.904
+      name: Overall F1
+    - type: precision
+      value: 0.907
+      name: Overall Precision
+    - type: recall
+      value: 0.902
+      name: Overall Recall
+---
+
+# DataFog PII-NER v1
+
+A token classification model for detecting **Personally Identifiable Information (PII)** in English text. Built on DeBERTa-v3-xsmall with character-level CNN features and a CRF decoding head for structured BIO tag prediction.
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Architecture | DeBERTa-v3-xsmall + CharCNN + CRF |
+| Parameters | ~22.7M total |
+| Labels | 89 BIO tags (40 entity types) |
+| Max sequence length | 256 tokens |
+| Training data | ~135K examples from 3 datasets |
+| Training hardware | NVIDIA A100 (Colab), BF16 mixed precision |
+| Framework | Transformers 5.0, PyTorch 2.x |
+
+## Architecture
+
+```
+Input text
+    |
+    v
+DeBERTa-v3-xsmall (70.7M pretrained params)
+    |
+    v
+Character CNN (3/4/5-gram filters)
+    |
+    v
+Gating Fusion (learned weighted combination)
+    |
+    v
+CRF Head (sequence-level decoding)
+    |
+    v
+89 BIO tag predictions
+```
+
+The CRF head enforces valid BIO tag sequences (e.g., I-PERSON can only follow B-PERSON or I-PERSON), which improves entity boundary detection compared to independent per-token classification.
+
+## Supported Entity Types (40 types, 4 tiers)
+
+### Tier 1 -- Critical PII
+SSN, Credit Card, Bank Account, Passport Number, Drivers License, Tax ID
+
+### Tier 2 -- High Sensitivity
+Person, Email, Phone, Date of Birth, Street Address, IP Address
+
+### Tier 3 -- Moderate Sensitivity
+Username, Date, Location, Organization, URL, License Plate, Age, Nationality, Gender, Ethnicity, Religion, Marital Status
+
+### Tier 4 -- Domain-Specific
+Medical Record, Employee ID, Student ID, Account Number, PIN, Password, Biometric, Vehicle ID, Device ID, Crypto Wallet, IBAN, Swift Code, Insurance Number, Salary, Criminal Record, Political Affiliation, Sexual Orientation, Health Condition, Genetic Data, Trade Union
+
+## Test Set Results
+
+| Metric | Value |
+|--------|-------|
+| **Overall F1** | **0.904** |
+| Overall Precision | 0.907 |
+| Overall Recall | 0.902 |
+
+### Tier Recall
+
+| Tier | Recall | Target |
+|------|--------|--------|
+| Tier 1 (Critical) | 0.722 | 0.98 |
+| Tier 2 (High) | 0.934 | 0.95 |
+| Tier 3 (Moderate) | 0.919 | 0.90 |
+| Tier 4 (Domain) | 0.866 | 0.85 |
+
+### Per-Entity F1 (All Types)
+
+| Entity Type | F1 | Recall |
+|-------------|-----|--------|
+| Biometric | 0.996 | 0.996 |
+| URL | 0.994 | 0.995 |
+| Email | 0.991 | 0.987 |
+| IP Address | 0.988 | 0.992 |
+| Date of Birth | 0.978 | 0.980 |
+| Vehicle ID | 0.964 | 0.989 |
+| Phone | 0.963 | 0.961 |
+| Employee ID | 0.962 | 0.959 |
+| License Plate | 0.960 | 0.952 |
+| Gender | 0.952 | 0.949 |
+| IBAN | 0.930 | 0.898 |
+| Swift Code | 0.926 | 0.980 |
+| Username | 0.924 | 0.912 |
+| Location | 0.922 | 0.908 |
+| Account Number | 0.908 | 0.917 |
+| Organization | 0.898 | 0.903 |
+| SSN | 0.891 | 0.858 |
+| Drivers License | 0.885 | 0.881 |
+| Password | 0.878 | 0.885 |
+| Date | 0.875 | 0.869 |
+| Person | 0.861 | 0.868 |
+| Credit Card | 0.862 | 0.839 |
+| Age | 0.851 | 0.861 |
+| Street Address | 0.834 | 0.817 |
+| Bank Account | 0.791 | 0.746 |
+| Tax ID | 0.665 | 0.624 |
+| Passport Number | 0.469 | 0.385 |
+| PIN | 0.432 | 0.302 |
+
+## Training Details
+
+- **Backbone LR:** 2e-5 (with AdamW eps=1.0 to prevent NaN)
+- **Head LR:** 1e-3 (50x faster than backbone)
+- **Warmup:** 10% of steps
+- **Epochs:** 10 (best checkpoint at epoch 5)
+- **Effective batch size:** 32
+- **Mixed precision:** BF16
+
+## Training Data
+
+Trained on a combined dataset of ~135K examples from:
+- [AI4Privacy PII Dataset](https://huggingface.co/datasets/ai4privacy/internationalised_pii_dataset)
+- [Nemotron PII](https://huggingface.co/datasets/ai4privacy/pii-masking-400k)
+- [Gretel PII Masking](https://huggingface.co/datasets/gretelai/gretel-pii-masking-en-v1)
+
+## Limitations
+
+- Tier 1 recall (0.722) is below the 0.98 target -- critical PII types like SSN, Credit Card, and Passport Number need improvement
+- Rare entity types (PIN, Passport Number, Tax ID) have low F1 due to limited training examples
+- English-only
+- Max 256 tokens per input (longer documents need chunking)
+- Custom architecture requires the `datafog-pii-ner` package for loading (not a standard HuggingFace token classifier)
+
+## Citation
+
+```bibtex
+@software{datafog_pii_ner_v1,
+  title={DataFog PII-NER v1: Token Classification for PII Detection},
+  author={DataFog},
+  year={2026},
+  url={https://github.com/DataFog/datafog-labs}
+}
+```
+
+## License
+
+Apache 2.0

config.json

@@ -0,0 +1,207 @@
+{
+  "architectures": [
+    "PiiNerModel"
+  ],
+  "backbone": "microsoft/deberta-v3-xsmall",
+  "char_cnn_filters": [
+    50,
+    50,
+    50
+  ],
+  "char_cnn_widths": [
+    3,
+    4,
+    5
+  ],
+  "char_embed_dim": 50,
+  "char_vocab_size": 256,
+  "dropout": 0.1,
+  "dtype": "float16",
+  "id2label": {
+    "0": "O",
+    "1": "B-SSN",
+    "2": "I-SSN",
+    "3": "B-CREDIT_CARD",
+    "4": "I-CREDIT_CARD",
+    "5": "B-BANK_ACCOUNT",
+    "6": "I-BANK_ACCOUNT",
+    "7": "B-PASSPORT_NUMBER",
+    "8": "I-PASSPORT_NUMBER",
+    "9": "B-DRIVERS_LICENSE",
+    "10": "I-DRIVERS_LICENSE",
+    "11": "B-TAX_ID",
+    "12": "I-TAX_ID",
+    "13": "B-PERSON",
+    "14": "I-PERSON",
+    "15": "B-EMAIL",
+    "16": "I-EMAIL",
+    "17": "B-PHONE",
+    "18": "I-PHONE",
+    "19": "B-DATE_OF_BIRTH",
+    "20": "I-DATE_OF_BIRTH",
+    "21": "B-STREET_ADDRESS",
+    "22": "I-STREET_ADDRESS",
+    "23": "B-IP_ADDRESS",
+    "24": "I-IP_ADDRESS",
+    "25": "B-USERNAME",
+    "26": "I-USERNAME",
+    "27": "B-DATE",
+    "28": "I-DATE",
+    "29": "B-LOCATION",
+    "30": "I-LOCATION",
+    "31": "B-ORGANIZATION",
+    "32": "I-ORGANIZATION",
+    "33": "B-URL",
+    "34": "I-URL",
+    "35": "B-LICENSE_PLATE",
+    "36": "I-LICENSE_PLATE",
+    "37": "B-AGE",
+    "38": "I-AGE",
+    "39": "B-NATIONALITY",
+    "40": "I-NATIONALITY",
+    "41": "B-GENDER",
+    "42": "I-GENDER",
+    "43": "B-ETHNICITY",
+    "44": "I-ETHNICITY",
+    "45": "B-RELIGION",
+    "46": "I-RELIGION",
+    "47": "B-MARITAL_STATUS",
+    "48": "I-MARITAL_STATUS",
+    "49": "B-MEDICAL_RECORD",
+    "50": "I-MEDICAL_RECORD",
+    "51": "B-EMPLOYEE_ID",
+    "52": "I-EMPLOYEE_ID",
+    "53": "B-STUDENT_ID",
+    "54": "I-STUDENT_ID",
+    "55": "B-ACCOUNT_NUMBER",
+    "56": "I-ACCOUNT_NUMBER",
+    "57": "B-PIN",
+    "58": "I-PIN",
+    "59": "B-PASSWORD",
+    "60": "I-PASSWORD",
+    "61": "B-BIOMETRIC",
+    "62": "I-BIOMETRIC",
+    "63": "B-VEHICLE_ID",
+    "64": "I-VEHICLE_ID",
+    "65": "B-DEVICE_ID",
+    "66": "I-DEVICE_ID",
+    "67": "B-CRYPTO_WALLET",
+    "68": "I-CRYPTO_WALLET",
+    "69": "B-IBAN",
+    "70": "I-IBAN",
+    "71": "B-SWIFT_CODE",
+    "72": "I-SWIFT_CODE",
+    "73": "B-INSURANCE_NUMBER",
+    "74": "I-INSURANCE_NUMBER",
+    "75": "B-SALARY",
+    "76": "I-SALARY",
+    "77": "B-CRIMINAL_RECORD",
+    "78": "I-CRIMINAL_RECORD",
+    "79": "B-POLITICAL_AFFILIATION",
+    "80": "I-POLITICAL_AFFILIATION",
+    "81": "B-SEXUAL_ORIENTATION",
+    "82": "I-SEXUAL_ORIENTATION",
+    "83": "B-HEALTH_CONDITION",
+    "84": "I-HEALTH_CONDITION",
+    "85": "B-GENETIC_DATA",
+    "86": "I-GENETIC_DATA",
+    "87": "B-TRADE_UNION",
+    "88": "I-TRADE_UNION"
+  },
+  "label2id": {
+    "O": 0,
+    "B-SSN": 1,
+    "I-SSN": 2,
+    "B-CREDIT_CARD": 3,
+    "I-CREDIT_CARD": 4,
+    "B-BANK_ACCOUNT": 5,
+    "I-BANK_ACCOUNT": 6,
+    "B-PASSPORT_NUMBER": 7,
+    "I-PASSPORT_NUMBER": 8,
+    "B-DRIVERS_LICENSE": 9,
+    "I-DRIVERS_LICENSE": 10,
+    "B-TAX_ID": 11,
+    "I-TAX_ID": 12,
+    "B-PERSON": 13,
+    "I-PERSON": 14,
+    "B-EMAIL": 15,
+    "I-EMAIL": 16,
+    "B-PHONE": 17,
+    "I-PHONE": 18,
+    "B-DATE_OF_BIRTH": 19,
+    "I-DATE_OF_BIRTH": 20,
+    "B-STREET_ADDRESS": 21,
+    "I-STREET_ADDRESS": 22,
+    "B-IP_ADDRESS": 23,
+    "I-IP_ADDRESS": 24,
+    "B-USERNAME": 25,
+    "I-USERNAME": 26,
+    "B-DATE": 27,
+    "I-DATE": 28,
+    "B-LOCATION": 29,
+    "I-LOCATION": 30,
+    "B-ORGANIZATION": 31,
+    "I-ORGANIZATION": 32,
+    "B-URL": 33,
+    "I-URL": 34,
+    "B-LICENSE_PLATE": 35,
+    "I-LICENSE_PLATE": 36,
+    "B-AGE": 37,
+    "I-AGE": 38,
+    "B-NATIONALITY": 39,
+    "I-NATIONALITY": 40,
+    "B-GENDER": 41,
+    "I-GENDER": 42,
+    "B-ETHNICITY": 43,
+    "I-ETHNICITY": 44,
+    "B-RELIGION": 45,
+    "I-RELIGION": 46,
+    "B-MARITAL_STATUS": 47,
+    "I-MARITAL_STATUS": 48,
+    "B-MEDICAL_RECORD": 49,
+    "I-MEDICAL_RECORD": 50,
+    "B-EMPLOYEE_ID": 51,
+    "I-EMPLOYEE_ID": 52,
+    "B-STUDENT_ID": 53,
+    "I-STUDENT_ID": 54,
+    "B-ACCOUNT_NUMBER": 55,
+    "I-ACCOUNT_NUMBER": 56,
+    "B-PIN": 57,
+    "I-PIN": 58,
+    "B-PASSWORD": 59,
+    "I-PASSWORD": 60,
+    "B-BIOMETRIC": 61,
+    "I-BIOMETRIC": 62,
+    "B-VEHICLE_ID": 63,
+    "I-VEHICLE_ID": 64,
+    "B-DEVICE_ID": 65,
+    "I-DEVICE_ID": 66,
+    "B-CRYPTO_WALLET": 67,
+    "I-CRYPTO_WALLET": 68,
+    "B-IBAN": 69,
+    "I-IBAN": 70,
+    "B-SWIFT_CODE": 71,
+    "I-SWIFT_CODE": 72,
+    "B-INSURANCE_NUMBER": 73,
+    "I-INSURANCE_NUMBER": 74,
+    "B-SALARY": 75,
+    "I-SALARY": 76,
+    "B-CRIMINAL_RECORD": 77,
+    "I-CRIMINAL_RECORD": 78,
+    "B-POLITICAL_AFFILIATION": 79,
+    "I-POLITICAL_AFFILIATION": 80,
+    "B-SEXUAL_ORIENTATION": 81,
+    "I-SEXUAL_ORIENTATION": 82,
+    "B-HEALTH_CONDITION": 83,
+    "I-HEALTH_CONDITION": 84,
+    "B-GENETIC_DATA": 85,
+    "I-GENETIC_DATA": 86,
+    "B-TRADE_UNION": 87,
+    "I-TRADE_UNION": 88
+  },
+  "max_char_len": 20,
+  "model_type": "pii_ner",
+  "transformers_version": "5.0.0",
+  "use_cache": false,
+  "num_labels": 89
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:888bf4c2b3f70475540abbfa30672ae2826cf4f1c3b09b02ec45c52fb56cd066
+size 143144240

test_results.json

@@ -0,0 +1,70 @@
+{
+  "eval_loss": 2.6675541400909424,
+  "eval_overall_f1": 0.9043580541465338,
+  "eval_overall_precision": 0.9072015660825143,
+  "eval_overall_recall": 0.9015323117921386,
+  "eval_type_account_number_f1": 0.907736863880844,
+  "eval_type_account_number_recall": 0.9172240802675585,
+  "eval_type_age_f1": 0.850771869639794,
+  "eval_type_age_recall": 0.8611111111111112,
+  "eval_type_bank_account_f1": 0.791044776119403,
+  "eval_type_bank_account_recall": 0.7464788732394366,
+  "eval_type_biometric_f1": 0.9962141698215252,
+  "eval_type_biometric_recall": 0.9956756756756757,
+  "eval_type_credit_card_f1": 0.8619495299356753,
+  "eval_type_credit_card_recall": 0.8391136801541426,
+  "eval_type_date_f1": 0.8748460591133004,
+  "eval_type_date_recall": 0.8694920440636474,
+  "eval_type_date_of_birth_f1": 0.9784499054820416,
+  "eval_type_date_of_birth_recall": 0.9803030303030303,
+  "eval_type_drivers_license_f1": 0.8849557522123893,
+  "eval_type_drivers_license_recall": 0.8807588075880759,
+  "eval_type_email_f1": 0.9905722429291821,
+  "eval_type_email_recall": 0.9873251748251748,
+  "eval_type_employee_id_f1": 0.9622997172478794,
+  "eval_type_employee_id_recall": 0.9586854460093897,
+  "eval_type_gender_f1": 0.9522240527182866,
+  "eval_type_gender_recall": 0.9490968801313628,
+  "eval_type_iban_f1": 0.9304556354916067,
+  "eval_type_iban_recall": 0.8981481481481481,
+  "eval_type_ip_address_f1": 0.987908643081057,
+  "eval_type_ip_address_recall": 0.9919064748201439,
+  "eval_type_license_plate_f1": 0.9595290654893304,
+  "eval_type_license_plate_recall": 0.9518248175182482,
+  "eval_type_location_f1": 0.9216815623965575,
+  "eval_type_location_recall": 0.9077986437141367,
+  "eval_type_organization_f1": 0.8982110448535131,
+  "eval_type_organization_recall": 0.9025635681533972,
+  "eval_type_passport_number_f1": 0.46875,
+  "eval_type_passport_number_recall": 0.38461538461538464,
+  "eval_type_password_f1": 0.8778082191780823,
+  "eval_type_password_recall": 0.8850828729281768,
+  "eval_type_person_f1": 0.8611754487550666,
+  "eval_type_person_recall": 0.867643841610151,
+  "eval_type_phone_f1": 0.9628220140515222,
+  "eval_type_phone_recall": 0.9608530528775927,
+  "eval_type_pin_f1": 0.43209876543209874,
+  "eval_type_pin_recall": 0.3017241379310345,
+  "eval_type_ssn_f1": 0.8910891089108911,
+  "eval_type_ssn_recall": 0.8583106267029973,
+  "eval_type_street_address_f1": 0.833598628627403,
+  "eval_type_street_address_recall": 0.8165027584552651,
+  "eval_type_swift_code_f1": 0.9259259259259259,
+  "eval_type_swift_code_recall": 0.9803921568627451,
+  "eval_type_tax_id_f1": 0.6650602409638554,
+  "eval_type_tax_id_recall": 0.6244343891402715,
+  "eval_type_url_f1": 0.9938633938100321,
+  "eval_type_url_recall": 0.9946595460614153,
+  "eval_type_username_f1": 0.9244669316949765,
+  "eval_type_username_recall": 0.9122681883024251,
+  "eval_type_vehicle_id_f1": 0.9642458100558661,
+  "eval_type_vehicle_id_recall": 0.9885452462772051,
+  "eval_tier_1_recall": 0.7222852935733847,
+  "eval_tier_2_recall": 0.934089055481893,
+  "eval_tier_3_recall": 0.918601849881968,
+  "eval_tier_4_recall": 0.8656847205124917,
+  "eval_runtime": 165.9273,
+  "eval_samples_per_second": 102.123,
+  "eval_steps_per_second": 3.194,
+  "epoch": 10.0
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bed6f2f807b6b79aa6aa3df98784b3a76b787829f82f533f3a1e66be070a519
+size 5265

training_config.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "microsoft/deberta-v3-xsmall",
+  "max_seq_len": 256,
+  "max_char_len": 20,
+  "dropout": 0.1,
+  "epochs": 10,
+  "batch_size": 32,
+  "gradient_accumulation_steps": 1,
+  "lr_backbone": 2e-05,
+  "lr_head": 0.001,
+  "warmup_ratio": 0.1,
+  "weight_decay": 0.01,
+  "fp16": false,
+  "bf16": true,
+  "val_ratio": 0.1,
+  "test_ratio": 0.1,
+  "seed": 42,
+  "output_dir": "/content/pii_ner_v1_output",
+  "run_name": "pii-ner-v1-full",
+  "_try_bf16": true
+}