Initial release

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+confusion_matrix_entity.png filter=lfs diff=lfs merge=lfs -text
+label_metrics_entity.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,143 @@
+---
+language:
+  - it
+license: apache-2.0
+tags:
+  - token-classification
+  - ner
+  - italian
+  - transformers
+  - pytorch
+datasets:
+  - custom
+metrics:
+  - f1
+  - precision
+  - recall
+base_model: colinglab/BureauBERTo
+pipeline_tag: token-classification
+widget:
+  - text: "Mario Rossi, nato il 15/03/1985, residente in Via Roma 123, 00100 Roma, codice fiscale RSSMRA85C15H501Z."
+    example_title: "Documento anagrafico"
+  - text: "Il paziente assume Tachipirina 1000mg due volte al giorno per 5 giorni."
+    example_title: "Documento medico"
+---
+
+# Nerone: Italian NER for Sensitive Data
+
+Named Entity Recognition model for extracting and classifying sensitive personal information from Italian documents.
+
+## Model Description
+
+Fine-tuned [BureauBERTo](https://huggingface.co/colinglab/BureauBERTo) (Italian BERT variant) for token classification with 70 entity types:
+
+- **Personal**: PERSON, AGE, GENDER, MARITAL_STATUS, PROFESSION, BLOOD_TYPE, FISCAL_CODE
+- **Geographic**: ADDRESS, COUNTRY, REGION, PROVINCE, MUNICIPALITY, ZIP_CODE, LATITUDE, LONGITUDE, ALTITUDE
+- **Contact**: PHONE, EMAIL, URL
+- **Financial**: MONEY_AMOUNT, PERCENTAGE, CARD_NUMBER, CVV, CHECK_NUMBER, ACCOUNT_NUMBER, IBAN, BIC, VAT_NUMBER, TAX_TYPE
+- **Medical**: DISEASE, MEDICINE, DOSAGE, FORM, MEDICAL_RECORD
+- **Legal/Administrative**: PASSPORT, DRIVER_LICENSE, LICENSE_NUMBER, LICENSE_PLATE, LAW, COURT, ACT_NUMBER, PROTOCOL_NUMBER, PROPERTY_REGIME
+- **Cadastral**: CADASTRAL_SHEET, CADASTRAL_PARCEL, CADASTRAL_MAP, CADASTRAL_SUB
+- **Technical**: IP, IMEI, MAC, UUID, VIN, OTP_CODE, PIN
+- **Codes**: ISBN, CIG_CODE, CUP_CODE, REA_CODE, SDI_CODE, ATC_CODE, ATECO_CODE, ICD_CODE
+- **Temporal**: DATE, DATE_RANGE, TIME, TIME_RANGE, YEAR, DURATION, FREQUENCY
+- **Misc**: ORGANIZATION
+
+## Dataset
+
+- **Total samples**: 122,625
+- **Split**: 70% train / 15% validation / 15% test
+- **Source**: Italian administrative documents
+
+## Training
+
+- **Base model**: colinglab/BureauBERTo
+- **Learning rate**: 4e-5
+- **Batch size**: 32
+- **Max sequence length**: 256
+
+## Evaluation Results
+
+| Metric    | Score |
+|-----------|-------|
+| F1        | 0.915 |
+| Precision | 0.895 |
+| Recall    | 0.936 |
+
+![Entity-level metrics](label_metrics_entity.png)
+
+![Confusion matrix](confusion_matrix_entity.png)
+
+## Usage
+
+```python
+from transformers import AutoModelForTokenClassification, AutoTokenizer, pipeline
+
+model = AutoModelForTokenClassification.from_pretrained("lcs06/nerone")
+tokenizer = AutoTokenizer.from_pretrained("lcs06/nerone")
+
+ner = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="first")
+
+text = """Il sottoscritto Mario Rossi, nato a Roma il 15/03/1985,
+residente in Via Garibaldi 42, 00153 Roma (RM),
+codice fiscale RSSMRA85C15H501Z,
+dichiara di essere titolare del conto corrente
+IBAN IT60X0542811101000000123456 presso Banca Intesa."""
+
+entities = ner(text)
+print(entities)
+```
+
+**Output:**
+```json
+[
+  {"entity_group": "PERSON", "score": 1.0, "word": "Mario Rossi", "start": 15, "end": 26},
+  {"entity_group": "MUNICIPALITY", "score": 1.0, "word": "Roma", "start": 35, "end": 39},
+  {"entity_group": "DATE", "score": 1.0, "word": "15/03/1985", "start": 43, "end": 53},
+  {"entity_group": "ADDRESS", "score": 1.0, "word": "Via Garibaldi 42, 00153 Roma (RM)", "start": 68, "end": 101},
+  {"entity_group": "FISCAL_CODE", "score": 1.0, "word": "RSSMRA85C15H501Z", "start": 118, "end": 134},
+  {"entity_group": "IBAN", "score": 0.99, "word": "IT60X0542811101000000123456", "start": 188, "end": 215},
+  {"entity_group": "ORGANIZATION", "score": 1.0, "word": "Banca Intesa", "start": 223, "end": 235}
+]
+```
+
+## Intended Use
+
+Designed for processing Italian administrative and legal documents to identify and classify sensitive personal data. Primary use cases:
+
+- Document anonymization
+- GDPR compliance
+- Data extraction from public administration documents
+
+## Limitations
+
+- Optimized for formal Italian text (administrative, legal, medical documents)
+- Performance may degrade on informal text, dialects, or non-standard formatting
+
+## Acknowledgements
+
+This model is fine-tuned from [BureauBERTo](https://huggingface.co/colinglab/BureauBERTo), developed by CoLingLab at the University of Pisa. BureauBERTo adapts [UmBERTo](https://huggingface.co/Musixmatch/umberto-commoncrawl-cased-v1) to Italian bureaucratic and administrative language.
+
+```bibtex
+@inproceedings{auriemma2023bureauberto,
+  title = {{BureauBERTo}: adapting {UmBERTo} to the {Italian} bureaucratic language},
+  author = {Auriemma, Serena and Madeddu, Mauro and Miliani, Martina and Bondielli, Alessandro and Passaro, Lucia C and Lenci, Alessandro},
+  booktitle = {Proceedings of the Italia Intelligenza Artificiale - Thematic Workshops (Ital IA 2023)},
+  series = {CEUR Workshop Proceedings},
+  volume = {3486},
+  pages = {240--248},
+  publisher = {CEUR-WS.org},
+  year = {2023},
+  url = {https://ceur-ws.org/Vol-3486/42.pdf}
+}
+```
+
+## Framework Versions
+
+- Transformers: 4.57.6
+- PyTorch: 2.11.0
+- Python: 3.13
+
+## License
+
+Apache 2.0

config.json

@@ -0,0 +1,249 @@
+{
+  "architectures": [
+    "CamembertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 5,
+  "classifier_dropout": 0.3,
+  "dtype": "float32",
+  "eos_token_id": 6,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.2,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "O",
+    "1": "B-ACCOUNT_NUMBER",
+    "2": "B-ACT_NUMBER",
+    "3": "B-ADDRESS",
+    "4": "B-AGE",
+    "5": "B-ALTITUDE",
+    "6": "B-ATC_CODE",
+    "7": "B-ATECO_CODE",
+    "8": "B-BIC",
+    "9": "B-BLOOD_TYPE",
+    "10": "B-CADASTRAL_MAP",
+    "11": "B-CADASTRAL_PARCEL",
+    "12": "B-CADASTRAL_SHEET",
+    "13": "B-CADASTRAL_SUB",
+    "14": "B-CARD_NUMBER",
+    "15": "B-CHECK_NUMBER",
+    "16": "B-CIG_CODE",
+    "17": "B-COUNTRY",
+    "18": "B-COURT",
+    "19": "B-CUP_CODE",
+    "20": "B-CVV",
+    "21": "B-DATE",
+    "22": "B-DATE_RANGE",
+    "23": "B-DISEASE",
+    "24": "B-DOSAGE",
+    "25": "B-DRIVER_LICENSE",
+    "26": "B-DURATION",
+    "27": "B-EMAIL",
+    "28": "B-FISCAL_CODE",
+    "29": "B-FORM",
+    "30": "B-FREQUENCY",
+    "31": "B-GENDER",
+    "32": "B-IBAN",
+    "33": "B-ICD_CODE",
+    "34": "B-IMEI",
+    "35": "B-IP",
+    "36": "B-ISBN",
+    "37": "B-LATITUDE",
+    "38": "B-LAW",
+    "39": "B-LICENSE_NUMBER",
+    "40": "B-LICENSE_PLATE",
+    "41": "B-LONGITUDE",
+    "42": "B-MAC",
+    "43": "B-MARITAL_STATUS",
+    "44": "B-MEDICAL_RECORD",
+    "45": "B-MEDICINE",
+    "46": "B-MONEY_AMOUNT",
+    "47": "B-MUNICIPALITY",
+    "48": "B-ORGANIZATION",
+    "49": "B-OTP_CODE",
+    "50": "B-PASSPORT",
+    "51": "B-PERCENTAGE",
+    "52": "B-PERSON",
+    "53": "B-PHONE",
+    "54": "B-PIN",
+    "55": "B-PROFESSION",
+    "56": "B-PROPERTY_REGIME",
+    "57": "B-PROTOCOL_NUMBER",
+    "58": "B-PROVINCE",
+    "59": "B-REA_CODE",
+    "60": "B-REGION",
+    "61": "B-SDI_CODE",
+    "62": "B-TAX_TYPE",
+    "63": "B-TIME",
+    "64": "B-TIME_RANGE",
+    "65": "B-URL",
+    "66": "B-UUID",
+    "67": "B-VAT_NUMBER",
+    "68": "B-VIN",
+    "69": "B-YEAR",
+    "70": "B-ZIP_CODE",
+    "71": "I-ADDRESS",
+    "72": "I-AGE",
+    "73": "I-BIC",
+    "74": "I-BLOOD_TYPE",
+    "75": "I-CADASTRAL_MAP",
+    "76": "I-CADASTRAL_PARCEL",
+    "77": "I-CADASTRAL_SHEET",
+    "78": "I-CADASTRAL_SUB",
+    "79": "I-CARD_NUMBER",
+    "80": "I-COUNTRY",
+    "81": "I-COURT",
+    "82": "I-DATE",
+    "83": "I-DATE_RANGE",
+    "84": "I-DISEASE",
+    "85": "I-DOSAGE",
+    "86": "I-DURATION",
+    "87": "I-EMAIL",
+    "88": "I-FORM",
+    "89": "I-FREQUENCY",
+    "90": "I-IBAN",
+    "91": "I-LAW",
+    "92": "I-LICENSE_NUMBER",
+    "93": "I-LICENSE_PLATE",
+    "94": "I-MAC",
+    "95": "I-MEDICAL_RECORD",
+    "96": "I-MEDICINE",
+    "97": "I-MONEY_AMOUNT",
+    "98": "I-MUNICIPALITY",
+    "99": "I-ORGANIZATION",
+    "100": "I-PERSON",
+    "101": "I-PHONE",
+    "102": "I-PROFESSION",
+    "103": "I-PROPERTY_REGIME",
+    "104": "I-PROVINCE",
+    "105": "I-REA_CODE",
+    "106": "I-REGION",
+    "107": "I-TIME",
+    "108": "I-TIME_RANGE"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B-ACCOUNT_NUMBER": 1,
+    "B-ACT_NUMBER": 2,
+    "B-ADDRESS": 3,
+    "B-AGE": 4,
+    "B-ALTITUDE": 5,
+    "B-ATC_CODE": 6,
+    "B-ATECO_CODE": 7,
+    "B-BIC": 8,
+    "B-BLOOD_TYPE": 9,
+    "B-CADASTRAL_MAP": 10,
+    "B-CADASTRAL_PARCEL": 11,
+    "B-CADASTRAL_SHEET": 12,
+    "B-CADASTRAL_SUB": 13,
+    "B-CARD_NUMBER": 14,
+    "B-CHECK_NUMBER": 15,
+    "B-CIG_CODE": 16,
+    "B-COUNTRY": 17,
+    "B-COURT": 18,
+    "B-CUP_CODE": 19,
+    "B-CVV": 20,
+    "B-DATE": 21,
+    "B-DATE_RANGE": 22,
+    "B-DISEASE": 23,
+    "B-DOSAGE": 24,
+    "B-DRIVER_LICENSE": 25,
+    "B-DURATION": 26,
+    "B-EMAIL": 27,
+    "B-FISCAL_CODE": 28,
+    "B-FORM": 29,
+    "B-FREQUENCY": 30,
+    "B-GENDER": 31,
+    "B-IBAN": 32,
+    "B-ICD_CODE": 33,
+    "B-IMEI": 34,
+    "B-IP": 35,
+    "B-ISBN": 36,
+    "B-LATITUDE": 37,
+    "B-LAW": 38,
+    "B-LICENSE_NUMBER": 39,
+    "B-LICENSE_PLATE": 40,
+    "B-LONGITUDE": 41,
+    "B-MAC": 42,
+    "B-MARITAL_STATUS": 43,
+    "B-MEDICAL_RECORD": 44,
+    "B-MEDICINE": 45,
+    "B-MONEY_AMOUNT": 46,
+    "B-MUNICIPALITY": 47,
+    "B-ORGANIZATION": 48,
+    "B-OTP_CODE": 49,
+    "B-PASSPORT": 50,
+    "B-PERCENTAGE": 51,
+    "B-PERSON": 52,
+    "B-PHONE": 53,
+    "B-PIN": 54,
+    "B-PROFESSION": 55,
+    "B-PROPERTY_REGIME": 56,
+    "B-PROTOCOL_NUMBER": 57,
+    "B-PROVINCE": 58,
+    "B-REA_CODE": 59,
+    "B-REGION": 60,
+    "B-SDI_CODE": 61,
+    "B-TAX_TYPE": 62,
+    "B-TIME": 63,
+    "B-TIME_RANGE": 64,
+    "B-URL": 65,
+    "B-UUID": 66,
+    "B-VAT_NUMBER": 67,
+    "B-VIN": 68,
+    "B-YEAR": 69,
+    "B-ZIP_CODE": 70,
+    "I-ADDRESS": 71,
+    "I-AGE": 72,
+    "I-BIC": 73,
+    "I-BLOOD_TYPE": 74,
+    "I-CADASTRAL_MAP": 75,
+    "I-CADASTRAL_PARCEL": 76,
+    "I-CADASTRAL_SHEET": 77,
+    "I-CADASTRAL_SUB": 78,
+    "I-CARD_NUMBER": 79,
+    "I-COUNTRY": 80,
+    "I-COURT": 81,
+    "I-DATE": 82,
+    "I-DATE_RANGE": 83,
+    "I-DISEASE": 84,
+    "I-DOSAGE": 85,
+    "I-DURATION": 86,
+    "I-EMAIL": 87,
+    "I-FORM": 88,
+    "I-FREQUENCY": 89,
+    "I-IBAN": 90,
+    "I-LAW": 91,
+    "I-LICENSE_NUMBER": 92,
+    "I-LICENSE_PLATE": 93,
+    "I-MAC": 94,
+    "I-MEDICAL_RECORD": 95,
+    "I-MEDICINE": 96,
+    "I-MONEY_AMOUNT": 97,
+    "I-MUNICIPALITY": 98,
+    "I-ORGANIZATION": 99,
+    "I-PERSON": 100,
+    "I-PHONE": 101,
+    "I-PROFESSION": 102,
+    "I-PROPERTY_REGIME": 103,
+    "I-PROVINCE": 104,
+    "I-REA_CODE": 105,
+    "I-REGION": 106,
+    "I-TIME": 107,
+    "I-TIME_RANGE": 108,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "camembert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.6",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 40310
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8167f2d881ffb666d82418da49fa395021e7ed44a9a7899fb9f0651a7f3e7690
+size 465997628

special_tokens_map.json

@@ -0,0 +1,55 @@
+{
+  "additional_special_tokens": [
+    "<s>NOTUSED",
+    "</s>NOTUSED"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}