Upload 9 files

README.md

@@ -0,0 +1,208 @@
+---
+language:
+- pt
+license: other
+license_name: custom-agplv3-dual-license
+license_link: https://huggingface.co/inesctec/CitiLink-SpanBERT-Coreference-pt/blob/main/LICENSE
+tags:
+- coreference-resolution
+- meeting-minutes
+- portuguese
+- spanbert
+- administrative-documents
+library_name: transformers
+base_model:
+- neuralmind/bert-base-portuguese-cased
+pipeline_tag: token-classification
+---
+
+# SpanBERT-Coreference: Coreference Resolution for Portuguese Municipal Meeting Minutes
+
+This model consists of a fine-tuned **SpanBERT** for coreference resolution in minutes of Portuguese municipal meetings, identifying when different expressions in a document refer to the same real-world entity.
+
+## Model Description
+
+The **SpanBERT-PT-Coreference** uses a mention-ranking architecture (Lee et al., 2017) on top of a SpanBERT encoder pre-trained in Portuguese, specifically optimized for the linguistic and formal structure of administrative minutes in Portugal. The model detects coreferent mention clusters across 17 entity categories, enabling consistent pseudonymization of sensitive personal data throughout long documents.
+
+### Key Features
+
+- 🏛️ **Specialized for Municipal Minutes**: Fine-tuned on authentic Portuguese council meeting minutes
+- 🔗 **Coreference-Aware Pseudonymization**: Links multiple mentions of the same entity, ensuring consistent ID assignment across the document
+- 📄 **Long Document Support**: Handles documents of up to ~28,000 tokens via sliding window encoding with stride overlap
+- ⚙️ **Mention-Ranking Architecture**: Scores all antecedent candidates for each mention span and clusters them via Union-Find
+
+## Model Details
+
+- **Base Model**: SpanBERT (Portuguese continued pre-training)
+- **Architecture**: Mention-ranking coreference (Lee et al., 2017)
+- **Parameters**: ~110M (encoder) + ~25M (coreference heads)
+- **Max Window Size**: 512 tokens (stride: 128)
+- **Fine-tuning Dataset**: 120 Portuguese meeting minutes (6 municipalities, ~1.3M tokens)
+- **Evaluation Metrics**: MUC, B³, CEAF-e, CoNLL F1, LEA
+- **Training Framework**: PyTorch + Transformers
+
+## Entity Types
+
+The model resolves coreference across 17 entity categories:
+
+| Entity Type | Description | Example |
+|------------|-------------|---------|
+| `PERSONAL-NAME` | Proper names of individuals | João Silva → João → ele |
+| `PERSONAL-ADMIN` | Administrative identifiers and process numbers | 5597/2023 → o processo |
+| `PERSONAL-POSITION` | Professional roles and political positions | Presidente da Câmara → o Presidente |
+| `PERSONAL-ADDRESS` | Addresses and street names | Rua das Flores n.º 10 → Rua das Flores |
+| `PERSONAL-LOCATION` | Cities, parishes, or geographic locations | Porto → o concelho |
+| `PERSONAL-DATE` | Dates of events or decisions | 20/05/2023 |
+| `PERSONAL-COMPANY` | Companies or private legal entities | Construções & Filho, Lda |
+| `PERSONAL-INFO` | Biographical or sensitive personal data | NIF, número de contribuinte |
+| `PERSONAL-DEGREE` | Academic titles or professional degrees | Licenciatura de Psicologia |
+| `PERSONAL-TIME` | References to specific times | 14:30h |
+| `PERSONAL-LICENSE` | License plates or registration numbers | 48-RF-99 |
+| `PERSONAL-JOB` | Person's profession or occupation | Professor |
+| `PERSONAL-VEHICLE` | Vehicle identification and models | Mercedes-Benz Classe S |
+| `PERSONAL-FACULTY` | Higher education institutions | Faculdade de Economia |
+| `PERSONAL-FAMILY` | Kinship or family relationships | Marido |
+| `PERSONAL-ARTISTIC` | Artistic names or pseudonyms | Pintura |
+| `PERSONAL-OTHER` | Miscellaneous personal information | Dados diversos |
+
+## Cluster Distribution (Training Set)
+
+| Entity Class | Number of Clusters |
+|-------------|-------------------|
+| `PERSONAL-ADMIN` | 273 |
+| `PERSONAL-NAME` | 259 |
+| `PERSONAL-POSITION` | 174 |
+| `PERSONAL-ADDRESS` | 115 |
+| `PERSONAL-LOCATION` | 64 |
+| `PERSONAL-DATE` | 49 |
+| `PERSONAL-OTHER` | 16 |
+| `PERSONAL-INFO` | 12 |
+| `PERSONAL-COMPANY` | 10 |
+| `PERSONAL-DEGREE` | 6 |
+| `PERSONAL-LICENSE` | 6 |
+| `PERSONAL-VEHICLE` | 4 |
+| `PERSONAL-JOB` | 4 |
+| `PERSONAL-TIME` | 3 |
+| `PERSONAL-FAMILY` | 2 |
+| `PERSONAL-ARTISTIC` | 1 |
+| **Total** | **998** |
+
+## How It Works
+
+The model uses a two-stage pipeline. First, entity spans are identified by a NER model ([inesctec/CitiLink-XLMR-Anonymization-pt](https://huggingface.co/inesctec/CitiLink-XLMR-Anonymization-pt)). Then, the coreference model groups spans that refer to the same entity, assigning consistent IDs throughout the document.
+
+**INPUT:**
+```
+João Paulo Rosinha Daniel apresentou o requerimento n.º 2683/20020627.
+O pedido de João Paulo, referente ao processo 2683/20020627, foi aprovado.
+```
+
+**OUTPUT:**
+```
+<NAME-1> apresentou o requerimento n.º <ADMIN-1>.
+O pedido de <NAME-1>, referente ao processo <ADMIN-1>, foi aprovado.
+```
+
+### Architecture Details
+
+The mention-ranking model represents each span as:
+
+```
+span_repr = [h_start ; h_end ; head_attn ; width_emb]
+           = [768 ; 768 ; 768 ; 20] = 2324 dims
+```
+
+Antecedent scoring uses:
+
+```
+score(i,j) = mention_score(i) + mention_score(j) + antecedent_score([vi; vj; vi*vj; dist_emb])
+```
+
+Clusters are built via Union-Find, restricted to spans of the same NER class and scores above threshold.
+
+## Results
+
+### Overall Performance (Test Set)
+
+| Metric | Precision (%) | Recall (%) | F1 Score (%) |
+|--------|--------------|-----------|-------------|
+| **MUC** | 90.20 | 92.80 | **91.50** |
+| **B³** | 86.70 | 92.50 | **89.50** |
+| **CEAF-e** | 89.20 | 74.40 | **81.20** |
+| **CoNLL F1** | — | — | **87.40** |
+| **LEA** | 64.80 | 88.20 | **74.80** |
+
+
+## Usage
+
+### Quick Start
+
+The simplest way to use the model:
+
+```python
+import torch
+import json
+from transformers import AutoTokenizer, AutoModel
+from huggingface_hub import hf_hub_download
+
+model_id = "inesctec/CitiLink-SpanBERT-Coreference-pt"
+
+# Load tokenizer and encoder
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+encoder   = AutoModel.from_pretrained(model_id)
+
+# Load coreference heads and config
+cabecas_path   = hf_hub_download(repo_id=model_id, filename="cabecas_coref.pt")
+coref_cfg_path = hf_hub_download(repo_id=model_id, filename="coref_config.json")
+
+with open(coref_cfg_path) as f:
+    coref_cfg = json.load(f)
+
+cabecas = torch.load(cabecas_path, map_location="cpu", weights_only=False)
+
+# Input text (already tokenized by whitespace)
+tokens = [
+    "João", "Paulo", "Rosinha", "Daniel", "apresentou", "o",
+    "requerimento", "n.º", "2683/20020627", ".",
+    "O", "pedido", "de", "João", "Paulo", "foi", "aprovado",
+    "ao", "abrigo", "do", "processo", "2683/20020627", "."
+]
+
+# NER spans already extracted by a NER model — (start, end, class)
+ner_spans = [
+    (0,  3,  "NAME"),   # "João Paulo Rosinha Daniel"
+    (8,  8,  "ADMIN"),  # "2683/20020627"
+    (13, 14, "NAME"),   # "João Paulo"
+    (21, 21, "ADMIN"),  # "2683/20020627" (segunda ocorrência)
+]
+
+# Tokenize for encoder
+inputs = tokenizer(tokens, is_split_into_words=True, return_tensors="pt")
+
+# Run encoder
+with torch.no_grad():
+    outputs = encoder(**inputs)
+
+print("Coreference model loaded successfully.")
+print(f"Hidden states shape: {outputs.last_hidden_state.shape}")
+
+```
+
+## Limitations
+
+- **Domain Specificity**: Best performance on administrative/governmental meeting minutes
+- **Language**: Trained exclusively on European Portuguese (PT-PT)
+- **Window Size**: Processes documents via 512-token sliding windows; very long-range coreference (>5,000 tokens apart) may be missed
+- **NER Dependency**: Requires entity spans from a NER model as input — does not perform span detection independently
+- **Same-class Restriction**: Only links mentions of the same NER entity class (e.g., NAME to NAME, never NAME to POSITION)
+- **Dataset Size**: Fine-tuned on 61 training documents; performance on highly specific sub-domains may vary
+
+**Version**: 1.0  
+**Last Updated**: 2026-06-25
+
+---
+## License
+
+This project uses a custom dual-license based on AGPL v3.
+
+See the full license terms here: [LICENSE](./LICENSE)

cabecas_coref.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f44f3a6118075ae948cda07a1eafc8ea5acf518d6a4caac00991cc68781aadb
+size 46720064

config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "directionality": "bidi",
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 28996
+}

coref_config.json

@@ -0,0 +1,26 @@
+{
+  "hidden_size": 768,
+  "ffnn_dim": 1024,
+  "max_span_width": 52,
+  "max_antecedents": 50,
+  "max_cluster_size": 12,
+  "dropout": 0.3,
+  "dist_buckets": [
+    1,
+    2,
+    3,
+    4,
+    5,
+    8,
+    16,
+    32,
+    64
+  ],
+  "ner_union_margin": 0.5,
+  "min_cluster_score": 0.3,
+  "epsilon": 0.00040307387826032937,
+  "domain": "atas municipais portuguesas",
+  "language": "pt",
+  "best_val_conll": 0.9187203268708114,
+  "best_epoch": 13
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4929f0963b466f25a25ed32dca2cc2d64e8a8462191077536dd50e130c086136
+size 433263448

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}