README.md

---
library_name: transformers
tags:
  - transformers
  - pytorch
  - bert
  - legal-domain
  - entity-classification
  - sequence-classification
  - NER
  - longformer
  - token-classification
  - label-studio
  - english
  - fine-tuned
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->


# Legal-BERT Base Entity Classifier

## Overview
A fine-tuned Longformer-based model for classifying legal entities (such as locations and dates) within the context of legal decision texts.
The model is based on `allenai/longformer` and is trained to predict the type of a marked entity span, given its context, using special entity markers `[E] ... [/E]`.

## Model Details
- **Model Name:** longformer-classifier-refinement-abb
- **Architecture:** Longformer (allenai/longformer)
- **Task:** Entity Classification (NER-style, entity-in-context classification)
- **Framework:** PyTorch, Hugging Face Transformers
- **Author:** S. Vercoutere

## Intended Use
- **Purpose:** Automatic classification of legal entities (e.g., location, date) in municipal or governmental decision documents.
- **Not Intended For:** General-purpose NER, non-legal domains, or tasks outside entity classification.

## Training Data
- **Source:** Annotated legal decision texts from Ghent/Freiburg/Bamberg.
- **Entity Types:**
  - Locations: `impact_location`, `context_location`
  - Dates: `publication_date`, `session_date`, `entry_date`, `expiry_date`, `legal_date`, `context_date`, `validity_period`, `context_period`
- **Preprocessing:**
  - XML-like tags in text, with entities wrapped in `<entity_type>...</entity_type>`.
  - For training, one entity per sample is marked with `[E] ... [/E]` in context.
  - Dataset balanced to max 5000 samples per label.

## Training Procedure
- **Model:** `nlpaueb/legal-bert-base-uncased`
- **Tokenization:** Hugging Face AutoTokenizer, with `[E]` and `[/E]` as additional special tokens.
- **Max Sequence Length:** 2048 (trained)
- **Batch Size:** 4
- **Optimizer:** AdamW
- **Learning Rate:** 2e-5
- **Epochs:** 10
- **Mixed Precision:** Yes (AMP)
- **Validation Split:** 20%
- **Evaluation Metrics:** Accuracy, F1, confusion matrix

## Evaluation

**Validation Accuracy:** 0.8454 (on held-out validation set)

**Detailed Entity-Level Evaluation:**

| Entity Label     | Precision | Recall | F1-score   | Support |
| ---------------- | --------- | ------ | ---------- | ------- |
| context_date     | 0.9272    | 0.9405 | 0.9338     | 975     |
| context_location | 0.9671    | 0.9751 | 0.9711     | 843     |
| context_period   | 0.9744    | 0.8321 | 0.8976     | 137     |
| entry_date       | 0.9528    | 0.9587 | 0.9557     | 484     |
| expiry_date      | 0.8980    | 0.9496 | 0.9231     | 139     |
| impact_location  | 0.9501    | 0.9559 | 0.9530     | 997     |
| legal_date       | 1.0000    | 0.9926 | 0.9963     | 943     |
| publication_date | 0.9501    | 0.9870 | 0.9682     | 386     |
| session_date     | 0.9597    | 0.9597 | 0.9597     | 347     |
| validity_period  | 0.9932    | 0.9379 | 0.9648     | 467     |
| **accuracy**     |           |        | **0.9601** | 5718    |
| **macro avg**    | 0.9572    | 0.9489 | 0.9523     | 5718    |
| **weighted avg** | 0.9606    | 0.9601 | 0.9601     | 5718    |


## Usage Example
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("svercoutere/longformer-classifier-refinement-abb")
model = AutoModelForSequenceClassification.from_pretrained("svercoutere/longformer-classifier-refinement-abb")

def classify_entity(entity_text, context_text):
    marked_text = context_text.replace(entity_text, f"[E] {entity_text} [/E]", 1)
    inputs = tokenizer(marked_text, return_tensors="pt", truncation=True, max_length=2048, padding="max_length")
    with torch.no_grad():
        outputs = model(**inputs)
    pred = torch.argmax(outputs.logits, dim=-1).item()
    return pred  # Map to label using label_encoder.classes_
```

## Limitations & Bias
- The model is trained on legal texts from specific municipalities and may not generalize to other domains or languages.
- Only entity types present in the training data are supported.
- The model expects entities to be marked with `[E] ... [/E]` in the input.

## Citation
If you use this model, please cite:

```
@misc{longformer-classifier-refinement-abb,
  author = {S. Vercoutere},
  title = {Longformer Entity Refinement},
  year = {2026},
  howpublished = {\url{https://huggingface.co/svercoutere/longformer-classifier-refinement-abb}}
}
```