File size: 4,601 Bytes
2f0f7fd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ---
library_name: transformers
tags:
- transformers
- pytorch
- bert
- legal-domain
- entity-classification
- sequence-classification
- NER
- longformer
- token-classification
- label-studio
- english
- fine-tuned
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
# Legal-BERT Base Entity Classifier
## Overview
A fine-tuned Longformer-based model for classifying legal entities (such as locations and dates) within the context of legal decision texts.
The model is based on `allenai/longformer` and is trained to predict the type of a marked entity span, given its context, using special entity markers `[E] ... [/E]`.
## Model Details
- **Model Name:** longformer-classifier-refinement-abb
- **Architecture:** Longformer (allenai/longformer)
- **Task:** Entity Classification (NER-style, entity-in-context classification)
- **Framework:** PyTorch, Hugging Face Transformers
- **Author:** S. Vercoutere
## Intended Use
- **Purpose:** Automatic classification of legal entities (e.g., location, date) in municipal or governmental decision documents.
- **Not Intended For:** General-purpose NER, non-legal domains, or tasks outside entity classification.
## Training Data
- **Source:** Annotated legal decision texts from Ghent/Freiburg/Bamberg.
- **Entity Types:**
- Locations: `impact_location`, `context_location`
- Dates: `publication_date`, `session_date`, `entry_date`, `expiry_date`, `legal_date`, `context_date`, `validity_period`, `context_period`
- **Preprocessing:**
- XML-like tags in text, with entities wrapped in `<entity_type>...</entity_type>`.
- For training, one entity per sample is marked with `[E] ... [/E]` in context.
- Dataset balanced to max 5000 samples per label.
## Training Procedure
- **Model:** `nlpaueb/legal-bert-base-uncased`
- **Tokenization:** Hugging Face AutoTokenizer, with `[E]` and `[/E]` as additional special tokens.
- **Max Sequence Length:** 2048 (trained)
- **Batch Size:** 4
- **Optimizer:** AdamW
- **Learning Rate:** 2e-5
- **Epochs:** 10
- **Mixed Precision:** Yes (AMP)
- **Validation Split:** 20%
- **Evaluation Metrics:** Accuracy, F1, confusion matrix
## Evaluation
**Validation Accuracy:** 0.8454 (on held-out validation set)
**Detailed Entity-Level Evaluation:**
| Entity Label | Precision | Recall | F1-score | Support |
| ---------------- | --------- | ------ | ---------- | ------- |
| context_date | 0.9272 | 0.9405 | 0.9338 | 975 |
| context_location | 0.9671 | 0.9751 | 0.9711 | 843 |
| context_period | 0.9744 | 0.8321 | 0.8976 | 137 |
| entry_date | 0.9528 | 0.9587 | 0.9557 | 484 |
| expiry_date | 0.8980 | 0.9496 | 0.9231 | 139 |
| impact_location | 0.9501 | 0.9559 | 0.9530 | 997 |
| legal_date | 1.0000 | 0.9926 | 0.9963 | 943 |
| publication_date | 0.9501 | 0.9870 | 0.9682 | 386 |
| session_date | 0.9597 | 0.9597 | 0.9597 | 347 |
| validity_period | 0.9932 | 0.9379 | 0.9648 | 467 |
| **accuracy** | | | **0.9601** | 5718 |
| **macro avg** | 0.9572 | 0.9489 | 0.9523 | 5718 |
| **weighted avg** | 0.9606 | 0.9601 | 0.9601 | 5718 |
## Usage Example
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("svercoutere/longformer-classifier-refinement-abb")
model = AutoModelForSequenceClassification.from_pretrained("svercoutere/longformer-classifier-refinement-abb")
def classify_entity(entity_text, context_text):
marked_text = context_text.replace(entity_text, f"[E] {entity_text} [/E]", 1)
inputs = tokenizer(marked_text, return_tensors="pt", truncation=True, max_length=2048, padding="max_length")
with torch.no_grad():
outputs = model(**inputs)
pred = torch.argmax(outputs.logits, dim=-1).item()
return pred # Map to label using label_encoder.classes_
```
## Limitations & Bias
- The model is trained on legal texts from specific municipalities and may not generalize to other domains or languages.
- Only entity types present in the training data are supported.
- The model expects entities to be marked with `[E] ... [/E]` in the input.
## Citation
If you use this model, please cite:
```
@misc{longformer-classifier-refinement-abb,
author = {S. Vercoutere},
title = {Longformer Entity Refinement},
year = {2026},
howpublished = {\url{https://huggingface.co/svercoutere/longformer-classifier-refinement-abb}}
}
```
|