svercoutere/longformer-ner-refinement-abb

Longformer NER ABB

Overview

This project provides a fine-tuned Longformer-based model for Named Entity Recognition (NER) in legal decision texts. The model is based on allenai/longformer-base-4096 and is trained to identify and classify entity spans (such as locations and dates) in legal documents.

Model Details

• Model Name: svercoutere/longformer-ner-refinement-abb
• Architecture: Longformer (allenai/longformer-base-4096)
• Task: Named Entity Recognition (NER)
• Framework: PyTorch, Hugging Face Transformers
• Author: S. Vercoutere

Intended Use

• Purpose: Automatic extraction and classification of legal entities (e.g., location, date) in municipal or governmental decision documents.
• Not Intended For: General-purpose NER, non-legal domains, or tasks outside entity extraction/classification.

Training Data

• Source: Annotated legal decision texts from Label Studio projects.
• Entity Types:
  • Locations: impact_location, context_location
  • Dates: publication_date, session_date, entry_date, expiry_date, legal_date, context_date, validity_period, context_period
• Preprocessing:
  • BIO tagging scheme for entity spans.
  • Dataset balanced to max N samples per label (see notebook for details).

Training Procedure

• Model: allenai/longformer-base-4096
• Tokenization: Hugging Face AutoTokenizer
• Max Sequence Length: 4096
• Batch Size: 4
• Optimizer: AdamW
• Learning Rate: 2e-5
• Epochs: 10
• Mixed Precision: Yes (AMP)
• Validation Split: 20%
• Evaluation Metrics: Precision, Recall, F1-score, Support (per entity type)

Evaluation

Validation F1-score: (see notebook output for actual value)

Detailed Entity-Level Evaluation:

Entity Label	Precision	Recall	F1-score	Support
CONTEXTUAL DATE	0.6723	0.7862	0.7248	407
CONTEXTUAL LOCATION	0.6240	0.5000	0.5552	156
CONTEXTUAL PERIOD	0.2143	0.4286	0.2857	7
ENTRY DATE	0.8462	0.8128	0.8291	203
EXPIRATION DATE	0.6000	0.6818	0.6383	22
LEGAL BASIS DATE	0.8854	0.9205	0.9026	151
PERIOD OF EFFECT	0.6222	0.7000	0.6588	120
PRIMARY LOCATION	0.7425	0.8253	0.7817	2358
PUBLICATION DATE	0.7500	0.8242	0.7853	91
SESSION DATE	0.6824	0.8788	0.7682	66
micro avg	0.7330	0.8051	0.7674	3581
macro avg	0.6639	0.7358	0.6930	3581
weighted avg	0.7344	0.8051	0.7670	3581

Replace the above with your actual NER results table from the notebook.

Usage Example

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("svercoutere/svercoutere/longformer-ner-refinement-abb")
model = AutoModelForTokenClassification.from_pretrained("svercoutere/svercoutere/longformer-ner-refinement-abb")

def predict_entities(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512, padding="max_length")
    with torch.no_grad():
        outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=-1)
    # Map predictions to entity labels using model.config.id2label
    return predictions

Limitations & Bias

• The model is trained on legal texts from specific municipalities and may not generalize to other domains or languages.
• Only entity types present in the training data are supported.

Citation

If you use this model, please cite:

@misc{svercoutere/longformer-ner-refinement-abb,
  author = {S. Vercoutere},
  title = {Longformer NER ABB},
  year = {2026},
  howpublished = {\url{https://huggingface.co/svercoutere/svercoutere/longformer-ner-refinement-abb}}
}

License

MIT