A Study of Temporal Fusion Strategies for Named Entity Recognition in Historical Texts
Abstract
Temporal metadata embedding through late fusion strategies improves named entity recognition performance in historical texts by enhancing model generalization across different time periods.
Temporal variation poses a unique challenge for named entity recognition (NER) in historical texts, where entities drift in surface form and salience across time. While language models (LMs) have made progress in various NLP tasks, their ability to reason about temporality, especially in diachronic contexts, remains limited or at least, questionable. In this paper, we systematically study how temporal metadata can be structurally embedded into NER models using a range of lightweight fusion strategies. We experiment with both absolute and relative temporal representations, injected into Transformer-based architectures via early or late fusion mechanisms such as cross-attention, adapters, and concatenation. Our evaluations on French and German historical datasets reveal that late fusion strategies yield more robust and temporally generalisable performance, particularly in early and noisy periods.
Community
Very cool idea @emanuelaboros !
Another interesting approach was to integrate temporal and other Metadata information into the encoder directly:
https://arxiv.org/abs/2211.10086
From @Kaspar and @davanstrien
Hello @stefan-it , thanks for pointing this out, I actually saw this paper some days ago (I know, 2022.., somehow slipped off my radar while doing these experiments).
The idea is actually very close in spirit, but on my side I tried using temporal metadata not just as extra text concatenated to the input, but as structured information injected into the model. It would definitely have been interesting to compare directly with their models. It's in my plan to do so.
Get this paper in your agent:
hf papers read 2606.27881 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper