---
license: apache-2.0
base_model: sentence-transformers/all-MiniLM-L6-v2
language:
  - en
library_name: pytorch
pipeline_tag: sentence-similarity
tags:
  - temporal-retrieval
  - information-retrieval
  - retrieval-augmented-generation
  - gaussian-embeddings
---

# TEMPS: temporal module

**TEMPS** (Temporal Embedding Model for Precise Search) adds a temporal score to any dense
retriever. This repository holds the trained temporal module from
*TEMPS: Temporal Sentence Embeddings for Temporal Information Retrieval*
(Hassani, Romero, Bouzeghoub, Jacquelinet, EMNLP 2026).

- **Code:** https://github.com/aldebaran-care/TEMPS
- **Weights:** `temps-weights.pth` (PyTorch `state_dict`, fp32)

## Model description

Dense retrievers match queries and passages on topic but not on time, so they return passages
that are on-topic yet temporally wrong. TEMPS attaches to a **frozen** semantic retriever and
scores how well two *anchored* texts (text + anchor date) align in time.

For a text and its anchor date, the module outputs a diagonal Gaussian over time. Its layers are:

- a frozen `all-MiniLM-L6-v2` backbone (up to 512 tokens)
- a sinusoidal anchor-date encoder
- attention pooling
- mean and variance heads

Two texts are compared with the KL-based score

```
ts(1, 2) = 1 / (1 + KL(p_1 || p_2))
```

which is fused with the semantic retriever's cosine score after per-query min-max
normalization:

```
s(1, 2) = (1 - alpha) * cos(E_1, E_2) + alpha * ts(1, 2)
```

`alpha` is set per benchmark on a held-out validation split: 0.6 for TimeQA, 0.8 for
TempReason and 0.3 for TS-Retriever.

## Training

No hand-labeled temporal data is used. Each anchored temporal expression is resolved to an
inclusive day interval `[a, b]` and moment-matched to a Gaussian
(`mu = (a + b + 1) / 2`, `sigma^2 = (b - a + 1)^2 / 12`). The pair label is
`1 / (1 + KL(p_I1 || p_I2))`, and the module is trained with a CoSENT ranking loss to reproduce
that ordering.

- **Data:** 60M automatically generated tuples (55M synthetic, 5M from Wikipedia text in the
  Multilingual MLM Temporal Tagging Resources)
- **Setup:** 3 epochs, global batch size 2,048, lr 3e-4, weight decay 0.01, 5% linear warmup,
  max 512 tokens, fp16, seed 42; about 4 hours on 8×A100-80GB

## Usage

The model class lives in the code repository. Clone it and run from its root:

```bash
git clone https://github.com/aldebaran-care/TEMPS && cd TEMPS
uv sync
```

```python
import torch
from torch.distributions import Normal, kl_divergence
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

from temporal_embeddings.model.gauss_model import GaussModel
from temporal_embeddings.utils.positional_encoding import positional_encoding

backbone = "sentence-transformers/all-MiniLM-L6-v2"
model = GaussModel(backbone).eval()
model.load_state_dict(torch.load(hf_hub_download("aldebaran-research/temps", "temps-weights.pth"), map_location="cpu"))
tokenizer = AutoTokenizer.from_pretrained(backbone)

@torch.inference_mode()
def encode(texts, anchor_dates):
    batch = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
    return model(**batch, dates=positional_encoding(anchor_dates))

query = encode(["Who led the company last spring?"], ["2021-11-09"])
docs = encode(["In April 2021 she was appointed CEO.", "He ran the firm from 1998 to 2003."],
              ["2021-11-09", "2021-11-09"])

# Temporal score 1 / (1 + KL(query || doc)), KL averaged over dimensions; higher = better match
kl = kl_divergence(Normal(query.mu, query.std), Normal(docs.mu, docs.std)).mean(dim=-1)
print(1 / (1 + kl))  # tensor([0.7331, 0.7130])
```

To reproduce the paper's evaluation, `uv run python scripts/download_model.py` saves this
checkpoint where the evaluation configs expect it; see the repository README.

## Results

On the held-out test splits (MRR):

| Benchmark | Best system | MRR | Gain over backbone |
|---|---|---|---|
| TimeQA (alpha=0.6) | SFR-Embedding-Mistral + TEMPS | 0.5360 | +31.9% |
| TempReason (alpha=0.8) | SFR-Embedding-Mistral + SUTime control | 0.6901 | +11.5% |
| TS-Retriever (alpha=0.3) | SFR-Embedding-Mistral + TEMPS | 0.8240 | +9.3% |

On TS-Retriever, `E5-base-v2 + TEMPS` lifts R@1 from 19.92 to 25.39 and P@1 from 58.63 to
69.14 over TSContriever. On TempReason, the rule-based SUTime control outperforms TEMPS; see
the paper for the full tables.

## Limitations

- English only.
- Covers expressions that can be anchored and resolved to an interval: absolute dates,
  deictic expressions and offsets, present references, seasons, durations and intervals.
  Discourse-level reasoning and event–event relations are out of scope.
- A passage of up to 512 tokens is reduced to a single Gaussian, which loses information on
  long or temporally mixed passages.
- Passages with no temporal expression were held out of training, so their Gaussians are
  uncalibrated; the semantic score is what ranks them.
- Results come from a single training run (seed 42).

## License

Apache-2.0, inherited from
[`sentence-transformers/all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2).
The code repository is MIT.

## Citation

```bibtex
@inproceedings{hassani-etal-2026-temps,
  title     = {{TEMPS}: Temporal Sentence Embeddings for Temporal Information Retrieval},
  author    = {Hassani, Mourad and Romero, Julien and Bouzeghoub, Amel and Jacquelinet, Christian},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2026},
  publisher = {Association for Computational Linguistics}
}
```

## Acknowledgments

This project was provided with computing and storage resources by GENCI at IDRIS thanks to the
grant 2025-105442 on the supercomputer Jean Zay's A100 partition.