Awakened: XLM-R FOL — Person–Place Relation Extraction

CLEF 2026 · HIPE Track · Task 1 · Team Awakened

Dragoș-Mitruț Vasile · Elena-Simona Apostol · Ciprian-Octavian Truică

Model Description

A fine-tuned XLM-RoBERTa-large for person–place relation extraction from historical documents, submitted as Run 2 (the efficiency-oriented system) of Team Awakened at HIPE-2026.

For each (person, place) pair the model predicts two labels:

at — is the person connected to the place? (TRUE / PROBABLE / FALSE)
isAt — does that connection hold around the document's date? (TRUE / FALSE)

The [CLS] representation of XLM-R is concatenated with 16 Wikidata knowledge-graph features (presence, temporal, and relation flags) and 6 text-pattern features (locative action, role, origin, time, departure, candidacy), then passed to two classification heads. Training uses a focal loss on the at head and an auxiliary logic-constrained loss that penalises inconsistent label pairs (e.g. at=FALSE with isAt=TRUE). A single multilingual model is trained jointly on German, English, and French.

Models

Property	Value
Base encoder	FacebookAI/xlm-roberta-large
Parameters	560,965,127
Languages	de, en, fr (single multilingual model)
KG features	16
Text features	6
Focal γ	1.0
λ_hard / λ_soft	0.05 / 0.02
FR Test A global	0.6076

Usage

This model has a custom architecture (encoder + feature fusion + dual heads), so it loads with trust_remote_code=True. Besides the text, it expects a 16-dim Wikidata KG feature vector and a 6-dim text-pattern feature vector per (person, place) pair. These are built from the cached Wikidata facts and the local context using the code in the GitHub repository (hipe_dataset_fol.py).

Loading the model

import torch
from transformers import AutoModel, AutoTokenizer

device = (
    torch.device("mps") if torch.backends.mps.is_available()
    else torch.device("cuda") if torch.cuda.is_available()
    else torch.device("cpu")
)

model = AutoModel.from_pretrained("DS4AI-UPB/<repo>", trust_remote_code=True).to(device).eval()
tokenizer = AutoTokenizer.from_pretrained("FacebookAI/xlm-roberta-large")

Running inference with real KG features

The KG and text-pattern features must match the ones the model was trained on, so reuse the repository's feature builders rather than reimplementing them:

from hipe_dataset_fol import build_kg_features, build_text_pattern_features, load_kg_caches

# load the cached Wikidata facts (shipped in the repo)
kg_pairs, kg_persons, kg_locations = load_kg_caches("kg_facts.jsonl")

# doc: {"text": ..., "language": "en"};  pair: QIDs, mentions, article_date
kg  = build_kg_features(pair["pers_qid"], pair["loc_qid"], pair["article_date"],
                        kg_pairs=kg_pairs, kg_persons=kg_persons, kg_locations=kg_locations)
fol = build_text_pattern_features(doc, pair)

enc = tokenizer(doc["text"], return_tensors="pt", truncation=True, max_length=320).to(device)
with torch.no_grad():
    out = model(
        input_ids=enc["input_ids"],
        attention_mask=enc["attention_mask"],
        kg_features=torch.tensor([kg], dtype=torch.float, device=device),
        fol_features=torch.tensor([fol], dtype=torch.float, device=device),
    )

at = out.at_logits.argmax(-1).item()      # 0=FALSE, 1=PROBABLE, 2=TRUE
isAt = out.isAt_logits.argmax(-1).item()  # 0=FALSE, 1=TRUE
if at == 0:        # hard rule: at=FALSE => isAt=FALSE
    isAt = 0

Requirements

pip install -U torch transformers safetensors

Requires transformers >= 4.40, torch >= 2.1. Inference runs on a single GPU (~16 GB VRAM is sufficient); CPU works for small batches.

Training Data

The HIPE-2026 sandbox split (German, English, French), provided by the organizers. Wikidata facts are cached from the public SPARQL endpoint and reused across runs; the cache is available in the code repository.

Intended Use

Intended: person–place relation extraction on historical newspaper text in German, English, and French, as the efficiency-oriented counterpart to the prompted Claude Sonnet 4 systems (Runs 1 and 3).
Out of scope: literary/narrative text (the model was trained on newspapers only); languages other than de/en/fr; production deployment without validation.

Limitations

Trained on newspapers; not applied to the literary Test B surprise set.
Requires external Wikidata KG features and text-pattern features at inference — not a standalone text-only classifier.
isAt (temporal grounding) is the weaker component; PROBABLE vs FALSE remains the hardest at distinction.

Citation

@inproceedings{Vasile2026Awakened,
    author    = {Vasile, Dragoș-Mitruț and Apostol, Elena-Simona and Truică, Ciprian-Octavian},
    title     = {Team Awakened at HIPE-2026: Knowledge-Augmented Prompting and Logic-Constrained Fine-Tuning for Person--Place Relation Extraction},
    booktitle = {Working Notes of CLEF 2026},
    month     = {September},
    year      = {2026}
}