---
title: README
emoji: 🏃
colorFrom: purple
colorTo: green
sdk: static
pinned: false
---

# UPC-HLE Cross-Encoder Models

Welcome to the UPC-HLE organization on HuggingFace!  
This repository hosts the cross-encoder models developed for our submission to [SemEval-2025 Task 7: Multilingual and Cross-lingual Fact-Checked Claim Retrieval](https://disai.eu/semeval-2025/). Our approach combines dense Text Embedding Models (TEMs) with a neural Cross-Encoder re-ranking stage to retrieve fact-checked claims for social-media posts in a multilingual setting. The dataset is a subset of [MultiClaim Dataset](https://zenodo.org/records/7737983).

---

## 🔍 Organization Overview

The SemEval-2025 Task 7 challenge requires retrieving previously fact-checked claims relevant to user queries (social-media posts) across multiple languages. Our solution uses a two-step pipeline:

1. **Initial Retrieval**  
   Encode the query (post) and all fact-check documents with a bi-encoder TEM to get dense embeddings. Retrieve the top 100 candidates by cosine similarity.

2. **Cross-Encoder Re-Ranking**  
   Fine-tune a transformer-based Cross-Encoder (Jina Reranker v2) on triplets of (post, candidate, label) using hard-negative sampling. Re-score the 100 candidates to produce the final ranked list.

This yields strong improvements over lexical baselines and standalone embeddings, especially in cross-lingual retrieval.

---

## 📦 Models in This Organization

These are the best performing models in the final solution for the shared task. The bi-encoder retrieval is performed using [Snowflake/snowflake-arctic-embed-l-v2.0](https://huggingface.co/Snowflake/snowflake-arctic-embed-l-v2.0)
 and, afterwards, the re-ranking is done using the models available in this repository, which use Jina V2 as base Cross Encoder.

 **Languages Available**

- **English** (eng)  
- **French** (fra)  
- **German** (deu)  
- **Portuguese** (por)  
- **Thai** (tha)  
- **Malay** (msa)  
- **Arabic** (ara)  
- **Polish** (pol)  
- **Turkish** (tur)
- **Crosslingual**

Thanks to its XLM-RoBERTa backbone, the model can encode sentences in any of these languages into the same 768-dimensional vector space. You can therefore use the **same** model instance to compute embeddings and cosine-similarity scores for queries and fact-check candidates **across** all test‐set languages, without loading separate language-specific checkpoints.

## 🚀 Usage

Install the 🤗 Sentence Transformers library and then load any model as follows:

```python
from sentence_transformers import CrossEncoder
rerank_model = CrossEncoder(reranker_model_name, num_labels=1, max_length=1024, trust_remote_code=True)
ranked_vals = rerank_model.rank(str_candidate, ls_factchecks, show_progress_bar=False)

```

---

## 📈 Performance

All scores are reported as success@10 (the fraction of queries where at least one correct fact-check appears in the top 10).


Model	Monolingual	Cross-lingual
mTEM-CE-Eng-SF	0.91	0.78
mTEM-CE-E5	0.91	0.75
mTEM-CE-Eng-E5	0.90	0.76
mTEM-LF-CE-Eng-E5	0.89	0.75
These results demonstrate the benefit of CE re-ranking over the baseline TEM alone ​

---

## 🛠 Training Details
Triplet Mining:
– Retrieve 100 candidates per post by cosine similarity.
– Label the gold fact-check as positive; sample up to 4 hard negatives where their TEM score ≥ 0.95 × positive score.

Hyperparameters (for mTEM-CE-Eng-SF):
– Batch size: 32
– Epochs: 5
– Learning rate: 2 × 10⁻⁵
– Warmup steps: 10% of total
– Candidates for re-ranking: 100

## 📚 Citation
If you use these models or our results, please cite:

@inproceedings{semeval2025task7upchle,
    title={UPC-HLE at SemEval-2025 Task 7: Multilingual Fact-Checked Claim Retrieval with Text Embedding Models and Cross-Encoder Re-Ranking},
    author={Alberto Becerra-Tome, Agustín Conesa},
    booktitle = {Proceedings of the 19th International Workshop on Semantic Evaluation},
    series = {SemEval 2025},
    year = {2025},
    address = {Vienna, Austria},
    month = {July},
    pages = {}
    doi= {}
}

## 📄 License
All models and code are released under the MIT License. See LICENSE for details.

Happy fact-checking!