README.md

---
license: mit
language:
- fr
tags:
- summarization
- abstractive-summarization
- barthez
- bge-m3
- named-entity-injection
library_name: pytorch
pipeline_tag: summarization
---

# SBARThez — Pre-trained checkpoints

This repository hosts the **trained model checkpoints** for **SBARThez**, the French
abstractive summarization model introduced in the paper
**"Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization"**
(LREC 2026).

SBARThez replaces the token embedding layer of [BARThez](https://huggingface.co/moussaKam/barthez)
with **sentence-level embeddings** (by default [BGE-M3](https://huggingface.co/BAAI/bge-m3)),
and adds an optional **Named Entity Injection (NEI)** module that prepends named-entity
tokens to the decoder input to improve the factual consistency of the generated summaries.

Model weights are hosted here on the Hugging Face Hub. **The full training and evaluation code, along with usage
instructions, lives in the GitHub repository.**

## Links

- 📄 **Paper:** *Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization*, LREC 2026 — [Link](https://hal.science/hal-05665423/)
- 💻 **Code (training + evaluation):** https://github.com/cchellaf/SBARThez

## Available checkpoints

| File | Training data | NEI module | Description |
|---|---|---|---|
| `sbarthez_nei_mlsum1.pth` | MLSUM (French) | ✅ | Trained on MLSUM with the NEI module. This is the **first-stage** model in the paper, intended as an initialization for further fine-tuning on other datasets. |
| `sbarthez_nei_orange1.pth` | OrangeSum | ✅ | Trained on OrangeSum with the NEI module. Training was **continued from `sbarthez_nei_mlsum1.pth`**. |

The MLSUM checkpoint serves as the first training stage and can be used to initialize
training on any other summarization dataset. The OrangeSum checkpoint was produced exactly
this way — by continuing training from the MLSUM model.

## Usage

See the [GitHub repository](https://github.com/cchellaf/SBARThez) for full instructions on
preprocessing, training, and inference. To download a checkpoint:

```python
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="cchellaf/sbarthez_nei",
    filename="sbarthez_nei_mlsum1.pth",
    local_dir="checkpoints",
)
```

## Citation

```bibtex
@inproceedings{el2026using,
  title={Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization},
  author={El Hammoud, Chaimae Chellaf and Mdhaffar, Salima and Est{\`e}ve, Yannick and Huet, St{\'e}phane},
  booktitle={The Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
  pages={9873--9883},
  year={2026}
}
```

## Contact

**Chaimae Chellaf El Hammoud** — chaimae.chellaf-el-hammoud@alumni.univ-avignon.fr