SBARThez — Pre-trained checkpoints
This repository hosts the trained model checkpoints for SBARThez, the French abstractive summarization model introduced in the paper "Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization" (LREC 2026).
SBARThez replaces the token embedding layer of BARThez with sentence-level embeddings (by default BGE-M3), and adds an optional Named Entity Injection (NEI) module that prepends named-entity tokens to the decoder input to improve the factual consistency of the generated summaries.
Model weights are hosted here on the Hugging Face Hub. The full training and evaluation code, along with usage instructions, lives in the GitHub repository.
Links
- 📄 Paper: Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization, LREC 2026 — Link
- 💻 Code (training + evaluation): https://github.com/cchellaf/SBARThez
Available checkpoints
| File | Training data | NEI module | Description |
|---|---|---|---|
sbarthez_nei_mlsum1.pth |
MLSUM (French) | ✅ | Trained on MLSUM with the NEI module. This is the first-stage model in the paper, intended as an initialization for further fine-tuning on other datasets. |
sbarthez_nei_orange1.pth |
OrangeSum | ✅ | Trained on OrangeSum with the NEI module. Training was continued from sbarthez_nei_mlsum1.pth. |
The MLSUM checkpoint serves as the first training stage and can be used to initialize training on any other summarization dataset. The OrangeSum checkpoint was produced exactly this way — by continuing training from the MLSUM model.
Usage
See the GitHub repository for full instructions on preprocessing, training, and inference. To download a checkpoint:
from huggingface_hub import hf_hub_download
path = hf_hub_download(
repo_id="cchellaf/sbarthez_nei",
filename="sbarthez_nei_mlsum1.pth",
local_dir="checkpoints",
)
Citation
@inproceedings{el2026using,
title={Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization},
author={El Hammoud, Chaimae Chellaf and Mdhaffar, Salima and Est{\`e}ve, Yannick and Huet, St{\'e}phane},
booktitle={The Fifteenth Language Resources and Evaluation Conference (LREC 2026)},
pages={9873--9883},
year={2026}
}
Contact
Chaimae Chellaf El Hammoud — chaimae.chellaf-el-hammoud@alumni.univ-avignon.fr