XLM-RoBERTa Arabic Quality Classifier

A text quality classifier for Arabic pretraining data, trained from XLM-RoBERTa. This model reproduces the FineWeb2-HQ approach (Messmer et al., 2025) for Arabic, as the original authors did not release their trained classifiers but did release their code.

For improved Arabic performance and inference speed, see mmBERT-Arabic-Quality-Classifier.

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="AdaMLLab/XLM-RoBERTa-Arabic-Quality-Classifier")
result = classifier("النص العربي هنا")

Citation

@misc{messmer2025fineweb2hq,
      title={Enhancing Multilingual LLM Pretraining with Model-Based Data Selection}, 
      author={Bettina Messmer and Vinko Sabolčec and Martin Jaggi},
      year={2025},
      eprint={2502.10361},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.10361}, 
}

@misc{alrashed2025mixminmatch,
      title={Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets}, 
      author={Sultan Alrashed and Francesco Orabona},
      year={2025},
      eprint={2512.18834v2},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.18834v2}, 
}