README.md

---
language:
- multilingual
license: apache-2.0
library_name: pytorch
datasets:
- Prosho/pear-data
base_model: facebook/xlm-roberta-xl
tags:
- machine-translation
- mt-evaluation
- quality-estimation
- reference-free
- pairwise-ranking
- mbr
- pear
- pear-xl
---

<div align="center">

<h1 style="font-family: 'Arial', sans-serif; font-size: 28px; font-weight: bold;">
  🍐 PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation
</h1>

[![ACL 2026](https://img.shields.io/badge/ACL-2026-2f6fdd)](https://2026.aclweb.org/)
[![ACL Anthology](https://img.shields.io/badge/ACL%20Anthology-paper-b31b1b)](https://aclanthology.org/2026.acl-long.1953/)
[![arXiv](https://img.shields.io/badge/arXiv-2601.18006-b31b1b)](https://arxiv.org/abs/2601.18006)
[![PyPI](https://img.shields.io/badge/PyPI-v1.1.1-blue)](https://pypi.org/project/pear-mt/)
[![GitHub](https://img.shields.io/badge/GitHub-pear-181717?logo=github)](https://github.com/prosho-97/pear)
[![Collection](https://img.shields.io/badge/%F0%9F%A4%97%20HF-PEAR%20collection-ffcc4d)](https://huggingface.co/collections/Prosho/pear)
[![License](https://img.shields.io/badge/License-Apache%202.0-green)](https://www.apache.org/licenses/LICENSE-2.0)

</div>

## Model variant

This repository contains the PEAR-XL checkpoint trained without Knowledge Distillation (KD).

## Installation

PEAR supports Python 3.11, 3.12, and 3.13. Install the published inference toolkit from PyPI:

```bash
python -m pip install pear-mt
```

The distribution is named `pear-mt`; the Python import package and primary CLI command are both named `pear`.

To install from source instead:

```bash
git clone https://github.com/prosho-97/pear.git
cd pear
python -m pip install .
```

## Quick start: pairwise QE scoring

```python
import pear

metric = pear.load_metric("pear-xl")  # resolves to this Hugging Face model

scores = pear.score_pairwise(
    metric,
    sources=["The cat is on the mat."],
    translations_a=["El gato está en la alfombra."],
    translations_b=["El gato está en el mapa."],
    batch_size=8,
    gpus="auto",
    progress_bar=True,
)
```

Positive scores prefer `translations_a`; negative scores prefer `translations_b`.

To score both candidate orders:

```python
scores = pear.score_pairwise(
    metric,
    sources=["The cat is on the mat."],
    translations_a=["El gato está en la alfombra."],
    translations_b=["El gato está en el mapa."],
    mode="both",
)
# {"forward": [...], "reverse": [...]}
```

## Reference-anchored PEAR-XL

PEAR-XL can also use a human reference, or any other anchor translation, as the
second candidate:

```python
scores = pear.score_reference_anchored(
    metric,
    sources=["The cat is on the mat."],
    translations=["El gato está en la alfombra."],
    references=["El gato está sobre la alfombra."],
    batch_size=8,
)
```

As in pairwise QE scoring, `mode="both"` is available for both-order reference-anchored inference.

## PEAR-XL for MBR decoding

```python
from pear.mbr import pear_utility_matrix, select_mbr_hypothesis

metric = pear.load_metric("pear-xl")
source = "Questa è una traduzione molto buona."
hypotheses = [
    "This is a good translation.",
    "This is a very good translation.",
    "This is a bad translation.",
]

utility = pear_utility_matrix(
    metric,
    source,
    hypotheses,
    mode="half",
    batch_size=8,
)
index, expected_utility = select_mbr_hypothesis(utility)
print(hypotheses[index], expected_utility)
```

Use `mode="full"` for all off-diagonal ordered pairs, or `mode="half"` to score only one triangular half and fill the opposite direction by PEAR antisymmetry.

## CLI examples

Pairwise TSV input must contain `src`, `mt_0`, and `mt_1` columns:

```bash
pear score --model pear-xl --input pairs.tsv --output scored.tsv --batch-size 8
pear score --hf-model Prosho/pear-xl --input pairs.tsv --output scored.tsv --batch-size 8
```

Reference-anchored TSV input must contain `src`, `mt`, and `ref` columns:

```bash
pear score --model pear-xl --mode reference --input refs.tsv --output scored.tsv --batch-size 8
```

MBR input JSONL rows must contain `src` and `hypotheses`:

```bash
pear mbr --model pear-xl --input nbest.jsonl --output selected.jsonl --utility half --batch-size 8
```

Use `--gpus 0` to force CPU inference. The default, `--gpus auto`, selects one available CUDA or MPS accelerator.

## Citation

If you use this model, please cite the PEAR paper:

```bibtex
@inproceedings{proietti-etal-2026-pear,
    title = "{PEAR}: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation",
    author = "Proietti, Lorenzo  and
      Grundkiewicz, Roman  and
      Post, Matt",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.1953/",
    doi = "10.18653/v1/2026.acl-long.1953",
    pages = "42189--42207",
    ISBN = "979-8-89176-390-6"
}
```

## Links

- Package: <https://pypi.org/project/pear-mt/>
- Paper: <https://aclanthology.org/2026.acl-long.1953/>
- Code: <https://github.com/prosho-97/pear>
- Dataset: <https://huggingface.co/datasets/Prosho/pear-data>
- Collection: <https://huggingface.co/collections/Prosho/pear>
- PEAR: <https://huggingface.co/Prosho/pear>