🍐 PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation

Installation

Install the PEAR inference toolkit from the GitHub repository:

git clone https://github.com/prosho-97/pear.git
cd pear
pip install .

Quick start: pairwise QE scoring

import pear

metric = pear.load_metric("pear")  # resolves to this Hugging Face model

scores = pear.score_pairwise(
    metric,
    sources=["The cat is on the mat."],
    translations_a=["El gato está en la alfombra."],
    translations_b=["El gato está en el mapa."],
    batch_size=16,
    gpus=1,
    progress_bar=True,
)

Positive scores prefer translations_a; negative scores prefer translations_b.

To score both candidate orders:

scores = pear.score_pairwise(
    metric,
    sources=["The cat is on the mat."],
    translations_a=["El gato está en la alfombra."],
    translations_b=["El gato está en el mapa."],
    mode="both",
)
# {"forward": [...], "reverse": [...]}

Reference-anchored PEAR

PEAR can also be used with a human reference, or any other anchor translation, as the second candidate:

scores = pear.score_reference_anchored(
    metric,
    sources=["The cat is on the mat."],
    translations=["El gato está en la alfombra."],
    references=["El gato está sobre la alfombra."],
    batch_size=16,
)

As in pairwise QE scoring, mode="both" is available for both-order reference-anchored inference.

PEAR for MBR decoding

from pear.mbr import pear_utility_matrix, select_mbr_hypothesis

metric = pear.load_metric("pear")
source = "Questa è una traduzione molto buona."
hypotheses = [
    "This is a good translation.",
    "This is a very good translation.",
    "This is a bad translation.",
]

utility = pear_utility_matrix(metric, source, hypotheses, mode="half", batch_size=16)
index, expected_utility = select_mbr_hypothesis(utility)
print(hypotheses[index], expected_utility)

Use mode="full" for all off-diagonal ordered pairs, or mode="half" to score only one triangular half and fill the opposite direction by PEAR antisymmetry.

CLI examples

Pairwise TSV input must contain src, mt_0, and mt_1 columns:

pear score --model pear --input pairs.tsv --output scored.tsv --batch-size 16
pear score --hf-model Prosho/pear --input pairs.tsv --output scored.tsv --batch-size 16

Reference-anchored TSV input must contain src, mt, and ref columns:

pear score --model pear --mode reference --input refs.tsv --output scored.tsv --batch-size 16

MBR input JSONL rows must contain src and hypotheses:

pear mbr --model pear --input nbest.jsonl --output selected.jsonl --utility half --batch-size 16

Citation

If you use this model, please cite the PEAR paper:

@inproceedings{proietti-etal-2026-pear,
    title = "{PEAR}: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation",
    author = "Proietti, Lorenzo  and
      Grundkiewicz, Roman  and
      Post, Matt",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.1953/",
    doi = "10.18653/v1/2026.acl-long.1953",
    pages = "42189--42207",
    ISBN = "979-8-89176-390-6"
}

Links

• Paper: https://aclanthology.org/2026.acl-long.1953/
• Code: https://github.com/prosho-97/pear
• Collection: https://huggingface.co/collections/Prosho/pear
• PEAR-XL: https://huggingface.co/Prosho/pear-xl