README.md

---
library_name: eole
tags:
- eole
- comet
- machine-translation-evaluation
- quality-estimation
- converted
base_model: Unbabel/wmt22-comet-da
license: apache-2.0
---

# WMT22 COMET DA (EOLE)

This is [Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da) converted to [EOLE](https://github.com/eole-nlp/eole) format.

No model weights were modified. This repository contains a format conversion for use with EOLE's native `transformer_encoder_scorer` implementation.

This is not an upstream `unbabel-comet` checkpoint layout. Use it with EOLE.

## Model Details

| | |
|---|---|
| Original model | [Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da) |
| EOLE architecture | `transformer_encoder_scorer` |
| Scoring type | `comet` |
| Class identifier | `regression_metric` |
| EOLE scorer | `EOLE-COMET` |
| Requires reference | Yes |
| Encoder | XLM-R large style encoder |

## Usage

Requires an EOLE version with native COMET scorer support.

### Validation Metric

```yaml
valid_metrics: ["EOLE-COMET"]
comet_model: eole-nlp/wmt22-comet-da-eole
comet_batch_size: 64
```

### Direct Scoring

```bash
eole predict \
  --model_path eole-nlp/wmt22-comet-da-eole \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --ref /path/to/ref.txt \
  --output /path/to/scores.txt \
  --with_score
```

By default, direct scoring writes one segment score per input line. To emit a
single aggregate system score, add `--score_level system`:

```bash
eole predict \
  --model_path eole-nlp/wmt22-comet-da-eole \
  --src /path/to/src.txt \
  --tgt /path/to/mt.txt \
  --ref /path/to/ref.txt \
  --output /path/to/system-score.txt \
  --with_score \
  --score_level system
```

`score_level: system` writes one numeric line containing the arithmetic mean of
the segment scores, similar to Unbabel COMET's `--only_system` mode but using
EOLE's score-file format.

## Conversion

Converted with:

```bash
eole convert COMET \
  --model Unbabel/wmt22-comet-da \
  --output wmt22-comet-da-eole
```

## Parity

This conversion was parity-checked against the upstream Unbabel COMET runtime. On a 100-line WMT17 slice, native EOLE scoring matched the upstream baseline within floating point tolerance:

| Metric | Value |
|---|---:|
| System score delta | `1.85e-08` |
| Sentence MAE | `5.42e-08` |
| Sentence max abs | `7.15e-07` |

## Original Model and Attribution

This model is a conversion of [Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da). The model weights were converted without modification.

This converted repository follows the upstream model license: Apache-2.0. Please refer to the original model card and license for training data, intended use, limitations, and citation details.

Unbabel COMET: https://github.com/Unbabel/COMET