Instructions to use eole-nlp/wmt22-comet-da-eole-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- COMET
How to use eole-nlp/wmt22-comet-da-eole-fp16 with COMET:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: eole
|
| 3 |
+
tags:
|
| 4 |
+
- eole
|
| 5 |
+
- comet
|
| 6 |
+
- machine-translation-evaluation
|
| 7 |
+
- quality-estimation
|
| 8 |
+
- converted
|
| 9 |
+
- fp16
|
| 10 |
+
base_model: Unbabel/wmt22-comet-da
|
| 11 |
+
license: apache-2.0
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# WMT22 COMET DA FP16 (EOLE)
|
| 15 |
+
|
| 16 |
+
This is [Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da) converted to [EOLE](https://github.com/eole-nlp/eole) format.
|
| 17 |
+
|
| 18 |
+
Model weights were dtype-cast to fp16 during EOLE conversion. This repository contains a format and dtype conversion for use with EOLE's native `transformer_encoder_scorer` implementation.
|
| 19 |
+
|
| 20 |
+
This is not an upstream `unbabel-comet` checkpoint layout. Use it with EOLE.
|
| 21 |
+
|
| 22 |
+
## Model Details
|
| 23 |
+
|
| 24 |
+
| | |
|
| 25 |
+
|---|---|
|
| 26 |
+
| Original model | [Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da) |
|
| 27 |
+
| EOLE architecture | `transformer_encoder_scorer` |
|
| 28 |
+
| Weight dtype | `fp16` |
|
| 29 |
+
| Scoring type | `comet` |
|
| 30 |
+
| Class identifier | `regression_metric` |
|
| 31 |
+
| EOLE scorer | `EOLE-COMET` |
|
| 32 |
+
| Requires reference | Yes |
|
| 33 |
+
| Encoder | XLM-R large style encoder |
|
| 34 |
+
|
| 35 |
+
## Usage
|
| 36 |
+
|
| 37 |
+
Requires an EOLE version with native COMET scorer support.
|
| 38 |
+
|
| 39 |
+
### Validation Metric
|
| 40 |
+
|
| 41 |
+
```yaml
|
| 42 |
+
valid_metrics: ["EOLE-COMET"]
|
| 43 |
+
comet_model: eole-nlp/wmt22-comet-da-eole-fp16
|
| 44 |
+
comet_batch_size: 64
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
### Direct Scoring
|
| 48 |
+
|
| 49 |
+
```bash
|
| 50 |
+
eole predict \
|
| 51 |
+
--model_path eole-nlp/wmt22-comet-da-eole-fp16 \
|
| 52 |
+
--src /path/to/src.txt \
|
| 53 |
+
--tgt /path/to/mt.txt \
|
| 54 |
+
--ref /path/to/ref.txt \
|
| 55 |
+
--output /path/to/scores.txt \
|
| 56 |
+
--with_score
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
By default, direct scoring writes one segment score per input line. To emit a
|
| 60 |
+
single aggregate system score, add `--score_level system`:
|
| 61 |
+
|
| 62 |
+
```bash
|
| 63 |
+
eole predict \
|
| 64 |
+
--model_path eole-nlp/wmt22-comet-da-eole-fp16 \
|
| 65 |
+
--src /path/to/src.txt \
|
| 66 |
+
--tgt /path/to/mt.txt \
|
| 67 |
+
--ref /path/to/ref.txt \
|
| 68 |
+
--output /path/to/system-score.txt \
|
| 69 |
+
--with_score \
|
| 70 |
+
--score_level system
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
`score_level: system` writes one numeric line containing the arithmetic mean of
|
| 74 |
+
the segment scores, similar to Unbabel COMET's `--only_system` mode but using
|
| 75 |
+
EOLE's score-file format.
|
| 76 |
+
|
| 77 |
+
## Conversion
|
| 78 |
+
|
| 79 |
+
Converted with:
|
| 80 |
+
|
| 81 |
+
```bash
|
| 82 |
+
eole convert COMET \
|
| 83 |
+
--model Unbabel/wmt22-comet-da \
|
| 84 |
+
--dtype fp16 \
|
| 85 |
+
--output wmt22-comet-da-eole-fp16
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
## Parity
|
| 89 |
+
|
| 90 |
+
The fp32 EOLE conversion was parity-checked against the upstream Unbabel COMET runtime. On a 100-line WMT17 slice, native EOLE scoring matched the upstream baseline within floating point tolerance:
|
| 91 |
+
|
| 92 |
+
| Metric | Value |
|
| 93 |
+
|---|---:|
|
| 94 |
+
| System score delta | `1.85e-08` |
|
| 95 |
+
| Sentence MAE | `5.42e-08` |
|
| 96 |
+
| Sentence max abs | `7.15e-07` |
|
| 97 |
+
|
| 98 |
+
The fp16 artifacts are intended to provide smaller EOLE model files and fp16 runtime behavior. Run the EOLE COMET parity harness against these fp16 repositories if exact fp16 deltas are required.
|
| 99 |
+
|
| 100 |
+
## Original Model and Attribution
|
| 101 |
+
|
| 102 |
+
This model is a conversion of [Unbabel/wmt22-comet-da](https://huggingface.co/Unbabel/wmt22-comet-da). The model weights were converted to EOLE format and dtype-cast to fp16.
|
| 103 |
+
|
| 104 |
+
This converted repository follows the upstream model license: Apache-2.0. Please refer to the original model card and license for training data, intended use, limitations, and citation details.
|
| 105 |
+
|
| 106 |
+
Unbabel COMET: https://github.com/Unbabel/COMET
|