| --- |
| pipeline_tag: translation |
| library_name: comet |
| language: |
| - multilingual |
| - af |
| - am |
| - ar |
| - as |
| - az |
| - be |
| - bg |
| - bn |
| - br |
| - bs |
| - ca |
| - cs |
| - cy |
| - da |
| - de |
| - el |
| - en |
| - eo |
| - es |
| - et |
| - eu |
| - fa |
| - fi |
| - fr |
| - fy |
| - ga |
| - gd |
| - gl |
| - gu |
| - ha |
| - he |
| - hi |
| - hr |
| - hu |
| - hy |
| - id |
| - is |
| - it |
| - ja |
| - jv |
| - ka |
| - kk |
| - km |
| - kn |
| - ko |
| - ku |
| - ky |
| - la |
| - lo |
| - lt |
| - lv |
| - mg |
| - mk |
| - ml |
| - mn |
| - mr |
| - ms |
| - my |
| - ne |
| - nl |
| - 'no' |
| - om |
| - or |
| - pa |
| - pl |
| - ps |
| - pt |
| - ro |
| - ru |
| - sa |
| - sd |
| - si |
| - sk |
| - sl |
| - so |
| - sq |
| - sr |
| - su |
| - sv |
| - sw |
| - ta |
| - te |
| - th |
| - tl |
| - tr |
| - ug |
| - uk |
| - ur |
| - uz |
| - vi |
| - xh |
| - yi |
| - zh |
| license: apache-2.0 |
| base_model: |
| - FacebookAI/xlm-roberta-large |
| --- |
| |
| # COMET-partial |
|
|
| This model is based on [COMET-early-exit](https://github.com/zouharvi/COMET-early-exit), which is a fork but not compatible with original Unbabel's COMET. |
| To run the model, you need to first install this version of COMET either with: |
| ```bash |
| pip install "git+https://github.com/zouharvi/COMET-early-exit#egg=comet-early-exit&subdirectory=comet_early_exit" |
| ``` |
| or in editable mode: |
| ```bash |
| git clone https://github.com/zouharvi/COMET-early-exit.git |
| cd COMET-early-exit |
| pip3 install -e comet_early_exit |
| ``` |
|
|
|
|
| This model is described in the appendix in the paper. |
| It is able to score even *incomplete* translations (i.e. prefixes of translations): |
| ```python |
| import comet_early_exit |
| model = comet_early_exit.load_from_checkpoint(comet_early_exit.download_model("zouharvi/COMET-partial")) |
| data = [ |
| { |
| "src": "I want to receive my food in 10 to 15 minutes.", |
| "mt": "Ich werde", |
| }, |
| { |
| "src": "I want to receive my food in 10 to 15 minutes.", |
| "mt": "Ich möchte", |
| }, |
| { |
| "src": "I want to receive my food in 10 to 15 minutes.", |
| "mt": "Ich möchte mein Essen in", |
| }, |
| { |
| "src": "I want to receive my food in 10 to 15 minutes.", |
| "mt": "Ich möchte mein Essen in 10 bis 15 Minuten erhalten.", |
| }, |
| { |
| "src": "I want to receive my food in 10 to 15 minutes.", |
| "mt": "Ich möchte mein Essen in 10 bis 15 Minuten bekommen.", |
| } |
| ] |
| model_output = model.predict(data, batch_size=8, gpus=1) |
| print("scores", model_output["scores"]) |
| ``` |
| Outputs (formatted): |
| ``` |
| scores 89.26 89.45 89.51 89.48 89.66 |
| ``` |
|
|
| This model is based on the work [Early-Exit and Instant Confidence Translation Quality Estimation](http://arxiv.org/abs/2502.14429) which can be cited as: |
| ``` |
| @misc{zouhar2025earlyexitinstantconfidencetranslation, |
| title={Early-Exit and Instant Confidence Translation Quality Estimation}, |
| author={Vilém Zouhar and Maike Züfle and Beni Egressy and Julius Cheng and Jan Niehues}, |
| year={2025}, |
| eprint={2502.14429}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CL}, |
| url={https://arxiv.org/abs/2502.14429}, |
| } |
| ``` |
