atella_finetuned: direct Runyankole ⇄ Ateso translation
atella_finetuned is a 89M-parameter encoder-decoder Transformer that translates Runyankole → Ateso and
Ateso → Runyankole directly, with one model and no English on either side. The direction is set by a prefix token on the
source (<2teo> / <2nyn>; the tag names the target language).
It is atella_base (pretrained only on machine-translated pairs) further
trained on the human-translated Runyankole–Ateso pairs of the SALT training set. Headline scores on the complete SALT test set are in Evaluation.
Quick start
pip install torch sentencepiece safetensors huggingface_hub
import sys
from huggingface_hub import snapshot_download
folder = snapshot_download("amDANIEL2024/atella_finetuned") # add token="hf_..." if the repo is still private
sys.path.insert(0, folder)
from atella import Translator
t = Translator(folder) # uses the GPU if there is one
ateso = t.translate("Okwombeka orugo kukatandika omu kwezi kw'okubanza 2000.", "nyn2teo")
print(ateso)
print(t.translate(ateso, "teo2nyn")) # back again
Directions are "nyn2teo" and "teo2nyn". translate takes a string or a list and accepts beam (default 4).
This is a custom PyTorch model, not a transformers model, so AutoModel will not load it.
Evaluation
Complete SALT sets (500 sentences each), beam 4, sacreBLEU 2.5.1. Test is the headline. SALT dev was used to stop training and choose the checkpoint, so dev scores are slightly optimistic.
| Split | Direction | System | n | BLEU | chrF | chrF++ |
|---|---|---|---|---|---|---|
| test | nyn→teo | model | 500 | 15.47 | 47.23 | 44.01 |
| test | nyn→teo | copy baseline (source unchanged) | 500 | 0.41 | 15.79 | 13.28 |
| test | teo→nyn | model | 500 | 10.19 | 45.46 | 40.60 |
| test | teo→nyn | copy baseline (source unchanged) | 500 | 0.40 | 14.87 | 12.69 |
| dev | nyn→teo | model | 500 | 16.27 | 48.33 | 45.11 |
| dev | nyn→teo | copy baseline (source unchanged) | 500 | 0.23 | 15.99 | 13.54 |
| dev | teo→nyn | model | 500 | 10.08 | 46.60 | 41.85 |
| dev | teo→nyn | copy baseline (source unchanged) | 500 | 0.22 | 15.13 | 12.99 |
Signatures: BLEU nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.5.1; chrF nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space:no|version:2.5.1; chrF++ nrefs:1|case:mixed|eff:yes|nc:6|nw:2|space:no|version:2.5.1.
The copy baseline outputs the source sentence unchanged and scores it against the reference. An English-pivot baseline was not run. Differences of 1 to 2 BLEU on a few hundred sentences are within noise.
Example outputs
The first 5 sentences of each SALT test file, not selected and not edited. The reference is the human translation. Mistakes are included; read the model output critically.
nyn → teo
| Source | Human reference | Model output |
|---|---|---|
| Eby'eby'afaashoni biriyo nibigaruza akatwe. | Ageutu ikampunin luka enape apolo bobo. | Euta erionget loka ikapun alemun ikapun idis. |
| Omuzibwe n'endwara erikukwata abantu baingi omu kyanga. | Erai adeka na ikamuni itunga luipu kotoma atutubet. | Erai adeka na esuronokini itunga luipu katutubet. |
| Obwoki burungi, nibwija kuruga omukyanga eki. | Esike lo ejok ebuni kosiep kalo. | Esike loajokan nes ebuni alomun kaiboisit kana. |
| Akaba akwise naavuga egaaro obwo nyowe ndikwiruka. | Auta ngesi aicak egali kauta eong akerit. | Auta ngesi airenge erot nautar eong akerit. |
| Aby'obushoboorozi nibahhiga bundi nuuma abastigazi enkwata-buta abaayeyereire akatare k'omu nju | Eutasi ipolisin aimo itelepai iarei luabilete ka akoko eiduka lokapolon. | Isinyikokinitos apedorosio ace itunga luitolosete ibiasaran ibiasaranakec. |
teo → nyn
| Source | Human reference | Model output |
|---|---|---|
| Ageutu ikampunin luka enape apolo bobo. | Eby'eby'afaashoni biriyo nibigaruza akatwe. | Kampuni z'omupiira zitandikire kugaruka kutunguuka. |
| Erai adeka na ikamuni itunga luipu kotoma atutubet. | Omuzibwe n'endwara erikukwata abantu baingi omu kyanga. | N'endwara erikukwata abantu baingi omu kyanga. |
| Esike lo ejok ebuni kosiep kalo. | Obwoki burungi, nibwija kuruga omukyanga eki. | Esike nungi neeza aha rubaju oru. |
| Auta ngesi aicak egali kauta eong akerit. | Akaba akwise naavuga egaaro obwo nyowe ndikwiruka. | Akaba naavuga egaari enyima yaniruka. |
| Eutasi ipolisin aimo itelepai iarei luabilete ka akoko eiduka lokapolon. | Aby'obushoboorozi nibahhiga bundi nuuma abastigazi enkwata-buta abaayeyereire akatare k'omu nju | Pooriisi eriyo neronda teleki ibiri z'okwiba duuka mpango. |
Model
| Architecture | pre-LN Transformer encoder-decoder, 6 + 6 layers, d_model 640, 8 heads, FFN 2560, GELU |
| Parameters | 89M (input embedding, decoder embedding and output projection are tied) |
| Tokenizer | shared SentencePiece unigram, 32,000 pieces, case preserved, digits split (spm.model); trained on Runyankole and Ateso text only |
| Context | 256 positions (sinusoidal); training pairs were limited to 128 pieces per side |
| Weights | model.safetensors (float32) |
Training
- Pretraining (
atella_base): 5 epochs, 17,426 optimizer steps on the Runyankole–Ateso pairs of tulati, both directions. In tulati every non-English column is a machine translation of one English column, made by Sunbird/translate-nllb-1.3b-salt; no human-written text is in it, so both sides of every pretraining pair are machine output. The English column was not used. Of 1,534,699 tulati rows, 127,539 (8.3%) were dropped because they were flagged empty, degenerate, looks-like-English or length-outlier in either language (flagged in Runyankole: 35,277; in Ateso: 104,732), then 1,015 identical-side pairs and 506 exact duplicates were removed, and 3 pair(s) that matched a SALT dev/test sentence; 1,405,636 pairs remained. See theatella_basecard for details. - Fine-tuning: the SALT training pairs (human translations) that have both languages, both directions, peak LR 6e-5, SALT dev loss checked every 100 steps, early stopping after 5 checks without improvement. The checkpoint with the lowest dev loss was kept (step 2,200). SALT test was not used in training or in choosing the checkpoint.
Limitations
- The pretraining pairs are machine output on both sides, so the base model inherits that translator's errors and style. The translator's output was flagged (empty, degenerate, looks-like-English or length-outlier) for 2.3% of tulati rows in Runyankole and 6.8% in Ateso. These rates describe the generated data, not this model. This card does not report how good the translator is for Runyankole or Ateso.
- Scores differ by direction: Ateso → Runyankole scores lower than Runyankole → Ateso on both dev and test (see Evaluation). The reason has not been established.
- SALT's Runyankole and Ateso sentences are independent human translations of the same English sentence, so a reference in one language is not a translation of the reference in the other. Differences of 1 to 2 BLEU on 500 sentences are within noise.
- Real human-written text beyond SALT (dialect, spelling, code-switching) may be handled worse than the benchmark suggests.
- Meaning can be dropped or changed, especially for numbers, names and rare words. Do not use it for medical, legal or other high-stakes content without human review.
- Sentence-level model: inputs longer than about 128 pieces are truncated or degrade.
- Outputs have not been reviewed by a reader of Runyankole or Ateso.
Files
model.safetensors weights · config.json architecture and training summary · spm.model tokenizer ·
atella.py standalone inference code · README.md this card.
License
Released under the MIT License (as are tulati, the translation model that generated the training data, and SALT).
Author
Amanyire Daniel (amDANIEL2024) · domusdeiug@gmail.com · https://domusdeiug.com
- Downloads last month
- 26
Model tree for amDANIEL2024/atella_finetuned
Base model
amDANIEL2024/atella_base