NLLB-600M fine-tuned for English, Swahili, Kikuyu and Kalenjin

A multilingual translation model fine-tuned from facebook/nllb-200-distilled-600M. It translates in all 12 directions between English, Swahili, Kikuyu and Kalenjin, with an emphasis on agricultural advisory content (crops, livestock, farm management).

Language Code to use
English eng_Latn
Swahili swh_Latn
Kikuyu kik_Latn
Kalenjin kln_Latn (new token, see below)

Kalenjin isn't one of NLLB's languages. This model adds the token kln_Latn (id 256204). Its embedding was initialised from Dholuo (luo_Latn) and then trained. Use kln_Latn like any other NLLB language code. The Kalenjin training data mixes the Nandi and Kipsigis dialects, and Kalenjin spelling isn't standardised.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

repo = "Biatus/multilingual_nllb_kln_kik"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

def translate(text, src_lang, tgt_lang):
    tokenizer.src_lang = src_lang
    inputs = tokenizer(text, return_tensors="pt")
    out = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),
        num_beams=4,
        max_new_tokens=128,
    )
    return tokenizer.batch_decode(out, skip_special_tokens=True)[0]

text = "How can I control fall armyworm in my maize farm?"
print(translate(text, "eng_Latn", "swh_Latn"))  # Ninawezaje kudhibiti viwavi jeshi katika shamba langu la mahindi?
print(translate(text, "eng_Latn", "kik_Latn"))
print(translate(text, "eng_Latn", "kln_Latn"))

Training

The model was trained in two stages, both as full fine-tunes in bf16 mixed precision with label-smoothed cross-entropy (0.1) and a cosine learning-rate schedule.

  1. 10 directions (English/Swahili ↔ Kikuyu/Kalenjin, plus Kikuyu ↔ Kalenjin), trained from the base NLLB model for up to 5 epochs at learning rate 5e-5.
  2. Continued fine-tuning with English ↔ Swahili added (12 directions). Training started from the stage-1 model and ran for 2 epochs at learning rate 2e-5. All stage-1 data was replayed alongside the new pairs so the existing directions weren't forgotten.

Data

Parallel sentences from scripted speech transcripts, general-domain corpora and reviewed agricultural advisory sentences. Cleaning included text normalisation, source-side English/Swahili language identification, pair-level filters, deduplication and, for English–Swahili, an NLLB-based filter for misaligned pairs. Kikuyu ↔ Kalenjin pairs are pivoted through shared English/Swahili source sentences.

Direction (each trained both ways) Training pairs
English – Kikuyu 84,123
English – Kalenjin 70,439
Swahili – Kikuyu 80,743
Swahili – Kalenjin 47,224
English – Swahili 48,679
Kikuyu – Kalenjin (pivot) 67,400

Train, dev and test were split by sentence group across all language pairs: no sentence in dev or test appears in training in any direction.

Evaluation

These are held-out test-set results, with beam search (4 beams), sacreBLEU and chrF++. There are about 2,000 sentences per direction (1,605 for Kikuyu ↔ Kalenjin).

Direction chrF++ BLEU
eng → kik 49.83 21.90
eng → kln 41.87 13.45
eng → swh 70.94 49.32
kik → eng 55.33 34.03
kik → kln 36.99 10.68
kik → swh 64.22 40.59
kln → eng 56.07 36.43
kln → kik 41.90 15.75
kln → swh 46.05 19.93
swh → eng 71.03 50.88
swh → kik 58.74 30.98
swh → kln 35.95 5.88
Average 52.41 27.48

For comparison, the base NLLB-200-distilled-600M scores 55.4 (eng → swh) and 53.2 (swh → eng) chrF++ on the same English–Swahili test set.

Limitations

  • Domain. The English–Swahili test set is drawn from agricultural advisory questions, so its scores are likely higher than you would get on general-domain Swahili. Most of the Swahili ↔ Kikuyu/Kalenjin test data comes from news and everyday speech.
  • Kalenjin quality. Translation into Kalenjin is the weakest part of the model, especially Swahili → Kalenjin. This partly reflects non-standard spelling and the mix of dialects in the references, which also depresses BLEU.
  • Numbers. Kikuyu and Kalenjin references transcribed from speech often spell out numbers in words, so the model may do the same.
  • Casing and punctuation. Some Kikuyu and Kalenjin outputs are lowercase without punctuation, following speech-derived training targets.
  • Unreviewed output. Don't rely on translations of safety-critical advice (e.g. pesticide or veterinary drug dosages) without review by a fluent speaker.

License

CC-BY-NC-4.0, inherited from the base NLLB-200 model. Non-commercial use only.

Downloads last month
34
Safetensors
Model size
0.6B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Biatus/multilingual_nllb_kln_kik

Finetuned
(479)
this model