Lordkiki/dialingua-bkv-mt

Dialingua — Bekwarra ↔ English

Fine-tune of facebook/m2m100_418M (MIT) for Bekwarra (ISO 639-3 bkv), a Bendi language of Cross River State, Nigeria, spoken by roughly 100,000 people.

Bekwarra is in no major multilingual model — not NLLB, not FLORES-200, not MADLAD. The language token __bkv__ was added to the tokenizer and seeded from Igbo before fine-tuning.

Honest scope

Trained on 1,068 verse pairs OCR'd from printed Old Testament portions. That is below the ~3,000 pairs where this stops being a demo.

• Scripture and formal register: works. Ibang h'uni ang'a de achi i na k'ufo uni irichom? → "What does a wise man do to the fool?"
• Everyday speech: unreliable. A Bekwarra proverb comes back as unrelated scripture. The model has never seen conversational Bekwarra.

Do not use this for anything consequential — medical, legal, or safety information — without a native speaker checking the output.

Usage

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Lordkiki/dialingua-bkv-mt")
model = AutoModelForSeq2SeqLM.from_pretrained("Lordkiki/dialingua-bkv-mt")

# The language maps are rebuilt from a hardcoded list on load, so the added
# code has to be re-registered or src_lang raises KeyError('bkv').
bkv_id = tok.convert_tokens_to_ids("__bkv__")
for attr, key, val in [("lang_code_to_token", "bkv", "__bkv__"),
                       ("lang_token_to_id", "__bkv__", bkv_id),
                       ("id_to_lang_token", bkv_id, "__bkv__"),
                       ("lang_code_to_id", "bkv", bkv_id)]:
    getattr(tok, attr)[key] = val

tok.src_lang = "bkv"
enc = tok("Ibang h'uni ang'a de achi i na k'ufo uni irichom?", return_tensors="pt")
out = model.generate(**enc, forced_bos_token_id=tok.convert_tokens_to_ids("__en__"),
                     num_beams=4, no_repeat_ngram_size=3)
print(tok.batch_decode(out, skip_special_tokens=True)[0])

Orthography

Bekwarra's apostrophe is phonemic — a letter, not punctuation. k'uchu, ng'amin, r'abe. Both the closed form and the spaced form (K' iye) occur in current writing and both are handled.

Training data

Old Testament portions in Bekwarra, OCR'd from scanned print. The underlying translations are © their rights holders; this model is released for research and language-preservation use. Settle permissions before commercial use.