Jisr-MT-50M-Dialects2En

13 Arabic dialects → English, one 49M-parameter model.

+12.63 chrF++ over its base on every dialect tested, reaching 76.2% of the 12B model that generated its training data — at 1/245th the size.

Results

UBC-NLP/alexandria test splits, beam 4. gemma-4-12B-it is the model that produced this model's training targets, so it is the effective teacher ceiling.

dialect	base 49M	this model	gain	gemma-4-12B	% of teacher
Syrian	37.19	51.48	+14.29	64.52	79.8%
Sudanese	35.38	47.79	+12.40	57.60	83.0%
Saudi	38.90	47.53	+8.64	63.16	75.3%
Lebanese	31.11	46.34	+15.23	55.72	83.2%
Tunisian	32.34	46.19	+13.85	64.42	71.7%
Palestinian	34.33	46.15	+11.81	55.71	82.8%
Egyptian	32.72	45.11	+12.39	61.08	73.9%
Yemeni	34.34	43.88	+9.55	57.36	76.5%
Libyan	32.65	41.50	+8.85	53.17	78.1%
Moroccan	25.71	40.60	+14.89	55.69	72.9%
weighted avg	32.66	45.29	+12.63	59.46	76.2%

Weighted by each dialect's share of the training corpus.

Why this direction works so much better than English → dialect

The same corpus trains a worse model in the reverse direction. For English → Egyptian it scored 34.96 chrF++ against 37.91 for a purpose-built human corpus.

The reason is which side is machine-generated. Here the source is genuine human speech (ASR transcripts) and the target is Gemma's English. Noise on the source side costs far less than noise on the target side, because the model is never asked to reproduce it. Going the other way, the model had to generate machine-translated Arabic.

This makes the run effectively sequence-level distillation of a 12B model into 49M — the same mechanism that let Jisr-15M-distill reach 83.5% of a 241M teacher.

Compared with frontier models

Same benchmark, 200 turns per dialect. gemma-4-12B-it produced this model's training targets; gpt-5.1 never saw this corpus and is an independent read.

dialect	this model (49M)	gpt-5.1	gemma-4-12B	% of gpt-5.1
Lebanese	46.34	53.16	55.72	87%
Palestinian	46.15	53.25	55.71	87%
Sudanese	47.79	55.23	57.60	87%
Syrian	51.48	59.80	64.52	86%
Libyan	41.50	49.55	53.17	84%
Yemeni	43.88	52.86	57.36	83%
Saudi	47.53	57.68	63.16	82%
Egyptian	45.11	56.44	61.08	80%
Tunisian	46.19	59.44	64.42	78%
Moroccan	40.60	52.61	55.69	77%
weighted avg	45.29	55.48	59.46	82%

A 49M model reaches 82% of gpt-5.1, within 7.4–12 chrF++ of it on every dialect, after 23 minutes of training on one GPU. The spread is tight (77–87%), unlike the English→dialect direction where the same family ranges more widely.

Two things worth noting

gpt-5.1 scores below gemma-4-12B on all 10 dialects (55.48 vs 59.46 weighted). Gemma generated this project's training English, so its style sits closer to conventional MT English — but Alexandria's English references are human-written and independent of that corpus, so this is a real result on a neutral benchmark rather than contamination. Treat it as surprising and worth a second look, not settled.

Moroccan is unremarkable here (gpt-5.1 52.61, mid-pack) while in the English→Moroccan direction gpt-5.1 collapses to 20.86 — below the 49M Jisr-MT-50M-Moroccan at 32.23. Frontier models can understand Darija; they struggle to produce it.

⚠️ It costs MSA

FLORES+ devtest, MSA ar→en, 1,012 sentences:

model	BLEU	chrF	chrF++
Jisr-MT-50M (base)	29.35	57.86	55.87
this model	23.18	52.59	50.65
	−6.17	−5.27	−5.22

The corpus included MSA→English replay at 0.3×, and it was not enough. The reasoning behind that ratio — that forgetting would be milder because the target language does not change — was only partly right: the model still lost 6.17 BLEU of MSA.

Use the base model for Modern Standard Arabic. Use this one for dialect. A future run should try 1× replay, which is what held MSA in place in the English→dialect models.

Usage

from transformers import AutoTokenizer, MarianMTModel

tok = AutoTokenizer.from_pretrained("oddadmix/Jisr-MT-50M-Dialects2En")
model = MarianMTModel.from_pretrained("oddadmix/Jisr-MT-50M-Dialects2En")

text = ">>eng<< العيال بيلعبوا كورة في الشارع بعد المدرسة"
out = model.generate(**tok(text, return_tensors="pt"), num_beams=4)
print(tok.decode(out[0], skip_special_tokens=True))

One tag, >>eng<<, and it is required. There is no per-dialect tag: the target is English for every input, so the model infers the dialect from the text itself. Arabic → English only — this model has no English → Arabic direction.

Training

Base	Jisr-MT-50M (49M, ar→en 29.35 BLEU)
Dialect data	1,272,744 pairs across 13 dialects
MSA replay	381,823 ar→en pairs (0.3×) — too little, see above
Total	1,654,567 rows, single >>eng<< tag
Budget	1 epoch, 25,852 steps, bs 64, lr 3e-4 cosine, bf16
Hardware	23 minutes

The base is Jisr-MT-50M, not Jisr-MT-50M-tatoeba. The tatoeba model has the family's best en→ar but its ar→en is broken (7.86 BLEU), and this direction depends on it entirely.

All 13 dialects were pooled into one model rather than trained separately, following the AllDialects result where small dialects gained most from sharing (Libyan +7.62 chrF++).

Limitations

• MSA regressed by 6.17 BLEU — see above.
• Arabic → English only.
• Algerian, Bahraini and Iraqi are unvalidated. UBC-NLP/alexandria has no config for them, so 3 of the 13 dialects this model claims have no external benchmark at all.
• Cannot exceed its teacher; it reaches 76.2% of gemma-4-12B-it and inherits its biases.
• Source register is spoken/conversational — the training data is ASR transcripts.
• 49M parameters: expect errors on rare named entities and long complex input.