---
license: mit
base_model: none
tags:
- arabizi
- transliteration
- arabic-nlp
- arabic-dialects
pretty_name: ArabiziKit trained layer
---

# ArabiziKit trained layer

The learned layer of the ArabiziKit hybrid transliteration system
(https://github.com/rb2625/arabizi-kit), which converts Arabizi — Arabic
written in Latin letters and digits (2 for hamza/qaf, 3 for ayn, 7 for ha,
5 for kha) — into Arabic script.

Three small, dependency-free components, all pure Python:

| component | file | what it does |
| --- | --- | --- |
| word reading table | `model.json` | Arabizi word → observed Arabic renderings with frequencies (1,155 entries), article-aware alignment (`el etnein` → `الاثنين`) |
| character trigram LM | `model.json` | Laplace-smoothed trigram over corpus references, reranking each word's candidates toward natural Arabic |
| Naive Bayes dialect classifier | `egyptian.json`, `levantine.json`, `maghrebi.json` | word tokens + Arabizi code markers; supplies the dialect hint automatically (only for dialects whose conventions change the engine's readings) |

## Training

Trained on the calibration set plus the pipeline train/dev splits
(`rb2625/arabizi-kit-corpus`) and public dialect text. Held-out test and
external sets are excluded from training.

## Usage

```bash
pip install arabizikit
arabizikit model train     # rebuild the model from the corpus
arabizikit "bach n9ra" --model              # learned layer on one sentence
arabizikit eval --data test.json --model    # reproduce the paper's numbers
```

## Results (exact@1 / hit@3 / CER, from the paper)

| set | rules, no hint | rules, oracle hint | learned layer |
| --- | --- | --- | --- |
| pipeline dev | 0.021 / 0.021 / 0.291 | 0.021 / 0.021 / 0.291 | 0.354 / 0.542 / 0.097 |
| pipeline test | 0.000 / 0.000 / 0.299 | 0.000 / 0.000 / 0.299 | 0.061 / 0.102 / 0.226 |
| Egyptian (external) | 0.376 / 0.526 / 0.236 | 0.376 / 0.526 / 0.236 | 0.348 / 0.502 / 0.248 |
| Levantine (external) | 0.296 / 0.429 / 0.076 | 0.296 / 0.429 / 0.076 | 0.270 / 0.360 / 0.096 |
| Moroccan Darija (external) | 0.035 / 0.057 / 0.235 | 0.088 / 0.115 / 0.180 | 0.053 / 0.080 / 0.207 |

## Paper

"ArabiziKit: An Open, Hybrid, Benchmark-Driven Arabizi to Arabic-Script
Transliteration System" (arXiv, cs.CL).