NUMALIA

NUMALIA (نوماليا) is a 5.19M-parameter tokenizer-free, byte-level dialect and language identifier for the Maghreb and North Africa, trained from scratch on 124,036 multi-dialect rows across 8 classes (algerian, moroccan, tunisian, libyan, hassaniya, msa, french, other_arabic).

On the 12,990-row sealed test split, NUMALIA reaches 91.79% accuracy and 91.59% macro F1 at 20.8 MB (float32), returning both a point label and a calibrated 99%-coverage conformal prediction set for every input.

Task 8-Class Maghrebi Dialect & Language Identification
Architecture Byte-Level ConvStem (k=5/9/13) + 6L SwiGLU Transformer + LDAM Cosine Classifier + Conformal Calibration
Parameters 5,191,048 (20.8 MB float32, zero tokenizer dependencies)
Input / Scripts Raw UTF-8 bytes (vocab 258, max 512 bytes) — Arabic script, Arabizi, French
Classes (8) algerian, moroccan, tunisian, libyan, hassaniya, msa, french, other_arabic
License (Weights) CC-BY-SA-4.0
Author(s) Abderahmane Ainouche, Algerian NLP Collective

Quickstart

Install dependencies:

pip install torch transformers

The architecture and identify API travel with the weights (modeling_numalia.py), so pass trust_remote_code=True:

from transformers import AutoModelForSequenceClassification

REPO = "algerian-nlp/NUMALIA"
model = AutoModelForSequenceClassification.from_pretrained(
    REPO,
    trust_remote_code=True,
).eval()

texts = [
    "واش راك خويا لابس عليك؟",                                    # Algerian Arabic script
    "wesh rak khoya cava chwiya?",                                # Arabizi (cross-Maghrebi)
    "اش خبارك اخاي لاباس عليك كولشي مزيان؟",                     # Moroccan Arabic script
    "chnawa a7welek ya sahbi?",                                   # Tunisian Arabizi
    "شنو جوك اليوم يا غالي؟",                                     # Libyan Arabic script
    "يكانك اتدير الفظة ماهي الاوقية يالتك اتكيس البنك الشعبي",    # Hassaniya
    "عقد مجلس الأمن الدولي جلسة طارئة لمناقشة الأوضاع الراهنة.",   # MSA
    "Le gouvernement a annoncé de nouvelles mesures économiques.", # French
]

for r in model.identify(texts):
    print(f"{r.language} ({r.confidence:.2%}) -> {list(r.prediction_set)}")

Representative output from the published release:

algerian (93.06%) -> ['algerian', 'moroccan']
algerian (77.02%) -> ['algerian', 'moroccan', 'tunisian', 'other_arabic']
moroccan (97.20%) -> ['moroccan']
tunisian (91.40%) -> ['tunisian']
libyan (88.10%) -> ['libyan']
hassaniya (83.50%) -> ['hassaniya']
msa (99.68%) -> ['msa']
french (99.40%) -> ['french']

Adjusting conformal coverage: At the default 99% target coverage (alpha = 0.01), any class with adjusted probability >= 1.96% enters prediction_set to preserve the coverage guarantee on borderline Maghrebi texts. Use r.language for the argmax single label, or lower target_coverage (e.g., 0.95) for tighter sets:

for r in model.identify(texts, target_coverage=0.95):
    print(f"{r.language} ({r.confidence:.2%}) -> {list(r.prediction_set)}")

Evaluation

Evaluated on sealed benchmark partitions from one training run and one deterministic evaluation pass (seed 42).

Test Split (12,990 Rows, 8 Classes)

Class Support Precision Recall F1
French (french) 1,500 99.40% 100.00% 99.70%
MSA (msa) 1,861 98.36% 99.68% 99.01%
Tunisian (tunisian) 2,000 96.81% 92.50% 94.60%
Other Arabic (other_arabic) 2,000 94.67% 86.15% 90.21%
Libyan (libyan) 1,078 83.98% 96.29% 89.71%
Hassaniya (hassaniya) 552 81.35% 94.02% 87.23%
Algerian (algerian) 1,999 91.89% 82.14% 86.74%
Moroccan (moroccan) 2,000 81.57% 89.85% 85.51%
Macro Average 12,990 91.00% 92.58% 91.59%
Overall Accuracy 12,990 — — 91.79%

Validation Split (9,389 Rows)

Metric Value
Accuracy 93.64%
Macro F1 94.08%
Conformal empirical coverage (99% target, alpha = 0.01) 99.13%
Conformal mean set size 2.02 classes

Note on anchor suites: DziriEval and MADAR do not span these 8 Maghrebi/Hassaniya/French classes, so no anchor-suite row applies.


Intended Use & Scope

Recommended uses:

  • Dialect routing, corpus provenance screening, and dataset filtering for Maghrebi NLP pipelines.
  • High-recall corpus curation using the 99%-coverage conformal set (retaining texts whenever algerian appears in prediction_set).

Out-of-scope uses:

  • Text generation (classification-only architecture).
  • Fine-grained identification of Eastern Arabic varieties (Levantine, Gulf, Egyptian, and Mesopotamian Arabic are grouped into other_arabic).
  • Automated decisions affecting individuals.

This model has not been evaluated for bias, toxicity, or factuality.


Architecture

Component Specification
Parameters 5,191,048 (20.8 MB float32)
Input Raw UTF-8 bytes, vocabulary 258, max 512 bytes (no tokenizer)
Conv Stem 3 parallel depthwise-separable branches (kernels 5 / 9 / 13, dim 128 each)
Transformer 6 layers, 8 heads, embed dim 256, SwiGLU FFN dim 704, RoPE positions
Pooling & Head Attentive pooling + scaled cosine classifier with LDAM per-class margins
Training Projection 128-dim projection head for supervised contrastive loss (training only)
Calibration Balanced Softmax prior shift (tau = 0.5) + split conformal quantile (q_hat = 0.9804)
Release Precision float32

Operating directly on UTF-8 bytes avoids subword fragmentation and out-of-vocabulary failures across Arabic script, Arabizi digits (3, 7, 9), and French orthography, while the multi-scale ConvStem captures local character, affix, and clitic n-grams before global Transformer self-attention.


Training Data

Built from 13 sources totaling 337,742 raw rows: 62,514 were removed by length/noise filtering (<3 words or >512 bytes), 17,752 exact duplicates were removed, and 146,415 rows were allocated (seed 42) into 124,036 train, 9,389 validation, and 12,990 sealed test rows (zero hash collisions between test and train/dev).

Source Domain / Dialect Tier Train Rows Share
evageon/IADD North African & Arab dialects Permissive (CC-BY-4.0) 29,734 23.97%
wikimedia/wikipedia-ar Modern Standard Arabic Restricted (CC-BY-SA-4.0) 16,743 13.50%
wikimedia/wikipedia-fr French articles (sample) Restricted (CC-BY-SA-4.0) 15,000 12.09%
zenodo/TUNIZI-v2 Tunisian Arabizi comments Permissive (CC-BY-4.0) 14,706 11.86%
ayoubkirouane/Algerian-Darija Algerian Darija Permissive (CC-BY-4.0) 11,061 8.92%
MouadJb/MYC Moroccan YouTube comments Unknown (research citation) 10,501 8.47%
LeMGarouani/MAC Moroccan tweets Unknown (research citation) 6,692 5.40%
Mansour-Essgaer/Libya-Telecom Libyan telecom tweets Unknown (research use) 6,005 4.84%
DarjaCore/algerian-darja-sample Algerian Darija Telegram Permissive (MIT) 5,929 4.78%
Mansour-Essgaer/Libyan-Resturant Libyan restaurant reviews Unknown (research use) 2,976 2.40%
Amin-tech99/ai-for-rim Hassaniya / Mauritanian Permissive (MIT) 2,505 2.02%
tunis-ai/tsac Tunisian social comments Unknown (research use) 1,378 1.11%
mendeley/HASSANIYA Hassaniya Facebook comments Permissive (CC-BY-4.0) 806 0.65%
Total 8 classes Mixed 124,036 100.00%
  • Class breakdown (train): algerian 20,001 (16.13%), moroccan 20,000 (16.12%), tunisian 20,000 (16.12%), other_arabic 20,000 (16.12%), msa 16,743 (13.50%), french 15,000 (12.09%), libyan 8,981 (7.24%), hassaniya 3,311 (2.67%).
  • Script breakdown (train): Arabic script 84,535 (68.15%), Latin / Arabizi 36,539 (29.46%), mixed 2,962 (2.39%).

Training Data License Composition

The model weights are released under CC-BY-SA-4.0. That grant applies to weights and code only and does not relicense the underlying training text.

Tier Train Rows Share Sources
Permissive 64,741 52.19% CC-BY-4.0 (IADD, TUNIZI-v2, Algerian-Darija, HASSANIYA), MIT (DarjaCore, ai-for-rim)
Restricted 31,743 25.59% CC-BY-SA-4.0 (Wikipedia Arabic & French)
Unknown 27,552 22.21% Research sources without formal open licenses (MYC, MAC, Libya-Telecom, Libyan-Resturant, TSAC)

Across all 146,415 split rows: 51.76% permissive (75,777), 25.61% restricted (37,499), and 22.63% unknown (33,139).


Training Details

Parameter Value
Objective Cross-entropy (label smoothing 0.05) + SupCon (weight 0.5, temp 0.05, bank 2,048) + Group-DRO (step 0.01) + LDAM margins
Optimizer AdamW (lr=3e-4, weight_decay=0.01, max_grad_norm=1.0)
Batch Size 12 groups × 10 samples = 120 per step
Schedule Cosine decay with warm-up (deferred share 0.5)
Epochs / Steps 8 epochs (8,264 steps; best validation checkpoint at step 6,966: 93.71% acc, 94.10% macro F1)
Augmentation Phrase-level mixing (p=0.35), case augmentation (p=0.10)
Inference Prior Shift Balanced Softmax logit adjustment (tau = 0.5)
Precision float32
Hardware & Duration 1× NVIDIA Tesla T4 (16 GB), 2.99 h (10,783 s)
Seed 42

Limitations

  • Single training run. All numbers reflect one run under seed 42; variance across seeds is unmeasured.
  • Cross-Maghrebi Arabizi overlap. Short greetings shared across borders (wesh, khoya, chwiya, sahbi) naturally produce multi-dialect conformal sets.
  • Low-resource Hassaniya coverage. Only 3,311 training rows (552 test rows); performance may degrade on Hassaniya text stylistically distant from the training sources.
  • Distribution-dependent conformal guarantee. The conformal quantile (q_hat = 0.9804) was calibrated on half of the 9,389-row validation split; the >= 99% coverage guarantee applies under exchangeability with that distribution.
  • 8-class boundary. Eastern Arabic varieties collapse into other_arabic.
  • No safety evaluation. Bias, toxicity, and factuality have not been evaluated.

Repository Contents

File Size Description / SHA-256
model.safetensors 20.8 MB Float32 weights (5,191,048 params) · SHA-256 33ebc03ba642064ce4f3663aa0a46f12dd5dd89eb0e9e065a40286606c5f5e8f
config.json 1.4 KB Architecture config, auto_map, and calibration constants
modeling_numalia.py 17.7 KB Self-contained model implementation and identify API
calibration.json 551 B Conformal q_hat, target coverage, prior-shift vector, and class labels
export_report.json 944 B Verified file sizes, SHA-256 checksums, parameter count, and reload parity (max_confidence_diff = 0.0)

Reload verification confirms zero missing or unexpected keys and exact prediction parity (max_confidence_diff = 0.0).


The name

NUMALIA folds the classical names of the peoples across North Africa and the Sahara into one word: NUmidia (Algeria), MAuretania (Morocco and Mauritania), LIbu (Libya), and Afri (Tunisia). The classifier carries their shared name because it models the continuum of Maghrebi and Hassaniya Arabic varieties side by side in a single calibrated space.


Citation

@misc{algerian_nlp_numalia_2026,
  title  = {NUMALIA: Byte-Level Dialect Identification for Maghrebi Arabic},
  author = {Ainouche, Abderahmane and Algerian NLP Collective},
  year   = {2026},
  url    = {https://huggingface.co/algerian-nlp/NUMALIA}
}

License

Weights and code are released under CC-BY-SA-4.0. Refer to the Training Data License Composition section above regarding the underlying training text before redistributing derivatives.

Downloads last month
49
Safetensors
Model size
5.19M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including algerian-nlp/NUMALIA

Evaluation results

  • Accuracy (test, 8 classes, single run) on LID benchmark (test, 12,990 rows)
    self-reported
    0.918
  • Macro F1 (test, 8 classes, single run) on LID benchmark (test, 12,990 rows)
    self-reported
    0.916
  • Accuracy (validation, 8 classes, single run) on LID benchmark (validation, 9,389 rows)
    self-reported
    0.936
  • Macro F1 (validation, 8 classes, single run) on LID benchmark (validation, 9,389 rows)
    self-reported
    0.941