Instructions to use algerian-nlp/NUMALIA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use algerian-nlp/NUMALIA with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="algerian-nlp/NUMALIA", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSequenceClassification model = AutoModelForSequenceClassification.from_pretrained("algerian-nlp/NUMALIA", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
NUMALIA
NUMALIA (نوماليا) is a 5.19M-parameter tokenizer-free, byte-level dialect and language identifier for the Maghreb and North Africa, trained from scratch on 124,036 multi-dialect rows across 8 classes (algerian, moroccan, tunisian, libyan, hassaniya, msa, french, other_arabic).
On the 12,990-row sealed test split, NUMALIA reaches 91.79% accuracy and 91.59% macro F1 at 20.8 MB (float32), returning both a point label and a calibrated 99%-coverage conformal prediction set for every input.
| Task | 8-Class Maghrebi Dialect & Language Identification |
| Architecture | Byte-Level ConvStem (k=5/9/13) + 6L SwiGLU Transformer + LDAM Cosine Classifier + Conformal Calibration |
| Parameters | 5,191,048 (20.8 MB float32, zero tokenizer dependencies) |
| Input / Scripts | Raw UTF-8 bytes (vocab 258, max 512 bytes) — Arabic script, Arabizi, French |
| Classes (8) | algerian, moroccan, tunisian, libyan, hassaniya, msa, french, other_arabic |
| License (Weights) | CC-BY-SA-4.0 |
| Author(s) | Abderahmane Ainouche, Algerian NLP Collective |
Quickstart
Install dependencies:
pip install torch transformers
The architecture and identify API travel with the weights (modeling_numalia.py), so pass trust_remote_code=True:
from transformers import AutoModelForSequenceClassification
REPO = "algerian-nlp/NUMALIA"
model = AutoModelForSequenceClassification.from_pretrained(
REPO,
trust_remote_code=True,
).eval()
texts = [
"واش راك خويا لابس عليك؟", # Algerian Arabic script
"wesh rak khoya cava chwiya?", # Arabizi (cross-Maghrebi)
"اش خبارك اخاي لاباس عليك كولشي مزيان؟", # Moroccan Arabic script
"chnawa a7welek ya sahbi?", # Tunisian Arabizi
"شنو جوك اليوم يا غالي؟", # Libyan Arabic script
"يكانك اتدير الفظة ماهي الاوقية يالتك اتكيس البنك الشعبي", # Hassaniya
"عقد مجلس الأمن الدولي جلسة طارئة لمناقشة الأوضاع الراهنة.", # MSA
"Le gouvernement a annoncé de nouvelles mesures économiques.", # French
]
for r in model.identify(texts):
print(f"{r.language} ({r.confidence:.2%}) -> {list(r.prediction_set)}")
Representative output from the published release:
algerian (93.06%) -> ['algerian', 'moroccan']
algerian (77.02%) -> ['algerian', 'moroccan', 'tunisian', 'other_arabic']
moroccan (97.20%) -> ['moroccan']
tunisian (91.40%) -> ['tunisian']
libyan (88.10%) -> ['libyan']
hassaniya (83.50%) -> ['hassaniya']
msa (99.68%) -> ['msa']
french (99.40%) -> ['french']
Adjusting conformal coverage: At the default 99% target coverage (alpha = 0.01), any class with adjusted probability >= 1.96% enters prediction_set to preserve the coverage guarantee on borderline Maghrebi texts. Use r.language for the argmax single label, or lower target_coverage (e.g., 0.95) for tighter sets:
for r in model.identify(texts, target_coverage=0.95):
print(f"{r.language} ({r.confidence:.2%}) -> {list(r.prediction_set)}")
Evaluation
Evaluated on sealed benchmark partitions from one training run and one deterministic evaluation pass (seed 42).
Test Split (12,990 Rows, 8 Classes)
| Class | Support | Precision | Recall | F1 |
|---|---|---|---|---|
French (french) |
1,500 | 99.40% | 100.00% | 99.70% |
MSA (msa) |
1,861 | 98.36% | 99.68% | 99.01% |
Tunisian (tunisian) |
2,000 | 96.81% | 92.50% | 94.60% |
Other Arabic (other_arabic) |
2,000 | 94.67% | 86.15% | 90.21% |
Libyan (libyan) |
1,078 | 83.98% | 96.29% | 89.71% |
Hassaniya (hassaniya) |
552 | 81.35% | 94.02% | 87.23% |
Algerian (algerian) |
1,999 | 91.89% | 82.14% | 86.74% |
Moroccan (moroccan) |
2,000 | 81.57% | 89.85% | 85.51% |
| Macro Average | 12,990 | 91.00% | 92.58% | 91.59% |
| Overall Accuracy | 12,990 | — | — | 91.79% |
Validation Split (9,389 Rows)
| Metric | Value |
|---|---|
| Accuracy | 93.64% |
| Macro F1 | 94.08% |
Conformal empirical coverage (99% target, alpha = 0.01) |
99.13% |
| Conformal mean set size | 2.02 classes |
Note on anchor suites: DziriEval and MADAR do not span these 8 Maghrebi/Hassaniya/French classes, so no anchor-suite row applies.
Intended Use & Scope
Recommended uses:
- Dialect routing, corpus provenance screening, and dataset filtering for Maghrebi NLP pipelines.
- High-recall corpus curation using the 99%-coverage conformal set (retaining texts whenever
algerianappears inprediction_set).
Out-of-scope uses:
- Text generation (classification-only architecture).
- Fine-grained identification of Eastern Arabic varieties (Levantine, Gulf, Egyptian, and Mesopotamian Arabic are grouped into
other_arabic). - Automated decisions affecting individuals.
This model has not been evaluated for bias, toxicity, or factuality.
Architecture
| Component | Specification |
|---|---|
| Parameters | 5,191,048 (20.8 MB float32) |
| Input | Raw UTF-8 bytes, vocabulary 258, max 512 bytes (no tokenizer) |
| Conv Stem | 3 parallel depthwise-separable branches (kernels 5 / 9 / 13, dim 128 each) |
| Transformer | 6 layers, 8 heads, embed dim 256, SwiGLU FFN dim 704, RoPE positions |
| Pooling & Head | Attentive pooling + scaled cosine classifier with LDAM per-class margins |
| Training Projection | 128-dim projection head for supervised contrastive loss (training only) |
| Calibration | Balanced Softmax prior shift (tau = 0.5) + split conformal quantile (q_hat = 0.9804) |
| Release Precision | float32 |
Operating directly on UTF-8 bytes avoids subword fragmentation and out-of-vocabulary failures across Arabic script, Arabizi digits (3, 7, 9), and French orthography, while the multi-scale ConvStem captures local character, affix, and clitic n-grams before global Transformer self-attention.
Training Data
Built from 13 sources totaling 337,742 raw rows: 62,514 were removed by length/noise filtering (<3 words or >512 bytes), 17,752 exact duplicates were removed, and 146,415 rows were allocated (seed 42) into 124,036 train, 9,389 validation, and 12,990 sealed test rows (zero hash collisions between test and train/dev).
| Source | Domain / Dialect | Tier | Train Rows | Share |
|---|---|---|---|---|
evageon/IADD |
North African & Arab dialects | Permissive (CC-BY-4.0) | 29,734 | 23.97% |
wikimedia/wikipedia-ar |
Modern Standard Arabic | Restricted (CC-BY-SA-4.0) | 16,743 | 13.50% |
wikimedia/wikipedia-fr |
French articles (sample) | Restricted (CC-BY-SA-4.0) | 15,000 | 12.09% |
zenodo/TUNIZI-v2 |
Tunisian Arabizi comments | Permissive (CC-BY-4.0) | 14,706 | 11.86% |
ayoubkirouane/Algerian-Darija |
Algerian Darija | Permissive (CC-BY-4.0) | 11,061 | 8.92% |
MouadJb/MYC |
Moroccan YouTube comments | Unknown (research citation) | 10,501 | 8.47% |
LeMGarouani/MAC |
Moroccan tweets | Unknown (research citation) | 6,692 | 5.40% |
Mansour-Essgaer/Libya-Telecom |
Libyan telecom tweets | Unknown (research use) | 6,005 | 4.84% |
DarjaCore/algerian-darja-sample |
Algerian Darija Telegram | Permissive (MIT) | 5,929 | 4.78% |
Mansour-Essgaer/Libyan-Resturant |
Libyan restaurant reviews | Unknown (research use) | 2,976 | 2.40% |
Amin-tech99/ai-for-rim |
Hassaniya / Mauritanian | Permissive (MIT) | 2,505 | 2.02% |
tunis-ai/tsac |
Tunisian social comments | Unknown (research use) | 1,378 | 1.11% |
mendeley/HASSANIYA |
Hassaniya Facebook comments | Permissive (CC-BY-4.0) | 806 | 0.65% |
| Total | 8 classes | Mixed | 124,036 | 100.00% |
- Class breakdown (train):
algerian20,001 (16.13%),moroccan20,000 (16.12%),tunisian20,000 (16.12%),other_arabic20,000 (16.12%),msa16,743 (13.50%),french15,000 (12.09%),libyan8,981 (7.24%),hassaniya3,311 (2.67%). - Script breakdown (train): Arabic script 84,535 (68.15%), Latin / Arabizi 36,539 (29.46%), mixed 2,962 (2.39%).
Training Data License Composition
The model weights are released under CC-BY-SA-4.0. That grant applies to weights and code only and does not relicense the underlying training text.
| Tier | Train Rows | Share | Sources |
|---|---|---|---|
| Permissive | 64,741 | 52.19% | CC-BY-4.0 (IADD, TUNIZI-v2, Algerian-Darija, HASSANIYA), MIT (DarjaCore, ai-for-rim) |
| Restricted | 31,743 | 25.59% | CC-BY-SA-4.0 (Wikipedia Arabic & French) |
| Unknown | 27,552 | 22.21% | Research sources without formal open licenses (MYC, MAC, Libya-Telecom, Libyan-Resturant, TSAC) |
Across all 146,415 split rows: 51.76% permissive (75,777), 25.61% restricted (37,499), and 22.63% unknown (33,139).
Training Details
| Parameter | Value |
|---|---|
| Objective | Cross-entropy (label smoothing 0.05) + SupCon (weight 0.5, temp 0.05, bank 2,048) + Group-DRO (step 0.01) + LDAM margins |
| Optimizer | AdamW (lr=3e-4, weight_decay=0.01, max_grad_norm=1.0) |
| Batch Size | 12 groups × 10 samples = 120 per step |
| Schedule | Cosine decay with warm-up (deferred share 0.5) |
| Epochs / Steps | 8 epochs (8,264 steps; best validation checkpoint at step 6,966: 93.71% acc, 94.10% macro F1) |
| Augmentation | Phrase-level mixing (p=0.35), case augmentation (p=0.10) |
| Inference Prior Shift | Balanced Softmax logit adjustment (tau = 0.5) |
| Precision | float32 |
| Hardware & Duration | 1× NVIDIA Tesla T4 (16 GB), 2.99 h (10,783 s) |
| Seed | 42 |
Limitations
- Single training run. All numbers reflect one run under seed 42; variance across seeds is unmeasured.
- Cross-Maghrebi Arabizi overlap. Short greetings shared across borders (
wesh,khoya,chwiya,sahbi) naturally produce multi-dialect conformal sets. - Low-resource Hassaniya coverage. Only 3,311 training rows (552 test rows); performance may degrade on Hassaniya text stylistically distant from the training sources.
- Distribution-dependent conformal guarantee. The conformal quantile (
q_hat = 0.9804) was calibrated on half of the 9,389-row validation split; the >= 99% coverage guarantee applies under exchangeability with that distribution. - 8-class boundary. Eastern Arabic varieties collapse into
other_arabic. - No safety evaluation. Bias, toxicity, and factuality have not been evaluated.
Repository Contents
| File | Size | Description / SHA-256 |
|---|---|---|
model.safetensors |
20.8 MB | Float32 weights (5,191,048 params) · SHA-256 33ebc03ba642064ce4f3663aa0a46f12dd5dd89eb0e9e065a40286606c5f5e8f |
config.json |
1.4 KB | Architecture config, auto_map, and calibration constants |
modeling_numalia.py |
17.7 KB | Self-contained model implementation and identify API |
calibration.json |
551 B | Conformal q_hat, target coverage, prior-shift vector, and class labels |
export_report.json |
944 B | Verified file sizes, SHA-256 checksums, parameter count, and reload parity (max_confidence_diff = 0.0) |
Reload verification confirms zero missing or unexpected keys and exact prediction parity (max_confidence_diff = 0.0).
The name
NUMALIA folds the classical names of the peoples across North Africa and the Sahara into one word: NUmidia (Algeria), MAuretania (Morocco and Mauritania), LIbu (Libya), and Afri (Tunisia). The classifier carries their shared name because it models the continuum of Maghrebi and Hassaniya Arabic varieties side by side in a single calibrated space.
Citation
@misc{algerian_nlp_numalia_2026,
title = {NUMALIA: Byte-Level Dialect Identification for Maghrebi Arabic},
author = {Ainouche, Abderahmane and Algerian NLP Collective},
year = {2026},
url = {https://huggingface.co/algerian-nlp/NUMALIA}
}
License
Weights and code are released under CC-BY-SA-4.0. Refer to the Training Data License Composition section above regarding the underlying training text before redistributing derivatives.
- Downloads last month
- 49
Collection including algerian-nlp/NUMALIA
Evaluation results
- Accuracy (test, 8 classes, single run) on LID benchmark (test, 12,990 rows)self-reported0.918
- Macro F1 (test, 8 classes, single run) on LID benchmark (test, 12,990 rows)self-reported0.916
- Accuracy (validation, 8 classes, single run) on LID benchmark (validation, 9,389 rows)self-reported0.936
- Macro F1 (validation, 8 classes, single run) on LID benchmark (validation, 9,389 rows)self-reported0.941