MedAdil
/

BERTal

@@ -1,206 +1,189 @@
 ---
-base_model: AbderrahmanSkiredj1/BERTouch
-library_name: peft
 tags:
-- base_model:adapter:AbderrahmanSkiredj1/BERTouch
 - lora
-- transformers
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]
-### Framework versions
-- PEFT 0.18.1

 ---
+language:
+- ar
+- fr
 tags:
+- text-classification
+- darija
+- arabic
+- moroccan-arabic
+- arabizi
+- nlp
 - lora
+- peft
+- bertouch
+license: mit
+base_model: AbderrahmanSkiredj1/BERTouch
+pipeline_tag: text-classification
 ---
+# 🇲🇦 BERTal — Classification Sémantique de Messages en Darija Marocaine
+> **Comprendre la Darija, l'Arabizi et le Français — là où les modèles classiques échouent.**
+BERTal est un modèle de classification de texte fine-tuné sur **BERTouch** via la méthode **LoRA**,
+spécialisé dans la compréhension sémantique des messages rédigés en **Darija marocaine**,
+en **Arabizi** (écriture latine du dialecte) et en **Français** — y compris leurs mélanges
+(*Code-switching*), phénomène omniprésent dans la communication numérique marocaine.
+---
+## 🎯 Tâche
+Classification automatique de messages en **7 catégories métier** :
+| Catégorie | Description |
+|---|---|
+| 🔵 Recrutement | Offres d'emploi, CV, entretiens, stages |
+| 🟢 Personnel | Famille, amis, échanges informels |
+| 🟠 Commercial | Vente, achat, prix, promotions |
+| 🟣 Administratif | Documents officiels, CNSS, CIN, commune |
+| 🩵 Education | Examens, cours, université, concours |
+| 🔴 Santé | Médecin, pharmacie, hôpital, médicaments |
+| ⚫ Autre | Sport, divertissement, actualités |
+---
+## 🧠 Pourquoi BERTal ?
+La **Darija marocaine** est l'une des langues les plus parlées au Maghreb,
+utilisée quotidiennement par plus de **40 millions de personnes**,
+mais quasi absente des ressources NLP existantes. Elle présente trois défis majeurs :
+- ❌ **Aucune orthographe standardisée** — chaque locuteur écrit à sa façon
+- ❌ **Arabizi omniprésent** — mélange de chiffres et de lettres latines (`3`, `7`, `9`)
+- ❌ **Code-switching permanent** — Darija + Français dans la même phrase
+BERTal relève ces défis en s'appuyant sur **BERTouch**, le seul modèle BERT
+pré-entraîné nativement sur la Darija marocaine, et en l'adaptant via **LoRA**
+sur un dataset multilingue de **351 573 phrases** couvrant les trois scripts.
+---
+## 📊 Performances
+| Évaluation | Score |
+|---|---|
+| Accuracy (validation, 52 736 phrases) | **81.34%** |
+| Accuracy (jeu de test inédit 70 phrases multilingues) | **74.3%** |
+| Baseline sans fine-tuning | 11.4% |
+| Gain total | **+62.9 points** |
+### Résultats par catégorie (jeu de test 70 phrases)
+| Catégorie | Score |
+|---|---|
+| Recrutement | 90% |
+| Santé | 90% |
+| Commercial | 80% |
+| Autre | 80% |
+| Personnel | 60% |
+| Administratif | 60% |
+| Education | 60% |
+---
+## 🗂️ Dataset d'entraînement
+Le modèle a été entraîné sur un dataset multilingue construit en 4 versions itératives
+à partir du corpus [ATLASIA](https://huggingface.co/datasets/atlasia/moroccan_darija_domain_classifier_dataset),
+enrichi par génération synthétique via **Gemini 2.5 Flash**.
+| Script | Lignes | % |
+|---|---|---|
+| Arabe (Darija) | 191 654 | 54.5% |
+| Arabizi | 107 567 | 30.6% |
+| Français | 52 352 | 14.9% |
+| **Total** | **351 573** | **100%** |
+Déséquilibre inter-classes : **1.35x** (quasi équilibré naturellement).
+---
+## ⚙️ Configuration LoRA
+```python
+LoraConfig(
+    task_type      = TaskType.SEQ_CLS,
+    r              = 16,
+    lora_alpha     = 32,
+    lora_dropout   = 0.1,
+    target_modules = ["query", "value"]
+)
+```
+- **Paramètres entraînables** : 595 207 / 135 793 934 (**0.44%**)
+- **Époques** : 3
+- **GPU** : Tesla T4 (Google Colab)
+- **Batch size** : 32
+---
+## 🚀 Utilisation
+```python
+import torch
+import torch.nn.functional as F
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+from peft import PeftModel
+CATEGORIES = [
+    'Recrutement', 'Personnel', 'Commercial',
+    'Administratif', 'Education', 'Sante', 'Autre'
+]
+label2id = {label: idx for idx, label in enumerate(CATEGORIES)}
+id2label = {idx: label for idx, label in enumerate(CATEGORIES)}
+# Charger le modèle
+tokenizer   = AutoTokenizer.from_pretrained("MedAdil/BERTal")
+base_model  = AutoModelForSequenceClassification.from_pretrained(
+    "AbderrahmanSkiredj1/BERTouch",
+    num_labels              = 7,
+    id2label                = id2label,
+    label2id                = label2id,
+    ignore_mismatched_sizes = True
+)
+model = PeftModel.from_pretrained(base_model, "MedAdil/BERTal")
+model.eval()
+# Classifier un message
+def classifier(texte: str) -> dict:
+    inputs  = tokenizer(texte, return_tensors="pt", truncation=True, max_length=128)
+    with torch.no_grad():
+        logits  = model(**inputs).logits
+    probas       = F.softmax(logits, dim=-1)[0]
+    scores       = {id2label[i]: round(float(p), 4) for i, p in enumerate(probas)}
+    label_predit = max(scores, key=scores.get)
+    return {"label": label_predit, "confidence": scores[label_predit], "scores": scores}
+# Exemples
+print(classifier("bghit nkhdem f had chrika IT"))
+# → {'label': 'Recrutement', 'confidence': 0.9935, ...}
+print(classifier("خصني رونديفو عند الطبيب"))
+# → {'label': 'Sante', 'confidence': 0.9904, ...}
+print(classifier("Je cherche un appartement à Oujda"))
+# → {'label': 'Commercial', 'confidence': 0.9942, ...}
+```
+---
+## 📝 Citation
+```bibtex
+@misc{mani2026bertal,
+  author    = {Mohammed Adil MANI},
+  title     = {BERTal: LoRA Fine-tuning of BERTouch for Multilingual
+               Moroccan Darija Message Classification},
+  year      = {2026},
+  publisher = {Hugging Face},
+  url       = {https://huggingface.co/MedAdil/BERTal}
+}
+```
+---
+*BERTal — Bringing NLP to the language of millions 🇲🇦*