| --- |
| library_name: transformers |
| datasets: |
| - HeshamHaroon/saudi-dialect-conversations |
| base_model: |
| - LiquidAI/LFM2.5-1.2B-Instruct |
| --- |
| |
| # Saudi Dialect LFM2.5 — Instruction-Tuned Arabic Dialect Model |
|
|
| ## Model Description |
|
|
| This model is a fine-tuned version of **Liquid AI**’s **LFM2.5‑1.2B‑Instruct**, adapted for Saudi dialect conversational generation. |
|
|
| The base model belongs to the LFM2.5 family — hybrid state-space + attention language models designed for **fast on-device inference**,low memory usage, and strong performance relative to size. It contains ~1.17B parameters, 32k context length, and supports multilingual generation including Arabic. |
|
|
| This fine-tuned variant specializes the model for **Saudi dialect conversational patterns**, improving fluency, dialect authenticity, and instruction following for regional Arabic use cases. |
|
|
| --- |
|
|
| ## Intended Use |
|
|
| ### Primary Use Cases |
|
|
| * Saudi dialect chatbots |
| * Customer support assistants |
| * Conversational agents |
| * Arabic NLP research |
| * Dialect-aware RAG pipelines |
| * Dialogue generation systems |
|
|
| ### Out-of-Scope Uses |
|
|
| * Legal/medical advice |
| * Safety-critical decision making |
| * High-precision knowledge tasks without retrieval |
| * Sensitive content generation |
|
|
| --- |
|
|
| ## Training Details |
|
|
| ### Base Model |
|
|
| * Architecture: Hybrid state-space + attention |
| * Parameters: ~1.17B |
| * Context length: 32,768 tokens |
| * Training tokens: ~28T |
| * Languages: Multilingual including Arabic |
|
|
| --- |
|
|
| ### Dataset |
|
|
| Fine-tuned on: |
|
|
| **Dataset:** |
| `HeshamHaroon/saudi-dialect-conversations` |
|
|
| **Domain:** |
| Conversational dialogue |
|
|
| **Language:** |
| Saudi dialect Arabic |
|
|
| **Format:** |
| Instruction → Response pairs |
|
|
| **Purpose:** |
| Increase dialect authenticity and conversational naturalness. |
|
|
| --- |
|
|
| ### Training Configuration |
|
|
| (Extracted from training notebook) |
|
|
| | Parameter | Value | |
| | --------------------- | ---------------------------- | |
| | Epochs | 4 | |
| | Learning Rate | 2e-4 | |
| | Batch Size | 16 | |
| | Gradient Accumulation | 4 | |
| | Optimizer | AdamW | |
| | LR Scheduler | Linear | |
| | Warmup Ratio | 0.03 | |
| | Sequence Length | 8096 | |
| | Precision | FP16 | |
| | Training Type | Supervised Fine-Tuning (SFT) | |
|
|
| --- |
|
|
| ### Training Procedure |
|
|
| Training was performed using: |
|
|
| * Transformers |
| * TRL SFTTrainer |
| * LoRA fine-tuning |
| * Mixed precision |
| * Gradient accumulation |
|
|
| The base model weights were adapted rather than retrained from scratch. |
|
|
| --- |
|
|
| ## Evaluation |
|
|
|
|
|
|
| Qualitative evaluation indicates: |
|
|
| * Improved dialect fluency |
| * Reduced MSA leakage |
| * Better conversational tone |
| * Higher lexical authenticity |
|
|
| Dialect-specific fine-tuning is known to significantly increase dialect generation accuracy and reduce standard-Arabic drift in Arabic LLMs. |
|
|
| --- |
|
|
| ## Performance Characteristics |
|
|
| **Strengths** |
|
|
| * Very fast inference |
| * Low memory footprint |
| * Strong conversational coherence |
| * Good instruction following |
|
|
| **Limitations** |
|
|
| * Smaller model → limited factual depth |
| * May hallucinate |
| * Less capable for complex reasoning vs larger models |
| * Dialect bias toward Saudi Arabic |
|
|
| --- |
|
|
| ## Bias, Risks, and Safety |
|
|
| Potential risks: |
|
|
| * Dialect bias |
| * Cultural bias from dataset |
| * Toxic outputs if prompted maliciously |
| * Hallucinated facts |
|
|
| Mitigations: |
|
|
| * Filtering dataset |
| * Instruction alignment |
| * Moderation layers recommended |
|
|
| --- |
|
|
| ## Hardware Requirements |
|
|
| Runs efficiently on: |
|
|
| * CPU inference (<1GB memory quantized) |
| * Mobile NPUs |
| * Edge devices |
|
|
| --- |
|
|
| ## Example Usage |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| |
| model_id = "AyoubChLin/lfm2.5-saudi-dialect" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id) |
| |
| prompt = "تكلم باللهجة السعودية عن القهوة" |
| |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=200) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| --- |
|
|
| ## Training Compute |
|
|
| * **GPU:** 1 × NVIDIA A100 (40 GB VRAM) |
| * **CPU:** 8 cores |
| * **RAM:** 16 GiB |
| * **Compute Environment:** Cloud training instance |
|
|
| --- |
|
|
| ## License |
|
|
| Same as base model license unless otherwise specified. |
|
|
| --- |
|
|
| ## Citation |
|
|
| If you use this model: |
|
|
| ``` |
| @misc{saudi-dialect-lfm2.5, |
| author = {Cherguelaine Ayoub}, |
| title = {Saudi Dialect LFM2.5}, |
| year = {2026}, |
| publisher = {Hugging Face} |
| } |
| ``` |
|
|
| --- |
|
|
| ## Acknowledgments |
|
|
| * Liquid AI for base model |
| * Dataset creators |
| * Open-source tooling ecosystem |
|
|
| --- |
