README.md

---
library_name: transformers
datasets:
- HeshamHaroon/saudi-dialect-conversations
base_model:
- LiquidAI/LFM2.5-1.2B-Instruct
---

# Saudi Dialect LFM2.5 — Instruction-Tuned Arabic Dialect Model

## Model Description

This model is a fine-tuned version of **Liquid AI**’s **LFM2.5‑1.2B‑Instruct**, adapted for Saudi dialect conversational generation.

The base model belongs to the LFM2.5 family — hybrid state-space + attention language models designed for **fast on-device inference**,low memory usage, and strong performance relative to size. It contains ~1.17B parameters, 32k context length, and supports multilingual generation including Arabic.

This fine-tuned variant specializes the model for **Saudi dialect conversational patterns**, improving fluency, dialect authenticity, and instruction following for regional Arabic use cases.

---

## Intended Use

### Primary Use Cases

* Saudi dialect chatbots
* Customer support assistants
* Conversational agents
* Arabic NLP research
* Dialect-aware RAG pipelines
* Dialogue generation systems

### Out-of-Scope Uses

* Legal/medical advice
* Safety-critical decision making
* High-precision knowledge tasks without retrieval
* Sensitive content generation

---

## Training Details

### Base Model

* Architecture: Hybrid state-space + attention
* Parameters: ~1.17B
* Context length: 32,768 tokens
* Training tokens: ~28T
* Languages: Multilingual including Arabic

---

### Dataset

Fine-tuned on:

**Dataset:**
`HeshamHaroon/saudi-dialect-conversations`

**Domain:**
Conversational dialogue

**Language:**
Saudi dialect Arabic

**Format:**
Instruction → Response pairs

**Purpose:**
Increase dialect authenticity and conversational naturalness.

---

### Training Configuration

(Extracted from training notebook)

| Parameter             | Value                        |
| --------------------- | ---------------------------- |
| Epochs                | 4                            |
| Learning Rate         | 2e-4                         |
| Batch Size            | 16                            |
| Gradient Accumulation | 4                            |
| Optimizer             | AdamW                        |
| LR Scheduler          | Linear                       |
| Warmup Ratio          | 0.03                         |
| Sequence Length       | 8096                         |
| Precision             | FP16                         |
| Training Type         | Supervised Fine-Tuning (SFT) |

---

### Training Procedure

Training was performed using:

* Transformers
* TRL SFTTrainer
* LoRA fine-tuning
* Mixed precision
* Gradient accumulation

The base model weights were adapted rather than retrained from scratch.

---

## Evaluation



Qualitative evaluation indicates:

* Improved dialect fluency
* Reduced MSA leakage
* Better conversational tone
* Higher lexical authenticity

Dialect-specific fine-tuning is known to significantly increase dialect generation accuracy and reduce standard-Arabic drift in Arabic LLMs.

---

## Performance Characteristics

**Strengths**

* Very fast inference
* Low memory footprint
* Strong conversational coherence
* Good instruction following

**Limitations**

* Smaller model → limited factual depth
* May hallucinate
* Less capable for complex reasoning vs larger models
* Dialect bias toward Saudi Arabic

---

## Bias, Risks, and Safety

Potential risks:

* Dialect bias
* Cultural bias from dataset
* Toxic outputs if prompted maliciously
* Hallucinated facts

Mitigations:

* Filtering dataset
* Instruction alignment
* Moderation layers recommended

---

## Hardware Requirements

Runs efficiently on:

* CPU inference (<1GB memory quantized)
* Mobile NPUs
* Edge devices

---

## Example Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "AyoubChLin/lfm2.5-saudi-dialect"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "تكلم باللهجة السعودية عن القهوة"

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Training Compute

* **GPU:** 1 × NVIDIA A100 (40 GB VRAM)
* **CPU:** 8 cores
* **RAM:** 16 GiB
* **Compute Environment:** Cloud training instance

---

## License

Same as base model license unless otherwise specified.

---

## Citation

If you use this model:

```
@misc{saudi-dialect-lfm2.5,
  author = {Cherguelaine Ayoub},
  title = {Saudi Dialect LFM2.5},
  year = {2026},
  publisher = {Hugging Face}
}
```

---

## Acknowledgments

* Liquid AI for base model
* Dataset creators
* Open-source tooling ecosystem

---