Update README.md
Browse files
README.md
CHANGED
|
@@ -2,4 +2,51 @@
|
|
| 2 |
language:
|
| 3 |
- ar
|
| 4 |
pipeline_tag: visual-question-answering
|
| 5 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
language:
|
| 3 |
- ar
|
| 4 |
pipeline_tag: visual-question-answering
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
# Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic
|
| 8 |
+
|
| 9 |
+
Dallah is an advanced multimodal large language model (MLLM) tailored for the Arabic language, with a specific focus on understanding and generating content across various Arabic dialects. Built upon the **LLaVA** framework and powered by the **LLaMA-2** architecture, Dallah integrates both textual and visual data to facilitate comprehensive multimodal interactions.
|
| 10 |
+
|
| 11 |
+
## Model Details
|
| 12 |
+
|
| 13 |
+
- **Architecture**: LLaVA-based multimodal model with LLaMA-2 backbone.
|
| 14 |
+
- **Languages Supported**: Modern Standard Arabic (MSA) and six major Arabic dialects.
|
| 15 |
+
- **Modalities**: Text and image.
|
| 16 |
+
|
| 17 |
+
## Training Data
|
| 18 |
+
|
| 19 |
+
Dallah was fine-tuned on a diverse dataset encompassing both textual and visual information:
|
| 20 |
+
- **Textual Data**: Includes MSA and six prominent Arabic dialects, ensuring the model's proficiency across different regional linguistic variations.
|
| 21 |
+
- **Visual Data**: Comprised of image-text pairs, enabling the model to process and generate content that integrates both modalities.
|
| 22 |
+
|
| 23 |
+
## Performance
|
| 24 |
+
|
| 25 |
+
Dallah demonstrates state-of-the-art performance in Arabic MLLMs:
|
| 26 |
+
- Excels in both MSA and dialectal Arabic benchmarks.
|
| 27 |
+
- Effectively handles complex multimodal interactions involving textual and visual elements.
|
| 28 |
+
|
| 29 |
+
## Applications
|
| 30 |
+
|
| 31 |
+
Dallah’s multimodal and dialect-aware capabilities make it suitable for a range of applications, including:
|
| 32 |
+
- **Multilingual Chatbots**: Enhancing user interactions by understanding and responding in specific Arabic dialects.
|
| 33 |
+
- **Content Creation**: Assisting in generating culturally and linguistically appropriate content for diverse Arabic-speaking audiences.
|
| 34 |
+
- **Educational Tools**: Supporting language learning by providing examples and explanations in various dialects.
|
| 35 |
+
- **Cultural Preservation**: Documenting and promoting the use of different Arabic dialects on digital platforms.
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
## Citation
|
| 39 |
+
|
| 40 |
+
If you use Dallah in your research or applications, please cite the following paper:
|
| 41 |
+
|
| 42 |
+
```bibtex
|
| 43 |
+
@inproceedings{alwajih2024dallah,
|
| 44 |
+
title={Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic},
|
| 45 |
+
author={Alwajih, Fakhraddin and Bhatia, Gagan and Abdul-Mageed, Muhammad},
|
| 46 |
+
booktitle={Proceedings of The Second Arabic Natural Language Processing Conference},
|
| 47 |
+
pages={320--336},
|
| 48 |
+
year={2024},
|
| 49 |
+
address={Bangkok, Thailand},
|
| 50 |
+
publisher={Association for Computational Linguistics},
|
| 51 |
+
url={https://aclanthology.org/2024.arabicnlp-1.27}
|
| 52 |
+
}
|