AfnanTS/ARBERTv2_ArLAMA

ARBERTv2_ArLAMA is a transformer-based Arabic language model fine-tuned on Masked Language Modeling (MLM) tasks. The model uses Knowledge Graphs (KGs) to enhance its understanding of semantic relations and improve its performance in various Arabic NLP tasks.

Uses

Direct Use

Filling masked tokens in Arabic text, particularly in contexts enriched with knowledge from KGs.

Downstream Use

Can be further fine-tuned for Arabic NLP tasks that require semantic understanding, such as text classification or question answering.

How to Get Started with the Model

from transformers import pipeline
fill_mask = pipeline("fill-mask", model="AfnanTS/ARBERTv2_ArLAMA")
fill_mask("اللغة [MASK] مهمة جدا."

Training Details

Training Data

Trained on the ArLAMA dataset, which is designed to represent Knowledge Graphs in natural language.

Training Procedure

Continued pre-training of ArBERTv2 using Masked Language Modeling (MLM) tasks, integrating structured knowledge from Knowledge Graphs.