language:
- ar
- fa
- he
- bn
- ru
- zh
- ko
- es
license: apache-2.0
library_name: transformers
pipeline_tag: fill-mask
tags:
- multilingual
- transliteration
- hate-speech
- xlm-roberta
- transfer-learning
- masked-language-modeling
XLM_Latin
Model Description
XLM_Latin is a transliteration-aware multilingual language model developed to investigate the impact of script unification on multilingual and cross-lingual hate speech detection. The model is based on the XLM-RoBERTa architecture and was further pretrained on multilingual transliterated corpora represented in a unified Latin script.
Intended Uses
The model is intended for:
- Multilingual hate speech detection
- Cross-lingual transfer learning
- Zero-shot learning
- Transliteration-based NLP research
- Multilingual representation learning
Languages
The model was trained on transliterated text from:
- Arabic
- Persian
- Hebrew
- Bengali
- Russian
- Chinese
- Korean
- Spanish
Training Data
The model was pretrained on 360,768 transliterated text samples collected from multiple sources, including news articles, sentiment datasets, social media content, and public corpora.
Model Architecture
- Base model: XLM-RoBERTa-base
- Model type: Masked Language Model (MLM)
- Tokenizer: SentencePiece (Unigram)
- Vocabulary size: 32,000
Evaluation
The model was evaluated using:
- Monolingual learning
- Cross-lingual transfer learning
- Zero-shot transfer learning
- Multilingual learning
Limitations
- Performance depends on transliteration quality.
- The model may inherit biases from training data.
- Intended primarily for research purposes.
Ethical Considerations
The model processes hate speech data that may contain offensive content. It should not be used as the sole decision-making system in high-risk applications.
How to Use the Model
Installation
Install the required libraries:
pip install transformers torch sentencepiece
Load the Model and Tokenizer
The pretrained XLM_Latin model can be loaded directly from the Hugging Face Hub:
from transformers import AutoTokenizer, AutoModelForMaskedLM
model_name = "GhadeerALbadani/XLM_Latin"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name)
Replace YOUR_USERNAME with your Hugging Face username or organization name.
Example: Tokenization
The tokenizer can be used to convert transliterated multilingual text into model inputs:
text = "ana la uhibbu hadha almujtama"
inputs = tokenizer(
text,
return_tensors="pt",
truncation=True,
padding=True
)
print(inputs)
Example: Extracting Contextual Representations
The model can be used to generate contextual embeddings for transliterated multilingual text:
from transformers import AutoTokenizer, AutoModel
model_name = "GhadeerALbadani/XLM_Latin"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
text = "ana la uhibbu hadha almujtama"
inputs = tokenizer(
text,
return_tensors="pt"
)
outputs = model(**inputs)
embeddings = outputs.last_hidden_state
print(embeddings.shape)
Example: Masked Language Prediction
Since XLM_Latin is pretrained using the Masked Language Modeling (MLM) objective, it can predict masked tokens:
from transformers import pipeline
fill_mask = pipeline(
"fill-mask",
model="GhadeerALbadani/XLM_Latin",
tokenizer="GhadeerALbadani/XLM_Latin"
)
result = fill_mask(
"ana <mask> hadha almujtama"
)
print(result)
Fine-Tuning for Hate Speech Detection
XLM_Latin is intended to serve as a pretrained backbone model and can be further fine-tuned for multilingual hate speech detection tasks:
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained(
"YOUR_USERNAME/XLM_Latin",
num_labels=2
)
The resulting fine-tuned model can then be used for monolingual, multilingual, cross-lingual, and zero-shot hate speech detection experiments.
Citation
@mastersthesis{XLM_Latin_2026,
title={A Multilingual Model for Detecting Hate Speech on Unknown Language Script},
author={Author Name},
year={2026},
school={University Name}
}