README.md

---
language:
- ar
- fa
- he
- bn
- ru
- zh
- ko
- es

license: apache-2.0
library_name: transformers
pipeline_tag: fill-mask

tags:
- multilingual
- transliteration
- hate-speech
- xlm-roberta
- transfer-learning
- masked-language-modeling
---

# XLM_Latin

## Model Description

XLM_Latin is a transliteration-aware multilingual language model developed
to investigate the impact of script unification on multilingual and
cross-lingual hate speech detection. The model is based on the
XLM-RoBERTa architecture and was further pretrained on multilingual
transliterated corpora represented in a unified Latin script.

## Intended Uses

The model is intended for:

- Multilingual hate speech detection
- Cross-lingual transfer learning
- Zero-shot learning
- Transliteration-based NLP research
- Multilingual representation learning

## Languages

The model was trained on transliterated text from:

- Arabic
- Persian
- Hebrew
- Bengali
- Russian
- Chinese
- Korean
- Spanish

## Training Data

The model was pretrained on 360,768 transliterated text samples collected
from multiple sources, including news articles, sentiment datasets,
social media content, and public corpora.

## Model Architecture

- Base model: XLM-RoBERTa-base
- Model type: Masked Language Model (MLM)
- Tokenizer: SentencePiece (Unigram)
- Vocabulary size: 32,000

## Evaluation

The model was evaluated using:

- Monolingual learning
- Cross-lingual transfer learning
- Zero-shot transfer learning
- Multilingual learning

## Limitations

- Performance depends on transliteration quality.
- The model may inherit biases from training data.
- Intended primarily for research purposes.

## Ethical Considerations

The model processes hate speech data that may contain offensive content.
It should not be used as the sole decision-making system in high-risk
applications.


## How to Use the Model

### Installation

Install the required libraries:

```bash
pip install transformers torch sentencepiece
```

### Load the Model and Tokenizer

The pretrained XLM_Latin model can be loaded directly from the Hugging Face Hub:

```python
from transformers import AutoTokenizer, AutoModelForMaskedLM

model_name = "GhadeerALbadani/XLM_Latin"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name)
```

Replace `YOUR_USERNAME` with your Hugging Face username or organization name.

---

### Example: Tokenization

The tokenizer can be used to convert transliterated multilingual text into model inputs:

```python
text = "ana la uhibbu hadha almujtama"

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    padding=True
)

print(inputs)
```

---

### Example: Extracting Contextual Representations

The model can be used to generate contextual embeddings for transliterated multilingual text:

```python
from transformers import AutoTokenizer, AutoModel

model_name = "GhadeerALbadani/XLM_Latin"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

text = "ana la uhibbu hadha almujtama"

inputs = tokenizer(
    text,
    return_tensors="pt"
)

outputs = model(**inputs)

embeddings = outputs.last_hidden_state

print(embeddings.shape)
```

---

### Example: Masked Language Prediction

Since XLM_Latin is pretrained using the Masked Language Modeling (MLM) objective, it can predict masked tokens:

```python
from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model="GhadeerALbadani/XLM_Latin",
    tokenizer="GhadeerALbadani/XLM_Latin"
)

result = fill_mask(
    "ana <mask> hadha almujtama"
)

print(result)
```

---

### Fine-Tuning for Hate Speech Detection

XLM_Latin is intended to serve as a pretrained backbone model and can be further fine-tuned for multilingual hate speech detection tasks:

```python
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    "YOUR_USERNAME/XLM_Latin",
    num_labels=2
)
```

The resulting fine-tuned model can then be used for monolingual, multilingual, cross-lingual, and zero-shot hate speech detection experiments.


## Citation

```bibtex
@mastersthesis{XLM_Latin_2026,
  title={A Multilingual Model for Detecting Hate Speech on Unknown Language Script},
  author={Author Name},
  year={2026},
  school={University Name}
}