SindhiLM: A Specialized GPT-2 Model for Sindhi

SindhiLM is a causal language model trained from scratch to provide high-quality text generation for the Sindhi language. It significantly outperforms general multilingual models by focusing specifically on Sindhi morphology and syntax.

Model Details

• Developed by: Aakash Meghwar
• Model type: Causal Language Model (GPT-2 architecture)
• Language: Sindhi (sd)
• Library Name: Transformers
• Base Model: openai-community/gpt2

Evaluation Results

The model was evaluated using Perplexity (PPL) on a held-out Sindhi test set. Lower perplexity indicates a better understanding of the language.

Model	Perplexity (Lower is Better)
mBERT (Baseline)	2,360,312
GPT-2 (Base)	500,000
SindhiLM (Ours)	212,503

Training Data

The model was trained on the Sindhi Mega Corpus, consisting of approximately 118 million tokens. This dataset includes diverse Sindhi literature, news, and web content.

How to Get Started

You can use this model directly with the Hugging Face transformers library:

from transformers import pipeline

# Load the model
generator = pipeline("text-generation", model="aakashMeghwar01/SindhiLM")

# Generate Sindhi text
prompt = "سنڌ جي ثقافت"
output = generator(prompt, max_new_tokens=20, do_sample=True, temperature=0.7)

print(output[0]['generated_text'])