Sumair-Parveiz/code-switching-codesaviours-si26-sumair

Language Identification Model for Code-Switched Roman Urdu & English

This model is a fine-tuned version of XLM-RoBERTa (base) trained for token-level language identification in Roman Urdu and English text.

Intended Use & Task

• Task: Token Classification (Language Identification)
• Labels:
  • 0 / URD: Roman Urdu token
  • 1 / ENG: English token
  • 2 / MIX: Code-switched / Hybrid / Localized token

Performance & Evaluation Metrics

Evaluated on test set tokens:

Class	Precision	Recall	F1-Score
URD	1.00	0.98	0.99
ENG	0.96	1.00	0.98
MIX	0.89	0.80	0.84
Accuracy	—	—	0.98

How to Use

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_id = "Sumair-Parveiz/code-switching-codesaviours-si26-sumair"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

nlp = pipeline("token-classification", model=model, tokenizer=tokenizer)
example = "Main abhi sochoing mode mein hoon"
print(nlp(example))