README.md

---
library_name: transformers
license: apache-2.0
base_model: google/muril-base-cased
tags:
- ner
- token-classification
- indic
- multilingual
- marathi
- bengali
- telugu
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: muril-ner-multilingual
  results: []
---

# muril-ner-multilingual 🏷️

This model is a fine-tuned version of **[google/muril-base-cased](https://huggingface.co/google/muril-base-cased)** for **Named Entity Recognition (NER)** across three major Indic languages: **Marathi (mr)**, **Bengali (bn)**, and **Telugu (te)**.

It uses a joint multilingual full fine-tuning strategy to extract three primary entity types:
- 👤 **PER** (Person)
- 🏢 **ORG** (Organization)
- 📍 **LOC** (Location)

## Model Description

- **Developed by:** Learner-sai
- **Model Type:** Token Classification (NER)
- **Base Model:** `google/muril-base-cased`
- **Languages:** Marathi (`mr`), Bengali (`bn`), Telugu (`te`)
- **License:** Apache 2.0

MuRIL (Multilingual Representations for Indian Languages) was adapted by adding a 7-class linear sequence classification head (`O`, `B-PER`, `I-PER`, `B-ORG`, `I-ORG`, `B-LOC`, `I-LOC`). All parameters were updated during joint training across all three target languages to leverage cross-lingual transfer.

## Intended Uses & Limitations

### Intended Uses 🎯
- Automated entity extraction from news articles, social media, and documents in Marathi, Bengali, and Telugu.
- Downstream NLP applications such as information retrieval, question answering, and knowledge graph construction for Indic languages.

### Limitations ⚠️
- **Grammatical Suffixes:** In agglutinative or highly inflected languages like Marathi and Telugu, locative or case suffixes (e.g., Marathi `-त` in "दिल्लीत") may sometimes be included inside the predicted entity span.
- **Entity Scope:** The model is trained exclusively on `PER`, `ORG`, and `LOC` tags; it will not recognize other categories like dates, monetary values, or product names.

## Training and Evaluation Data

The model was trained on a combined dataset comprising annotated sentences across Marathi, Bengali, and Telugu.
- **Label Schema:** IOB2 format (`B-`, `I-`, `O`) with 7 total classes.
- **Token Alignment:** Subword tokenization artifacts were handled using `-100` label masking on non-initial subwords to ensure clean cross-entropy loss calculation.

## Training Procedure

### Training Hyperparameters
- **Learning Rate:** `3e-05`
- **Train Batch Size:** 16
- **Eval Batch Size:** 32
- **Epochs:** 3
- **Optimizer:** `AdamW (fused)` with $\beta_1=0.9, \beta_2=0.999, \epsilon=1\text{e-}08$
- **LR Scheduler:** Linear with 937 warmup steps (~10% of total steps)
- **Mixed Precision:** Native FP16 (`fp16=True`)
- **Seed:** 42

### Evaluation Metrics

Evaluated on the validation split using `seqeval` (entity-level span matching):

| Epoch | Training Loss | Validation Loss | Precision | Recall | Entity F1 🏆 | Token Accuracy |
| :---: | :-----------: | :-------------: | :-------: | :----: | :---------: | :------------: |
| 1.0   | 0.2845        | 0.2822          | 0.7172    | 0.7725 | 0.7438      | 0.9320         |
| 2.0   | 0.2024        | 0.2335          | 0.7392    | 0.7723 | 0.7554      | 0.9346         |
| **3.0**| **0.1804**    | **0.2328**      | **0.7382**| **0.7771** | **0.7572**  | **0.9352**     |

## Framework Versions

- **Transformers:** 5.14.1
- **PyTorch:** 2.11.0+cu128
- **Datasets:** 2.21.0
- **Tokenizers:** 0.22.2