|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- open-thoughts/OpenThoughts-114k |
|
|
language: |
|
|
- te |
|
|
pipeline_tag: fill-mask |
|
|
--- |
|
|
|
|
|
Model Card for Telugu BERT Model |
|
|
This model is a BERT-based language model trained for Masked Language Modeling (MLM) in Telugu. It is designed to understand and generate Telugu text effectively. |
|
|
|
|
|
Model Details |
|
|
Model Description |
|
|
Developed by: MATHI |
|
|
Model type: Transformer-based Masked Language Model (MLM) |
|
|
Language(s) (NLP): Telugu |
|
|
License: MIT |
|
|
Model Sources |
|
|
Repository: Hugging Face Model Repo |
|
|
Demo : Colab Notebook |
|
|
|
|
|
Uses |
|
|
Direct Use |
|
|
This model can be used for: Text completion in Telugu |
|
|
Fill-mask prediction (predict missing words in a sentence) |
|
|
Pretraining or fine-tuning for Telugu NLP tasks |
|
|
|
|
|
Downstream Use |
|
|
Fine-tuned versions of this model can be used for: |
|
|
Named Entity Recognition (NER) |
|
|
Sentiment Analysis |
|
|
Machine Translation |
|
|
Text Summarization |
|
|
|
|
|
Out-of-Scope Use |
|
|
Not suitable for real-time dialogue generation |
|
|
Not trained for code-mixing (Telugu + English) |
|
|
|
|
|
Bias, Risks, and Limitations |
|
|
The model may reflect biases present in the training data. |
|
|
Accuracy may vary for dialectal variations of Telugu. |
|
|
May generate incorrect or misleading predictions. |
|
|
|
|
|
Recommendations |
|
|
Users should verify the model's outputs before relying on them for critical applications. |
|
|
|
|
|
How to Get Started with the Model |
|
|
Use the code below to get started: |
|
|
from transformers import AutoModelForMaskedLM, AutoTokenizer, pipeline |
|
|
model_name = "Mathiarasi/TMod" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForMaskedLM.from_pretrained(model_name) |
|
|
fill_mask = pipeline("fill-mask", model=model, tokenizer=tokenizer) |
|
|
print(fill_mask("మక్దూంపల్లి పేరుతో చాలా [MASK] ఉన్నాయి.")) |
|
|
|
|
|
Training Details |
|
|
Training Data |
|
|
The model is trained on a Telugu corpus containing diverse text sources. |
|
|
Data preprocessing included text normalization, cleaning, and tokenization. |
|
|
Training Procedure |
|
|
Preprocessing |
|
|
Used WordPiece Tokenizer with a vocabulary of 30,000 tokens. |
|
|
|
|
|
Training Hyperparameters |
|
|
Batch Size: 16 |
|
|
Learning Rate: 5e-5 |
|
|
Epochs: 3 |
|
|
Optimizer: AdamW |
|
|
|
|
|
Speeds, Sizes, Times |
|
|
Testing Data |
|
|
Evaluated on a held-out dataset of Telugu text. |
|
|
|
|
|
Technical Specifications |
|
|
Model Architecture and Objective |
|
|
|
|
|
Model Type: BERT (Bidirectional Encoder Representations from Transformers) |
|
|
|
|
|
Training Objective: Masked Language Modeling (MLM) |
|
|
|
|
|
Dataset library: datasets |
|
|
|
|
|
Citation |
|
|
If you use this model, please cite: |
|
|
|
|
|
@article{Mathiarasi2025, |
|
|
title={Telugu BERT: A Transformer-Based Language Model for Telugu}, |
|
|
author={Mathiarasi}, |
|
|
journal={Hugging Face Models}, |
|
|
year={2025} |
|
|
} |
|
|
|
|
|
Model Card Authors : MATHIARASI |
|
|
|
|
|
Model Card Contact |
|
|
For questions, contact mathiarasie1710@gmail.com |
