Showrov5843/banglat5-dialect-to-standard

BanglaT5 Regional Bangla Text to Standard Bangla Text Converter

This model was fine-tuned on a custom dataset to convert Rangpuria/Pabnaia/Barishailla/Sylheti/Khulna/Chittagonian/Noakhali/etc. dialects to Standard Bengali. This is the Undergraduate Thesis work of Showrov Azam, B.Sc. in C.S.E ,Daffodil International University

How to use

Important: You must use the csebuetnlp/normalizer.

!pip install git+[https://github.com/csebuetnlp/normalizer](https://github.com/csebuetnlp/normalizer)
from transformers import pipeline
from normalizer import normalize

model_id = "Showrov5843/banglat5-dialect-to-standard"
pipe = pipeline("translation", model=model_id, tokenizer=model_id)

input_text = "তোমার ডায়ালেক্ট এখানে দিন"
normalized_text = normalize(input_text)

print(pipe("normalize: " + normalized_text))