base_model: ai4bharat/indictrans2-en-indic-dist-200M
library_name: transformers
pipeline_tag: translation
license: mit
language:
- en
- mr
metrics:
- bleu
- chrf
- comet
tags:
- generated_from_trainer
- punctuation-robustness
- indictrans2
model-index:
- name: iitb-en-indic-without-punct
results:
- task:
type: translation
name: English to Marathi Translation
dataset:
name: Virām (PEM)
type: viram
metrics:
- type: bleu
value: 10.1304
name: BLEU
- type: chrf
value: 32.6831
name: chrF++
- type: comet
value: 0.5427
name: COMET
iitb-en-indic-without-punct
This model is a fine-tuned version of ai4bharat/indictrans2-en-indic-dist-200M designed to improve punctuation robustness in English-to-Marathi machine translation.
It was introduced in the paper Assessing and Improving Punctuation Robustness in English-Marathi Machine Translation.
Model Description
Traditional machine translation systems often struggle when punctuation is missing or ambiguous in the source text. This checkpoint represents Approach 2 from the associated research, where the IndicTrans2 model was directly fine-tuned on the IITB-ENG-MAR dataset with all English source punctuations removed. This allows the model to implicitly learn the context required to resolve semantic and structural ambiguities when punctuation is absent.
- Paper: Assessing and Improving Punctuation Robustness in English-Marathi Machine Translation
- Repository: Viram_Marathi (GitHub)
- Demo: Punctuation Robust Translation Demo
- Base Model: ai4bharat/indictrans2-en-indic-dist-200M
Intended Uses & Limitations
This model is intended for translating English text into Marathi, particularly in scenarios where the source English text might lack proper punctuation or contain punctuation-induced ambiguities.
Training and Evaluation Data
The model was fine-tuned on a modified version of the English-Marathi parallel corpus from IIT Bombay.
It was evaluated on the Virām (formerly PEM) diagnostic benchmark, which consists of 54 manually curated, punctuation-ambiguous instances designed to assess how well MT systems preserve meaning when punctuation is varied or missing.
Training Procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 8
Evaluation Results
| Metric | Value |
|---|---|
| BLEU | 10.1304 |
| chrF++ | 32.6831 |
| COMET | 0.5427 |
| Loss | 0.3722 |
Framework versions
- Transformers 4.53.2
- Pytorch 2.4.0a0+f70bd71a48.nv24.06
- Datasets 2.21.0
- Tokenizers 0.21.4
Citation
If you use this model or the Virām benchmark, please cite:
@article{shejole2025assessing,
title={Assessing and Improving Punctuation Robustness in English-Marathi Machine Translation},
author={Shejole, Kaustubh and others},
journal={arXiv preprint arXiv:2601.09725},
year={2025}
}