README.md

---
title: Multilingual Transliteration
emoji: 🌐
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.8.0
app_file: app.py
pinned: false
---

# Multilingual Transliteration Model

This project implements a multilingual transliteration model (English -> Hindi, Bengali, Tamil) using a fine-tuned mT5 model. It focuses on optimization using CTranslate2 for fast inference and provides a Gradio-based web interface.

## Project Structure
- `src/`: Source code for training, optimization, and deployment.
- `data/`: Directory for storing datasets (train/test/val).
- `models/`: Directory for saving trained and optimized models.
- `requirements.txt`: Python dependencies.

## Setup

1.  **Clone the repository:**
    ```bash
    git clone <repo_url>
    cd <repo_name>
    ```

2.  **Create a virtual environment (optional but recommended):**
    ```bash
    python -m venv venv
    .\venv\Scripts\activate  # Windows
    # source venv/bin/activate # Linux/Mac
    ```

3.  **Install dependencies:**
    ```bash
    pip install -r requirements.txt
    ```

## Usage

### 1. Data Preparation
Generate dummy data for training:
```bash
python src/prepare_data.py
```

### 2. Training
Train the mT5 model:
```bash
python src/train.py
```

### 3. Optimization
Optimize the trained model using CTranslate2 and benchmark:
```bash
python src/optimize.py
```

### 4. Run Demo
Launch the Gradio app:
```bash
python src/app.py
```

## Approach
- **Model:** `google/mt5-small` is used as the base model due to its multilingual capabilities and efficiency.
- **Optimization:** CTranslate2 is used to quantize and optimize the model for faster CPU/GPU inference.
- **Deployment:** Gradio provides a simple and interactive UI for the model.