README.md

metadata

title: Audio/Video Translation Toolkit
emoji: 🚀
colorFrom: indigo
colorTo: purple
sdk: gradio
python_version: 3.10.0
app_file: app.py
tags:
  - translation
  - audio
  - video
  - speech-synthesis
  - voice-cloning
  - gradio

🚀 Audio/Video Translation Toolkit

This application provides a complete pipeline for translating the audio of a video or audio file from English to various Indian languages. It handles everything from vocal separation and transcription to translation, speech synthesis, and voice cloning.

---

## Key Features 🛠️

• 🎬 Full Video Translation: Upload a video, and the app will extract the audio, translate it, and merge it back into the original video.
• 🎵 Full Audio Translation: Translate standalone audio files.
• 🗣️ Vocal Separation: Isolate vocals from background music before processing.
• ✍️ Transcription & Pace Detection: Uses Whisper to transcribe the audio and determine the original speaker's pace.
• 🌐 Multi-Lingual Translation: Translate text to Tamil, Telugu, or Hindi using either local models or the Sarvam API.
• 🔊 Speech Synthesis: Generate new speech in the target language using models from ai4bharat.
• 🧬 Voice Cloning: Clone the voice from the original speaker onto the newly synthesized audio for a more natural result.

---

## How to Use the Main Pipeline

1. Navigate to the Translate Video or Translate Audio tab.
2. Upload your file.
3. Select the Target Language.
4. Click the Translate button and wait for the process to complete.

---