|
|
| # 📖 Detailed Code Explanation: Multi-Language Translator |
|
|
| This document explains the **Multi-Language Translator**, detailing each component and step of the provided Python code. |
|
|
| --- |
|
|
| ## 📝 Overview |
|
|
| **Purpose** |
| Translate English text into multiple languages dynamically selected by users. |
|
|
| **Tech Stack** |
| - **Translation Model**: Hugging Face Transformers (`Meta's NLLB-200`) |
| - **Language Codes**: FLORES-200 dataset |
| - **User Interface**: Gradio Blocks interface |
| - **Data Handling**: JSON file for language codes |
|
|
| --- |
|
|
| ## ⚙️ Setup & Dependencies |
|
|
| Install necessary libraries: |
|
|
| ``` |
| pip install transformers gradio torch |
| ``` |
|
|
| Ensure you have the `language.json` file (FLORES-200 codes) in your project directory. |
|
|
| --- |
|
|
| ## 🔍 Detailed Block-by-Block Code Explanation |
|
|
| ### 1. Imports |
|
|
| ``` |
| import torch |
| import gradio as gr |
| import json |
| from transformers import pipeline |
| ``` |
|
|
| **Detailed Explanation:** |
| - `torch`: Essential for model computations and optimizations. |
| - `gradio`: Creates simple yet powerful web-based user interfaces. |
| - `json`: Reads and processes language codes from JSON files. |
| - `pipeline`: Loads NLP models for tasks like translation easily. |
|
|
| ### 2. Load Translation Model |
|
|
| ``` |
| text_translator = pipeline("translation", model="facebook/nllb-200-distilled-600M", |
| torch_dtype=torch.bfloat16) |
| ``` |
|
|
| **Detailed Explanation:** |
| - Loads the `NLLB-200` translation model capable of translating between 200+ languages. |
| - Optionally optimizes performance via `torch_dtype=torch.bfloat16` if supported. |
|
|
| ### 3. Load Language Codes from JSON |
|
|
| ``` |
| with open('language.json', 'r') as file: |
| language_data = json.load(file) |
| ``` |
|
|
| **Detailed Explanation:** |
| - Reads language names and their corresponding FLORES-200 codes from `language.json`. |
| - Prepares the app to support a dynamic range of languages. |
|
|
| ### 4. Helper Function to Retrieve FLORES Codes |
|
|
| ``` |
| def get_FLORES_code_from_language(language): |
| for entry in language_data: |
| if entry['Language'].lower() == language.lower(): |
| return entry['FLORES-200 code'] |
| return None |
| ``` |
|
|
| **Detailed Explanation:** |
| - Matches selected language to its FLORES-200 code. |
| - Returns the correct code for translation or provides an error if not found. |
|
|
| ### 5. Translation Function |
|
|
| ``` |
| def translate_text(text, destination_language): |
| dest_code = get_FLORES_code_from_language(destination_language) |
| if not dest_code: |
| return f"❌ Language code for '{destination_language}' not found." |
| translation = text_translator(text, src_lang="eng_Latn", tgt_lang=dest_code) |
| return translation[0]["translation_text"] |
| ``` |
|
|
| **Detailed Explanation:** |
| - Translates provided English text into the user-selected language. |
| - Gracefully handles missing language codes. |
|
|
| ### 6. Gradio Interface |
|
|
| ``` |
| language_list = [entry["Language"] for entry in language_data] |
| |
| with gr.Blocks(theme=gr.themes.Default()) as demo: |
| gr.Markdown("# 🌐 Multilingual Translator — Powered by Meta NLLB-200") |
| gr.Markdown("Translate English text into 200+ languages.") |
| |
| with gr.Row(): |
| input_text = gr.Textbox(label="📝 English Input", lines=6, placeholder="Type your English sentence here...") |
| output_lang = gr.Dropdown(choices=language_list, label="🌍 Target Language") |
| |
| output_text = gr.Textbox(label="✅ Translated Output", lines=4) |
| |
| translate_btn = gr.Button("🔄 Translate") |
| |
| translate_btn.click(fn=translate_text, inputs=[input_text, output_lang], outputs=output_text) |
| |
| gr.Markdown("---") |
| gr.Markdown("🧠 Model: `facebook/nllb-200-distilled-600M` |
| 🌐 Language codes from [FLORES-200](https://github.com/facebookresearch/flores)") |
| |
| demo.launch() |
| ``` |
|
|
| **Detailed Explanation:** |
| - User-friendly interface to input English text and select target language. |
| - Interactive, real-time translation upon button click. |
| - Displays the translated text clearly. |
|
|
| --- |
|
|
| ## 🚀 Core Concepts |
|
|
| | Concept | Importance | |
| |--------------------------------|------------------------------------------------| |
| | **NLLB-200 Translation Model** | Provides accurate multilingual translations | |
| | **FLORES-200 Dataset** | Ensures accurate language-code mapping | |
| | **Gradio Interface** | Quick creation of user-friendly web interfaces | |
| | **Dynamic JSON Data Loading** | Easily updates supported languages | |
|
|
| --- |
|
|
| ## 🔄 Alternative Models & Possible Enhancements |
|
|
| - **Alternate Translation Models**: |
| ``` |
| pipeline("translation", model="Helsinki-NLP/opus-mt-en-de") # Example: English-German translation |
| pipeline("translation", model="t5-base") |
| ``` |
|
|
| - **Additional UI Features**: |
| - Speech-to-text for input. |
| - Audio playback for translated text. |
| - Option to download translated text. |
|
|
| - **Performance Improvements**: |
| - GPU support for faster translations. |
| - Offline caching of translations for commonly translated phrases. |
|
|
|
|
