# TRIEM AI - Code Explanation Guide 🎓 This document explains every part of the **TRIEM (Tribal Responsive Intelligent Empowerment Model)** project. It is written in simple English to help you understand the code structure, functions, and logic for your presentation or viva. --- ## 📂 Project Structure Overview Think of this project like a **human body**: * **`server.py` (The Heart):** Manages everything, receives requests, and coordinates actions. * **`src/brain.py` (The Brain):** Thinks, decides what to say, and uses AI models (Groq/Gemini). * **`src/asr_provider.py` (The Ears):** Listens to audio and converts it to text (ASR). * **`src/tts_provider.py` (The Mouth):** Converts text responses back into speech (TTS). * **`src/mt_provider.py` (The Translator):** Translates between Santali and English. * **`web/` (The Face):** What the user sees and interacts with (HTML/CSS/JS). --- ## 1. `server.py` (The Main Controller) ### 🟢 Purpose This is the **Flask Web Server**. It acts as the bridge between the user's browser and the AI models. When you click "Record" or "Upload", this file receives the audio and manages the entire pipeline. ### 🔗 Imports * `flask`: To create the web server. * `soundfile`, `os`: To handle audio files. * `src.brain`, `src.asr_provider`, etc.: Imports our custom AI modules. ### ⚙️ Main Functions #### `init_models()` * **What it does:** Starts up all the AI engines (ASR, Translate, TTS) when the app launches. * **Why important:** Without this, the app is "brain dead" and can't process anything. #### `process_audio()` (The Most Important Function!) * **What it does:** This is the **Master Pipeline**. 1. **Receives Audio:** Takes the user's voice file. 2. **Cleans Audio:** Uses FFmpeg to remove background noise. 3. **ASR (Listen):** Converts Santali Audio -> Santali Text. 4. **MT (Translate):** Converts Santali Text -> English Text. 5. **Brain (Think):** Sends English Text to AI (Brain) to get an answer. 6. **MT (Translate Back):** Converts English Answer -> Santali Answer. 7. **TTS (Speak):** Converts Santali Answer -> Audio. * **Input:** Audio File (`.wav`). * **Output:** JSON data containing the text and the path to the response audio. #### `save_interaction()` * **What it does:** Saves the chat (Question & Answer) to `chat_history.json`. * **Why important:** Allows the user to see their past conversations. ### 📝 Key Code Explanation ```python # This line ensures we use the best available device (GPU if possible) for PyTorch os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True" ``` --- ## 2. `src/brain.py` (The Intelligence) ### 🟢 Purpose This file decides **what to say**. It connects to advanced AI models (like Google Gemini or Groq) to generate smart answers. It also handles the **offline/online** logic. ### 🔗 Imports * `groq`: For ultra-fast online AI responses. * `google.genai`: For Google Gemini AI (backup). * `src.faq_database`: To check if we already know the answer. ### ⚙️ Main Functions #### `get_ai_response(text, santali_text, ...)` * **What it does:** The core thinking function. * **Logic Flow:** 1. **Check FAQ:** "Have I answered this before?" If yes, return the saved answer instantly (High Speed!). 2. **Check Internet:** If online, try **Groq** (Fastest AI). 3. **Fallback:** If Groq fails, try **Gemini** or **Ollama** (Offline). * **Input:** User's text (in English and Santali). * **Output:** The smartest answer the AI can generate. #### `normalize_text(text)` * **What it does:** Cleans up text by removing weird symbols. * **Why:** Helps the computer match questions better (e.g., "Hello!" becomes "hello"). --- ## 3. `src/config_hf.py` (The Settings) ### 🟢 Purpose This is a **configuration file**. It stores "hard-coded" settings like Model Names, API Keys (loaded securely), and Audio Settings. ### 📝 Key Code ```python # These are the specific Hugging Face models we use for Santali ASR_MODEL_NAME = "ai4bharat/indic-conformer-600m-multilingual" TTS_MODEL_NAME = "ai4bharat/indic-parler-tts" ``` * **Why important:** If we want to change the AI model later, we only change it here, not everywhere in the code. --- ## 4. `src/asr_provider.py` (The Ears 👂) ### 🟢 Purpose **ASR = Automatic Speech Recognition**. This file converts spoken Santali audio into written Santali text. ### ⚙️ Main Functions #### `transcribe(audio_file_path)` * **What it does:** Loads the audio, sends it to the `IndicConformer` model, and returns the text. * **Technical Detail:** It resamples audio to 16,000 Hz because that's what the AI expects. --- ## 5. `src/mt_provider.py` (The Translator 🔄) ### 🟢 Purpose **MT = Machine Translation**. Since most smart AIs understand English best, we translate Santali to English to "think", and then translate the answer back to Santali. ### ⚙️ Main Functions #### `translate(text, src_lang, tgt_lang)` * **What it does:** Translates text from Source Language -> Target Language. * **Input:** Text, e.g., "Johar" (Santali). * **Output:** Translated Text, e.g., "Hello" (English). * **Key Logic:** It uses `ai4bharat/indictrans2` models which are state-of-the-art for Indian languages. --- ## 6. `src/tts_provider.py` (The Mouth 🗣️) ### 🟢 Purpose **TTS = Text to Speech**. This file takes the final Santali text answer and turns it into a human-like voice. ### ⚙️ Main Functions #### `speak_to_file(text)` * **What it does:** Uses the `ParlerTTS` model to generate audio. * **Why important:** It allows the tribal user (who might not read) to **hear** the answer. * **Special Trick:** It uses a "description" prompt ("A female speaker...") to control how the voice sounds. --- ## 7. `src/faq_database.py` (The Memory 🧠) ### 🟢 Purpose This is a **Local Cache**. It remembers questions and answers using a simple database (`SQLite`). ### ⚙️ Main Functions #### `get_answer(question)` * **What it does:** Searches the database. * **Fuzzy Matching:** It uses `difflib` to find answers even if the question is slightly different (e.g., "What is your name?" vs "What's ur name?"). #### `add_entry(question, answer)` * **What it does:** Saves a new Q/A pair to the database so the AI learns and gets faster next time. --- ## 8. Web Interface Files (The Face 🖥️) ### `web/templates/index.html` * **Purpose:** The **Skeleton** of the website. * **Contains:** The Record button, Chat window, Visualizer canvas, and Dropdowns. * **Key Element:** `` - This creates the cool audio wave animation. ### `web/static/style.css` * **Purpose:** The **Makeup** (Design). * **Features:** * **Glassmorphism:** The inputs look like frosted glass (`backdrop-filter: blur`). * **Animations:** The pulsing ring around the record button. * **Responsive:** Makes sure it looks good on mobile and PC. ### `web/static/script.js` * **Purpose:** The **Muscles** (Action). * **Key Logic:** * **`startRecording()`**: Asks browser for microphone access and starts recording. * **`sendAudioToBackend()`**: Sends the recorded blob to `server.py` and waits for the result. * **`drawVisualizer()`**: Draws the dancing bars based on voice volume. * **`updateInputMode()`**: Switches between "Microphone" and "File Upload" modes. --- ## 9. Helper Files ### `Start_TRIEM_App.bat` * **Purpose:** A one-click launcher for Windows. You double-click this, and it opens the server and browser automatically. ### `requirements.txt` / `requirements_hf.txt` * **Purpose:** A shopping list for Python. It tells the computer which libraries (like `torch`, `flask`, `transformers`) need to be installed for the code to run. --- ## 🌟 Summary of the Flow 1. **User** speaks into the **Browser** (`script.js`). 2. **Server** (`server.py`) gets the audio. 3. **ASR** (`asr_provider.py`) types it out. 4. **MT** (`mt_provider.py`) translates to English. 5. **Brain** (`brain.py`) thinks of an answer. 6. **MT** translates answer back to Santali. 7. **TTS** (`tts_provider.py`) speaks the answer. 8. **User** hears the response! This structure ensures the AI is **Modular** (easy to fix parts separately) and **Scalable** (can add more languages later).