| ---
|
| title: veun
|
| app_file: app.py
|
| sdk: gradio
|
| sdk_version: 5.41.0
|
| ---
|
| # 🎥 AI Video Chat Assistant with RAG Knowledge System
|
|
|
| An intelligent, multi-modal chatbot that can analyze video content, answer your questions, and remember past interactions using a sophisticated Retrieval-Augmented Generation (RAG) system.
|
|
|
| 
|
|
|
| ---
|
|
|
| ## ✨ Features
|
|
|
| - **Intelligent Video Analysis**: Upload a video, and the assistant will use the Gemini 1.5 Flash model to understand its content.
|
| - **Multi-Modal Chat**: Ask questions about the uploaded video, and receive detailed, context-aware answers.
|
| - **Persistent Memory**:
|
| - **Short-Term Memory**: Uses **ChromaDB** to remember the conversation history within a single session.
|
| - **Long-Term Knowledge**: Uses a **FAISS Vector Store** to create a persistent, searchable knowledge base from all interactions (video analyses, Q&A), enabling cross-session insights.
|
| - **RAG-Powered Context**: Follow-up questions are enhanced with relevant context retrieved from both the current conversation and the long-term knowledge base.
|
| - **Interactive UI**: A user-friendly interface built with **Gradio**, featuring distinct sections for video interaction and knowledge base management.
|
| - **Robust Backend**: Powered by **FastAPI**, providing a scalable and efficient API.
|
| - **Debugging & Management**: The UI includes tools to directly query the knowledge base, add test data, and view system statistics.
|
|
|
| ---
|
|
|
| ## 🏗️ Architecture
|
|
|
| The application operates on a decoupled frontend-backend model. The Gradio UI serves as a pure client, making HTTP requests to the FastAPI backend, which houses all the AI logic, data processing, and state management.
|
|
|
| The core of the architecture is its **Dual-Memory System**:
|
| 1. **ChromaDB for Conversational Context**: Provides fast, session-specific memory. It answers the question, "What have we been talking about *right now*?".
|
| 2. **FAISS for Enduring Knowledge**: Creates a permanent, long-term knowledge base from key insights. It answers the question, "What has the assistant learned from *all past interactions*?".
|
|
|
| ### System Flow Diagram
|
|
|
| This diagram illustrates how a user request flows through the system, interacting with the dual-memory stores and the Gemini AI model.
|
|
|
| 
|
|
|
| ## ⚙️ Setup and Installation
|
|
|
| Follow these steps to get the application running on your local machine.
|
|
|
| ### 1. Prerequisites
|
| - Python 3.8 or higher
|
| - Git
|
|
|
| ### 2. Clone the Repository
|
| ```bash
|
| git clone [https://github.com/your-username/ai-video-chat-assistant.git](https://github.com/your-username/ai-video-chat-assistant.git)
|
| cd ai-video-chat-assistant
|
| ```
|
|
|
| ### 3. Create a Virtual Environment
|
| It's highly recommended to use a virtual environment.
|
| ```bash
|
| # For Windows
|
| python -m venv venv
|
| venv\Scripts\activate
|
|
|
| # For macOS/Linux
|
| python3 -m venv venv
|
| source venv/bin/activate
|
| ```
|
|
|
| ### 4. Install Dependencies
|
| Install all the required packages from the `requirements.txt` file.
|
| ```bash
|
| pip install -r requirements.txt
|
| ```
|
|
|
| ### 5. Set Up Environment Variables
|
| Create a file named `.env` in the root directory of the project. This file will hold your Gemini API key.
|
|
|
| ```
|
| # .env
|
| GEMINI_API_KEY="YOUR_GEMINI_API_KEY_HERE"
|
| ```
|
|
|
| ---
|
|
|
| ## 🚀 Running the Application
|
|
|
| Once the setup is complete, you can start the application with a single command:
|
|
|
| ```bash
|
| python app/main.py
|
| ```
|
|
|
| You will see output indicating that the FastAPI backend and Gradio frontend are running:
|
|
|
| ```
|
| 🚀 Starting AI Video Chat Assistant...
|
| 📊 FastAPI Backend: [http://127.0.0.1:8000](http://127.0.0.1:8000)
|
| 🎨 Gradio Frontend: [http://127.0.0.1:7860](http://127.0.0.1:7860)
|
| 🧠 RAG System: with RAG Integration
|
| ======================================================================
|
| ```
|
|
|
| - Open your web browser and navigate to **`http://127.0.0.1:7860`** to use the application.
|
|
|
| ---
|
|
|
| ## 📖 How to Use
|
|
|
| 1. **Upload a Video**: Click the "🎬 Click to Upload Video" button and select a video file.
|
| 2. **Ask a Question**: Once the video is uploaded, the text box will become active. Type a question about the video (e.g., "What is happening in this video?") and press Enter or click "Send 🚀".
|
| 3. **Chat**: The AI will respond. You can ask follow-up questions. The system will remember the context of the current conversation.
|
| 4. **Explore the Knowledge Base**:
|
| - Click the **"🧠 Knowledge Base"** tab.
|
| - **Search**: Type a query into the search box to find relevant information from all past interactions.
|
| - **Debug**: Use the **"🎯 Add Test Data"** button to populate the RAG store for testing and **"📊 Show Stats"** to see its current state.
|
| 5. **Start a New Conversation**: Click the **"🔄 New Conversation"** button to clear the current state and start fresh with a new session ID.
|
|
|
| ---
|
|
|
| ## 📡 API Endpoints
|
|
|
| The FastAPI backend exposes several endpoints. You can test them at `http://127.0.0.1:8000/docs`.
|
|
|
| | Method | Endpoint | Description |
|
| |--------|------------------------|-----------------------------------------------------------------------------|
|
| | `POST` | `/chat/video` | Main endpoint to chat with a video. Requires a video file and a prompt. |
|
| | `POST` | `/chat/text` | Handles text-only follow-up questions. |
|
| | `POST` | `/rag/query` | Directly queries the FAISS RAG knowledge base. |
|
| | `GET` | `/rag/stats` | Retrieves comprehensive statistics about the RAG system. |
|
| | `POST` | `/rag/debug` | Adds pre-defined test data to the RAG vector store. |
|
| | `POST` | `/rag/reinitialize` | Forces a re-initialization of the RAG vector store. |
|
| | `GET` | `/health` | A simple health check endpoint to verify system status. |
|
|
|
| ---
|
|
|
| ## 🧪 Debugging and Testing
|
|
|
| The project includes standalone scripts for testing the RAG system independently.
|
|
|
| - **`scripts/debug_rag.py`**: A comprehensive test suite that checks imports, adds data, runs test queries, and offers an interactive query mode. Run it with `python scripts/debug_rag.py`.
|
| - **`scripts/query_rag.py`**: A simple script to perform a similarity search on the RAG vector store. Modify the `query` variable inside the script and run with `python scripts/query_rag.py`.
|
| |
