diff --git "a/README.md" "b/README.md" --- "a/README.md" +++ "b/README.md" @@ -7,175 +7,149 @@ colorFrom: blue colorTo: purple sdk_version: 4.44.0 --- -# Chance RAG: Advanced Retrieval-Augmented Generation System - -## Table of Contents -1. [System Architecture](#1-system-architecture) -2. [Data Flow](#2-data-flow) -3. [Detailed Component Descriptions](#3-detailed-component-descriptions) - - [3.1 Document Processor](#31-document-processor) - - [3.2 MistralRAGChatbot](#32-mistralragchatbot) - - [3.3 Retrieval Engine](#33-retrieval-engine) - - [3.4 Reranking Engine](#34-reranking-engine) - - [3.5 Response Generator](#35-response-generator) -4. [User Interface](#4-user-interface) -5. [Performance Optimization](#5-performance-optimization) -6. [Error Handling and Logging](#6-error-handling-and-logging) -7. [Future Enhancements](#7-future-enhancements) -8. [Conclusion](#8-conclusion) - -## 1. System Architecture - -[Diagram 1: System Architecture] -+-------------------+ +-------------------+ +-------------------+ -| PDF Document | | User Interface | | Mistral AI API | -| | | (Gradio UI) | | | -+--------+----------+ +--------+----------+ +--------+----------+ -| | | -v v v -+-------------------+ +-------------------+ +-------------------+ -| Document Processor| | MistralRAGChatbot | | Response Generator| -| | | | | | -+--------+----------+ +--------+----------+ +--------+----------+ -| ^ ^ -v | | -+-------------------+ +-------------------+ +-------------------+ -| Vector Database | | Retrieval Engine | | Reranking Engine | -| (Annoy Index) | | | | | -+-------------------+ +-------------------+ +-------------------+ - -## 2. Data Flow - -[Diagram 2: Data Flow] -+-------------+ +-----------------+ +------------------+ -| PDF Upload | --> | Text Extraction | --> | Chunk Generation | -+-------------+ +-----------------+ +------------------+ -| -v -+-------------+ +-----------------+ +------------------+ -| User Query | --> | Query Embedding | --> | Document Retrieval| -+-------------+ +-----------------+ +------------------+ -| -v -+-------------+ +-----------------+ +------------------+ -| Reranking | --> | Context Creation| --> | Response Generation| -+-------------+ +-----------------+ +------------------+ -| -v -+------------------+ -| Display Response | -+------------------+ - -## 3. Detailed Component Descriptions - -### 3.1 Document Processor - -The Document Processor handles PDF files and prepares them for use in the RAG system. - -Key functions: -- `store_embeddings_in_vector_db`: Processes PDFs and stores embeddings. -- `split_text_into_chunks`: Divides text into manageable chunks. - -### 3.2 MistralRAGChatbot - -This core class orchestrates the entire RAG process. - -Key methods: -- `generate_response_with_rag`: Coordinates retrieval, reranking, and response generation. -- `retrieve_documents`: Fetches relevant documents using various methods. -- `rerank_documents`: Applies reranking algorithms to improve relevance. - -### 3.3 Retrieval Engine - -The Retrieval Engine uses multiple methods to find relevant documents. - -Methods: -1. Annoy (Approximate Nearest Neighbors) -2. TF-IDF (Term Frequency-Inverse Document Frequency) -3. BM25 (Best Matching 25) -4. Word2Vec -5. Euclidean Distance -6. Jaccard Similarity - -[Diagram 3: Retrieval Methods Comparison] -Method | Speed | Accuracy | Memory Usage ----------|-------|----------|-------------- -Annoy | Fast | Good | Low -TF-IDF | Fast | Moderate | Moderate -BM25 | Fast | Good | Low -Word2Vec | Slow | Good | High -Euclidean| Fast | Moderate | Low -Jaccard | Slow | Moderate | Low - -### 3.4 Reranking Engine - -The Reranking Engine applies advanced algorithms to improve the relevance of retrieved documents. - -Methods: -1. Advanced Fusion -2. Reciprocal Rank Fusion -3. Weighted Score Fusion -4. Semantic Similarity Reranking - -[Diagram 4: Reranking Process] -+-------------------+ +-------------------+ +-------------------+ -| Retrieved Docs | --> | Reranking Methods | --> | Reranked Docs | -+-------------------+ +-------------------+ +-------------------+ -| -+--------+--------+ -| | -+-------v------+ +-------v------+ -| Score Fusion | | Semantic Sim | -+--------------+ +--------------+ - -### 3.5 Response Generator - -Utilizes the Mistral AI API to generate human-like responses based on the retrieved and reranked context. - -Key function: -- `build_prompt`: Constructs the prompt for the Mistral AI model. - -## 4. User Interface - -The Gradio-based user interface provides an intuitive way to interact with the Chance RAG system. - -Components: -1. PDF Upload -2. User Query Input -3. Response Style Selection -4. Retrieval Methods Selection -5. Reranking Methods Selection -6. Chunk Size and Overlap Adjustment -7. Response Display - -## 5. Performance Optimization - -To improve the system's performance, consider the following: +# **Chance RAG Documentation** -1. Implement caching for embeddings and frequently retrieved documents. -2. Use parallel processing for document retrieval and reranking. -3. Optimize chunk size and overlap based on document characteristics. -4. Implement adaptive retrieval method selection based on query type. +## **Overview:** -## 6. Error Handling and Logging +ChanceRAG is a Retrieval-Augmented Generation (RAG) application designed to process documents (such as Documents and Docs) and retrieve relevant information to provide detailed and accurate responses based on user queries. The system leverages various retrieval techniques, including vector embeddings, TF-IDF, BM25, and Word2Vec, and re-ranking methods such as advanced fusion, reciprocal rank fusion, and semantic similarity. The application integrates with Mistral’s embedding model for generating embeddings and employs Annoy for efficient retrieval using angular distance. -The system includes error handling and logging mechanisms to ensure robustness and facilitate debugging. +## **Data Flow:** +![Data Flow](images/Data_Flow.png) + +### **1\. Document Processing and Embedding Storage:** + +* ### The user uploads a document (PDF), which is split into smaller chunks to stay within token limits. + +* ### Text from each page is extracted, chunked, and transformed into embeddings using the Mistral embedding model. + +* ### These embeddings are then stored in a vector database. + +### **2\. Query Handling and Retrieval:** + +* ### Upon receiving a query, the system creates embeddings for the query and employs various retrieval methods, including Annoy, TF-IDF, BM25, and Word2Vec, to fetch the most relevant document chunks. + +### **3\. Re-ranking and Fusion:** + +* ### Retrieved document chunks are re-ranked using methods such as reciprocal rank fusion, advanced fusion retrieval, weighted score fusion, and semantic similarity. + +* ### The highest-ranked results are used to generate a final response. + +### **4\. Response Generation:** + +* ### Based on the retrieved context, ChanceRAG generates detailed, tailored responses using the Mistral AI API. + +* ### Users can customize the response style (e.g., Detailed, Concise, Creative, or Technical), retrieval methods, reranking strategies, chunk size, and overlap. + +## **Components of the ChanceRAG System:** + +### **1\. Document Processor:** + +* **store\_embeddings\_in\_vector\_db**: Processes and stores embeddings for PDFs. +* **split\_text\_into\_chunks**: Splits text into manageable chunks. + +### **2\. MistralRAGChatbot:** + +* **generate\_response\_with\_rag**: Manages the entire RAG process, including retrieval, reranking, and response generation. +* **retrieve\_documents**: Fetches relevant document chunks using different retrieval methods. +* **rerank\_documents**: Enhances retrieval relevance with reranking algorithms. + +### **3\. Retrieval Engine:** + +* Utilizes methods like Annoy, TF-IDF, BM25, Word2Vec, Euclidean Distance, and Jaccard Similarity to identify relevant content. + +| Method | Speed | Accuracy | Memory Usage | +| :---: | :---: | :---: | :---: | +| Annoy | Fast | Good | Low | +| TF-IDF | Fast | Moderate | Moderate | +| BM25 | Fast | Good | Low | +| Word2Vec | Slow | Good | High | +| Euclidean | Fast | Moderate | Low | +| Jaccard | Slow | Moderate | Low | + + Retrieval Methods Comparison + +### **4\. Reranking Engine:** + +* Applies reranking methods like advanced fusion, reciprocal rank fusion, weighted score fusion, and semantic similarity to ensure the most relevant documents are prioritized. + **Methods:** +* Advanced Fusion +* Reciprocal Rank Fusion +* Weighted Score Fusion +* Semantic Similarity Reranking + + +### **5\. Response Generator:** + +* ### The **build\_prompt** function constructs prompts for the Mistral AI model to generate human-like responses based on retrieved and reranked content. + +## **User Interface:** + +Built with Gradio, the interface offers easy interaction with the system. +Components include: + +* PDF Upload +* User Query Input +* Response Style Selection +* Retrieval Methods Selection +* Reranking Methods Selection +* Chunk Size and Overlap Adjustment +* Response Display. + +![Landing Page](images/Landing_Page.png) + +![Response_Style](images/Response_Style.png) + +![Retrieval Methods](images/Retrieval_Methods.png) + +![Reranking Methods](images/Reranking_Methods.png) + +![RAG Response](images/RAG_Response.png) + + +## **Performance Optimization:** + +To enhance performance: + +* Implement caching for embeddings and frequently retrieved documents. +* Use parallel processing for retrieval and reranking. +* Optimize chunk size and overlap based on document characteristics. +* Adapt retrieval methods based on query type. + +## **Metrics:** + +* **Precision Score:** The system achieved an average precision score of 80%, reflecting its accuracy in identifying relevant results. +* **Hit Rate:** The hit rate was true, indicating successful identification of correct answers within the system's outputs. +* **NDCG (Normalized Discounted Cumulative Gain):** The ranking quality was measured at 5, showcasing how well the system ranked relevant information. +* **Hallucination:** No hallucinations were detected, confirming that the system provided factual and relevant responses. +* **Correctness:** The correctness of the system’s outputs was validated as true, ensuring reliability in the information provided. + +## **Error Handling and Logging:** + +The system includes error handling and logging mechanisms to ensure robustness and facilitate debugging. Key aspects: -- Exception handling in critical functions -- Logging of important events and errors -- User-friendly error messages in the interface -## 7. Future Enhancements +* Exception handling in critical functions +* Logging of important events and errors +* User-friendly error messages in the interface + +## **Future Enhancements:** + +* Support for multi-document processing. +* Dynamic model selection based on query complexity. +* User feedback integration to improve retrieval and reranking. +* Multilingual support for handling queries in different languages. +* Advanced analytics to track performance and usage. + +## **Conclusion:** + +The Chance RAG system provides a powerful and flexible solution for context-aware question answering based on documents. By leveraging multiple retrieval and reranking methods, along with the advanced language capabilities of the Mistral AI API, it offers highly relevant and coherent responses to user queries. +This documentation provides a comprehensive overview of the system architecture, components, and processes. For specific implementation details, refer to the inline comments and docstrings in the code. -1. Multi-document support: Allow processing of multiple PDFs simultaneously. -2. Dynamic model selection: Choose the most appropriate Mistral AI model based on the query complexity. -3. User feedback integration: Incorporate user feedback to improve retrieval and reranking over time. -4. Multilingual support: Extend the system to handle multiple languages. -5. Advanced analytics: Implement usage analytics and performance metrics tracking. +[image1]: -## 8. Conclusion +[image2]: -The Chance RAG system provides a powerful and flexible solution for context-aware question answering based on PDF documents. By leveraging multiple retrieval and reranking methods, along with the advanced language capabilities of the Mistral AI API, it offers highly relevant and coherent responses to user queries. +[image3]: -This documentation provides a comprehensive overview of the system architecture, components, and processes. For specific implementation details, refer to the inline comments and docstrings in the code. \ No newline at end of file +[image4]: \ No newline at end of file