PROJECT_OVERVIEW.md

Martechsol Chat Assistant - Project Overview

This document explains how your RAG (Retrieval-Augmented Generation) chatbot works in simple terms.

1. The "Brain" (The AI Model)

The assistant uses a hybrid LLM approach:

• Qwen 2.5 32B (specifically qwen/qwen3-32b) as the main reasoning and generation engine.
• Llama 3.1 8B (specifically llama-3.1-8b-instant) as a lightning-fast model for query rewriting and expansion.
• Provider: Both are powered by Groq, enabling near-instantaneous responses.

2. How it Works (The "RAG" Process)

Instead of just relying on general knowledge, this bot "reads" your documents to give specific answers.

1. Reading: It looks at your files in the docs/ folder (PDFs and Text files).
2. Memorizing: It breaks the text into small chunks and converts them into mathematical "vectors" (using the bge-small-en-v1.5 model).
3. Searching: When you ask a question, it expands the query using Llama 3.1 8B, then performs a Hybrid Search combining Dense vectors (FAISS) and Keyword search (BM25).
4. Reranking: It deeply evaluates the top retrieved chunks using bge-reranker-base to ensure maximum relevance.
5. Answering: It sends your question along with the most relevant document parts to the Qwen 2.5 AI, which then writes a highly precise, formatted reply.

3. The Architecture

Frontend (The Face)

• Gradio: This is the clean, chat-like interface you see. It is hosted on Hugging Face Spaces.
• WordPress Addon: A custom HTML/CSS/JS wrapper that lets you embed the chat as a beautiful floating button on your website.

Backend (The Engine)

• FastAPI: A high-performance Python framework that connects everything. It manages the messages, handles the document search, and talks to the AI provider.
• Uvicorn: The lightning-fast server that runs the FastAPI code.

4. Key Features

• Humanistic Replies: The bot is programmed to be polite, conversational, and professional.
• Context-Aware: It remembers the last few messages in the conversation so you can ask follow-up questions.
• Greeting Support: It can handle "Hi" and "Hello" naturally before getting down to business.
• Safe & Grounded: It is instructed to only answer based on your documents to prevent "hallucinations" (making things up).