PROJECT_TECHNICAL_SPEC.md

# Martechsol HR Assistant: Project Technical Specification & Workflow

This document provides a comprehensive overview of the **Martechsol HR Assistant**, covering its architecture, the models used, API integrations, and the end-to-end data workflow.

---

## 1. Project Architecture Overview

The system is a high-performance **Retrieval-Augmented Generation (RAG)** application designed to provide precise, grounded answers based on employee handbooks and HR documents.

### High-Level Stack:
- **Language**: Python 3.10+
- **API Framework**: FastAPI
- **Server**: Uvicorn
- **UI**: Gradio (for direct testing/Hugging Face Spaces)
- **Web Integration**: Custom HTML/CSS/JS "Floating Icon" for WordPress/External sites.
- **Persistence**: FAISS (Vector DB) & Local Session Storage.

---

## 2. Models & API Infrastructure

The project utilizes 4 distinct models to ensure a balance between speed, reasoning capability, and accuracy.

| Model Component | Model Name / ID | Provider / Engine | Purpose |
| :--- | :--- | :--- | :--- |
| **Main LLM (Brain)** | `qwen/qwen3-32b` | **Groq** | Reasoning, formatting, and generating the final response. |
| **Query Rewriter** | `llama-3.1-8b-instant` | **Groq** | Expanding user queries into multiple search terms (used as fallback). |
| **Embedding Model** | `BAAI/bge-small-en-v1.5` | Sentence Transformers | Converting text chunks into mathematical vectors for search. |
| **Reranker** | `ms-marco-MiniLM-L-6-v2` | Cross-Encoder | Deeply evaluating the relevance of retrieved document chunks. |

### APIs Connected:
1.  **Groq API**: Powers the LLM inference (Qwen3 and Llama 3.1). Groq is chosen for its extreme speed (LPUs).
2.  **SMTP (Gmail/Custom)**: Used for sending OTPs or notifications (if configured).
3.  **Hugging Face**: Used for hosting the Gradio interface and downloading model weights for Embeddings/Reranking.

---

## 3. The RAG Pipeline Workflow

When a user sends a message, the system follows this exact sequence:

### Phase 1: Query Processing
1.  **Normalization**: The message is cleaned and converted to a cache key (MD5 hash).
2.  **Cache Lookup**: If the exact question was asked recently, the answer is returned instantly from an in-memory LRU cache.
3.  **Local Expansion**: The query is expanded into up to 3 targeted search queries using a deterministic keyword map (e.g., "timing" → "office hours", "workday schedule"). This saves 3-5 seconds of LLM latency.

### Phase 2: Retrieval (Search)
1.  **Vector Search**: The system uses **FAISS** to find the most mathematically similar text chunks in the `docs/` folder.
2.  **Keyword Search (BM25)**: Complements vector search by finding exact word matches.
3.  **Hybrid Filtering**: Chunks are initially filtered by a relevance threshold (RRF).

### Phase 3: Intelligence & Reranking
1.  **Cross-Encoding**: The top retrieved chunks are sent to the **Reranker model**. Unlike standard search, the reranker looks at the *actual meaning* of the chunk relative to the question.
2.  **Context Construction**: The most relevant chunks (capped at 1500 words to avoid token limits) are assembled into a context block.

### Phase 4: Generation (The Final Answer)
1.  **Prompt Injection**: The question, context, and a pruned chat history (last 2 turns) are wrapped in a strict **Master System Prompt**.
2.  **LLM Execution**: The request is sent to `qwen/qwen3-32b`.
    - *Optimization*: The instruction `/no_think` is added to skip internal chain-of-thought, reducing response time by 40-60%.
3.  **Post-Processing**:
    - Strips `<think>` tags.
    - Removes conversational filler (e.g., "Based on the documents provided...").
    - Enforces HTML formatting (bolding key terms, `<br>` for lists).

---

## 4. Backend Components & Directory Structure

### Core Files:
- `app/main.py`: Entry point for the FastAPI application.
- `app/core/config.py`: Centralized configuration (API keys, model names, chunk sizes).
- `app/services/rag_pipeline.py`: Orchestrates the flow from query to final answer.
- `app/services/llm.py`: Handles communication with Groq and prompt engineering.
- `app/services/vector_store.py`: Manages FAISS indexing and retrieval.
- `app/services/session_store.py`: Manages user history and OTP sessions.

### Data Flow:
1.  **Input**: User Query (HTTP POST to `/chat` or Gradio WebSocket).
2.  **Processing**: FastAPI → RAG Pipeline Service → LLM Service.
3.  **Output**: JSON Response with `reply` and `retrieved_chunks` (metadata).

---

## 5. Summary of Key Features
- **Deterministic Intent Detection**: Instantly recognizes topics like "leaves" or "salary" to pull the right data without LLM delay.
- **Thinking Model Suppression**: Uses the power of Qwen3/DeepSeek while disabling their slow "thinking" phase for the final user response.
- **Hybrid Search**: Combines the precision of keywords with the "vibe" search of vectors.
- **Strict Formatting**: Enforces list styles and word counts to ensure a professional HR tone.