PROJECT_TECHNICAL_SPEC.md

Martechsol HR Assistant: Project Technical Specification & Workflow

This document provides a comprehensive overview of the Martechsol HR Assistant, covering its architecture, the models used, API integrations, and the end-to-end data workflow.

---

1. Project Architecture Overview

The system is a high-performance Retrieval-Augmented Generation (RAG) application designed to provide precise, grounded answers based on employee handbooks and HR documents.

High-Level Stack:

• Language: Python 3.10+
• API Framework: FastAPI
• Server: Uvicorn
• UI: Gradio (for direct testing/Hugging Face Spaces)
• Web Integration: Custom HTML/CSS/JS "Floating Icon" for WordPress/External sites.
• Persistence: FAISS (Vector DB) & Local Session Storage.

---

2. Models & API Infrastructure

The project utilizes 4 distinct models to ensure a balance between speed, reasoning capability, and accuracy.

Model Component	Model Name / ID	Provider / Engine	Purpose
Main LLM (Brain)	qwen/qwen3-32b	Groq	Reasoning, formatting, and generating the final response.
Query Rewriter	llama-3.1-8b-instant	Groq	Expanding user queries into multiple search terms (used as fallback).
Embedding Model	BAAI/bge-small-en-v1.5	Sentence Transformers	Converting text chunks into mathematical vectors for search.
Reranker	ms-marco-MiniLM-L-6-v2	Cross-Encoder	Deeply evaluating the relevance of retrieved document chunks.

APIs Connected:

1. Groq API: Powers the LLM inference (Qwen3 and Llama 3.1). Groq is chosen for its extreme speed (LPUs).
2. SMTP (Gmail/Custom): Used for sending OTPs or notifications (if configured).
3. Hugging Face: Used for hosting the Gradio interface and downloading model weights for Embeddings/Reranking.

---

3. The RAG Pipeline Workflow

When a user sends a message, the system follows this exact sequence:

Phase 1: Query Processing

1. Normalization: The message is cleaned and converted to a cache key (MD5 hash).
2. Cache Lookup: If the exact question was asked recently, the answer is returned instantly from an in-memory LRU cache.
3. Local Expansion: The query is expanded into up to 3 targeted search queries using a deterministic keyword map (e.g., "timing" → "office hours", "workday schedule"). This saves 3-5 seconds of LLM latency.

Phase 2: Retrieval (Search)

1. Vector Search: The system uses FAISS to find the most mathematically similar text chunks in the docs/ folder.
2. Keyword Search (BM25): Complements vector search by finding exact word matches.
3. Hybrid Filtering: Chunks are initially filtered by a relevance threshold (RRF).

Phase 3: Intelligence & Reranking

1. Cross-Encoding: The top retrieved chunks are sent to the Reranker model. Unlike standard search, the reranker looks at the actual meaning of the chunk relative to the question.
2. Context Construction: The most relevant chunks (capped at 1500 words to avoid token limits) are assembled into a context block.

Phase 4: Generation (The Final Answer)

1. Prompt Injection: The question, context, and a pruned chat history (last 2 turns) are wrapped in a strict Master System Prompt.
2. LLM Execution: The request is sent to qwen/qwen3-32b.
   • Optimization: The instruction /no_think is added to skip internal chain-of-thought, reducing response time by 40-60%.
3. Post-Processing:
   • Strips <think> tags.
   • Removes conversational filler (e.g., "Based on the documents provided...").
   • Enforces HTML formatting (bolding key terms, <br> for lists).

---

4. Backend Components & Directory Structure

Core Files:

• app/main.py: Entry point for the FastAPI application.
• app/core/config.py: Centralized configuration (API keys, model names, chunk sizes).
• app/services/rag_pipeline.py: Orchestrates the flow from query to final answer.
• app/services/llm.py: Handles communication with Groq and prompt engineering.
• app/services/vector_store.py: Manages FAISS indexing and retrieval.
• app/services/session_store.py: Manages user history and OTP sessions.

Data Flow:

1. Input: User Query (HTTP POST to /chat or Gradio WebSocket).
2. Processing: FastAPI → RAG Pipeline Service → LLM Service.
3. Output: JSON Response with reply and retrieved_chunks (metadata).

---

5. Summary of Key Features

• Deterministic Intent Detection: Instantly recognizes topics like "leaves" or "salary" to pull the right data without LLM delay.
• Thinking Model Suppression: Uses the power of Qwen3/DeepSeek while disabling their slow "thinking" phase for the final user response.
• Hybrid Search: Combines the precision of keywords with the "vibe" search of vectors.
• Strict Formatting: Enforces list styles and word counts to ensure a professional HR tone.