docs/Software Requirements Specification.md

#  **Software Requirements Specification (SRS)**

## **1. Introduction**

This document outlines the technical specifications for an Agentic RAG system built on Hugging Face **SmolAgents**, using **pgvector** for storage and **TruLens** for evaluation.

## **2. System Architecture**

The system follows a modular "Code-First" Agent architecture:

1. **Orchestrator:** smolagents.CodeAgent (interprets user query and writes execution code).  
2. **Tool Layer:** Custom pgvector retriever tool with integrated reranking logic.  
3. **Storage Layer:** PostgreSQL with the vector extension.  
4. **Evaluation Layer:** TruCustomApp recorder with OpenAI/HF/Gemini feedback providers.

## **3. Detailed Component Specifications**

### **3.1 Data Layer (PostgreSQL + pgvector)**

* **Schema:**  
  * id: UUID (Primary Key)  
  * content: TEXT (Document chunk)  
  * embedding: VECTOR(384)  
  * metadata: JSONB (Source URL, page number, etc.)  
* **Search Type:** HNSW or IVFFlat index for sub-100ms vector similarity lookups.

### **3.2 Retrieval & Reranking Tool**

* **First Stage:** Dense retrieval via pgvector (Cosine Similarity) returning Top 20 results.  
* **Second Stage:** Reranking via CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2') — ~85 MB, CPU-friendly, no GPU required.  
* **Logic:** The tool takes a string query, computes the embedding, queries PG, reranks the 20 results, and returns a formatted string of the top 5.

### **3.3 Agentic Logic (SmolAgents)**

The system MUST provide a configuration toggle to switch between the following Model Providers:

* **Option A: OpenAI Ecosystem**  
  * Includes standard OpenAI (GPT-4o, GPT-4o-mini) and **Azure OpenAI** instances.  
  * Uses OpenAIServerModel or LiteLLMModel for integration.  
* **Option B: Google Gemini Family**  
  * Includes gemini-3.1-flash-lite-preview and gemini-flash-lite-latest variants.  
  * Uses native Google Generative AI integration or LiteLLMModel.  
* **Reasoning:** The agent is configured with max_steps=5 to allow for query expansion or multi-step searching if the first attempt fails.

### **3.4 Evaluation Framework (TruLens 2.x OTEL)**

The backend uses **TruApp** with OTEL-instrumented replay, **RAG Triad** metrics (Context Relevance, Groundedness, Answer Relevance), and an LLM-as-judge whose provider is configured separately from the agent (`TRULENS_PROVIDER`: OpenAI, Azure OpenAI, Gemini, or none). Evaluation runs **asynchronously** after the HTTP response; the worker synchronizes on TruLens feedback completion up to **`TRULENS_FEEDBACK_TIMEOUT`** before persisting to `evaluation_results`. Retrieval context for scoring prefers the **authoritative reranked top-5 tool string** when the retriever ran.

* **Metrics (The RAG Triad):**  
  * Context relevance: User query vs. reranked retrieval context.  
  * Groundedness: Final answer vs. reranked retrieval context.  
  * Answer relevance: User query vs. final answer.

## **4. Technical Constraints**

* **Environment:** Python 3.10+.  
* **API-First Architecture:** Inference for the primary Orchestrator and Evaluation Judge is handled via external APIs. Local compute is reserved for Embedding and Reranking models.  
* **Credential Management:** Secure handling of API keys via environment variables:  
  * OPENAI_API_KEY / AZURE_OPENAI_API_KEY  
  * AZURE_OPENAI_ENDPOINT  
  * GOOGLE_API_KEY  
* **Connectivity:** The execution environment requires outbound HTTPS access to api.openai.com, \*.openai.azure.com, and generativelanguage.googleapis.com.  
* **Memory (Hybrid):** Reranking model (Stage 2) is CPU-only (~85 MB RAM); no GPU required. LLM reasoning is offloaded to the cloud.  
* **Security:** Agent code execution must be sandboxed. API keys must not be logged or exposed in the TruLens dashboard traces.

## **5. Performance Requirements**

* **Reranking Latency:** \< 500ms for 20 document pairs (local inference).  
* **Vector Query:** \< 50ms for 100k vectors.  
* **Inference Latency:** Target \< 3s for LLM generation steps; depends on provider (OpenAI vs. Gemini) and network overhead.  
* **Evaluation Overhead:** TruLens recording should be asynchronous or sampled to avoid blocking the user response.