docs/rag_retrieval_documentation.md

RAG API Data Flow & Retrieval Architecture

This document tracks the detailed Data Flow of the RAG (Retrieval-Augmented Generation) API, with a specific focus on the Retrieval Logic. Rather than just listing HTTP endpoints, this document explains the underlying methods, conceptual flow, and how the Domain Models, Ports, Use Cases, and Infrastructure Adapters interact to fetch, rerank, and summarize enterprise news data.

---

🏗️ 1. Architecture Overview (Hexagonal Architecture)

The RAG API relies on Hexagonal Architecture (Ports and Adapters). It strongly separates business logic from infrastructure frameworks.

• Domain/Models: The central, pure data structures representing the state (e.g., ChatRequest, User).
• Ports (Interfaces): Abstract definitions of what the system needs to do (e.g., VectorStorePort, LlmPort).
• Use Cases: The actual business logic where the retrieval steps, filtering, and flow occur.
• Adapters: The concrete implementation of Ports using external technologies (e.g., Qdrant, OpenAI, Redis, Postgres).

---

📂 2. File Directory Breakdown & Responsibilities

src/api/ (Primary Adapters / The Front Door)

• routes/rag.py: Exposes the /chat and /chat/stream endpoints. Role: Accepts the incoming HTTP payload, validates the JWT token (via Depends(get_current_user)), and forwards the request directly to the AgentRouterUseCase.
• dependencies.py: The Dependency Injection container. Role: Wires the concrete Infrastructure Adapters (e.g., QdrantAdapter, BgeEmbedderAdapter) to their respective Ports, and injects them into the Use Cases. Ensures components are instantiated only once.

src/core/domain/ (Core Data)

• schemas.py: Defines Pydantic validation models. Role: Houses ChatRequest (contains query, top_k, session_id, source_filter, etc.) which acts as the transport object through the system.

src/core/ports/ (The Interfaces)

• embedder_port.py: Defines encode_query().
• vector_store_port.py: Defines search().
• reranker_port.py: Defines rerank().
• llm_port.py: Defines generate() and generate_stream().
• cache_port.py: Defines get(), set(), and generate_exact_hash().

src/core/use_cases/ (The Business Logic Engine)

• agent_router_use_case.py: Role: The gateway. Analyzes the user's intent. Routes the request to AccountUseCase (if the user is asking about personal profile data) or RagChatUseCase (if asking about news).
• rag_chat_use_case.py: Role: The Heavy Lifter. Responsible for the entire Retrieval Logic flow. Contains methods like _extract_intents, _build_context, _limit_context, and _compress_document.
• account_use_case.py: Role: A secondary flow for handling user-specific DB aggregations (billing, history) rather than searching Vector DBs.

src/infrastructure/adapters/ (Concrete Infrastructure)

• redis_adapter.py: Role: Connects to the caching layer to prevent duplicate LLM processing calls.
• qdrant_adapter.py: Role: Orchestrates the query_points API call to Qdrant, fusing Dense and Sparse vector retrieval (Hybrid Search).
• bge_embedder_adapter.py: Role: Instantiates the massive BGE-M3 model (using FlagEmbedding). Converts text strings into multi-dimensional arrays (Dense and Lexical Sparse weights).
• bge_reranker_adapter.py: Role: Uses a Cross-Encoder to compare the user query and the retrieved documents string-by-string for absolute semantic precision.
• openai_adapter.py / ollama_adapter.py: Role: Connects to an external OpenAI API or Local Llama-3 instance to generate text.

---

🌊 3. The Retrieval Logic: Step-by-Step Data Flow Example

Scenario: A user submits the query: "What happened with Apple stock recently?"

Step 1: Ingestion & Intent Routing (agent_router_use_case.py)

1. Input: ChatRequest(query="What happened with Apple stock recently?", top_k=5)
2. Action: The API endpoint passes this to the AgentRouterUseCase.
3. LLM Classification: The Router asks the LLM: "Is this a NEWS search or an ACCOUNT search?"
4. Output: The LLM outputs NEWS. The Router forwards the request to the RagChatUseCase.

Step 2: Semantic Caching (redis_adapter.py)

1. Action: cache_port.generate_exact_hash() calculates an SHA-256 hash or deterministic key for the query string.
2. Check: Does this key exist in Redis?
3. If Yes: Return the answer instantly (0ms LLM time).
4. If No: Proceed with the expensive pipeline.

Step 3: Self-Query Extraction (rag_chat_use_case.py -> _extract_intents())

1. Action: The LLM analyzes the user's natural language query to dynamically extract metadata and physical parameters for the vector database.
2. Example Prompting: The LLM is provided with a system prompt like: "Extract the temporal constraints and target sources from the user query into JSON format. Valid sources: ['reuters', 'bloomberg']."
3. Execution: The LLM analyzes "What happened with Apple stock recently?"
4. Output Deduction: From the word "recently", it deduces the temporal boundary and constructs the following JSON structure:

   {
       "days_back": 3,
       "source": null
   }
   

5. Mapping: The RagChatUseCase parses this JSON. If days_back is present, it constructs a Qdrant models.Filter to physically exclude older documents from the multidimensional search space before the costly vector math occurs.

Step 4: Embedding / Vectorization (bge_embedder_adapter.py)

1. Action: encode_query() is called.
2. Model Processing: The BGE-M3 model tokenizes the string.
3. Output: Returns a Dict containing:
   • dense: [0.123, -0.456, 0.789, ... 1024 dimensions]
   • sparse: {"indices": [102, 451, ...], "values": [0.92, 0.44, ...]}

Step 5: Hybrid Vector Search (qdrant_adapter.py)

1. Action: Passes the query_vectors and the days_back=3 filter into vector_store_port.search().
2. Qdrant Processing: Qdrant performs a Fusion Query (Reciprocal Rank Fusion - RRF). It fetches the top 20 nearest neighbors from BOTH the Dense mathematical space AND the Sparse keyword space.
3. Output: Returns a List of raw SearchResult documents.

Step 6: Temporal Bias Scoring (rag_chat_use_case.py -> _build_context())

1. Action: Evaluates the published_at metadata of every hit.
2. Calculation: It deliberately decays the score of older articles via a mathematical multiplier (e.g., score_multiplier = max(0.5, 1.0 - (days_old / 60))).
3. Output: A dynamically re-scored list, preferring fresh data.

Step 7: Cross-Encoder Reranking (bge_reranker_adapter.py)

1. Action: For the top 20 remaining documents, the Reranker pairs the Query + Document Text together ([[query, doc1], [query, doc2]]).
2. Model Processing: The HuggingFace FlagReranker calculates exact semantic overlap.
3. Output: Returns the strict Top 5 (top_k) documents, guaranteed to be specifically relevant.

Step 8: Contextual Compression (rag_chat_use_case.py -> _limit_context())

1. Action: _limit_context uses tiktoken to count how many tokens the Top 5 documents contain.
2. Check: Are they over the 3000 Token limit?
3. Compression Loop: If they are over the limit, it calls _compress_document().
4. LLM Summarization: Passes the overflowing document string to the LLM with the instruction: "Extract pure facts... relevant to the query." The massive document strings are squashed down to bullet-point facts.
5. Output: A tightly packed context_text string ready for generation.

Step 9: Final Generation (llm_port.py)

1. Action: The packed context_text, the User query, and the recent Chat History are combined into the Final Prompt.
2. Model Processing: The LLM interprets the compressed context.
3. Output: The Final string ("Apple stock surged 4% after the latest earnings report...").
4. Cleanup: This answer is saved to both Postgres (chat_history_db) and Redis (cache), and returned to the API client.

---

📈 4. A4 Analysis and Future Updates

A4 Analysis (Current System Standing)

Dimension	Analysis & Findings
Resilience & Scalability	High. The Hexagonal architecture successfully decoupled Qdrant, Postgres, and the LLMs. We can swap OpenAiAdapter for OllamaAdapter simply by changing one dependency provider without touching the Business Logic flow. Missing dependencies (e.g., FlagEmbedding) gracefully utilize dummy fallbacks avoiding hard API crashes.
Retrieval Accuracy	Exceptional. We utilize a 3-Stage filtering mechanism: Semantic similarity (Dense), Lexical accuracy (Sparse), and absolute context alignment (Reranker). The addition of dynamic Temporal Biasing prevents the hallucination of historical news as current events.
Cost & Latency Management	Optimized. The implementation of Redis Semantic Caching guarantees that recursive identical intent avoids LLM round-trip costs. The AgentRouterUseCase ensures unrelated general questions (Account, Billing) never touch expensive Vector DB aggregations.
Memory Constraint Handling	Innovative. By employing _compress_document, the system prevents context-window truncation, ensuring critical tail-end entities still influence the LLM's final generation.

Proposed Future Updates (Roadmap)

1. Semantic Cache Refinement: Currently, the RedisAdapter relies on an exact SHA-256 string hash. Update: Calculate an actual LLM embedding of the prompt (Dense Vector) and store it in Redis. Use a Cosine-Similarity threshold (>0.95) to intercept semantically identical (but textually different) questions (e.g., "Apple stock" vs "AAPL share price").
2. Analytic Trend Fusion Enhancement: In _build_context, we fetch trending entities from ClickHouse. Update: Send these trending entities into the Agent Router so the system can proactively recommend or correlate user interactions with macroeconomic spikes before they ask.
3. Ollama Deployment Readiness: Test the bge_embedder_adapter and bge_reranker_adapter simultaneously against an active OllamaAdapter container to benchmark hardware-level VRAM bottlenecks on local inference machines.
4. Knowledge Graph Integration: Extract Triples (Subject-Predicate-Object) during the _compress_document step to progressively construct a Graph Database (Neo4j) alongside the Vector DB (Qdrant) for Multi-Hop reasoning queries in the future.