app/backend/ARCHITECTURE.md

FinBot Backend Architecture - Complete Guide

Overview

The FinBot RAG backend is organized around a layered architecture with clear separation of concerns. Each module handles a specific domain of the system, allowing for maintainability, testability, and scalability.

REQUEST
  ↓
[main.py] - FastAPI endpoints
  ↓
[pipeline/rag_pipeline.py] - Orchestration
  ├─ [guardrails/input_guards.py] - Validate queries
  ├─ [routing/router.py] - Route query to collection
  ├─ [retrieval/rbac_retriever.py] - RBAC-enforced retrieval
  ├─ [Groq API] - Generate answer
  ├─ [SentenceTransformer] - Generate embeddings
  └─ [guardrails/output_guards.py] - Validate response
  ↓
RESPONSE

---

Directory Structure & File Organization

1. Root Level Files

main.py (FastAPI Application)

• Purpose: HTTP API entry point
• Responsibility:
  • Define endpoints (routes)
  • Request/response validation
  • CORS setup
  • Lifecycle management
• Key Endpoints:
  • POST /api/chat - Main chat endpoint
  • GET /api/health - System health check
  • GET /api/users/{username} - Get user info
  • POST /admin/create-user - Admin user creation
  • POST /admin/ingest - Document ingestion trigger

Key Pattern: Controllers/Handlers that delegate to services

---

config.py (Configuration & Constants)

• Purpose: Centralize all configuration
• Contains:
  • User roles enum (EMPLOYEE, FINANCE, ENGINEERING, MARKETING, C_LEVEL)
  • Document collections enum (GENERAL, FINANCE, ENGINEERING, MARKETING, HR)
  • CRITICAL: ROLE_COLLECTION_ACCESS mapping (defines RBAC rules)
  • Demo users for testing
  • LLM config (model, temperature, tokens)
  • Retrieval config (top_k, score_threshold)

Key Pattern: Single source of truth for all constants

Example RBAC Rule:

ROLE_COLLECTION_ACCESS = {
    "employee": ["general"],
    "finance": ["general", "finance"],
    "engineering": ["general", "engineering"],
    "c_level": ["general", "finance", "engineering", "marketing", "hr"],
}

---

vector_store.py (Qdrant Vector Database)

• Purpose: Interface to Qdrant vector database (Cloud or Local)
• Responsibility:
  • Connect to Qdrant Cloud for persistent, shared storage
  • Fallback to local persistent storage for disconnected development
  • Create/manage vector collections and enforce RBAC filters
• Key Metadata Fields:
  • access_roles: Which roles can access this chunk
  • collection_name: Which collection (finance, engineering, etc.)
  • source_file: Original document
  • chunk_position: Position in hierarchical structure

Key Pattern: Singleton pattern (single instance per app)

---

metadata_schema.py (Data Models)

• Purpose: Pydantic models for data validation
• Key Classes:
  • Chunk - Represents a searchable document chunk
  • RAGResponse - Full pipeline response
  • QueryMetadata - Metadata about the query
  • RetrievalResult - Retrieval layer output

Key Pattern: Schema validation & type safety

---

2. Pipeline Module (pipeline/)

rag_pipeline.py (Orchestration Engine)

• Purpose: Orchestrate the entire RAG flow
• Thought Process:
  • A query goes through 5 distinct stages
  • Each stage has a specific responsibility
  • Each stage can fail independently and is logged

5-Stage Pipeline:

Stage 1: Input Validation (Guardrails)

Query → rate_limit check → injection detection → OffTopic check → PII check
        ↓ If fails, return error immediately

Stage 2: Semantic Routing

Query → Router (semantic-router) → Select collection
        "Show me sales data" → Route to FINANCE collection
        "How does the API work" → Route to ENGINEERING collection

Stage 3: RBAC-Enforced Retrieval

User Role + Selected Collection → Check access → Query vector store
        Employee wants FINANCE → DENIED
        Finance user wants FINANCE → ALLOWED → Retrieve chunks

Stage 4: LLM Generation

Question + Retrieved Chunks → Groq (Llama 3.3 70B) → Generate answer
        Uses retrieved chunks as context (RAG)

Stage 5: Output Validation (Guardrails)

Generated Answer → Check for hallucinations → Check for missing citations
        ↓ If issues detected, flag in response

Return Complete Response:

• answer: The generated response
• sources: Which chunks were used
• route: Which collection was queried
• accessible_collections: What user can access
• guardrail_flags: Any warnings/issues detected
• rbac_denied: Was access denied?

Key Pattern: Pipeline Pattern (chain of processors)

---

3. Routing Module (routing/)

router.py (Semantic Query Routing)

• Purpose: Determine which collection a query should search
• Technology: SemanticRouter (ML-based routing)
• Examples:

  "What are Q4 financials?" → FINANCE
  "How do I set up the API?" → ENGINEERING
  "What's our market strategy?" → MARKETING
  "What are company policies?" → GENERAL
  

semantic_router_config.py (Router Training Data)

• Purpose: Define routes and training examples
• Contents: Route definitions with example queries for each route
• How It Works: Semantic router learns from examples to categorize new queries

Key Pattern: Configuration-driven machine learning

---

4. Retrieval Module (retrieval/)

rbac_retriever.py (RBAC-Enforced Vector Search)

• Purpose: Retrieve chunks while enforcing access control
• Critical Logic:

  1. Get user's accessible collections (from config)
  2. Validate requested collections against user's access
  3. Search vector store ONLY in allowed collections
  4. Return chunks user is authorized to see
  

Key Principle: RBAC filter is applied at vector store level, not post-processing

Examples:

• Employee asks for FINANCE data → Denied, no chunks returned
• Finance user asks for FINANCE data → Allowed, chunks returned with access roles verified

Key Pattern: Authorization layer (middleware pattern)

---

user_auth.py (User Management)

• Purpose: User authentication & authorization
• Responsibility:
  • Store user profiles (role, department, etc.)
  • Map roles to accessible collections (using config.py)
  • Validate user roles
• Demo Users: Pre-defined users for testing

Key Pattern: Identity & Permissions service

---

5. Guardrails Module (guardrails/)

input_guards.py (Input Validation)

• Purpose: Validate and sanitize user input BEFORE processing
• Checks:
  • Rate Limiting: Max queries per user per time period
  • Injection Detection: SQL/prompt injection attempts
  • Off-Topic Detection: Is query relevant to knowledge base?
  • PII Detection: Does query ask for sensitive data?

Examples:

Query: "; DROP TABLE users; --" 
→ Detected as injection → Rejected

Query: "What's my credit card number?"
→ Detected as PII request → Rejected

Query: "Tell me a joke"
→ Detected as off-topic → Rejected

Query: "Show me Q4 sales"
→ Passes all checks → Continue to routing

Key Pattern: Defense-in-depth (multiple checks)

---

output_guards.py (Output Validation)

• Purpose: Validate LLM response BEFORE returning to user
• Checks:
  • Hallucination Detection: Is answer grounded in source documents?
  • Citation Quality: Are sources properly cited?
  • Completeness: Does answer address the query?

Examples:

Answer contains facts not in source docs
→ Flag as potential hallucination → Warn user

Answer references sources that weren't used
→ Flag as citation error → Warn user

Key Pattern: Quality assurance layer

---

6. Ingestion Module (ingestion/)

docling_parser.py (Document Parsing)

• Purpose: Parse complex documents (PDF, DOCX, Markdown)
• Responsibility:
  • Convert documents to structured text
  • Preserve document hierarchy (sections, subsections, etc.)
  • Extract metadata (titles, headings, structure)
• Output: Parsed document with hierarchical structure

Key Pattern: Standard parser pattern

---

hierarchical_chunker.py (Smart Chunking)

• Purpose: Break documents into optimal chunks
• Thought Process Behind Chunking:

  Raw Document (10+ pages)
    ↓
  Split by sections (respects hierarchy)
    ↓
  Split by semantic meaning (paragraphs, lists)
    ↓
  Create recursive chunks (overlap for context)
    ↓
  Tag chunks with metadata (section, source, role access)
    ↓
  Final Chunks (good context, minimal overlap)
  

Why Hierarchical?

• Maintains document structure
• Preserves context (related info together)
• Enables collection-level access control
• Improves retrieval relevance

Key Pattern: Recursive chunking

---

document_ingester.py (Orchestration of Ingestion)

• Purpose: Coordinate parsing → chunking → embedding → storage
• Pipeline:

  Document → Parse (docling_parser)
           → Chunk (hierarchical_chunker)
           → Generate embeddings (SentenceTransformer locally)
           → Tag with access roles (from config)
           → Store in vector DB (Qdrant)
  

Key Pattern: Pipeline pattern applied to ingestion

---

Design Patterns Used

1. Layered Architecture

Each layer has a specific responsibility and depends on layers below, but not above:

API Layer (main.py)
    ↓
Business Logic Layer (pipeline/)
    ↓
Data Access Layer (retrieval/, vector_store/)
    ↓
External Services (Groq, Qdrant)

2. Singleton Pattern

Single instances of expensive resources:

• Vector store (get_vector_store())
• RAG pipeline (get_rag_pipeline())
• User manager (get_user_manager())

3. Pipeline Pattern

Processes flow through stages:

• RAG pipeline (input → routing → retrieval → LLM → output)
• Ingestion pipeline (parse → chunk → embed → store)

4. Factory Pattern

Create instances via factory functions:

pipeline = get_rag_pipeline()
retriever = get_rbac_retriever()
router = get_router()

5. Configuration-Driven Design

Behavior controlled by config.py:

• Collection access rules
• User roles
• LLM settings
• No hardcoded values

6. Authorization Layer

RBAC enforced at retrieval layer:

• Not post-filtering
• Vetted at vector store level
• Cannot bypass

---

Data Flow Example: User Query

User: "Show me the Q4 sales report"
Role: finance

1. REQUEST
   POST /api/chat
   { "query": "Show me the Q4 sales report", "user_role": "finance" }

2. MAIN.PY (FastAPI)
   Validates request format, calls pipeline.answer_query()

3. PIPELINE - STAGE 1: INPUT GUARDS
   ✓ Not a rate limit violation
   ✓ Not an injection attack
   ✓ Not off-topic
   ✓ No PII request

4. PIPELINE - STAGE 2: ROUTING
   Query → Router → "This is about SALES/FINANCE"
   Route: FINANCE collection

5. PIPELINE - STAGE 3: RBAC RETRIEVAL
   User role: finance
   Requested collection: FINANCE
   ✓ finance role CAN access FINANCE collection
   Query Qdrant ONLY in FINANCE collection
   Returns: [chunk1, chunk2, chunk3] (Q4 sales data)

6. PIPELINE - STAGE 4: LLM GENERATION
   Prompt: context + query + instructions
   "Based on the Q4 sales data below, answer: Show me the Q4 sales report"
   Groq generates comprehensive answer

7. PIPELINE - STAGE 5: OUTPUT GUARDS
   ✓ Answer is grounded in source documents
   ✓ Sources are properly cited
   ✓ No hallucinations detected

8. RESPONSE
   {
     "answer": "Q4 sales totaled $4.2M...",
     "sources": [chunk1, chunk2, chunk3],
     "route": "finance",
     "user_role": "finance",
     "accessible_collections": ["general", "finance"],
     "guardrail_flags": [],
     "rbac_denied": false
   }

---

RBAC Enforcement Example

Scenario 1: Authorized Access

User: emp_john
Role: employee
Query: "Company policies"

RBAC Check:
- Employee can access: [general]
- Query routed to: general ✓
- Allowed collections: general ✓
→ Retrieval succeeds

Scenario 2: Unauthorized Access

User: emp_john
Role: employee
Query: "What are company financials?"

RBAC Check:
- Employee can access: [general]
- Query routed to: finance ✗
- Allowed collections: general ✗
→ RBAC DENIED
→ No chunks retrieved
→ Response: "You don't have access to financial data"

---

Key Design Decisions

1. Why Semantic Routing?

• Automatically routes queries to right collection
• No manual labeling needed
• Scales with new collections

2. Why Hierarchical Chunking?

• Preserves document context
• Enables collection-level access control
• Improves relevance

3. Why RBAC at Vector Store Level?

• Cannot be bypassed
• Retrieval Engine: RBAC-aware vector search
• Generation Engine: Groq (Llama 3.3 70B)
• Security Layer: Input/Output Guardrails
• Defense in depth
• Prevents malicious input
• Ensures answer quality
• Auditable (logged)

4. Why Separate Input/Output Guards?

• Defense in depth
• Prevents malicious input
• Ensures answer quality
• Auditable (logged)

5. Why Singleton Pattern?

• Vector store connections are expensive
• LLM client setup is expensive
• Router models take time to load
• Reuse same instance across requests

---

Summary

The backend is architected as a layered pipeline where:

1. Configuration (config.py) is the single source of truth for RBAC rules
2. API (main.py) is the thin HTTP layer
3. Pipeline (pipeline/) orchestrates the flow
4. Guardrails protect against bad input and bad output
5. Routing directs to correct collection
6. Retrieval enforces access control
7. Ingestion prepares documents for search

Each component is focused, testable, and replaceable. This design enables building a robust, secure RAG system that demonstrates enterprise-grade RBAC and quality assurance patterns.