Spaces:
Sleeping
Sleeping
File size: 13,991 Bytes
f9c215a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 | # Architecture Overview
This document explains the architecture of the RAG (Retrieval-Augmented Generation) chatbot for the Agentic AI eBook.
## System Overview
The system follows a standard RAG pattern: documents are chunked and embedded into a vector database during ingestion, then at query time, relevant chunks are retrieved and used to generate grounded answers.
### Key Components
1. **Ingestion Pipeline** (`app/ingest.py`) - Processes the PDF, creates chunks, generates embeddings, and stores in Pinecone
2. **Vector Store** (`app/vectorstore.py`) - Wrapper around Pinecone for storing and retrieving vectors
3. **RAG Pipeline** (`app/rag_pipeline.py`) - LangGraph-based pipeline for query processing
4. **Streamlit UI** (`streamlit_app/app.py`) - Web interface for user interactions
---
## Architecture Diagram
```
INGESTION FLOW
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ PDF │───▶│ Extract │───▶│ Clean │───▶│ Chunk │ │
│ │ File │ │ Text │ │ Text │ │ (500 tokens, │ │
│ │ │ │ by Page │ │ │ │ 50 overlap) │ │
│ └──────────┘ └──────────┘ └──────────┘ └────────┬─────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Pinecone │◀───│ Upsert │◀───│ Embeddings │ │
│ │ Vector Store │ │ Vectors │ │ (MiniLM-L6-v2) │ │
│ │ │ │ │ │ 384 dims │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
QUERY FLOW
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ ┌──────────┐ │
│ │ User │ │
│ │ Query │ │
│ └────┬─────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ LANGGRAPH PIPELINE │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Embed │──▶│ Retrieve │──▶│ Calculate │ │ │
│ │ │ Query │ │ Top-K │ │ Confidence │ │ │
│ │ │ │ │ Chunks │ │ │ │ │
│ │ └─────────────┘ └──────┬──────┘ └──────┬──────┘ │ │
│ │ │ │ │ │
│ │ ▼ ▼ │ │
│ │ ┌─────────────────────────────┐ │ │
│ │ │ Generate Answer │ │ │
│ │ │ │ │ │
│ │ │ ┌─────────────────────┐ │ │ │
│ │ │ │ If OpenAI Key: │ │ │ │
│ │ │ │ → LLM Generation │ │ │ │
│ │ │ │ (grounded prompt) │ │ │ │
│ │ │ ├─────────────────────┤ │ │ │
│ │ │ │ Else: │ │ │ │
│ │ │ │ → Extractive Mode │ │ │ │
│ │ │ │ (return chunks) │ │ │ │
│ │ │ └─────────────────────┘ │ │ │
│ │ └─────────────┬───────────────┘ │ │
│ │ │ │ │
│ └──────────────────────────────────┼───────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ RESPONSE │ │
│ │ { │ │
│ │ "final_answer": "...", │ │
│ │ "retrieved_chunks": [...], │ │
│ │ "confidence": 0.92 │ │
│ │ } │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ STREAMLIT UI │ │
│ │ ┌──────────────────┐ ┌───────────────────────────────────┐ │ │
│ │ │ Chat Interface │ │ Retrieved Chunks Panel │ │ │
│ │ │ - Question box │ │ - Chunk text │ │ │
│ │ │ - Answer card │ │ - Page numbers │ │ │
│ │ │ - Confidence │ │ - Relevance scores │ │ │
│ │ └──────────────────┘ └───────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Design Decisions
### 1. Chunking Strategy
We use **500 tokens** as the target chunk size with **50-100 token overlap**. This provides:
- Enough context for meaningful retrieval
- Overlap ensures important information spanning chunk boundaries isn't lost
- Token counting via tiktoken ensures consistent chunk sizes across different text densities
**Chunk ID Format**: `pdfpage_{page}_chunk_{index}` - This makes it easy to trace retrieved content back to the source PDF page for verification.
### 2. Embedding Model Choice
We use **sentence-transformers/all-MiniLM-L6-v2**:
- Open source and free (no API costs)
- Small model (384 dimensions) = fast inference and lower storage costs
- Good quality for semantic similarity tasks
- Can run entirely on CPU
Trade-off: Larger models like OpenAI's ada-002 (1536 dims) may provide better retrieval quality, but MiniLM offers excellent cost/performance ratio for this use case.
### 3. LangGraph Pipeline
The RAG pipeline uses LangGraph for orchestration because:
- Clear separation of pipeline stages (embed → retrieve → generate)
- Easy to add/modify nodes (e.g., reranking, query expansion)
- Built-in state management
- Aligns with modern LLM application patterns
### 4. Dual-Mode Answer Generation
The system supports two modes:
**LLM Generation Mode** (with OpenAI key):
- Uses GPT-3.5-turbo for natural language generation
- System prompt strictly instructs the model to only use provided chunks
- Produces more readable, synthesized answers
**Extractive Fallback Mode** (no API key):
- Returns relevant chunks directly with minimal formatting
- Always works, even offline
- Ensures the app is functional without paid APIs
This design choice ensures the application is **always functional** regardless of API availability.
### 5. Confidence Score Computation
Confidence is computed from retrieval similarity scores:
```python
# Normalize cosine similarity from [-1, 1] to [0, 1]
normalized = (score + 1) / 2
# Use maximum normalized score as confidence
confidence = max(normalized_scores)
```
This gives users an intuitive sense of how well the retrieved chunks match their query.
---
## File Structure
```
rag-eAgenticAI/
├── app/
│ ├── __init__.py # Package exports
│ ├── ingest.py # PDF → chunks → embeddings → Pinecone
│ ├── vectorstore.py # Pinecone wrapper (create, upsert, query)
│ ├── rag_pipeline.py # LangGraph pipeline + answer generation
│ └── utils.py # Chunking, cleaning, confidence calculation
│
├── streamlit_app/
│ ├── app.py # Main Streamlit application
│ └── assets/ # Static assets (images, CSS)
│
├── samples/
│ ├── sample_queries.txt # Example questions to test
│ └── expected_responses.md # Expected JSON response format
│
├── infra/
│ └── hf_space_readme_template.md # Hugging Face Spaces config
│
├── data/ # PDF files and generated chunks (gitignored)
│
├── README.md # Main documentation
├── architecture.md # This file
├── requirements.txt # Python dependencies
├── LICENSE # MIT License
└── .gitignore # Git ignore rules
```
---
## Data Flow Summary
1. **Ingestion** (run once):
- PDF → pdfplumber → raw text by page
- Text → clean_text() → cleaned text
- Cleaned text → chunk_text() → chunks with metadata
- Chunks → SentenceTransformer → embeddings
- Embeddings → Pinecone upsert → stored vectors
2. **Query** (each user question):
- Question → SentenceTransformer → query embedding
- Query embedding → Pinecone query → top-k chunks
- Chunks + scores → compute_confidence() → confidence score
- Chunks + question → LLM/extractive → final answer
- Answer + chunks + confidence → JSON response → Streamlit UI
|