Hitan2004 commited on
Commit
8cf3090
Β·
1 Parent(s): cfa3903

Add README.md for project documentation

Browse files

Added comprehensive documentation for the Agentic Corrective RAG project, including setup instructions, architecture, tech stack, and key features.

Files changed (1) hide show
  1. README.md +166 -0
README.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Agentic Corrective RAG β€” Document Q&A
2
+
3
+ ![CI](https://github.com/Hitan547/agentic-corrective-rag/actions/workflows/ci.yml/badge.svg)
4
+ ![Python](https://img.shields.io/badge/python-3.10%2B-blue)
5
+ ![LLM](https://img.shields.io/badge/LLM-LLaMA%203.3%2070B-orange)
6
+ ![Framework](https://img.shields.io/badge/framework-LangGraph-green)
7
+
8
+ A production-aware document Q&A system that answers questions **only from your uploaded documents** β€” not from the model's imagination. Built with hybrid retrieval, cross-encoder reranking, and a self-correcting LangGraph agent that automatically retries if the answer isn't grounded in the source material.
9
+
10
+ ---
11
+
12
+ ## What It Does
13
+
14
+ Upload any PDF or TXT file, ask a question, and get an answer backed by:
15
+ - The exact source chunks it used
16
+ - A validation verdict (PASS/FAIL)
17
+ - How many self-correction retries were needed
18
+
19
+ ---
20
+
21
+ ## Architecture
22
+
23
+ ```
24
+ PDF/TXT Upload
25
+ β”‚
26
+ β–Ό
27
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
28
+ β”‚ Ingestion Pipeline β”‚
29
+ β”‚ PyMuPDF β†’ Chunking β†’ Embeddingsβ”‚
30
+ β”‚ FAISS Index + BM25 Index β”‚
31
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
32
+ β”‚
33
+ β–Ό
34
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
35
+ β”‚ Hybrid Retrieval β”‚
36
+ β”‚ FAISS (dense) + BM25 (sparse) β”‚
37
+ β”‚ β†’ RRF Fusion β”‚
38
+ β”‚ β†’ Cross-Encoder Reranking β”‚
39
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
40
+ β”‚
41
+ β–Ό
42
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
43
+ β”‚ Corrective RAG Agent β”‚
44
+ β”‚ LangGraph StateGraph β”‚
45
+ β”‚ Generate β†’ Validate β†’ Retry β”‚
46
+ β”‚ (up to 3 automatic retries) β”‚
47
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
48
+ β”‚
49
+ β–Ό
50
+ Streamlit UI + FastAPI Backend
51
+ ```
52
+
53
+ ---
54
+
55
+ ## Tech Stack
56
+
57
+ | Layer | Technology |
58
+ |---|---|
59
+ | LLM | LLaMA 3.3 70B via Groq API |
60
+ | Agent Framework | LangGraph (StateGraph) |
61
+ | Dense Retrieval | FAISS + `all-MiniLM-L6-v2` |
62
+ | Sparse Retrieval | BM25 (rank-bm25) |
63
+ | Reranker | `cross-encoder/ms-marco-MiniLM-L-6-v2` |
64
+ | PDF Parsing | PyMuPDF (fitz) |
65
+ | Backend | FastAPI |
66
+ | Frontend | Streamlit |
67
+ | Testing | pytest (unit + integration) |
68
+ | CI/CD | GitHub Actions |
69
+
70
+ ---
71
+
72
+ ## Key Features
73
+
74
+ - **Hybrid Search** β€” combines FAISS semantic search and BM25 keyword search, fused with Reciprocal Rank Fusion (RRF)
75
+ - **Cross-Encoder Reranking** β€” re-scores top candidates by reading query + chunk together for higher precision
76
+ - **Self-Correcting Agent** β€” LangGraph pipeline automatically detects hallucinations and retries up to 3 times
77
+ - **Hallucination Validation** β€” a second LLM call checks every answer against the source context before returning it
78
+ - **Session Memory** β€” remembers last 5 turns of conversation per session
79
+ - **Background Indexing** β€” document ingestion runs in the background without blocking the API
80
+ - **CI/CD** β€” unit tests run automatically on every push via GitHub Actions
81
+
82
+ ---
83
+
84
+ ## Project Structure
85
+
86
+ ```
87
+ agentic-corrective-rag/
88
+ β”œβ”€β”€ agent.py # LangGraph corrective RAG agent
89
+ β”œβ”€β”€ retriever.py # Hybrid retrieval + RRF + reranking
90
+ β”œβ”€β”€ ingestion.py # PDF/TXT ingestion + FAISS/BM25 indexing
91
+ β”œβ”€β”€ main.py # FastAPI backend
92
+ β”œβ”€β”€ app.py # Streamlit frontend
93
+ β”œβ”€β”€ config.py # Configuration and constants
94
+ β”œβ”€β”€ requirements.txt
95
+ β”œβ”€β”€ tests/
96
+ β”‚ β”œβ”€β”€ test_unit.py # Unit tests (CI)
97
+ β”‚ └── test_integration.py # Integration tests (local only)
98
+ └── .github/
99
+ └── workflows/
100
+ └── ci.yml # GitHub Actions CI pipeline
101
+ ```
102
+
103
+ ---
104
+
105
+ ## Setup
106
+
107
+ **1. Clone the repo**
108
+ ```bash
109
+ git clone https://github.com/Hitan547/agentic-corrective-rag.git
110
+ cd agentic-corrective-rag
111
+ ```
112
+
113
+ **2. Install dependencies**
114
+ ```bash
115
+ pip install -r requirements.txt
116
+ ```
117
+
118
+ **3. Set up environment**
119
+ ```bash
120
+ # Create a .env file
121
+ echo "GROQ_API_KEY=your_key_here" > .env
122
+ ```
123
+ Get your free API key at [console.groq.com](https://console.groq.com)
124
+
125
+ **4. Run the backend**
126
+ ```bash
127
+ uvicorn main:app --reload --port 8080
128
+ ```
129
+
130
+ **5. Run the frontend** (new terminal)
131
+ ```bash
132
+ streamlit run app.py
133
+ ```
134
+
135
+ **6. Open** `http://localhost:8501`, upload a PDF, click **Index documents**, then ask questions.
136
+
137
+ ---
138
+
139
+ ## Running Tests
140
+
141
+ ```bash
142
+ # Unit tests (fast, no API needed)
143
+ python -m pytest tests/test_unit.py -v
144
+
145
+ # Integration tests (requires GROQ_API_KEY)
146
+ python -m pytest tests/test_integration.py -v -m integration
147
+ ```
148
+
149
+ ---
150
+
151
+ ## How the Agent Works
152
+
153
+ 1. **Generate** β€” LLaMA 3.3 70B answers using only the retrieved chunks
154
+ 2. **Validate** β€” a second LLM call checks if every claim is supported by the context
155
+ 3. **Retry** β€” if validation fails, the agent retries with the failure reason as feedback
156
+ 4. **Stop** β€” returns the answer after PASS or after 3 retries
157
+
158
+ ---
159
+
160
+ ## Author
161
+
162
+ **Hitan K** β€” Final-year CS undergraduate (AI specialization)
163
+
164
+ [![LinkedIn](https://img.shields.io/badge/LinkedIn-hitan--k-blue)](https://linkedin.com/in/hitan-k)
165
+ [![GitHub](https://img.shields.io/badge/GitHub-Hitan547-black)](https://github.com/Hitan547)
166
+ [![HuggingFace](https://img.shields.io/badge/HuggingFace-Hitan2004-yellow)](https://huggingface.co/Hitan2004)