README.md

📄 Gemini RAG Backend System (FastAPI)

Production-grade Retrieval-Augmented Generation (RAG) backend built with FastAPI, FAISS (ANN), and Google Gemini — featuring hybrid retrieval, HNSW indexing, cross-encoder reranking, evaluation logging, and analytics.

This repository demonstrates how modern AI backend systems are actually built in industry.

🚀 What This Project Is

This is a full RAG backend system that:

Ingests large PDF/TXT documents

Builds vector indexes with Approximate Nearest Neighbor (ANN) search

Answers questions using grounded LLM responses

Tracks confidence, known/unknown answers, and usage analytics

Supports production constraints (file limits, caching, logging)

The project evolved from RAG v1 → RAG v2, adding real-world scalability and observability.

✨ Key Features (RAG v2)

📥 Document Ingestion

Upload PDF and TXT files

Sentence-aware chunking with overlap

Page-level metadata for citations

🔍 Retrieval (Hybrid + ANN)

FAISS HNSW ANN index for scalable similarity search

Cosine similarity via normalized embeddings

Keyword boosting for lexical relevance

🧠 Reranking (Quality Boost)

Cross-Encoder (ms-marco-MiniLM) reranking

Improves relevance beyond raw vector similarity

Mimics production search stacks (retrieve → rerank)

🤖 LLM Generation

Google Gemini 2.5 Flash

Strict grounding: answers only from retrieved context

Honest fallback: "I don't know" when unsupported

📊 Evaluation & Monitoring

Logs every query:

retrieved chunk count

confidence score

known vs unknown answers

JSONL logs for offline analysis

Built-in analytics dashboard

📈 Analytics Dashboard

Total queries

Knowledge rate

Average confidence

Unknown query tracking

Recent query history

Dark / Light mode UI

🛡️ Production Safeguards

File upload size limits (configurable)

API quota handling

Caching to reduce LLM calls

Clean error handling

Persistent vector store


🏗️ System Architecture


Frontend (HTML / JS)
        ↓

FastAPI Backend
        ↓

Document Ingestion (PDF / TXT)
        ↓

Sentence Chunking + Metadata
        ↓

Embeddings (SentenceTransformers)
        ↓

FAISS ANN Index (HNSW)
        ↓

Hybrid Retrieval (Vector + Keyword)
        ↓

Cross-Encoder Reranking
        ↓

Prompt Assembly
        ↓

Google Gemini LLM
        ↓

Answer + Confidence + Citations
        ↓

Evaluation Logging + Analytics



🧠 Core Concepts Demonstrated

Retrieval-Augmented Generation (RAG)

Why pure LLMs hallucinate

How grounding fixes factual accuracy

Vector search vs keyword search

Hybrid retrieval strategies

Approximate Nearest Neighbor (ANN)

Why brute-force search fails at scale

HNSW indexing for fast similarity search

efConstruction vs efSearch trade-offs

Reranking

Why top-K vectors ≠ best answers

Cross-encoder reranking for relevance

Industry-standard retrieval pipelines

Evaluation & Observability

Measuring known vs unknown

Confidence as a heuristic, not truth

Logging for iterative improvement

Analytics-driven RAG tuning

Real Backend Engineering

API limits & retries

Persistent storage

Clean Git hygiene

Incremental system evolution


🛠️ Tech Stack

Backend

Python

FastAPI

FAISS (HNSW ANN)

SentenceTransformers

Cross-Encoder (MS MARCO)

Google Gemini API

PyPDF

python-dotenv

Frontend

HTML

CSS

Vanilla JavaScript (Fetch API)

Tooling & Platform

VS Code

Git & GitHub

Docker

Hugging Face Spaces (deployment)

Virtual Environments (venv)



⚙️ Setup & Run Locally

1️⃣ Clone Repository

git clone https://github.com/LVVignesh/gemini-rag-fastapi.git

cd gemini-rag-fastapi

2️⃣ Create Virtual Environment

python -m venv venv

venv\Scripts\activate

3️⃣ Install Dependencies

pip install -r requirements.txt

4️⃣ Configure Environment Variables

GEMINI_API_KEY=your_api_key_here

5️⃣ Run Server

uvicorn main:app --reload



⚠️ Known Limitations

Scanned/image-only PDFs require OCR (not included)

Confidence score is heuristic

Very large corpora may require:

batch ingestion

sharding

background workers



🚀 Live Demo

👉 Hugging Face Spaces
https://huggingface.co/spaces/lvvignesh2122/Gemini-Rag-Fastapi-Pro

📜 License

MIT License