README.md

metadata

title: Document RAG GPT
emoji: 📄
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.16.0
app_file: app.py
pinned: false
license: mit

📄 RAG Document Chat

A Retrieval-Augmented Generation (RAG) chat application. Ask questions answered strictly from document content, with guardrails against prompt injection and out-of-scope queries. The app ships preloaded with the Constitution of India so it works immediately, and you can also upload your own PDF, DOCX, or TXT files.

Built-in dataset

On startup the app indexes a small corpus (in data/):

• Fundamental-Rights.pdf — Part III, Articles 12–35
• Directive-Principles.docx — Part IV, Articles 36–51
• Preamble-and-Fundamental-Duties.txt — Preamble and Article 51A

This exercises all three file loaders (PDF, DOCX, TXT). The bare text of the Constitution is freely reproducible in India (Copyright Act, 1957, s. 52); it was compiled from public sources via build_corpus.py and is provided for demonstration (see indiacode.nic.in for the authoritative current text).

Sample questions: "What does Article 21 protect?", "What is the right against exploitation?", "List the Fundamental Duties", "What does the Preamble say?". An out-of-scope question like "What's the weather today?" is correctly refused.

Document scope (by design)

The assistant answers from one document set at a time:

• Default: the built-in Constitution of India corpus described above.
• After upload: once you upload and process your own files, the system uses only those documents for the rest of the session; the built-in corpus is no longer searched.

This single-source behaviour is an intentional design choice — when a user brings their own documents, answers should be grounded solely in those documents, with no blending from an unrelated default corpus. Starting a new session (or restarting the Space) reverts to the built-in corpus.

Features

• Multi-format upload — PDF, DOCX (paragraphs + tables), and TXT.
• RAG pipeline — chunking → embeddings → vector search → grounded generation.
• Guardrails
  • Input validation (empty / too-short / too-long).
  • Direct prompt-injection detection on the user's query.
  • Indirect prompt-injection defence: retrieved text is passed to the model as clearly delimited, untrusted data that it is instructed never to obey.
  • Out-of-scope handling: if retrieval similarity is below a threshold, the app answers "I couldn't find that in the uploaded documents" without calling the LLM.
• Session isolation — each user's index lives in their own session state.
• Provider-agnostic LLM — Groq (default, free tier), OpenAI, or Gemini, all via the OpenAI-compatible API.
• Conversation memory — the last few turns are sent to the model, so follow-up questions that refer back are understood in the context of the conversation.

How it works

1. Load documents and extract text (pypdf, python-docx).
2. Chunk text into ~900-character, paragraph-aware pieces with overlap.
3. Embed chunks with all-MiniLM-L6-v2 (Sentence-Transformers), normalized.
4. Store vectors in an in-memory FAISS index (IndexFlatIP = cosine similarity).
5. Answer: validate → injection check → retrieve top-k → scope check → generate a grounded answer from the retrieved excerpts only.

Project structure

app.py                 # Gradio UI + orchestration (loads built-in corpus at startup)
build_corpus.py        # (dev) regenerates the data/ files from public sources
data/                  # built-in corpus: Constitution of India (PDF, DOCX, TXT)
rag/
  document_loader.py   # PDF / DOCX / TXT extraction + validation
  chunker.py           # paragraph-aware chunking with overlap
  embeddings.py        # Sentence-Transformers wrapper (lazy singleton)
  vector_store.py      # FAISS in-memory store
  llm.py               # OpenAI-compatible client (Groq/OpenAI/Gemini)
  guardrails.py        # input validation + injection detection
  pipeline.py          # build_index() and answer()
requirements.txt
.env.example

Local setup

git clone <your-repo-url>
cd rag-document-chat
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env       # then edit .env and add your API key
python app.py              # opens at http://127.0.0.1:7860

Getting a free API key (Groq)

Create one at https://console.groq.com/keys (free tier, no credit card), then set GROQ_API_KEY in .env. To use a different provider, set LLM_PROVIDER to openai or gemini and provide the matching key.

Environment variables

Variable	Default	Description
LLM_PROVIDER	groq	groq, openai, or gemini
GROQ_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY	—	key for the chosen provider
LLM_MODEL	provider default	override the model name
EMBEDDING_MODEL	sentence-transformers/all-MiniLM-L6-v2	embedding model
RELEVANCE_THRESHOLD	0.25	min cosine score to treat a match as in-scope

Deploy to Hugging Face Spaces

1. Create a new Space at https://huggingface.co/new-space — choose Gradio as the SDK.
2. Push this repository to the Space (or upload the files). The included README front matter sets sdk: gradio and app_file: app.py. If the build reports the SDK version is unavailable, change sdk_version to the latest 6.x shown in the Space settings and update gradio== in requirements.txt to match.
3. In Settings → Variables and secrets, add a secret named GROQ_API_KEY (and LLM_PROVIDER if not using Groq).
4. The Space builds automatically and serves the app.

Notes & limitations

• The index is in memory and resets when the Space restarts or the session ends.
• Scanned/image-only PDFs yield no text (no OCR); the app reports this clearly.
• The injection filter uses high-signal heuristics; the model-side instruction is the primary defence against instructions hidden inside documents.
• Conversation memory is applied at the generation step only: recent turns help the model interpret a follow-up, but retrieval still embeds the raw question. For follow-ups whose meaning depends entirely on earlier turns, conversational query rewriting — rephrasing the follow-up into a standalone question before retrieval — would be the more robust approach.