README.md

metadata

title: Fast RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
storage: true

High-Performance RAG Chatbot (FastAPI + FAISS)

Production-style document QA chatbot using:

• FastAPI API service
• FAISS vector search
• SentenceTransformer embeddings (BAAI/bge-small-en-v1.5 by default)
• Groq (preferred) or Hugging Face LLM APIs
• Optional Gradio chat UI

Features

• Loads .pdf and .txt files from docs/
• Cleans extracted text and chunks into semantic windows
  • Chunk size: 420 tokens (word-level approximation)
  • Overlap: 80 tokens
• Builds FAISS index and saves it locally
• Re-indexes only when docs change (fingerprint-based cache)
• Retrieves top-k relevant chunks only (default k=4)
• Strict anti-hallucination prompt
• Health endpoint with docs/index status
• Retrieval logging (source + similarity score)
• CORS controls for website integration
• Optional API key auth for /chat
• In-memory rate limiting per client IP
• Query embedding cache for repeated questions
• Docker + docker-compose deployment

Project Structure

app/main.py - FastAPI app and endpoints
app/services/document_loader.py - PDF/TXT ingestion and cleaning
app/services/chunker.py - token-window chunking
app/services/embeddings.py - embedding model wrapper
app/services/vector_store.py - FAISS index and retrieval
app/services/llm.py - Groq/HF LLM clients and prompt
app/services/rag_pipeline.py - end-to-end chat flow
app/ui_gradio.py - optional web chat UI

Setup

1. Install dependencies:

pip install -r requirements.txt

2. Configure environment:

copy .env.example .env

Then set your keys in .env:

• GROQ_API_KEY (if using Groq)
• HF_API_KEY (if using Hugging Face)
• Optional:
  • API_KEY for request auth (send as x-api-key)
  • CORS_ALLOW_ORIGINS as comma-separated origins
  • RATE_LIMIT_REQUESTS and RATE_LIMIT_WINDOW_SECONDS

3. Add documents:

• Put your .pdf and .txt files in docs/

Run API

uvicorn app.main:app --host 0.0.0.0 --port 8000

Endpoints

GET /health

Returns status and index readiness.

POST /chat

Request:

{
  "message": "What are the key points?",
  "history": []
}

Response:

{
  "reply": "Answer based on retrieved context.",
  "retrieved_chunks": [
    {
      "id": "...",
      "source": "...",
      "text": "...",
      "score": 0.83
    }
  ]
}

Optional UI

Start API first, then:

python -m app.ui_gradio

By default, Gradio now runs in direct RAG mode (no localhost API dependency). If you set RAG_API_URL, it will call that external FastAPI endpoint instead.

Deployment Notes

• Works as backend for websites (REST API is frontend-agnostic)
• Persist data/index/ volume in production
• Prefer Groq provider for low latency
• Keep top_k small (3-5) for speed and lower prompt tokens
• Protect /chat with API_KEY in production
• Set strict CORS_ALLOW_ORIGINS instead of *

Docker Deployment

Build and run:

docker compose up --build -d

Health check:

curl http://localhost:8000/health

Chat call with API key:

curl -X POST http://localhost:8000/chat ^
  -H "Content-Type: application/json" ^
  -H "x-api-key: YOUR_API_KEY" ^
  -d "{\"message\":\"What does the handbook say about leave policy?\",\"history\":[]}"

Hugging Face Spaces (Recommended: Gradio Space)

Use these settings in your Space:

• SDK: Gradio
• App file: app.py
• Python version: 3.10+ (3.11 recommended)

Add Space Secrets:

• GROQ_API_KEY (or HF_API_KEY)
• Optional: LLM_PROVIDER, GROQ_MODEL, HF_MODEL, TOP_K

Upload project files (excluding .env) and include your knowledge files inside docs/.