| ---
|
| title: Fast RAG Chatbot
|
| emoji: 🤖
|
| colorFrom: blue
|
| colorTo: indigo
|
| sdk: docker
|
| pinned: false
|
| storage: true
|
| ---
|
|
|
| # High-Performance RAG Chatbot (FastAPI + FAISS)
|
|
|
| Production-style document QA chatbot using:
|
| - FastAPI API service
|
| - FAISS vector search
|
| - SentenceTransformer embeddings (`BAAI/bge-small-en-v1.5` by default)
|
| - Groq (preferred) or Hugging Face LLM APIs
|
| - Optional Gradio chat UI
|
|
|
| ## Features
|
|
|
| - Loads `.pdf` and `.txt` files from `docs/`
|
| - Cleans extracted text and chunks into semantic windows
|
| - Chunk size: 420 tokens (word-level approximation)
|
| - Overlap: 80 tokens
|
| - Builds FAISS index and saves it locally
|
| - Re-indexes only when docs change (fingerprint-based cache)
|
| - Retrieves top-k relevant chunks only (default k=4)
|
| - Strict anti-hallucination prompt
|
| - Health endpoint with docs/index status
|
| - Retrieval logging (source + similarity score)
|
| - CORS controls for website integration
|
| - Optional API key auth for `/chat`
|
| - In-memory rate limiting per client IP
|
| - Query embedding cache for repeated questions
|
| - Docker + docker-compose deployment
|
|
|
| ## Project Structure
|
|
|
| `app/main.py` - FastAPI app and endpoints
|
| `app/services/document_loader.py` - PDF/TXT ingestion and cleaning
|
| `app/services/chunker.py` - token-window chunking
|
| `app/services/embeddings.py` - embedding model wrapper
|
| `app/services/vector_store.py` - FAISS index and retrieval
|
| `app/services/llm.py` - Groq/HF LLM clients and prompt
|
| `app/services/rag_pipeline.py` - end-to-end chat flow
|
| `app/ui_gradio.py` - optional web chat UI
|
|
|
| ## Setup
|
|
|
| 1. Install dependencies:
|
|
|
| ```bash
|
| pip install -r requirements.txt
|
| ```
|
|
|
| 2. Configure environment:
|
|
|
| ```bash
|
| copy .env.example .env
|
| ```
|
|
|
| Then set your keys in `.env`:
|
| - `GROQ_API_KEY` (if using Groq)
|
| - `HF_API_KEY` (if using Hugging Face)
|
| - Optional:
|
| - `API_KEY` for request auth (send as `x-api-key`)
|
| - `CORS_ALLOW_ORIGINS` as comma-separated origins
|
| - `RATE_LIMIT_REQUESTS` and `RATE_LIMIT_WINDOW_SECONDS`
|
|
|
| 3. Add documents:
|
| - Put your `.pdf` and `.txt` files in `docs/`
|
|
|
| ## Run API
|
|
|
| ```bash
|
| uvicorn app.main:app --host 0.0.0.0 --port 8000
|
| ```
|
|
|
| ## Endpoints
|
|
|
| ### `GET /health`
|
| Returns status and index readiness.
|
|
|
| ### `POST /chat`
|
|
|
| Request:
|
| ```json
|
| {
|
| "message": "What are the key points?",
|
| "history": []
|
| }
|
| ```
|
|
|
| Response:
|
| ```json
|
| {
|
| "reply": "Answer based on retrieved context.",
|
| "retrieved_chunks": [
|
| {
|
| "id": "...",
|
| "source": "...",
|
| "text": "...",
|
| "score": 0.83
|
| }
|
| ]
|
| }
|
| ```
|
|
|
| ## Optional UI
|
|
|
| Start API first, then:
|
|
|
| ```bash
|
| python -m app.ui_gradio
|
| ```
|
|
|
| By default, Gradio now runs in direct RAG mode (no localhost API dependency).
|
| If you set `RAG_API_URL`, it will call that external FastAPI endpoint instead.
|
|
|
| ## Deployment Notes
|
|
|
| - Works as backend for websites (REST API is frontend-agnostic)
|
| - Persist `data/index/` volume in production
|
| - Prefer Groq provider for low latency
|
| - Keep `top_k` small (3-5) for speed and lower prompt tokens
|
| - Protect `/chat` with `API_KEY` in production
|
| - Set strict `CORS_ALLOW_ORIGINS` instead of `*`
|
|
|
| ## Docker Deployment
|
|
|
| Build and run:
|
|
|
| ```bash
|
| docker compose up --build -d
|
| ```
|
|
|
| Health check:
|
|
|
| ```bash
|
| curl http://localhost:8000/health
|
| ```
|
|
|
| Chat call with API key:
|
|
|
| ```bash
|
| curl -X POST http://localhost:8000/chat ^
|
| -H "Content-Type: application/json" ^
|
| -H "x-api-key: YOUR_API_KEY" ^
|
| -d "{\"message\":\"What does the handbook say about leave policy?\",\"history\":[]}"
|
| ```
|
|
|
| ## Hugging Face Spaces (Recommended: Gradio Space)
|
|
|
| Use these settings in your Space:
|
| - **SDK**: Gradio
|
| - **App file**: `app.py`
|
| - **Python version**: 3.10+ (3.11 recommended)
|
|
|
| Add Space Secrets:
|
| - `GROQ_API_KEY` (or `HF_API_KEY`)
|
| - Optional: `LLM_PROVIDER`, `GROQ_MODEL`, `HF_MODEL`, `TOP_K`
|
|
|
| Upload project files (excluding `.env`) and include your knowledge files inside `docs/`.
|
|
|
