---
title: Fast RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
storage: true
---

# High-Performance RAG Chatbot (FastAPI + FAISS)

Production-style document QA chatbot using:
- FastAPI API service
- FAISS vector search
- SentenceTransformer embeddings (`BAAI/bge-small-en-v1.5` by default)
- Groq (preferred) or Hugging Face LLM APIs
- Optional Gradio chat UI

## Features

- Loads `.pdf` and `.txt` files from `docs/`
- Cleans extracted text and chunks into semantic windows
  - Chunk size: 420 tokens (word-level approximation)
  - Overlap: 80 tokens
- Builds FAISS index and saves it locally
- Re-indexes only when docs change (fingerprint-based cache)
- Retrieves top-k relevant chunks only (default k=4)
- Strict anti-hallucination prompt
- Health endpoint with docs/index status
- Retrieval logging (source + similarity score)
- CORS controls for website integration
- Optional API key auth for `/chat`
- In-memory rate limiting per client IP
- Query embedding cache for repeated questions
- Docker + docker-compose deployment

## Project Structure

`app/main.py` - FastAPI app and endpoints  
`app/services/document_loader.py` - PDF/TXT ingestion and cleaning  
`app/services/chunker.py` - token-window chunking  
`app/services/embeddings.py` - embedding model wrapper  
`app/services/vector_store.py` - FAISS index and retrieval  
`app/services/llm.py` - Groq/HF LLM clients and prompt  
`app/services/rag_pipeline.py` - end-to-end chat flow  
`app/ui_gradio.py` - optional web chat UI

## Setup

1. Install dependencies:

```bash
pip install -r requirements.txt
```

2. Configure environment:

```bash
copy .env.example .env
```

Then set your keys in `.env`:
- `GROQ_API_KEY` (if using Groq)
- `HF_API_KEY` (if using Hugging Face)
- Optional:
  - `API_KEY` for request auth (send as `x-api-key`)
  - `CORS_ALLOW_ORIGINS` as comma-separated origins
  - `RATE_LIMIT_REQUESTS` and `RATE_LIMIT_WINDOW_SECONDS`

3. Add documents:
- Put your `.pdf` and `.txt` files in `docs/`

## Run API

```bash
uvicorn app.main:app --host 0.0.0.0 --port 8000
```

## Endpoints

### `GET /health`
Returns status and index readiness.

### `POST /chat`

Request:
```json
{
  "message": "What are the key points?",
  "history": []
}
```

Response:
```json
{
  "reply": "Answer based on retrieved context.",
  "retrieved_chunks": [
    {
      "id": "...",
      "source": "...",
      "text": "...",
      "score": 0.83
    }
  ]
}
```

## Optional UI

Start API first, then:

```bash
python -m app.ui_gradio
```

By default, Gradio now runs in direct RAG mode (no localhost API dependency).
If you set `RAG_API_URL`, it will call that external FastAPI endpoint instead.

## Deployment Notes

- Works as backend for websites (REST API is frontend-agnostic)
- Persist `data/index/` volume in production
- Prefer Groq provider for low latency
- Keep `top_k` small (3-5) for speed and lower prompt tokens
- Protect `/chat` with `API_KEY` in production
- Set strict `CORS_ALLOW_ORIGINS` instead of `*`

## Docker Deployment

Build and run:

```bash
docker compose up --build -d
```

Health check:

```bash
curl http://localhost:8000/health
```

Chat call with API key:

```bash
curl -X POST http://localhost:8000/chat ^
  -H "Content-Type: application/json" ^
  -H "x-api-key: YOUR_API_KEY" ^
  -d "{\"message\":\"What does the handbook say about leave policy?\",\"history\":[]}"
```

## Hugging Face Spaces (Recommended: Gradio Space)

Use these settings in your Space:
- **SDK**: Gradio
- **App file**: `app.py`
- **Python version**: 3.10+ (3.11 recommended)

Add Space Secrets:
- `GROQ_API_KEY` (or `HF_API_KEY`)
- Optional: `LLM_PROVIDER`, `GROQ_MODEL`, `HF_MODEL`, `TOP_K`

Upload project files (excluding `.env`) and include your knowledge files inside `docs/`.