README.md

metadata

title: RAG Document Q&A API
emoji: 📚
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860

RAG-Powered Document Q&A API

A production-style Retrieval-Augmented Generation API for asking grounded questions over uploaded PDF or text documents. The backend is FastAPI, the vector search layer is FAISS, embeddings come from Hugging Face sentence-transformers, and answer generation uses Hugging Face Inference Providers.

The implementation intentionally avoids LangChain and LlamaIndex so the RAG pipeline is visible end to end.

What It Does

• Accepts .pdf and .txt uploads through POST /ingest.
• Extracts text, chunks it with overlap, and embeds each chunk.
• Stores one local FAISS index plus metadata per uploaded document.
• Retrieves the most relevant chunks for a question.
• Sends only those chunks as context to a Hugging Face chat model.
• Returns the answer plus source chunks and similarity scores.
• Serves a minimal browser UI from the FastAPI app.

Architecture

Browser UI / cURL
  -> FastAPI routes
    -> document loader
    -> chunker
    -> sentence-transformer embedder
    -> FAISS document store
    -> Hugging Face router chat completions

Each ingested document gets a local directory under DATA_DIR containing:

• index.faiss
• metadata.json

Restarting the API reloads existing document indexes from DATA_DIR.

Local Setup

Use Python 3.11. The requirements pin CPU PyTorch wheels to avoid pulling GPU/CUDA packages.

cd /path/to/RAG
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Edit .env and set your Hugging Face token:

HF_API_KEY=your_huggingface_token

Your token needs permission to make Inference Provider calls.

Default models:

• Embeddings: sentence-transformers/all-MiniLM-L6-v2
• Generation: meta-llama/Llama-3.2-1B-Instruct

Run Locally With FastAPI

cd /path/to/RAG
source .venv/bin/activate
uvicorn app.main:app --host 127.0.0.1 --port 8011 --reload

Open:

• Frontend: http://127.0.0.1:8011/
• API docs: http://127.0.0.1:8011/docs
• Health check: http://127.0.0.1:8011/health

If you open frontend/index.html directly from disk, pass the API URL:

file:///path/to/RAG/frontend/index.html?api=http://127.0.0.1:8011

Demo Flow

Use a short document with facts that are easy to verify, such as a project brief, policy note, or article excerpt.

1. Start the API and open the frontend.
2. Upload a .txt or .pdf file.
3. Copy the returned document ID if you want to test from cURL.
4. Ask a question that is answered directly in the document.
5. Ask a question that is not in the document and confirm the assistant refuses to invent an answer.
6. Expand the source chunks in the response and compare them with the generated answer.

The key behavior to show is that the answer is grounded in retrieved chunks, not in the model's general memory.

API Usage

Ingest a document:

curl -X POST http://127.0.0.1:8011/ingest \
  -F "file=@sample.txt"

Query a document:

curl -X POST http://127.0.0.1:8011/query \
  -H "Content-Type: application/json" \
  -d '{
    "doc_id": "returned-doc-id",
    "question": "What are the key findings?",
    "top_k": 5
  }'

Check metadata:

curl http://127.0.0.1:8011/docs/returned-doc-id

Delete a document:

curl -X DELETE http://127.0.0.1:8011/docs/returned-doc-id

Hugging Face Spaces

This repo is ready for a Hugging Face Docker Space. Hugging Face Docker Spaces support FastAPI-style apps and expose the app on port 7860.

Create a new Space:

1. Go to Hugging Face Spaces and create a Space.
2. Choose Docker as the SDK.
3. Push this repo to the Space repository, or connect your GitHub repo if you prefer.
4. Add a Space secret:
   • Name: HF_API_KEY
   • Value: your Hugging Face token
5. Make sure the Space uses the included Dockerfile.

The app will run:

uvicorn app.main:app --host 0.0.0.0 --port 7860

Once the Space is running, users can visit:

https://<your-username>-<your-space-name>.hf.space/

API docs will be available at:

https://<your-username>-<your-space-name>.hf.space/docs

Notes for Spaces:

• Do not commit .env; use Space secrets for HF_API_KEY.
• Free Spaces may sleep and restart, so local FAISS data can disappear unless persistent storage is enabled.
• The first ingest can be slow because the embedding model may need to download.
• If the generation model is unsupported for your HF account/provider settings, change GENERATION_MODEL in the Space variables.

Configuration

HF_API_KEY=your_huggingface_inference_api_key
EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
GENERATION_MODEL=meta-llama/Llama-3.2-1B-Instruct
CHUNK_SIZE=512
CHUNK_OVERLAP=64
TOP_K_DEFAULT=5
DATA_DIR=data
MAX_UPLOAD_MB=10
HF_TIMEOUT_SECONDS=60

Tests

source .venv/bin/activate
pytest

The tests mock model inference and FAISS where needed so core API and pipeline behavior can be checked without downloading models or calling external services.

Limitations

• No authentication; this is intended as a portfolio/demo app.
• FAISS indexes are local files, not a distributed vector database.
• Chunking uses whitespace tokenization for clarity rather than tokenizer-specific token counts.
• Query generation requires a valid Hugging Face token with Inference Provider access.
• Hugging Face Spaces without persistent storage may lose uploaded document indexes after restart.

References

• Hugging Face Docker Spaces documentation: https://huggingface.co/docs/hub/spaces-sdks-docker
• Hugging Face Inference Providers documentation: https://huggingface.co/docs/inference-providers/main/index