README.md

metadata

title: RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.3.0
app_file: app.py
pinned: false

Hugging Face RAG Chatbot

A Retrieval-Augmented Generation (RAG) chatbot that can ingest PDFs and URLs, then answer questions based on the ingested documents. Built for deployment on Hugging Face Spaces.

Features

• 📄 PDF Ingestion: Upload and process PDF documents
• 🌐 URL Ingestion: Extract and process content from web URLs
• 🔍 Vector Search: Semantic search using sentence transformers
• 💬 Chatbot Interface: Interactive Gradio interface for querying documents
• 🚀 Hugging Face Ready: Configured for easy deployment to Hugging Face Spaces

Setup

Local Development

1. Install dependencies:

   pip install -r requirements.txt
   

2. Run the application:

   python app.py
   

3. Access the interface:

   • Open your browser to http://localhost:7860

Usage

1. Ingest Documents (Run this first or periodically to update):

   • Add PDF files to the pdfs/ folder
   • Edit ingest_documents.py and add your URLs to the URLS list
   • Run the ingestion script:

     py ingest_documents.py
     

   • Wait for processing to complete (this creates/updates the vector store)

2. Chat with Documents:

   • Run the chatbot app:

     py app.py
     

   • Open your browser to http://localhost:7860
   • Toggle "Use RAG" to enable/disable document retrieval
   • Ask questions about your ingested documents
   • The chatbot will retrieve relevant context and generate answers

Deployment to Hugging Face Spaces

Option 1: Using Hugging Face CLI

1. Install Hugging Face CLI:

   pip install huggingface_hub
   

2. Login to Hugging Face:

   huggingface-cli login
   

3. Create a new Space:

   • Go to https://huggingface.co/new-space
   • Choose a name and select "Gradio" as the SDK
   • Create the space

4. Clone and push your code:

   git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   cd YOUR_SPACE_NAME
   # Copy your files here
   git add .
   git commit -m "Initial commit"
   git push
   

Option 2: Using Git

1. Initialize git repository:

   git init
   git add .
   git commit -m "Initial commit"
   

2. Add Hugging Face remote:

   git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   git push -u origin main
   

Required Files for Hugging Face Spaces

Your Space needs these files:

• app.py - Main Gradio application
• requirements.txt - Python dependencies
• README.md - This file (optional but recommended)

Optional: Add app.py to README

For Hugging Face Spaces, you can also add a app.py reference in your README:

---
title: RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.0.0
app_file: app.py
pinned: false
---

Configuration

Setting Up Hugging Face Token (Required)

The chatbot uses Hugging Face Inference API to access high-quality models. You need to set up an API token:

1. Get your token:

   • Go to https://huggingface.co/settings/tokens
   • Create a new token with "Read" permissions
   • Copy the token

2. For local development:

   • Set environment variable: export HF_TOKEN=your_token_here (Linux/Mac)
   • Or: set HF_TOKEN=your_token_here (Windows)
   • Or create a .env file with HF_TOKEN=your_token_here

3. For Hugging Face Spaces:

   • Go to your Space → Settings → Secrets
   • Add a new secret: Name = HF_TOKEN, Value = your token
   • The app will automatically use this token

Chatbot Model and Finding Available Models

The chatbot uses the Hugging Face Inference API. Which models you can use depends on which providers you have enabled.

How to see which models are available to you:

1. Browse models that support Inference API

   • https://huggingface.co/inference/models — lists providers and models
   • https://huggingface.co/models?inference_provider=hf-inference — filter Hub models by “HF Inference API”

2. Enable providers (required)
   See “How to enable a provider and model” below.

3. Pick a chat model

   • From the links above, choose a text generation / chat model that’s supported by a provider you enabled. Note its model id (e.g. meta-llama/Llama-3.2-1B-Instruct).

4. Use it in the app

   • In app.py, pass that model id when creating the chatbot:

   chatbot = RAGChatbot(model_name="meta-llama/Llama-3.2-1B-Instruct")  # use an id you enabled
   

   • The app also tries fallbacks (Phi-2, Zephyr, Qwen) by default; if none are available, enable a provider that supports at least one of them, or set model_name to a model you enabled as above.

How to enable a provider and model

1. Log in to Hugging Face.

2. Open Inference Provider settings (one of these, depending on the current UI):

   • https://huggingface.co/settings/inference-providers
   • https://huggingface.co/settings/inference-api

3. Enable a provider

   • On that page you’ll see a list of providers (e.g. Hugging Face, Together, Groq, etc.).
   • Turn on the provider that serves your model (e.g. Together for ServiceNow-AI/Apriel-1.6-15b-Thinker:together).
   • You can set the order of providers; “auto” uses this order to pick which provider handles the request.

4. Credits / billing

   • Free accounts get a small amount of monthly credits; usage is deducted from that.
   • If you use a third-party provider (e.g. Together), you can either use HF-routed billing (credits on your HF account) or add a custom provider API key (e.g. Together API key) in the same settings so that provider is billed directly.

5. Confirm the model

   • Browse models for that provider:
     Together models on the Hub
   • Open the model page (e.g. ServiceNow-AI/Apriel-1.6-15b-Thinker) and check the inference widget; if you see “Together” and can run it, that model is available with your enabled provider.

6. Use it in this app

   • The app default is already ServiceNow-AI/Apriel-1.6-15b-Thinker:together.
   • Ensure Together is enabled in your Inference Provider settings and you have credits (or a Together API key). Then restart the app and send a message.

Embedding Model

The default embedding model is all-mpnet-base-v2, which provides high-quality embeddings for better retrieval.

To change the embedding model, edit both app.py and ingest_documents.py:

# In app.py
chatbot = RAGChatbot(embedding_model="all-mpnet-base-v2")

# In ingest_documents.py
ingestion = DocumentIngestion(embedding_model="all-mpnet-base-v2")

Note: If you change the embedding model, you must re-run ingest_documents.py to rebuild the vector store.

Ingestion Parameters

The ingestion system uses optimized parameters:

• Chunk size: 600 characters (for precise retrieval)
• Chunk overlap: 150 characters (to avoid cutting sentences)
• Retrieval count: 5 chunks (for comprehensive context)

These parameters are set in ingestion.py and can be adjusted if needed.

Project Structure

.
├── app.py                  # Main Gradio chatbot application
├── ingest_documents.py     # Standalone script to ingest PDFs and URLs
├── ingestion.py            # Document ingestion and vector store module
├── requirements.txt        # Python dependencies
├── README.md              # This file
├── pdfs/                  # Folder for PDF files (add your PDFs here)
│   └── README.md
└── data/
    └── vector_store/     # Saved vector store (created after ingestion)
        ├── index.faiss
        ├── documents.pkl
        └── embeddings.pkl

How It Works

The chatbot uses Retrieval-Augmented Generation (RAG):

1. Document Ingestion: PDFs and URLs are processed into chunks and embedded using sentence transformers
2. Vector Search: When you ask a question, the system searches for the most relevant document chunks
3. Answer Generation: The retrieved context is sent to Mistral-7B-Instruct via Inference API, which synthesizes a coherent answer based on the context

This approach combines the accuracy of document retrieval with the natural language capabilities of a large language model.

Limitations

• Vector store is stored locally (not persistent on Hugging Face Spaces by default)
• Large documents may take time to process
• Some URLs may be blocked or require authentication
• Requires HF_TOKEN for Inference API access (free tier available)
• If you change embedding model or chunk parameters, you must re-run ingestion

Troubleshooting

Out of Memory Errors

• Use smaller models
• Reduce chunk size in ingestion.py
• Process fewer documents at once

URL Fetching Issues

• Some websites block automated requests
• Try different URLs or use PDF uploads instead

Inference API Issues

• Verify your HF_TOKEN is set correctly
• Check that the token has "Read" permissions
• Ensure you have API access (free tier available)
• If you get rate limit errors, you may need to upgrade your Hugging Face account

Ingestion Issues

• If you changed embedding model or chunk parameters, re-run ingest_documents.py
• Ensure you have enough disk space for the vector store
• Large documents may take time to process

License

This project is open source and available under the MIT License.