DEPLOYMENT.md

Deploying to Hugging Face Spaces

This guide will walk you through deploying your RAG chatbot to Hugging Face Spaces.

Prerequisites

1. Hugging Face Account: Sign up at https://huggingface.co/join
2. Access Token: Get your token from https://huggingface.co/settings/tokens
   • Create a token with "Read" permissions
   • This token is required for the Inference API to work

Step-by-Step Deployment

Method 1: Using the Web Interface (Easiest)

1. Create a New Space:

   • Go to https://huggingface.co/new-space
   • Fill in:
     • Space name: Choose a name (e.g., my-rag-chatbot)
     • SDK: Select Gradio
     • Hardware: Choose based on your needs:
       • CPU basic: Free, good for testing
       • CPU upgrade: Better performance
       • GPU: If you need faster model inference
     • Visibility: Public or Private
   • Click Create Space

2. Upload Your Files:

   • In your new Space, click the Files and versions tab
   • Click Add file → Upload files
   • Upload these files:
     • app.py
     • ingestion.py
     • requirements.txt
     • README.md (optional but recommended)
     • pdfs/ folder (if you want to include sample PDFs)
     • ingest_documents.py (optional, for manual ingestion)

3. Set Up HF_TOKEN Secret (REQUIRED):

   • Go to your Space → Settings → Secrets
   • Click New secret
   • Name: HF_TOKEN
   • Value: Paste your Hugging Face access token
   • Click Add secret
   • Important: Without this token, the chatbot will not work as it needs the Inference API

4. Important Notes:

   • Vector Store: The data/vector_store/ folder is in .gitignore and won't be uploaded. You have two options:
     • Option A: Run ingest_documents.py on the Space after deployment (via the Space's terminal)
     • Option B: Upload the vector store files manually if they're not too large
   • PDFs: If your PDFs are large (>50MB), consider hosting them elsewhere or using Hugging Face Datasets

5. Wait for Build: Hugging Face will automatically:

   • Install dependencies from requirements.txt
   • Start your Gradio app
   • Your Space will be live at: https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME

Method 2: Using Git (Recommended for Updates)

1. Install Git (if not already installed):

   • Windows: Download from https://git-scm.com/download/win
   • Or use Git that comes with GitHub Desktop

2. Install Hugging Face CLI:

   py -m pip install huggingface_hub
   

3. Login to Hugging Face:

   huggingface-cli login
   

   Enter your access token when prompted.

4. Create a New Space (via web interface):

   • Go to https://huggingface.co/new-space
   • Create the space with Gradio SDK
   • Note your space name (e.g., YOUR_USERNAME/my-rag-chatbot)

5. Initialize Git in Your Project:

   cd C:\Users\DanielSimeone\Desktop\testing-hugging-face
   git init
   git add app.py ingestion.py requirements.txt README.md ingest_documents.py pdfs/
   git commit -m "Initial commit for Hugging Face Space"
   

6. Add Hugging Face Remote and Push:

   git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   git push -u origin main
   

7. For Future Updates:

   git add .
   git commit -m "Update description"
   git push
   

Post-Deployment Setup

Setting Up HF_TOKEN Secret

This is REQUIRED for the chatbot to work!

1. Go to your Space on Hugging Face
2. Click on Settings (gear icon)
3. Scroll down to Secrets section
4. Click New secret
5. Enter:
   • Name: HF_TOKEN
   • Value: Your Hugging Face access token (from https://huggingface.co/settings/tokens)
6. Click Add secret
7. The token will be automatically available to your app via os.environ.get("HF_TOKEN")

Note: The token must have "Read" permissions. The app uses it to access the Inference API for Mistral-7B-Instruct.

Setting Up the Vector Store on Hugging Face Spaces

Since the vector store isn't included in the repository, you need to create it on the Space:

1. Option A: Use the Space Terminal (if available):

   • Go to your Space → Settings → Enable Embedded Gradio SDK
   • Or use the Space's built-in terminal/console
   • Run: python ingest_documents.py

2. Option B: Upload Vector Store Files:

   • If your vector store files are small enough:

• Upload data/vector_store/index.faiss
  • Upload data/vector_store/documents.pkl
  • Upload data/vector_store/embeddings.pkl
  • The app will automatically load them on startup

3. Option C: Pre-build in a Script:
   • Create a setup.py or modify the Space to run ingestion on first launch
   • This is more complex but ensures the vector store is always ready

Required Files for Deployment

Your Space needs these files:

• ✅ app.py - Main Gradio application
• ✅ ingestion.py - Document ingestion module
• ✅ requirements.txt - Python dependencies
• ✅ README.md - Documentation (optional but recommended)
• ⚠️ data/vector_store/ - Will be created on the Space
• ⚠️ pdfs/ - Optional, include if you want sample PDFs

Configuration for Hugging Face Spaces

Model Configuration

The chatbot now uses Mistral-7B-Instruct-v0.2 via Hugging Face Inference API. This means:

• No local model loading: Faster startup, no need for GPU
• Hosted inference: The model runs on Hugging Face's infrastructure
• Requires HF_TOKEN: Must be set in Space secrets (see above)

The model is configured in app.py and can be changed if needed:

chatbot = RAGChatbot(model_name="mistralai/Mistral-7B-Instruct-v0.2")

App Configuration

The current app.py is configured for Spaces:

1. Port: Uses os.environ.get("PORT", 7860) - Spaces automatically sets this
2. Server name: Uses 0.0.0.0 (required for Spaces)
3. Share: Set to False (Spaces provides its own sharing)

The launch code is already configured correctly:

port = int(os.environ.get("PORT", 7860))
app.launch(
    share=False,
    server_name="0.0.0.0",
    server_port=port,
    theme=MinimalistTheme()
)

Optional: Add README Frontmatter

Add this to the top of your README.md for better Space display:

---
title: RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.3.0
app_file: app.py
pinned: false
---

Troubleshooting

Build Fails

• Check that all dependencies in requirements.txt are correct
• Ensure Python version compatibility (Spaces uses Python 3.10 by default)
• Check the build logs in your Space's Logs tab

Vector Store Not Loading

• Verify the vector store files are in the correct location
• Check file permissions
• Ensure the path in app.py is correct (should be data/vector_store)

Inference API Issues

• "HF_TOKEN not set" error: Make sure you've added the HF_TOKEN secret in Space settings
• API rate limits: Free tier has rate limits; upgrade if you need more requests
• Model access errors: Verify your token has "Read" permissions
• Connection errors: Check that the Inference API is accessible from your Space

Memory Issues

• If you get out-of-memory errors, consider:
  • Using a smaller embedding model (e.g., all-MiniLM-L6-v2 instead of all-mpnet-base-v2)
  • Reducing chunk size in ingestion.py
  • Processing fewer documents at once
  • Upgrading to a Space with more memory

Note: Since the model runs via Inference API, memory issues are less likely than with local model loading.

Updating Your Space

After making changes locally:

git add .
git commit -m "Description of changes"
git push

Hugging Face will automatically rebuild your Space.

Sharing Your Space

Once deployed, your Space will be available at:

https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME

You can share this URL with others!

Next Steps

• Add more PDFs to the pdfs/ folder
• Update URLs in ingest_documents.py
• Customize the theme further
• Add more features to the chatbot