DEPLOYMENT.md

# Deploying to Hugging Face Spaces

This guide will walk you through deploying your RAG chatbot to Hugging Face Spaces.

## Prerequisites

1. **Hugging Face Account**: Sign up at https://huggingface.co/join
2. **Access Token**: Get your token from https://huggingface.co/settings/tokens
   - Create a token with "Read" permissions
   - This token is required for the Inference API to work

## Step-by-Step Deployment

### Method 1: Using the Web Interface (Easiest)

1. **Create a New Space**:
   - Go to https://huggingface.co/new-space
   - Fill in:
     - **Space name**: Choose a name (e.g., `my-rag-chatbot`)
     - **SDK**: Select **Gradio**
     - **Hardware**: Choose based on your needs:
       - **CPU basic**: Free, good for testing
       - **CPU upgrade**: Better performance
       - **GPU**: If you need faster model inference
     - **Visibility**: Public or Private
   - Click **Create Space**

2. **Upload Your Files**:
   - In your new Space, click the **Files and versions** tab
   - Click **Add file** → **Upload files**
   - Upload these files:
     - `app.py`
     - `ingestion.py`
     - `requirements.txt`
     - `README.md` (optional but recommended)
     - `pdfs/` folder (if you want to include sample PDFs)
     - `ingest_documents.py` (optional, for manual ingestion)

3. **Set Up HF_TOKEN Secret (REQUIRED)**:
   - Go to your Space → **Settings** → **Secrets**
   - Click **New secret**
   - Name: `HF_TOKEN`
   - Value: Paste your Hugging Face access token
   - Click **Add secret**
   - **Important**: Without this token, the chatbot will not work as it needs the Inference API

4. **Important Notes**:
   - **Vector Store**: The `data/vector_store/` folder is in `.gitignore` and won't be uploaded. You have two options:
     - **Option A**: Run `ingest_documents.py` on the Space after deployment (via the Space's terminal)
     - **Option B**: Upload the vector store files manually if they're not too large
   - **PDFs**: If your PDFs are large (>50MB), consider hosting them elsewhere or using Hugging Face Datasets

5. **Wait for Build**: Hugging Face will automatically:
   - Install dependencies from `requirements.txt`
   - Start your Gradio app
   - Your Space will be live at: `https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME`

### Method 2: Using Git (Recommended for Updates)

1. **Install Git** (if not already installed):
   - Windows: Download from https://git-scm.com/download/win
   - Or use Git that comes with GitHub Desktop

2. **Install Hugging Face CLI**:
   ```bash
   py -m pip install huggingface_hub
   ```

3. **Login to Hugging Face**:
   ```bash
   huggingface-cli login
   ```
   Enter your access token when prompted.

4. **Create a New Space** (via web interface):
   - Go to https://huggingface.co/new-space
   - Create the space with Gradio SDK
   - Note your space name (e.g., `YOUR_USERNAME/my-rag-chatbot`)

5. **Initialize Git in Your Project**:
   ```bash
   cd C:\Users\DanielSimeone\Desktop\testing-hugging-face
   git init
   git add app.py ingestion.py requirements.txt README.md ingest_documents.py pdfs/
   git commit -m "Initial commit for Hugging Face Space"
   ```

6. **Add Hugging Face Remote and Push**:
   ```bash
   git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   git push -u origin main
   ```

7. **For Future Updates**:
   ```bash
   git add .
   git commit -m "Update description"
   git push
   ```

## Post-Deployment Setup

### Setting Up HF_TOKEN Secret

**This is REQUIRED for the chatbot to work!**

1. Go to your Space on Hugging Face
2. Click on **Settings** (gear icon)
3. Scroll down to **Secrets** section
4. Click **New secret**
5. Enter:
   - **Name**: `HF_TOKEN`
   - **Value**: Your Hugging Face access token (from https://huggingface.co/settings/tokens)
6. Click **Add secret**
7. The token will be automatically available to your app via `os.environ.get("HF_TOKEN")`

**Note**: The token must have "Read" permissions. The app uses it to access the Inference API for Mistral-7B-Instruct.

### Setting Up the Vector Store on Hugging Face Spaces

Since the vector store isn't included in the repository, you need to create it on the Space:

1. **Option A: Use the Space Terminal** (if available):
   - Go to your Space → **Settings** → Enable **Embedded Gradio SDK**
   - Or use the Space's built-in terminal/console
   - Run: `python ingest_documents.py`

2. **Option B: Upload Vector Store Files**:
   - If your vector store files are small enough:
- Upload `data/vector_store/index.faiss`
   - Upload `data/vector_store/documents.pkl`
   - Upload `data/vector_store/embeddings.pkl`
   - The app will automatically load them on startup

3. **Option C: Pre-build in a Script**:
   - Create a `setup.py` or modify the Space to run ingestion on first launch
   - This is more complex but ensures the vector store is always ready

## Required Files for Deployment

Your Space needs these files:

- ✅ `app.py` - Main Gradio application
- ✅ `ingestion.py` - Document ingestion module
- ✅ `requirements.txt` - Python dependencies
- ✅ `README.md` - Documentation (optional but recommended)
- ⚠️ `data/vector_store/` - Will be created on the Space
- ⚠️ `pdfs/` - Optional, include if you want sample PDFs

## Configuration for Hugging Face Spaces

### Model Configuration

The chatbot now uses **Mistral-7B-Instruct-v0.2** via Hugging Face Inference API. This means:
- **No local model loading**: Faster startup, no need for GPU
- **Hosted inference**: The model runs on Hugging Face's infrastructure
- **Requires HF_TOKEN**: Must be set in Space secrets (see above)

The model is configured in `app.py` and can be changed if needed:
```python
chatbot = RAGChatbot(model_name="mistralai/Mistral-7B-Instruct-v0.2")
```

### App Configuration

The current `app.py` is configured for Spaces:

1. **Port**: Uses `os.environ.get("PORT", 7860)` - Spaces automatically sets this
2. **Server name**: Uses `0.0.0.0` (required for Spaces)
3. **Share**: Set to `False` (Spaces provides its own sharing)

The launch code is already configured correctly:
```python
port = int(os.environ.get("PORT", 7860))
app.launch(
    share=False,
    server_name="0.0.0.0",
    server_port=port,
    theme=MinimalistTheme()
)
```

### Optional: Add README Frontmatter

Add this to the top of your `README.md` for better Space display:

```yaml
---
title: RAG Chatbot
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.3.0
app_file: app.py
pinned: false
---
```

## Troubleshooting

### Build Fails

- Check that all dependencies in `requirements.txt` are correct
- Ensure Python version compatibility (Spaces uses Python 3.10 by default)
- Check the build logs in your Space's **Logs** tab

### Vector Store Not Loading

- Verify the vector store files are in the correct location
- Check file permissions
- Ensure the path in `app.py` is correct (should be `data/vector_store`)

### Inference API Issues

- **"HF_TOKEN not set" error**: Make sure you've added the `HF_TOKEN` secret in Space settings
- **API rate limits**: Free tier has rate limits; upgrade if you need more requests
- **Model access errors**: Verify your token has "Read" permissions
- **Connection errors**: Check that the Inference API is accessible from your Space

### Memory Issues

- If you get out-of-memory errors, consider:
  - Using a smaller embedding model (e.g., `all-MiniLM-L6-v2` instead of `all-mpnet-base-v2`)
  - Reducing chunk size in `ingestion.py`
  - Processing fewer documents at once
  - Upgrading to a Space with more memory

**Note**: Since the model runs via Inference API, memory issues are less likely than with local model loading.

## Updating Your Space

After making changes locally:

```bash
git add .
git commit -m "Description of changes"
git push
```

Hugging Face will automatically rebuild your Space.

## Sharing Your Space

Once deployed, your Space will be available at:
```
https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
```

You can share this URL with others!

## Next Steps

- Add more PDFs to the `pdfs/` folder
- Update URLs in `ingest_documents.py`
- Customize the theme further
- Add more features to the chatbot