feat: migrate from Ollama to Hugging Face Inference API

- Replace Ollama with HF Inference API in llm_handler.py
- Update config.py with HF model settings
- Remove ollama dependencies, add huggingface_hub
- Create app.py as HF Spaces entry point
- Add HF token validation in main.py
- Create README_HF.md with HF Spaces YAML frontmatter
- Add comprehensive documentation:
- DEPLOYMENT_GUIDE.md: Step-by-step deployment instructions
- ENV_SETUP.md: Environment variables documentation
- MIGRATION_SUMMARY.md: Complete migration overview
- test_hf_integration.py: Integration test suite

Ready for deployment to Hugging Face Spaces with free, permanent hosting.

DEPLOYMENT_GUIDE.md

@@ -0,0 +1,309 @@
+# Hugging Face Spaces Deployment Guide
+
+This guide walks you through deploying the RAG system to Hugging Face Spaces for free, permanent hosting.
+
+## Prerequisites
+
+- Hugging Face account ([sign up here](https://huggingface.co/join))
+- Git installed on your local machine
+- Code migrated to use HF Inference API (already done ✅)
+
+## Step 1: Test Locally (Recommended)
+
+Before deploying, test the application with HF Inference API locally:
+
+### 1.1 Get Your HF Token
+
+1. Go to [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens)
+2. Click "New token"
+3. Name it (e.g., "RAG System")
+4. Select `read` permissions
+5. Click "Generate"
+6. Copy the token (starts with `hf_`)
+
+### 1.2 Set Environment Variable
+
+```bash
+# Linux/Mac
+export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+# Windows (PowerShell)
+$env:HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+```
+
+### 1.3 Install Dependencies
+
+```bash
+cd /Users/v.hirenko/Desktop/DevHubVault/my-ai-projects/rag-python-rag
+source venv/bin/activate  # or venv\Scripts\activate on Windows
+pip install -r requirements.txt
+```
+
+### 1.4 Run Tests
+
+```bash
+python test_hf_integration.py
+```
+
+If all tests pass, proceed to deployment!
+
+### 1.5 Test the Application
+
+```bash
+python app.py
+```
+
+Open http://localhost:7860 and test with a few questions.
+
+## Step 2: Create Hugging Face Space
+
+### 2.1 Create New Space
+
+1. Go to [https://huggingface.co/new-space](https://huggingface.co/new-space)
+2. Fill in the details:
+   - **Owner**: Your username (e.g., `monsara`)
+   - **Space name**: `rag-python-rag`
+   - **License**: MIT
+   - **Select the Space SDK**: Gradio
+   - **Space hardware**: CPU basic (free)
+   - **Space visibility**: Public (or Private if you prefer)
+3. Click "Create Space"
+
+### 2.2 Note Your Space URL
+
+Your Space will be available at:
+```
+https://huggingface.co/spaces/YOUR_USERNAME/rag-python-rag
+```
+
+## Step 3: Configure Space Secrets
+
+### 3.1 Add HF_TOKEN Secret
+
+1. Go to your Space page
+2. Click on "Settings" (gear icon)
+3. Scroll down to "Repository secrets"
+4. Click "Add a secret"
+5. Fill in:
+   - **Name**: `HF_TOKEN`
+   - **Value**: Your HF token from Step 1.1
+6. Click "Add"
+
+⚠️ **Important**: The Space will NOT work without this secret!
+
+## Step 4: Push Code to Space
+
+You have two options:
+
+### Option A: Direct Git Push (Recommended)
+
+```bash
+cd /Users/v.hirenko/Desktop/DevHubVault/my-ai-projects/rag-python-rag
+
+# Add HF Space as remote
+git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/rag-python-rag
+# Replace YOUR_USERNAME with your actual HF username
+
+# Push to HF Space
+git push hf main
+```
+
+If prompted for credentials:
+- **Username**: Your HF username
+- **Password**: Your HF token (the same one you created)
+
+### Option B: Link GitHub Repository
+
+1. In your Space Settings
+2. Find "Linked repositories"
+3. Click "Link a GitHub repository"
+4. Select `monsara/rag-python-rag`
+5. The Space will automatically sync with your GitHub repo
+
+## Step 5: Verify Deployment
+
+### 5.1 Check Build Logs
+
+1. Go to your Space page
+2. Click on "Logs" tab
+3. Watch the build process
+4. Look for:
+   ```
+   ✅ HF_TOKEN validated successfully
+   ✅ RAG pipeline setup complete!
+   Running on local URL:  http://0.0.0.0:7860
+   ```
+
+### 5.2 Test the Space
+
+1. Once "Running" status appears
+2. Click on the Space URL
+3. Try example questions
+4. Verify answers are generated correctly
+
+## Step 6: Update README (Optional)
+
+Replace the Space README with the HF-specific one:
+
+```bash
+cd /Users/v.hirenko/Desktop/DevHubVault/my-ai-projects/rag-python-rag
+
+# Backup current README
+mv README.md README_LOCAL.md
+
+# Use HF README
+cp README_HF.md README.md
+
+# Commit and push
+git add README.md
+git commit -m "Update README for HF Spaces"
+git push hf main
+```
+
+## Troubleshooting
+
+### Issue: "HF_TOKEN not found"
+
+**Solution**: Make sure you added `HF_TOKEN` to Space secrets (Step 3.1)
+
+### Issue: "Model not found" or "Rate limit exceeded"
+
+**Solution**: 
+- Check if you're using the correct model name in `config.py`
+- Free tier has rate limits (~1000 requests/hour)
+- Consider upgrading to HF Pro ($9/month)
+
+### Issue: "Build failed"
+
+**Solution**:
+1. Check the build logs for specific errors
+2. Verify `requirements.txt` has all dependencies
+3. Make sure Python version is compatible (3.9+)
+
+### Issue: "Application crashes on startup"
+
+**Solution**:
+1. Check if `app.py` is set as the entry point in Space settings
+2. Verify all imports are correct
+3. Check logs for Python errors
+
+### Issue: Slow responses
+
+**Solution**:
+- Free tier uses shared infrastructure
+- Upgrade to better hardware (paid)
+- Or optimize chunk size and retrieval count
+
+## Monitoring
+
+### View Logs
+
+```bash
+# Real-time logs
+# Go to Space page → Logs tab
+```
+
+### Check Usage
+
+1. Go to your HF profile
+2. Click on "Usage & billing"
+3. View API usage statistics
+
+## Updating the Space
+
+### Update Code
+
+```bash
+cd /Users/v.hirenko/Desktop/DevHubVault/my-ai-projects/rag-python-rag
+
+# Make your changes
+git add .
+git commit -m "Your update message"
+git push hf main
+```
+
+The Space will automatically rebuild.
+
+### Update Dependencies
+
+Edit `requirements.txt`, then:
+
+```bash
+git add requirements.txt
+git commit -m "Update dependencies"
+git push hf main
+```
+
+### Update Configuration
+
+Edit `config.py`, then push changes as above.
+
+## Cost Considerations
+
+### Free Tier (Current Setup)
+
+- ✅ **Cost**: $0/month
+- ✅ **Hosting**: Unlimited
+- ⚠️ **Rate limits**: ~1000 requests/hour
+- ⚠️ **Performance**: Shared CPU
+- ⚠️ **Tokens**: 1024 max per response
+
+### Upgrade Options
+
+**HF Pro ($9/month)**:
+- Higher rate limits
+- Faster inference
+- Priority support
+- Better hardware options
+
+**Dedicated Hardware**:
+- CPU Upgrade: $0.03/hour (~$22/month)
+- GPU T4: $0.60/hour (~$432/month)
+- GPU A10G: $1.05/hour (~$756/month)
+
+## Security Best Practices
+
+1. **Never commit HF_TOKEN** to Git
+2. **Use Space secrets** for all sensitive data
+3. **Rotate tokens** periodically
+4. **Monitor usage** for unexpected spikes
+5. **Set rate limits** in your application
+
+## Next Steps
+
+After successful deployment:
+
+1. ✅ Test thoroughly with various questions
+2. ✅ Share the Space URL with users
+3. ✅ Monitor logs for errors
+4. ✅ Gather user feedback
+5. ✅ Iterate and improve
+
+## Support
+
+If you need help:
+
+1. Check [HF Spaces documentation](https://huggingface.co/docs/hub/spaces)
+2. Visit [HF Community forums](https://discuss.huggingface.co/)
+3. Open an issue on [GitHub](https://github.com/monsara/rag-python-rag/issues)
+
+---
+
+## Quick Reference
+
+**Space URL**: `https://huggingface.co/spaces/YOUR_USERNAME/rag-python-rag`
+
+**Settings**: `https://huggingface.co/spaces/YOUR_USERNAME/rag-python-rag/settings`
+
+**Logs**: `https://huggingface.co/spaces/YOUR_USERNAME/rag-python-rag/logs`
+
+**Push command**:
+```bash
+git push hf main
+```
+
+**Update secret**: Space Settings → Repository secrets → Edit
+
+---
+
+Good luck with your deployment! 🚀

ENV_SETUP.md

@@ -0,0 +1,193 @@
+# Environment Variables Setup
+
+This document describes the environment variables needed to run the RAG system.
+
+## Required Variables
+
+### HF_TOKEN (Required)
+
+Your Hugging Face API token for accessing the Inference API.
+
+**How to get it:**
+1. Go to [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens)
+2. Click "New token"
+3. Give it a name (e.g., "RAG System")
+4. Select `read` permissions
+5. Click "Generate"
+6. Copy the token (starts with `hf_`)
+
+**Set it:**
+
+```bash
+# Linux/Mac
+export HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+# Windows (PowerShell)
+$env:HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+
+# Windows (CMD)
+set HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+```
+
+**For Hugging Face Spaces:**
+- Go to Space Settings → Repository secrets
+- Add secret: Name=`HF_TOKEN`, Value=your token
+
+## Optional Variables
+
+### HF_MODEL
+
+The Hugging Face model to use for text generation.
+
+**Default:** `meta-llama/Llama-3.2-3B-Instruct`
+
+**Other options:**
+- `mistralai/Mistral-7B-Instruct-v0.2`
+- `meta-llama/Meta-Llama-3-8B-Instruct`
+
+```bash
+export HF_MODEL=meta-llama/Llama-3.2-3B-Instruct
+```
+
+### EMBEDDING_MODEL
+
+The sentence transformer model for embeddings (runs locally).
+
+**Default:** `all-MiniLM-L6-v2`
+
+```bash
+export EMBEDDING_MODEL=all-MiniLM-L6-v2
+```
+
+### CHROMA_PERSIST_DIR
+
+Directory for ChromaDB vector database storage.
+
+**Default:** `./chroma_db`
+
+```bash
+export CHROMA_PERSIST_DIR=./chroma_db
+```
+
+### DOCUMENTS_DIR
+
+Directory containing source documents.
+
+**Default:** `./documents`
+
+```bash
+export DOCUMENTS_DIR=./documents
+```
+
+### PROCESSED_DOCS_DIR
+
+Directory for processed markdown documents.
+
+**Default:** `./processed_docs`
+
+```bash
+export PROCESSED_DOCS_DIR=./processed_docs
+```
+
+### DEFAULT_N_RESULTS
+
+Number of context chunks to retrieve for each query.
+
+**Default:** `5`
+
+```bash
+export DEFAULT_N_RESULTS=5
+```
+
+### SIMILARITY_THRESHOLD
+
+Minimum similarity score for retrieved chunks.
+
+**Default:** `0.5`
+
+```bash
+export SIMILARITY_THRESHOLD=0.5
+```
+
+### CHUNK_SIZE
+
+Size of text chunks for processing.
+
+**Default:** `1000`
+
+```bash
+export CHUNK_SIZE=1000
+```
+
+### CHUNK_OVERLAP
+
+Overlap between consecutive chunks.
+
+**Default:** `200`
+
+```bash
+export CHUNK_OVERLAP=200
+```
+
+### GRADIO_SHARE
+
+Whether to create a public Gradio share link.
+
+**Default:** `False` (set to `True` for 72-hour public link)
+
+```bash
+export GRADIO_SHARE=True
+```
+
+### GRADIO_SERVER_PORT
+
+Port for the Gradio server.
+
+**Default:** `7860`
+
+```bash
+export GRADIO_SERVER_PORT=7860
+```
+
+## Complete Example
+
+Create a `.env` file in the project root:
+
+```bash
+# Required
+HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+
+# Optional (uncomment to override defaults)
+# HF_MODEL=meta-llama/Llama-3.2-3B-Instruct
+# EMBEDDING_MODEL=all-MiniLM-L6-v2
+# CHROMA_PERSIST_DIR=./chroma_db
+# DOCUMENTS_DIR=./documents
+# PROCESSED_DOCS_DIR=./processed_docs
+# DEFAULT_N_RESULTS=5
+# SIMILARITY_THRESHOLD=0.5
+# CHUNK_SIZE=1000
+# CHUNK_OVERLAP=200
+# GRADIO_SHARE=False
+# GRADIO_SERVER_PORT=7860
+```
+
+Then load it before running:
+
+```bash
+# Using python-dotenv (recommended)
+pip install python-dotenv
+# Add to your script: from dotenv import load_dotenv; load_dotenv()
+
+# Or manually source it
+source .env  # Linux/Mac
+```
+
+## Verification
+
+To verify your environment is set up correctly:
+
+```bash
+python -c "import os; print('HF_TOKEN:', 'SET' if os.getenv('HF_TOKEN') else 'NOT SET')"
+```
+
+Should output: `HF_TOKEN: SET`

MIGRATION_SUMMARY.md

@@ -0,0 +1,185 @@
+# Migration Summary: Ollama → Hugging Face Spaces
+
+## Overview
+
+Successfully migrated the RAG system from local Ollama to Hugging Face Inference API for free, permanent cloud hosting.
+
+## Changes Made
+
+### 1. Core Files Modified
+
+#### `llm_handler.py` ✅
+- **Before**: Used `ollama` library for local LLM inference
+- **After**: Uses `huggingface_hub.InferenceClient` for cloud inference
+- **Key changes**:
+  - Replaced `ollama.Client` with `InferenceClient`
+  - Updated streaming logic for HF API
+  - Added token authentication
+  - Changed model to `meta-llama/Llama-3.2-3B-Instruct`
+
+#### `config.py` ✅
+- **Removed**:
+  ```python
+  OLLAMA_MODEL = "llama3.2"
+  OLLAMA_BASE_URL = "http://localhost:11434"
+  ```
+- **Added**:
+  ```python
+  HF_MODEL = "meta-llama/Llama-3.2-3B-Instruct"
+  HF_TOKEN = os.getenv("HF_TOKEN", "")
+  ```
+
+#### `requirements.txt` ✅
+- **Removed**:
+  - `ollama==0.4.4`
+  - `langchain-ollama==0.2.2`
+- **Added**:
+  - `huggingface_hub==0.20.3`
+
+#### `main.py` ✅
+- Added HF token validation on startup
+- Updated error messages
+- Removed Ollama-specific checks
+
+### 2. New Files Created
+
+#### `app.py` ✅
+- Entry point for Hugging Face Spaces
+- Validates HF_TOKEN from environment/secrets
+- Provides clear setup instructions if token missing
+- Launches Gradio interface with HF-specific settings
+
+#### `README_HF.md` ✅
+- Hugging Face Spaces README with YAML frontmatter
+- Setup instructions for HF Spaces
+- Usage examples
+- Architecture diagram
+- Rate limits and limitations
+- Links to documentation
+
+#### `ENV_SETUP.md` ✅
+- Complete environment variables documentation
+- Step-by-step token setup guide
+- Examples for different platforms
+- Verification commands
+
+#### `test_hf_integration.py` ✅
+- Comprehensive test suite
+- Tests HF token, imports, API connection
+- Tests LLM handler and vector store
+- Provides clear pass/fail results
+
+#### `DEPLOYMENT_GUIDE.md` ✅
+- Step-by-step deployment instructions
+- Local testing guide
+- HF Spaces setup process
+- Troubleshooting section
+- Cost considerations
+- Security best practices
+
+### 3. Files Unchanged
+
+- `document_converter.py` ✅ (no changes needed)
+- `text_splitter.py` ✅ (no changes needed)
+- `vector_store.py` ✅ (no changes needed)
+- `README.md` ✅ (kept for local development)
+- All documentation files ✅
+
+## Architecture Comparison
+
+### Before (Ollama)
+```
+User → Gradio → Vector Store → Ollama (Local) → Response
+```
+
+### After (HF Spaces)
+```
+User → Gradio → Vector Store → HF Inference API (Cloud) → Response
+```
+
+## Key Differences
+
+| Aspect | Ollama (Before) | HF Spaces (After) |
+|--------|----------------|-------------------|
+| **Hosting** | Local only | Cloud (free) |
+| **LLM** | llama3.2 (local) | Llama-3.2-3B-Instruct (cloud) |
+| **Setup** | Install Ollama + model | Just HF token |
+| **Cost** | Free (local compute) | Free (with rate limits) |
+| **Availability** | Only when PC on | 24/7 |
+| **Rate Limits** | None | ~1000 req/hour |
+| **Scalability** | Limited by hardware | Managed by HF |
+
+## Testing Checklist
+
+Before deploying to HF Spaces, run:
+
+```bash
+# 1. Set HF token
+export HF_TOKEN=hf_your_token_here
+
+# 2. Install dependencies
+pip install -r requirements.txt
+
+# 3. Run tests
+python test_hf_integration.py
+
+# 4. Test locally
+python app.py
+```
+
+## Deployment Steps
+
+1. ✅ **Get HF Token**: https://huggingface.co/settings/tokens
+2. ✅ **Test Locally**: Run `python test_hf_integration.py`
+3. ⏳ **Create Space**: https://huggingface.co/new-space
+4. ⏳ **Add Secret**: Space Settings → Repository secrets → `HF_TOKEN`
+5. ⏳ **Push Code**: `git push hf main`
+6. ⏳ **Verify**: Check Space URL and test
+
+## Next Steps
+
+### Immediate
+1. Get Hugging Face token
+2. Run local tests
+3. Create HF Space
+4. Deploy and verify
+
+### Future Enhancements
+- [ ] Add file upload functionality
+- [ ] Support multiple documents
+- [ ] Add conversation history
+- [ ] Custom model selection
+- [ ] Advanced filtering
+- [ ] Export conversations
+
+## Resources
+
+- **HF Spaces Docs**: https://huggingface.co/docs/hub/spaces
+- **HF Inference API**: https://huggingface.co/docs/api-inference/
+- **Get Token**: https://huggingface.co/settings/tokens
+- **Pricing**: https://huggingface.co/pricing
+
+## Rollback Plan
+
+If you need to revert to Ollama:
+
+```bash
+# Checkout previous commit
+git log --oneline  # Find commit before migration
+git checkout <commit-hash>
+
+# Or restore specific files
+git checkout HEAD~1 llm_handler.py config.py requirements.txt
+```
+
+## Support
+
+- **Documentation**: See `DEPLOYMENT_GUIDE.md`
+- **Issues**: https://github.com/monsara/rag-python-rag/issues
+- **HF Community**: https://discuss.huggingface.co/
+
+---
+
+**Migration completed**: ✅ All code changes done
+**Ready to deploy**: ⏳ Awaiting HF Space creation and token setup
+**Status**: Ready for testing and deployment

README_HF.md

@@ -0,0 +1,173 @@
+---
+title: RAG Python System
+emoji: 🤖
+colorFrom: blue
+colorTo: green
+sdk: gradio
+sdk_version: 4.44.1
+app_file: app.py
+pinned: false
+license: mit
+---
+
+# RAG Python System 🤖
+
+A local-first Retrieval-Augmented Generation (RAG) system that enables intelligent question-answering over your documents using Hugging Face Inference API, ChromaDB, and Gradio.
+
+## Features
+
+- 📄 **Document Processing**: Converts PDF, DOCX, and TXT files to searchable format
+- 🔍 **Semantic Search**: Uses sentence transformers for accurate context retrieval
+- 🤖 **AI-Powered Answers**: Leverages Llama 3.2 3B via Hugging Face Inference API
+- 💬 **Interactive UI**: Clean Gradio interface for easy interaction
+- 🎯 **Source Citations**: Provides references to source documents
+
+## How It Works
+
+1. **Document Ingestion**: Upload or use pre-loaded documents (currently includes "Think Python" guide)
+2. **Semantic Chunking**: Documents are split into meaningful chunks
+3. **Vector Embeddings**: Text chunks are converted to embeddings using `all-MiniLM-L6-v2`
+4. **Context Retrieval**: Relevant chunks are retrieved based on your question
+5. **Answer Generation**: Llama 3.2 generates answers using the retrieved context
+
+## Setup Instructions
+
+### For Hugging Face Spaces Deployment
+
+1. **Fork or Duplicate this Space**
+   - Click the three dots menu → "Duplicate Space"
+
+2. **Get Your Hugging Face Token**
+   - Go to [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens)
+   - Create a new token with `read` permissions
+   - Copy the token (starts with `hf_`)
+
+3. **Add Token to Space Secrets**
+   - Go to your Space Settings
+   - Navigate to "Repository secrets"
+   - Add a new secret:
+     - **Name**: `HF_TOKEN`
+     - **Value**: Your token from step 2
+   - Click "Add"
+
+4. **Restart the Space**
+   - The Space will automatically rebuild and start
+
+### For Local Development
+
+1. **Clone the Repository**
+   ```bash
+   git clone https://github.com/monsara/rag-python-rag.git
+   cd rag-python-rag
+   ```
+
+2. **Create Virtual Environment**
+   ```bash
+   python -m venv venv
+   source venv/bin/activate  # On Windows: venv\Scripts\activate
+   ```
+
+3. **Install Dependencies**
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+4. **Set Environment Variable**
+   ```bash
+   export HF_TOKEN=hf_your_token_here
+   ```
+
+5. **Run the Application**
+   ```bash
+   python app.py
+   ```
+
+6. **Access the Interface**
+   - Open your browser to `http://localhost:7860`
+
+## Usage Examples
+
+Try asking questions like:
+
+- "How do if-else statements work in Python?"
+- "What are the different types of loops in Python?"
+- "How do you handle errors in Python?"
+- "Explain Python functions with examples"
+- "What is object-oriented programming in Python?"
+
+## Architecture
+
+```
+User Query → Gradio UI → Vector Store (ChromaDB) → Context Retrieval
+                                                          ↓
+                                                   HF Inference API
+                                                          ↓
+                                                   Llama 3.2 3B
+                                                          ↓
+                                                   Formatted Response
+```
+
+## Tech Stack
+
+- **Frontend**: Gradio 4.44.1
+- **LLM**: Llama 3.2 3B Instruct (via Hugging Face Inference API)
+- **Embeddings**: all-MiniLM-L6-v2 (Sentence Transformers)
+- **Vector DB**: ChromaDB
+- **Document Processing**: PyMuPDF, python-docx
+- **Text Splitting**: LangChain Text Splitters
+
+## Rate Limits
+
+**Free Tier (Hugging Face Inference API):**
+- ~1000 requests/hour
+- 1024 max tokens per response
+- Shared infrastructure
+
+**For Production:**
+Consider upgrading to [Hugging Face Pro](https://huggingface.co/pricing) ($9/month) for:
+- Higher rate limits
+- Faster inference
+- Priority support
+
+## Limitations
+
+- Currently uses a single pre-loaded document ("Think Python")
+- Free tier has rate limits
+- Response quality depends on context relevance
+- Max 1024 tokens per response
+
+## Roadmap
+
+- [ ] File upload functionality
+- [ ] Multiple document support
+- [ ] Conversation history
+- [ ] Custom model selection
+- [ ] Advanced filtering options
+- [ ] Export conversation feature
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## License
+
+MIT License - see LICENSE file for details
+
+## Links
+
+- **GitHub Repository**: [monsara/rag-python-rag](https://github.com/monsara/rag-python-rag)
+- **Documentation**: See [README.md](README.md) for detailed technical documentation
+- **Hugging Face**: [Get your API token](https://huggingface.co/settings/tokens)
+
+## Support
+
+If you encounter issues:
+
+1. Check that `HF_TOKEN` is set correctly in Space secrets
+2. Verify your token has `read` permissions
+3. Check the Space logs for error messages
+4. Open an issue on GitHub
+
+---
+
+Built with ❤️ using Hugging Face, Gradio, and ChromaDB

app.py

@@ -0,0 +1,91 @@
+"""
+Hugging Face Spaces Entry Point
+This file is the main entry point for running the RAG system on HF Spaces
+"""
+import os
+import sys
+import logging
+
+# Configure logging
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+)
+logger = logging.getLogger(__name__)
+
+
+def validate_hf_token():
+    """
+    Validate that HF_TOKEN is set
+    
+    Returns:
+        True if token is valid, False otherwise
+    """
+    token = os.getenv("HF_TOKEN", "")
+    
+    if not token:
+        logger.error(
+            "❌ HF_TOKEN not found! Please set it in Space Settings → Repository secrets"
+        )
+        return False
+    
+    if not token.startswith("hf_"):
+        logger.error(
+            "❌ Invalid HF_TOKEN format. Token should start with 'hf_'"
+        )
+        return False
+    
+    logger.info("✅ HF_TOKEN validated successfully")
+    return True
+
+
+def main():
+    """
+    Main entry point for Hugging Face Spaces
+    """
+    logger.info("🚀 Starting RAG System on Hugging Face Spaces...")
+    
+    # Validate HF token
+    if not validate_hf_token():
+        logger.error(
+            "\n" + "="*60 + "\n"
+            "SETUP REQUIRED:\n"
+            "1. Go to your Space Settings\n"
+            "2. Navigate to 'Repository secrets'\n"
+            "3. Add a new secret:\n"
+            "   - Name: HF_TOKEN\n"
+            "   - Value: Your Hugging Face token from https://huggingface.co/settings/tokens\n"
+            "4. Restart the Space\n"
+            + "="*60
+        )
+        sys.exit(1)
+    
+    # Import main application after token validation
+    from main import rag_system, create_gradio_interface
+    
+    # Setup the pipeline
+    logger.info("Setting up RAG pipeline...")
+    success = rag_system.setup_pipeline(force_rebuild=False)
+    
+    if not success:
+        logger.error("Failed to setup RAG pipeline. Please check the logs.")
+        sys.exit(1)
+    
+    # Create and launch Gradio interface
+    logger.info("Creating Gradio interface...")
+    interface = create_gradio_interface()
+    
+    logger.info("Launching web interface on Hugging Face Spaces...")
+    
+    # Launch without share (not needed on HF Spaces)
+    # HF Spaces automatically provides the public URL
+    interface.queue().launch(
+        share=False,  # No need for share on HF Spaces
+        server_name="0.0.0.0",
+        server_port=7860,
+        show_error=True,
+    )
+
+
+if __name__ == "__main__":
+    main()

config.py

@@ -2,6 +2,7 @@
 Configuration file for RAG System
 Contains all settings and parameters for the document processing pipeline
 """
+import os
 from pathlib import Path
 from typing import List
 
@@ -35,9 +36,9 @@ EMBEDDING_DIMENSION = 384
 CHROMA_COLLECTION_NAME = "document_embeddings"
 CHROMA_DISTANCE_METRIC = "cosine"
 
-# LLM Configuration
-OLLAMA_MODEL = "llama3.2"
-OLLAMA_BASE_URL = "http://localhost:11434"
+# LLM Configuration - Hugging Face Inference API
+HF_MODEL = "meta-llama/Llama-3.2-3B-Instruct"
+HF_TOKEN = os.getenv("HF_TOKEN", "")
 
 # Retrieval Configuration
 DEFAULT_N_RESULTS = 5
@@ -65,7 +66,7 @@ GRADIO_CONFIG = {
         "What is object-oriented programming in Python?",
     ],
     "theme": "default",
-    "share": False,  # Set to True to create a public link
+    "share": True,  # Creates public link for 72 hours  # Set to True to create a public link
 }
 
 # Test Document URL (Think Python book)

llm_handler.py

@@ -1,14 +1,15 @@
 """
 LLM Handler Module
-Manages interaction with Ollama LLM for answer generation
+Manages interaction with Hugging Face Inference API for answer generation
 """
-import ollama
+import os
 from typing import Generator, Dict, List
 import logging
+from huggingface_hub import InferenceClient
 
 from config import (
-    OLLAMA_MODEL,
-    OLLAMA_BASE_URL,
+    HF_MODEL,
+    HF_TOKEN,
     SYSTEM_PROMPT,
     PROMPT_TEMPLATE,
 )
@@ -20,56 +21,28 @@ logger = logging.getLogger(__name__)
 
 class LLMHandler:
     """
-    Handles LLM interactions using Ollama
+    Handles LLM interactions using Hugging Face Inference API
     """
     
-    def __init__(self, model: str = OLLAMA_MODEL):
+    def __init__(self, model: str = HF_MODEL, token: str = None):
         """
         Initialize the LLM handler
         
         Args:
-            model: Name of the Ollama model to use
+            model: Name of the Hugging Face model to use
+            token: HF API token (if not provided, will use HF_TOKEN from config)
         """
         self.model = model
-        self.client = ollama.Client(host=OLLAMA_BASE_URL)
-        logger.info(f"Initialized LLM handler with model: {model}")
-        
-        # Verify model is available
-        try:
-            self.verify_model()
-        except Exception as e:
-            logger.error(f"Failed to verify model: {e}")
-            raise
-    
-    def verify_model(self) -> bool:
-        """
-        Verify that the model is available in Ollama
+        self.token = token or HF_TOKEN
         
-        Returns:
-            True if model is available
-        """
-        try:
-            models = self.client.list()
-            available_models = [m.model for m in models.models]
-            
-            # Check if model name matches any available model
-            model_available = any(
-                self.model in model_name
-                for model_name in available_models
+        if not self.token:
+            raise ValueError(
+                "Hugging Face token not found. Please set HF_TOKEN environment variable "
+                "or pass it to the constructor."
             )
-            
-            if model_available:
-                logger.info(f"Model {self.model} is available")
-                return True
-            else:
-                logger.error(
-                    f"Model {self.model} not found. "
-                    f"Available models: {available_models}"
-                )
-                return False
-        except Exception as e:
-            logger.error(f"Error verifying model: {e}")
-            raise
+        
+        self.client = InferenceClient(token=self.token)
+        logger.info(f"Initialized LLM handler with model: {model}")
     
     def generate_answer(
         self,
@@ -88,24 +61,30 @@ class LLMHandler:
         Returns:
             Generated answer
         """
-        # Format the prompt
-        prompt = PROMPT_TEMPLATE.format(
-            context=context,
-            question=question
-        )
+        # Format the prompt with system prompt, context, and question
+        full_prompt = f"{SYSTEM_PROMPT}\n\n{PROMPT_TEMPLATE.format(context=context, question=question)}"
         
         try:
-            response = self.client.generate(
-                model=self.model,
-                prompt=prompt,
-                system=SYSTEM_PROMPT,
-                stream=stream,
-            )
-            
             if not stream:
-                return response['response']
-            else:
+                response = self.client.text_generation(
+                    prompt=full_prompt,
+                    model=self.model,
+                    max_new_tokens=1024,
+                    temperature=0.7,
+                    top_p=0.95,
+                    stream=False,
+                )
                 return response
+            else:
+                # Return generator for streaming
+                return self.client.text_generation(
+                    prompt=full_prompt,
+                    model=self.model,
+                    max_new_tokens=1024,
+                    temperature=0.7,
+                    top_p=0.95,
+                    stream=True,
+                )
         
         except Exception as e:
             logger.error(f"Error generating answer: {e}")
@@ -126,23 +105,22 @@ class LLMHandler:
         Yields:
             Generated text tokens
         """
-        # Format the prompt
-        prompt = PROMPT_TEMPLATE.format(
-            context=context,
-            question=question
-        )
+        # Format the prompt with system prompt, context, and question
+        full_prompt = f"{SYSTEM_PROMPT}\n\n{PROMPT_TEMPLATE.format(context=context, question=question)}"
         
         try:
-            stream = self.client.generate(
+            stream = self.client.text_generation(
+                prompt=full_prompt,
                 model=self.model,
-                prompt=prompt,
-                system=SYSTEM_PROMPT,
+                max_new_tokens=1024,
+                temperature=0.7,
+                top_p=0.95,
                 stream=True,
+                details=False,
             )
             
-            for chunk in stream:
-                if 'response' in chunk:
-                    yield chunk['response']
+            for token in stream:
+                yield token
         
         except Exception as e:
             logger.error(f"Error streaming answer: {e}")

main.py

@@ -2,11 +2,12 @@
 Main RAG Application
 Combines all components and provides a Gradio web interface
 """
+import os
 import gradio as gr
 import logging
 from typing import Generator
 
-from config import GRADIO_CONFIG, DEFAULT_N_RESULTS
+from config import GRADIO_CONFIG, DEFAULT_N_RESULTS, HF_TOKEN
 from document_converter import download_test_document, convert_all_documents
 from text_splitter import process_all_documents
 from vector_store import VectorStore, retrieve_context
@@ -183,6 +184,16 @@ def main():
     """
     logger.info("🚀 Starting RAG System...")
     
+    # Validate HF token
+    if not HF_TOKEN:
+        logger.warning(
+            "⚠️  HF_TOKEN not set. Please set the HF_TOKEN environment variable.\n"
+            "Get your token from: https://huggingface.co/settings/tokens"
+        )
+        logger.info("Continuing without HF token validation (may fail during LLM calls)...")
+    else:
+        logger.info("✅ HF_TOKEN found")
+    
     # Setup the pipeline
     logger.info("Setting up RAG pipeline...")
     success = rag_system.setup_pipeline(force_rebuild=False)

requirements.txt

@@ -12,9 +12,8 @@ langchain-text-splitters==0.3.2
 # Vector Database
 chromadb==0.5.23
 
-# LLM Integration
-ollama==0.4.4
-langchain-ollama==0.2.2
+# LLM Integration - Hugging Face
+huggingface_hub==0.20.3
 
 # Web Interface (max version for Python 3.9)
 gradio==4.44.1

test_hf_integration.py

@@ -0,0 +1,185 @@
+"""
+Test script for Hugging Face Inference API integration
+Run this to verify the setup before deploying to HF Spaces
+"""
+import os
+import sys
+import logging
+
+# Configure logging
+logging.basicConfig(
+    level=logging.INFO,
+    format='%(asctime)s - %(levelname)s - %(message)s'
+)
+logger = logging.getLogger(__name__)
+
+
+def test_hf_token():
+    """Test if HF_TOKEN is set"""
+    logger.info("Testing HF_TOKEN...")
+    token = os.getenv("HF_TOKEN", "")
+    
+    if not token:
+        logger.error("❌ HF_TOKEN not found!")
+        logger.info("Please set it: export HF_TOKEN=hf_your_token_here")
+        return False
+    
+    if not token.startswith("hf_"):
+        logger.error("❌ Invalid HF_TOKEN format (should start with 'hf_')")
+        return False
+    
+    logger.info(f"✅ HF_TOKEN found: {token[:10]}...")
+    return True
+
+
+def test_imports():
+    """Test if all required packages are installed"""
+    logger.info("\nTesting imports...")
+    
+    required_packages = [
+        ("huggingface_hub", "Hugging Face Hub"),
+        ("gradio", "Gradio"),
+        ("chromadb", "ChromaDB"),
+        ("sentence_transformers", "Sentence Transformers"),
+        ("langchain_text_splitters", "LangChain Text Splitters"),
+    ]
+    
+    all_ok = True
+    for package, name in required_packages:
+        try:
+            __import__(package)
+            logger.info(f"✅ {name} installed")
+        except ImportError:
+            logger.error(f"❌ {name} not installed")
+            all_ok = False
+    
+    return all_ok
+
+
+def test_hf_api():
+    """Test Hugging Face Inference API connection"""
+    logger.info("\nTesting HF Inference API...")
+    
+    try:
+        from huggingface_hub import InferenceClient
+        from config import HF_TOKEN, HF_MODEL
+        
+        if not HF_TOKEN:
+            logger.error("❌ Cannot test API without HF_TOKEN")
+            return False
+        
+        client = InferenceClient(token=HF_TOKEN)
+        
+        # Test with a simple prompt
+        logger.info(f"Testing model: {HF_MODEL}")
+        logger.info("Sending test request...")
+        
+        response = client.text_generation(
+            prompt="Say 'Hello, World!' and nothing else.",
+            model=HF_MODEL,
+            max_new_tokens=20,
+            temperature=0.1,
+        )
+        
+        logger.info(f"✅ API Response: {response}")
+        return True
+        
+    except Exception as e:
+        logger.error(f"❌ API Test failed: {e}")
+        return False
+
+
+def test_llm_handler():
+    """Test the LLM handler module"""
+    logger.info("\nTesting LLM Handler...")
+    
+    try:
+        from llm_handler import LLMHandler
+        
+        llm = LLMHandler()
+        logger.info("✅ LLM Handler initialized")
+        
+        # Test answer generation
+        logger.info("Testing answer generation...")
+        test_question = "What is 2+2?"
+        test_context = "Basic arithmetic: 2+2 equals 4."
+        
+        answer = llm.generate_answer(test_question, test_context, stream=False)
+        logger.info(f"✅ Generated answer: {answer[:100]}...")
+        
+        return True
+        
+    except Exception as e:
+        logger.error(f"❌ LLM Handler test failed: {e}")
+        return False
+
+
+def test_vector_store():
+    """Test vector store initialization"""
+    logger.info("\nTesting Vector Store...")
+    
+    try:
+        from vector_store import VectorStore
+        
+        vs = VectorStore()
+        logger.info("✅ Vector Store initialized")
+        
+        stats = vs.get_collection_stats()
+        logger.info(f"✅ Collection stats: {stats}")
+        
+        return True
+        
+    except Exception as e:
+        logger.error(f"❌ Vector Store test failed: {e}")
+        return False
+
+
+def main():
+    """Run all tests"""
+    logger.info("="*60)
+    logger.info("RAG System - Hugging Face Integration Tests")
+    logger.info("="*60)
+    
+    tests = [
+        ("HF Token", test_hf_token),
+        ("Package Imports", test_imports),
+        ("HF API Connection", test_hf_api),
+        ("LLM Handler", test_llm_handler),
+        ("Vector Store", test_vector_store),
+    ]
+    
+    results = {}
+    for test_name, test_func in tests:
+        try:
+            results[test_name] = test_func()
+        except Exception as e:
+            logger.error(f"❌ {test_name} crashed: {e}")
+            results[test_name] = False
+    
+    # Summary
+    logger.info("\n" + "="*60)
+    logger.info("Test Summary")
+    logger.info("="*60)
+    
+    for test_name, passed in results.items():
+        status = "✅ PASS" if passed else "❌ FAIL"
+        logger.info(f"{status} - {test_name}")
+    
+    all_passed = all(results.values())
+    
+    logger.info("="*60)
+    if all_passed:
+        logger.info("🎉 All tests passed! Ready to deploy to HF Spaces.")
+        logger.info("\nNext steps:")
+        logger.info("1. Create a new Space at https://huggingface.co/new-space")
+        logger.info("2. Choose 'Gradio' as SDK")
+        logger.info("3. Add HF_TOKEN to Space secrets")
+        logger.info("4. Push code to the Space repository")
+        return 0
+    else:
+        logger.error("❌ Some tests failed. Please fix the issues before deploying.")
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())