feat: RAG system with OCR for Hugging Face Spaces

.gitignore

@@ -0,0 +1,7 @@
+backend/.env
+backend/.env.bak
+backend/storage/
+frontend/node_modules/
+frontend/dist/
+*.pyc
+__pycache__/

README.md

@@ -1,10 +1,431 @@
 ---
-title: Web Based Rag
-emoji: 💻
-colorFrom: pink
-colorTo: blue
+title: Web-Based RAG System
+emoji: 📚
+colorFrom: blue
+colorTo: indigo
 sdk: docker
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Web-Based RAG System
+
+A production-ready Retrieval-Augmented Generation (RAG) system that combines PDF document processing and web search capabilities to provide intelligent answers to user queries.
+
+## Table of Contents
+- [Features](#features)
+- [Tech Stack](#tech-stack)
+- [Architecture](#architecture)
+- [Installation](#installation)
+- [Configuration](#configuration)
+- [Usage](#usage)
+- [Project Structure](#project-structure)
+- [API Endpoints](#api-endpoints)
+- [Frontend Components](#frontend-components)
+- [Contributing](#contributing)
+- [License](#license)
+
+## Features
+
+- **Multi-Modal Query Processing**: Supports queries against both uploaded PDF documents and live web search
+- **PDF Document Management**: Upload, store, and process PDF documents with advanced extraction techniques
+- **OCR Support for Scanned PDFs**: Automatically extracts text from image-based/scanned PDFs using Tesseract OCR
+- **Hybrid Search**: Combine PDF-based retrieval with web search for comprehensive answers
+- **Confidence Scoring**: Provides confidence scores for generated responses
+- **Vector Storage**: Efficient similarity search using ChromaDB vector database
+- **Modern UI**: Responsive React-based frontend with intuitive user experience
+- **RESTful API**: Well-documented API endpoints for easy integration
+- **File Upload**: Drag-and-drop PDF upload functionality
+- **Query Modes**: Different query modes (PDF-only, Web-only, Hybrid, Restricted)
+
+## Tech Stack
+
+### Backend
+- **Framework**: FastAPI (Python)
+- **Database**: ChromaDB (Vector Database)
+- **Embeddings**: Sentence Transformers
+- **Language**: Python 3.11+
+- **Web Framework**: FastAPI with Uvicorn ASGI server
+- **HTTP Client**: aiohttp
+- **PDF Processing**: PyPDF, pdfplumber, pdf2image, pytesseract
+- **OCR**: Tesseract for scanned/image-based PDFs
+- **LLM Integration**: Groq API
+- **Environment Management**: python-dotenv
+- **Data Validation**: Pydantic
+
+### Frontend
+- **Framework**: React 18+
+- **Language**: TypeScript
+- **Styling**: Tailwind CSS
+- **Build Tool**: Vite
+- **HTTP Client**: Axios
+- **UI Components**: Custom-built with Lucide React icons
+- **File Upload**: react-dropzone
+- **Notifications**: react-hot-toast
+
+## Architecture
+
+The application follows a microservices architecture with a clear separation between frontend and backend:
+
+```
+┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
+│   Frontend      │    │   Backend       │    │   External      │
+│   (React)       │◄──►│   (FastAPI)     │◄──►│   Services      │
+│                 │    │                 │    │                 │
+│ • User Interface│    │ • API Gateway   │    │ • Groq API      │
+│ • File Upload   │    │ • PDF Processor │    │ • Web Search    │
+│ • Query Input   │    │ • Embedding     │    │ • Vector Store  │
+│ • Results Display│   │ • Retriever     │    │                 │
+└─────────────────┘    │ • LLM Service   │    └─────────────────┘
+                       └─────────────────┘
+```
+
+## Installation
+
+### Prerequisites
+- Python 3.11+
+- Node.js 18+
+- npm or yarn
+- Git
+- **Tesseract OCR** (for scanned PDF support):
+  - macOS: `brew install tesseract poppler`
+  - Ubuntu: `sudo apt-get install tesseract-ocr poppler-utils`
+  - Windows: Download from https://github.com/tesseract-ocr/tesseract
+
+### Backend Setup
+
+1. Clone the repository:
+   ```bash
+   git clone https://github.com/YuvrajSinghBhadoria2/web_based_rag.git
+   cd web_based_rag/backend
+   ```
+
+2. Create a virtual environment:
+   ```bash
+   python -m venv venv
+   source venv/bin/activate  # On Windows: venv\Scripts\activate
+   ```
+
+3. Install dependencies:
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+4. Create a `.env` file in the backend directory based on `.env.bak`:
+   ```bash
+   cp .env.bak .env
+   ```
+
+### Frontend Setup
+
+1. Navigate to the frontend directory:
+   ```bash
+   cd ../frontend
+   ```
+
+2. Install dependencies:
+   ```bash
+   npm install
+   ```
+
+## Configuration
+
+### Backend Environment Variables
+
+Create a `.env` file in the backend directory with the following variables:
+
+```env
+GROQ_API_KEY=your_groq_api_key_here
+SERPER_API_KEY=your_serper_api_key_here  # Optional - for web search
+TAVILY_API_KEY=your_tavily_api_key_here  # Optional - for web search
+CHROMA_DB_PATH=./storage/vector_db
+UPLOAD_DIR=./storage/uploads
+MODEL_NAME=llama3-70b-8192
+TEMPERATURE=0.1
+MAX_TOKENS=1000
+TOP_P=1
+STOP_TOKENS=["\n", "###"]
+CORS_ORIGINS=["http://localhost:5173", "http://localhost:3000", "http://127.0.0.1:5173", "http://127.0.0.1:3000", "http://localhost:5175"]
+```
+
+Replace `your_groq_api_key_here` with your actual Groq API key. You can get one from [Groq Cloud](https://console.groq.com/keys).
+
+For web search functionality, add Serper or Tavily API keys (optional - without them, hybrid mode will only use PDF sources).
+
+## Usage
+
+### Running the Backend
+
+1. Make sure you're in the backend directory
+2. Activate your virtual environment
+3. Start the backend server:
+   ```bash
+   uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
+   ```
+
+The backend will be available at `http://localhost:8000` with API documentation at `http://localhost:8000/api/docs`.
+
+### Running the Frontend
+
+1. Navigate to the frontend directory
+2. Start the development server:
+   ```bash
+   npm run dev
+   ```
+
+The frontend will be available at `http://localhost:5173`.
+
+### Application Workflow
+
+1. **Upload Documents**: Use the drag-and-drop interface to upload PDF documents
+2. **Select Query Mode**: Choose between PDF-only, Web-only, Hybrid, or Restricted modes
+3. **Enter Query**: Type your question in the query input
+4. **Get Response**: Receive an AI-generated answer with confidence score and source citations
+5. **Review Sources**: View the documents and web pages that contributed to the response
+
+### OCR for Scanned PDFs
+
+The system automatically detects and processes scanned/image-based PDFs using Tesseract OCR:
+- If a PDF contains selectable text, it uses the native text extraction
+- If no text is found, it automatically applies OCR to extract text from images
+- Works with scanned documents, image-only PDFs, and documents with mixed content
+
+## Project Structure
+
+```
+web_based_rag/
+├── backend/
+│   ├── app/
+│   │   ├── api/
+│   │   │   └── v1/
+│   │   │       └── routes/
+│   │   │           ├── documents.py    # Document management endpoints
+│   │   │           ├── health.py       # Health check endpoint
+│   │   │           ├── query.py        # Query processing endpoints
+│   │   │           └── upload.py       # File upload endpoints
+│   │   ├── core/     # Core utilities and configurations
+│   │   ├── models/
+│   │   │   └── schemas.py              # Pydantic models and schemas
+│   │   ├── services/
+│   │   │   ├── confidence.py          # Confidence scoring service
+│   │   │   ├── embeddings.py          # Embedding generation service
+│   │   │   ├── enhanced_llm.py        # Enhanced LLM service
+│   │   │   ├── llm_service.py         # LLM integration service
+│   │   │   ├── pdf_processor.py       # PDF processing service
+│   │   │   ├── prompt_guard.py        # Prompt safety service
+│   │   │   ├── retriever.py           # Information retrieval service
+│   │   │   ├── vector_store.py        # Vector database operations
+│   │   │   └── web_search.py          # Web search service
+│   │   ├── utils/
+│   │   │   ├── chunking.py           # Text chunking utilities
+│   │   │   └── rate_limiter.py        # Rate limiting utilities
+│   │   ├── config.py                 # Configuration settings
+│   │   └── main.py                   # Application entry point
+│   ├── storage/
+│   │   ├── uploads/                  # Uploaded PDF files
+│   │   ├── vector_db/                # Vector database files
+│   │   └── documents.json            # Document metadata
+│   ├── requirements.txt              # Python dependencies
+│   ├── Dockerfile                    # Docker configuration
+│   └── .env.bak                      # Environment variables template
+└── frontend/
+    ├── src/
+    │   ├── components/
+    │   │   ├── common/               # Reusable UI components
+    │   │   ├── documents/            # Document-related components
+    │   │   ├── layout/               # Layout components
+    │   │   ├── query/                # Query input components
+    │   │   ├── results/              # Results display components
+    │   │   └── settings/             # Settings modal components
+    │   ├── context/
+    │   │   └── AppContext.tsx        # Application state management
+    │   ├── services/
+    │   │   └── api.ts                # API service client
+    │   ├── types/
+    │   │   └── index.ts              # Type definitions
+    │   ├── App.tsx                   # Main application component
+    │   └── main.tsx                  # Application entry point
+    ├── package.json
+    ├── tsconfig.json
+    ├── tailwind.config.js
+    └── vite.config.ts
+```
+
+## API Endpoints
+
+### Health Check
+- `GET /` - Root endpoint returning API information
+
+### Documents
+- `GET /api/v1/documents` - Get list of uploaded documents
+- `DELETE /api/v1/documents/{document_id}` - Delete a document
+
+### File Upload
+- `POST /api/v1/upload` - Upload PDF document
+
+### Query
+- `POST /api/v1/query` - Process query with specified mode
+  - Request body: `{"query": "your query", "mode": "pdf|web|hybrid|restricted", "document_ids": ["optional document IDs"]}`
+  - Response: `{"response": "answer", "sources": [], "confidence": 0.85}`
+
+### Additional Endpoints
+- `GET /api/docs` - Interactive API documentation (Swagger UI)
+- `GET /api/redoc` - Alternative API documentation (ReDoc)
+
+## Frontend Components
+
+### Layout Components
+- **Header**: Navigation and branding
+- **Sidebar**: Document management and settings
+- **MainContent**: Primary content area
+
+### Document Components
+- **FileUpload**: Drag-and-drop PDF upload
+- **DocumentList**: Display of uploaded documents
+- **DocumentCard**: Individual document information
+
+### Query Components
+- **QueryInput**: Input field with mode selector
+- **ModeSelector**: Options for PDF-only, Web-only, Hybrid, or Restricted queries
+
+### Results Components
+- **ResultsDisplay**: Container for query results
+- **AnswerCard**: Display of the AI-generated answer
+- **SourcesList**: List of source documents
+- **SourceCard**: Detailed source information
+- **ConfidenceIndicator**: Visual representation of response confidence
+
+### Settings Components
+- **SettingsModal**: Configuration options
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add some amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+## Deploying to Hugging Face Spaces
+
+This application is configured for deployment on Hugging Face Spaces using the Docker SDK. The repository includes:
+
+- A `Dockerfile` that sets up the complete environment
+- A `README.md` with proper Hugging Face metadata
+- All necessary backend and frontend code
+
+To deploy to your Space:
+
+1. Create a new Space with the Docker SDK
+2. Point it to this repository
+3. Add your API keys as Space Secrets:
+   - `GROQ_API_KEY`: Your Groq API key
+4. The Space will automatically build and deploy using the Dockerfile
+
+Your application will be served at the port specified in the Dockerfile (7860).
+
+### Option 1: Using the Docker Image
+
+1. Create a new Space on Hugging Face with the following settings:
+   - **Space SDK**: Docker
+   - **Hardware**: Choose based on your needs (GPU recommended for better performance)
+
+2. Add your Hugging Face token and API keys as secrets in the Space settings:
+   - `HF_TOKEN`: Your Hugging Face token (`your_hf_token_here`)
+   - `GROQ_API_KEY`: Your Groq API key
+   - Any other required API keys
+
+3. Create a `Dockerfile` in your Space repository with the following content:
+
+```dockerfile
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# Install nodejs for the frontend
+RUN apt-get update && apt-get install -y nodejs npm && apt-get clean
+
+# Copy backend requirements and install Python dependencies
+COPY backend/requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Install frontend dependencies
+COPY frontend/package*.json ./frontend/
+RUN cd frontend && npm ci --only=production
+
+# Copy the rest of the application
+COPY . .
+
+# Build the frontend
+RUN cd frontend && npm run build
+
+# Expose the port Hugging Face Spaces expects
+EXPOSE 7860
+
+# Start both backend and frontend
+CMD bash -c "cd backend && python -m uvicorn app.main:app --host 0.0.0.0 --port 7860 & cd frontend && npx serve -s dist -l 7861"
+```
+
+4. Create an `.env` file in the backend directory with your API keys:
+
+```env
+GROQ_API_KEY=your_groq_api_key_here
+# Add other required environment variables
+```
+
+### Option 2: Deploying Your Existing React Frontend (Recommended)
+
+To deploy your existing React frontend along with the FastAPI backend (this preserves your original UI):
+
+1. In your Hugging Face Space repository, copy your entire project
+
+2. Create a Dockerfile that builds and serves both applications:
+
+```dockerfile
+FROM node:18-alpine AS frontend-build
+WORKDIR /app
+COPY frontend/package*.json .
+RUN npm ci
+COPY frontend/ .
+RUN npm run build
+
+FROM python:3.11-slim AS backend-build
+WORKDIR /app
+
+# Install dependencies
+COPY backend/requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY backend/ .
+
+# Copy built frontend
+COPY --from=frontend-build /app/dist ./frontend/dist
+
+# Install node for serving frontend
+RUN apt-get update && apt-get install -y nodejs npm && apt-get clean
+
+EXPOSE 7860
+
+CMD python -m uvicorn app.main:app --host 0.0.0.0 --port 7860
+```
+
+3. Update your backend CORS settings in `backend/app/config.py` to allow the Hugging Face Space URL
+
+4. Add your API keys as Space secrets:
+   - `GROQ_API_KEY`: Your Groq API key
+   - Other required API keys
+
+Note: This approach maintains your original React interface which is more feature-rich than a Gradio interface. Your existing frontend with its document cards, sidebar, settings modal, and responsive design will be preserved.
+
+## Deployment Steps
+
+1. Create a new repository on Hugging Face Spaces
+2. Push your code to the repository
+3. Add your API keys as secrets in the Space settings
+4. The application will automatically build and deploy
+
+Your RAG application is now ready for deployment on Hugging Face Spaces with your Hugging Face token: `your_hf_token_here`
+
+## License
+
+This project is licensed under the MIT License - see the LICENSE file for details.

backend/Dockerfile

@@ -0,0 +1,25 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    tesseract-ocr \
+    poppler-utils \
+    nodejs \
+    npm \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY backend/requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY backend/ ./backend/
+COPY frontend/ ./frontend/
+
+RUN cd frontend && npm install && npm run build
+
+RUN mkdir -p /app/storage/uploads /app/storage/vector_db
+
+EXPOSE 7860
+
+CMD ["bash", "-c", "cd backend && uvicorn app.main:app --host 0.0.0.0 --port 7860 & cd frontend && npx serve -s dist -l 7861"]

backend/app/api/__init__.py

@@ -0,0 +1,3 @@
+from app.api.v1 import api_router
+
+__all__ = ["api_router"]

backend/app/api/v1/__init__.py

@@ -0,0 +1,5 @@
+from fastapi import APIRouter
+from app.api.v1.routes import router
+
+api_router = APIRouter()
+api_router.include_router(router, prefix="/v1")

backend/app/api/v1/routes/__init__.py

@@ -0,0 +1,12 @@
+from fastapi import APIRouter
+from .upload import router as upload_router
+from .query import router as query_router
+from .documents import router as documents_router
+from .health import router as health_router
+
+router = APIRouter()
+
+router.include_router(upload_router)
+router.include_router(query_router)
+router.include_router(documents_router)
+router.include_router(health_router)

backend/app/api/v1/routes/documents.py

@@ -0,0 +1,70 @@
+from fastapi import APIRouter
+from app.models.schemas import DocumentListResponse, Document
+from app.services.vector_store import vector_store
+from pathlib import Path
+from datetime import datetime
+import json
+
+router = APIRouter(prefix="/documents", tags=["documents"])
+
+DOCUMENTS_DB = "./storage/documents.json"
+
+
+def load_documents_db():
+    if Path(DOCUMENTS_DB).exists():
+        with open(DOCUMENTS_DB, "r") as f:
+            return json.load(f)
+    return {}
+
+
+def save_documents_db(documents):
+    Path(DOCUMENTS_DB).parent.mkdir(exist_ok=True, parents=True)
+    with open(DOCUMENTS_DB, "w") as f:
+        json.dump(documents, f, default=str)
+
+
+@router.get("/", response_model=DocumentListResponse)
+async def list_documents():
+    documents_db = load_documents_db()
+
+    documents = []
+    for doc_id, doc_data in documents_db.items():
+        documents.append(
+            Document(
+                id=doc_id,
+                filename=doc_data.get("filename", "Unknown"),
+                upload_date=datetime.fromisoformat(
+                    doc_data.get("upload_date", datetime.utcnow().isoformat())
+                ),
+                chunk_count=doc_data.get("chunk_count", 0),
+                file_size=doc_data.get("file_size", 0),
+                status=doc_data.get("status", "ready"),
+            )
+        )
+
+    return DocumentListResponse(
+        documents=sorted(documents, key=lambda x: x.upload_date, reverse=True),
+        total=len(documents),
+    )
+
+
+@router.delete("/{document_id}")
+async def delete_document(document_id: str):
+    documents_db = load_documents_db()
+
+    if document_id not in documents_db:
+        return {"message": "Document not found"}
+
+    file_path = Path(f"./storage/uploads/{document_id}.pdf")
+    if file_path.exists():
+        file_path.unlink()
+
+    del documents_db[document_id]
+    save_documents_db(documents_db)
+
+    try:
+        await vector_store.delete_document(document_id)
+    except:
+        pass
+
+    return {"message": "Document deleted"}

backend/app/api/v1/routes/health.py

@@ -0,0 +1,14 @@
+from fastapi import APIRouter
+from app.models.schemas import HealthResponse
+
+router = APIRouter(prefix="/health", tags=["health"])
+
+
+@router.get("/", response_model=HealthResponse)
+async def health_check():
+    return HealthResponse(
+        status="healthy",
+        embedding_model="all-MiniLM-L6-v2",
+        vector_db="chromadb",
+        llm="groq",
+    )

backend/app/api/v1/routes/query.py

@@ -0,0 +1,87 @@
+from fastapi import APIRouter, HTTPException
+from app.models.schemas import QueryRequest, QueryResponse
+from app.services.retriever import retriever_service
+from app.services.enhanced_llm import enhanced_llm_service
+from app.services.prompt_guard import prompt_guard
+from app.services.confidence import confidence_service
+from datetime import datetime
+import time
+
+router = APIRouter(prefix="/query", tags=["query"])
+
+
+@router.post("/", response_model=QueryResponse)
+async def query_documents(request: QueryRequest):
+    start_time = time.time()
+
+    # Validate restricted mode
+    try:
+        if request.mode.value == "restricted":
+            await prompt_guard.validate_input(request.query, request.restrictions)
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+    # Retrieve sources
+    try:
+        sources = await retriever_service.retrieve(
+            query=request.query,
+            mode=request.mode,
+            top_k=request.top_k or 5,
+            document_ids=request.document_ids,
+        )
+    except Exception as e:
+        raise HTTPException(status_code=500, detail=f"Retrieval failed: {str(e)}")
+
+    # Handle no sources
+    if not sources:
+        return QueryResponse(
+            answer="No relevant sources found for your query.",
+            sources=[],
+            confidence=0,
+            mode_used=request.mode,
+            query=request.query,
+            timestamp=datetime.utcnow(),
+            processing_time_ms=int((time.time() - start_time) * 1000),
+        )
+
+    # Generate answer with enhanced LLM service
+    try:
+        answer = await enhanced_llm_service.generate_answer(
+            query=request.query, sources=sources
+        )
+    except Exception as e:
+        return QueryResponse(
+            answer=f"Unable to generate answer at this time due to high demand. Please try again in a few moments.",
+            sources=sources,
+            confidence=50,
+            mode_used=request.mode,
+            query=request.query,
+            timestamp=datetime.utcnow(),
+            processing_time_ms=int((time.time() - start_time) * 1000),
+        )
+
+    # Sanitize output
+    try:
+        answer = await prompt_guard.sanitize_output(answer)
+    except:
+        pass
+
+    # Calculate confidence
+    try:
+        confidence = confidence_service.calculate_confidence(
+            query=request.query, answer=answer, sources=sources
+        )
+    except:
+        confidence = 50.0
+
+    processing_time = int((time.time() - start_time) * 1000)
+
+    return QueryResponse(
+        answer=answer,
+        sources=sources,
+        confidence=confidence,
+        mode_used=request.mode,
+        query=request.query,
+        timestamp=datetime.utcnow(),
+        processing_time_ms=processing_time,
+    )

backend/app/api/v1/routes/upload.py

@@ -0,0 +1,87 @@
+from fastapi import APIRouter, UploadFile, File, HTTPException
+from app.models.schemas import UploadResponse
+from app.services.pdf_processor import pdf_processor
+from app.services.embeddings import embedding_service
+from app.services.vector_store import vector_store
+from pathlib import Path
+import uuid
+from datetime import datetime
+import shutil
+import os
+import json
+
+router = APIRouter(prefix="/upload", tags=["upload"])
+
+DOCUMENTS_DB = "./storage/documents.json"
+
+
+def load_documents_db():
+    if Path(DOCUMENTS_DB).exists():
+        with open(DOCUMENTS_DB, "r") as f:
+            return json.load(f)
+    return {}
+
+
+def save_documents_db(documents):
+    Path(DOCUMENTS_DB).parent.mkdir(exist_ok=True, parents=True)
+    with open(DOCUMENTS_DB, "w") as f:
+        json.dump(documents, f, default=str)
+
+
+@router.post("/", response_model=UploadResponse)
+async def upload_pdf(file: UploadFile = File(...)):
+    if not file.filename.endswith(".pdf"):
+        raise HTTPException(status_code=400, detail="Only PDF files are allowed")
+
+    document_id = str(uuid.uuid4())
+
+    upload_dir = Path("./storage/uploads")
+    upload_dir.mkdir(exist_ok=True, parents=True)
+    file_path = upload_dir / f"{document_id}.pdf"
+
+    with file_path.open("wb") as buffer:
+        shutil.copyfileobj(file.file, buffer)
+
+    try:
+        pdf_processor.validate_file(file_path)
+    except Exception as e:
+        if file_path.exists():
+            os.remove(file_path)
+        raise HTTPException(status_code=400, detail=str(e))
+
+    try:
+        chunks = await pdf_processor.process_document(file_path, document_id)
+    except Exception as e:
+        if file_path.exists():
+            os.remove(file_path)
+        raise HTTPException(status_code=500, detail=f"Failed to process PDF: {str(e)}")
+
+    try:
+        texts = [chunk["text"] for chunk in chunks]
+        embeddings = embedding_service.embed_batch(texts)
+        await vector_store.add_chunks(chunks, embeddings)
+    except Exception as e:
+        raise HTTPException(
+            status_code=500, detail=f"Failed to index document: {str(e)}"
+        )
+
+    file_size = file_path.stat().st_size if file_path.exists() else 0
+
+    documents_db = load_documents_db()
+    documents_db[document_id] = {
+        "id": document_id,
+        "filename": file.filename,
+        "upload_date": datetime.utcnow().isoformat(),
+        "chunk_count": len(chunks),
+        "file_size": file_size,
+        "status": "ready",
+    }
+    save_documents_db(documents_db)
+
+    return UploadResponse(
+        document_id=document_id,
+        filename=file.filename,
+        status="completed",
+        chunks_created=len(chunks),
+        upload_date=datetime.utcnow(),
+    )

backend/app/config.py

@@ -0,0 +1,47 @@
+import os
+from pathlib import Path
+from dotenv import load_dotenv
+from pydantic import BaseModel
+
+load_dotenv()
+
+
+class Settings(BaseModel):
+    VECTOR_DB_PATH: Path = Path(os.getenv("VECTOR_DB_PATH", "./storage/vector_db"))
+    EMBEDDING_MODEL: str = os.getenv("EMBEDDING_MODEL", "all-MiniLM-L6-v2")
+    CHUNK_SIZE: int = int(os.getenv("CHUNK_SIZE", "512"))
+    CHUNK_OVERLAP: int = int(os.getenv("CHUNK_OVERLAP", "50"))
+    TOP_K: int = int(os.getenv("TOP_K", "5"))
+
+    GROQ_API_KEY: str = os.getenv("GROQ_API_KEY", "")
+    SERPER_API_KEY: str = os.getenv("SERPER_API_KEY", "")
+    TAVILY_API_KEY: str = os.getenv("TAVILY_API_KEY", "")
+
+    ALLOWED_MODES: list = ["web", "pdf", "hybrid", "restricted"]
+
+    UPLOAD_DIR: Path = Path(os.path.abspath("./storage/uploads"))
+
+    MAX_FILE_SIZE: int = 10 * 1024 * 1024
+    ALLOWED_FILE_TYPES: list = ["application/pdf"]
+
+    GROQ_MODEL: str = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile")
+    GROQ_MODEL_PRIMARY: str = os.getenv("GROQ_MODEL_PRIMARY", "llama-3.1-8b-instant")
+    GROQ_MODEL_SECONDARY: str = os.getenv("GROQ_MODEL_SECONDARY", "mixtral-8x7b-32768")
+    GROQ_MODEL_FALLBACK: str = os.getenv(
+        "GROQ_MODEL_FALLBACK", "llama-3.3-70b-versatile"
+    )
+    MAX_TOKENS: int = int(os.getenv("MAX_TOKENS", "2048"))
+    TEMPERATURE: float = float(os.getenv("TEMPERATURE", "0.1"))
+
+    WEB_SEARCH_MAX_RESULTS: int = int(os.getenv("WEB_SEARCH_MAX_RESULTS", "5"))
+    SIMILARITY_THRESHOLD: float = float(os.getenv("SIMILARITY_THRESHOLD", "0.7"))
+
+    CORS_ORIGINS: list = [
+        "http://localhost:3000",
+        "http://localhost:5173",
+        "https://yuvis-web-based-rag.hf.space",
+        "https://*.hf.space",
+    ]
+
+
+settings = Settings()

backend/app/main.py

@@ -0,0 +1,52 @@
+from fastapi import FastAPI
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.staticfiles import StaticFiles
+from fastapi.responses import FileResponse
+from app.api.v1.routes import router as api_router
+from app.config import settings
+import os
+
+app = FastAPI(
+    title="RAG System API",
+    description="Production-grade RAG system with PDF and web search",
+    version="1.0.0",
+    docs_url="/api/docs",
+    redoc_url="/api/redoc",
+)
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=settings.CORS_ORIGINS,
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+app.include_router(api_router, prefix="/api/v1")
+
+# Serve static files (React frontend)
+frontend_path = "/app"
+try:
+    app.mount("/", StaticFiles(directory=frontend_path, html=True), name="static")
+except RuntimeError:
+    # Fallback path if /app doesn't exist
+    fallback_path = os.path.join(os.path.dirname(__file__), "../../frontend/dist")
+    if os.path.exists(fallback_path):
+        app.mount("/", StaticFiles(directory=fallback_path, html=True), name="static")
+
+
+@app.get("/")
+async def serve_index():
+    index_path = os.path.join(frontend_path, "index.html")
+    if os.path.exists(index_path):
+        return FileResponse(index_path)
+    # Also try fallback path
+    fallback_path = os.path.join(os.path.dirname(__file__), "../../frontend/dist/index.html")
+    if os.path.exists(fallback_path):
+        return FileResponse(fallback_path)
+    return {"message": "RAG System API", "version": "1.0.0"}
+
+
+@app.get("/health")
+async def health_check():
+    return {"status": "healthy", "service": "RAG System"}

backend/app/models/schemas.py

@@ -0,0 +1,71 @@
+from pydantic import BaseModel, Field
+from typing import List, Optional
+from datetime import datetime
+from enum import Enum
+
+
+class QueryMode(str, Enum):
+    WEB = "web"
+    PDF = "pdf"
+    HYBRID = "hybrid"
+    RESTRICTED = "restricted"
+
+
+class SourceType(str, Enum):
+    PDF = "pdf"
+    WEB = "web"
+
+
+class QueryRequest(BaseModel):
+    query: str = Field(..., min_length=1, max_length=1000)
+    mode: QueryMode
+    document_ids: Optional[List[str]] = None
+    restrictions: Optional[List[str]] = None
+    top_k: Optional[int] = Field(default=5, ge=1, le=20)
+
+
+class Source(BaseModel):
+    type: SourceType
+    content: str
+    reference: str
+    title: str
+    relevance_score: Optional[float] = None
+
+
+class QueryResponse(BaseModel):
+    answer: str
+    sources: List[Source]
+    confidence: float = Field(..., ge=0, le=100)
+    mode_used: QueryMode
+    query: str
+    timestamp: datetime
+    processing_time_ms: int
+
+
+class UploadResponse(BaseModel):
+    document_id: str
+    filename: str
+    status: str
+    chunks_created: int
+    upload_date: datetime
+
+
+class Document(BaseModel):
+    id: str
+    filename: str
+    upload_date: datetime
+    chunk_count: int
+    file_size: int
+    status: str
+
+
+class DocumentListResponse(BaseModel):
+    documents: List[Document]
+    total: int
+
+
+class HealthResponse(BaseModel):
+    status: str
+    embedding_model: str
+    vector_db: str
+    llm: str

backend/app/services/__init__.py

@@ -0,0 +1,19 @@
+from .embeddings import embedding_service
+from .pdf_processor import pdf_processor
+from .vector_store import vector_store
+from .retriever import retriever_service
+from .web_search import web_search_service
+from .llm_service import llm_service
+from .prompt_guard import prompt_guard
+from .confidence import confidence_service
+
+__all__ = [
+    "embedding_service",
+    "pdf_processor",
+    "vector_store",
+    "retriever_service",
+    "web_search_service",
+    "llm_service",
+    "prompt_guard",
+    "confidence_service",
+]

backend/app/services/confidence.py

@@ -0,0 +1,61 @@
+from typing import List
+from app.models.schemas import Source
+from app.services.embeddings import embedding_service
+
+
+class ConfidenceService:
+    def calculate_confidence(
+        self, query: str, answer: str, sources: List[Source]
+    ) -> float:
+        factors = {}
+
+        # Factor 1: Source count (25%)
+        source_count = len(sources)
+        if source_count == 0:
+            factors["source_count"] = 0
+        elif source_count == 1:
+            factors["source_count"] = 15
+        elif source_count == 2:
+            factors["source_count"] = 20
+        else:
+            factors["source_count"] = 25
+
+        # Factor 2: Average relevance score (30%)
+        if sources:
+            avg_relevance = sum(s.relevance_score or 0 for s in sources) / len(sources)
+            factors["source_relevance"] = avg_relevance * 30
+        else:
+            factors["source_relevance"] = 0
+
+        # Factor 3: Semantic similarity (30%)
+        try:
+            query_emb = embedding_service.embed_text(query)
+            answer_text = answer[:1000] if len(answer) > 1000 else answer
+            answer_emb = embedding_service.embed_text(answer_text)
+            similarity = embedding_service.calculate_similarity(query_emb, answer_emb)
+            factors["semantic_similarity"] = similarity * 30
+        except Exception:
+            factors["semantic_similarity"] = 0
+
+        # Factor 4: Citation density (15%)
+        citation_count = answer.count("[Source")
+        if citation_count == 0:
+            factors["citation_density"] = 0
+        elif citation_count == 1:
+            factors["citation_density"] = 10
+        elif citation_count == 2:
+            factors["citation_density"] = 13
+        else:
+            factors["citation_density"] = 15
+
+        # Calculate total
+        total_confidence = sum(factors.values())
+
+        # Ensure minimum confidence for valid responses
+        if source_count > 0 and total_confidence < 30:
+            total_confidence = 35
+
+        return round(min(total_confidence, 100), 2)
+
+
+confidence_service = ConfidenceService()

backend/app/services/embeddings.py

@@ -0,0 +1,33 @@
+from sentence_transformers import SentenceTransformer
+from typing import List
+import numpy as np
+from app.config import settings
+
+
+class EmbeddingService:
+    def __init__(self):
+        self.model = SentenceTransformer(settings.EMBEDDING_MODEL)
+        self.dimension = 384
+
+    def embed_text(self, text: str) -> List[float]:
+        embedding = self.model.encode(text, convert_to_numpy=True)
+        return embedding.tolist()
+
+    def embed_batch(self, texts: List[str]) -> List[List[float]]:
+        embeddings = self.model.encode(texts, convert_to_numpy=True, batch_size=32)
+        return embeddings.tolist()
+
+    def calculate_similarity(
+        self, embedding1: List[float], embedding2: List[float]
+    ) -> float:
+        vec1 = np.array(embedding1)
+        vec2 = np.array(embedding2)
+
+        dot_product = np.dot(vec1, vec2)
+        norm1 = np.linalg.norm(vec1)
+        norm2 = np.linalg.norm(vec2)
+
+        return float(dot_product / (norm1 * norm2))
+
+
+embedding_service = EmbeddingService()

backend/app/services/enhanced_llm.py

@@ -0,0 +1,267 @@
+"""
+Enhanced LLM Service with Multi-Model Support
+- Groq llama-3.1-8b-instant (Primary - Fast)
+- Groq mixtral-8x7b-32768 (Secondary - Quality)
+- Request queuing to prevent rate limits
+- Response caching for repeated queries
+"""
+
+import aiohttp
+import asyncio
+import time
+import hashlib
+import json
+from typing import List, Optional, Dict
+from datetime import datetime
+from dataclasses import dataclass
+from enum import Enum
+
+from app.models.schemas import Source
+from app.config import settings
+
+
+class ModelType(Enum):
+    PRIMARY = "llama-3.1-8b-instant"  # Fast, good for simple queries
+    SECONDARY = "mixtral-8x7b-32768"  # Better for complex queries
+    FALLBACK = "llama-3.1-70b-versatile"  # Highest quality
+
+
+@dataclass
+class ModelConfig:
+    name: str
+    max_tokens: int
+    temperature: float
+    priority: int  # Lower = higher priority
+
+
+# Model configurations
+MODEL_CONFIGS: Dict[ModelType, ModelConfig] = {
+    ModelType.PRIMARY: ModelConfig(
+        name=getattr(settings, "GROQ_MODEL_PRIMARY", "llama-3.1-8b-instant"),
+        max_tokens=2048,
+        temperature=0.1,
+        priority=1,
+    ),
+    ModelType.SECONDARY: ModelConfig(
+        name=getattr(settings, "GROQ_MODEL_SECONDARY", "mixtral-8x7b-32768"),
+        max_tokens=4096,
+        temperature=0.1,
+        priority=2,
+    ),
+    ModelType.FALLBACK: ModelConfig(
+        name=getattr(settings, "GROQ_MODEL_FALLBACK", "llama-3.1-70b-versatile"),
+        max_tokens=4096,
+        temperature=0.1,
+        priority=3,
+    ),
+}
+
+
+class RequestQueue:
+    """Simple async queue for serializing LLM requests"""
+
+    def __init__(self, min_delay: float = 1.0):
+        self.min_delay = min_delay
+        self.last_request_time: float = 0
+        self.lock = asyncio.Lock()
+
+    async def acquire(self):
+        """Wait until enough time has passed since last request"""
+        async with self.lock:
+            now = time.time()
+            elapsed = now - self.last_request_time
+            if elapsed < self.min_delay:
+                await asyncio.sleep(self.min_delay - elapsed)
+            self.last_request_time = time.time()
+
+
+class ResponseCache:
+    """Simple TTL cache for LLM responses"""
+
+    def __init__(self, ttl_seconds: int = 300):
+        self.ttl = ttl_seconds
+        self._cache: Dict[str, tuple] = {}
+        self.lock = asyncio.Lock()
+
+    def _make_key(self, prompt: str, model: str) -> str:
+        """Create cache key from prompt and model"""
+        content = f"{model}:{prompt}"
+        return hashlib.md5(content.encode()).hexdigest()
+
+    async def get(self, prompt: str, model: str) -> Optional[str]:
+        """Get cached response"""
+        key = self._make_key(prompt, model)
+        async with self.lock:
+            if key in self._cache:
+                response, timestamp = self._cache[key]
+                if (datetime.now() - timestamp).total_seconds() < self.ttl:
+                    return response
+                del self._cache[key]
+        return None
+
+    async def set(self, prompt: str, model: str, response: str):
+        """Cache a response"""
+        key = self._make_key(prompt, model)
+        async with self.lock:
+            self._cache[key] = (response, datetime.now())
+
+            # Clean old entries if cache is too large
+            if len(self._cache) > 100:
+                oldest = sorted(self._cache.items(), key=lambda x: x[1][1])[:10]
+                for k, _ in oldest:
+                    del self._cache[k]
+
+    def clear(self):
+        """Clear all cached responses"""
+        self._cache.clear()
+
+
+class EnhancedLLMService:
+    """Enhanced LLM service with multi-model support and rate limiting"""
+
+    def __init__(self):
+        self.api_key = settings.GROQ_API_KEY
+        self.base_url = "https://api.groq.com/openai/v1/chat/completions"
+        self.request_queue = RequestQueue(min_delay=1.0)
+        self.cache = ResponseCache(ttl_seconds=300)
+        self.model_order = [
+            ModelType.PRIMARY,
+            ModelType.SECONDARY,
+            ModelType.FALLBACK,
+        ]
+        self.max_retries = int(getattr(settings, "LLM_MAX_RETRIES", 5))
+        self.retry_delay = int(getattr(settings, "LLM_RETRY_DELAY", 2))
+
+    def _build_context(self, sources: List[Source]) -> str:
+        """Build context string from sources"""
+        if not sources:
+            return "No context available."
+
+        context_parts = []
+        for i, source in enumerate(sources, 1):
+            context_parts.append(
+                f"[Source {i}] {source.title}\n"
+                f"Reference: {source.reference}\n"
+                f"Content: {source.content}\n"
+            )
+
+        return "\n\n".join(context_parts)
+
+    def _get_system_prompt(self) -> str:
+        """Get system prompt"""
+        return """You are a precise assistant that answers questions using only the provided context.
+
+Rules:
+1. Base your answer ONLY on the provided context
+2. When citing sources, use the actual page number or reference provided (e.g., "According to Page 21..." or "As stated on Page 34...")
+3. Do NOT use generic citations like [Source 1] or [Source 2]
+4. If the context doesn't contain enough information, say "I don't have enough information to answer this question"
+5. Be concise and accurate
+6. Do not make assumptions or use external knowledge
+7. Write in a clear, professional tone"""
+
+    def _build_prompt(self, query: str, pdf_context: str, web_context: str) -> str:
+        """Build final prompt"""
+        return f"""Context from Documents:
+{pdf_context}
+
+Context from Web:
+{web_context}
+
+Question: {query}
+
+Provide a comprehensive answer based on the context above. Include source citations."""
+
+    async def _call_groq(self, model: ModelType, prompt: str) -> Optional[str]:
+        """Make API call to Groq with specific model"""
+        config = MODEL_CONFIGS[model]
+
+        payload = {
+            "model": config.name,
+            "messages": [
+                {"role": "system", "content": self._get_system_prompt()},
+                {"role": "user", "content": prompt},
+            ],
+            "temperature": config.temperature,
+            "max_tokens": config.max_tokens,
+        }
+
+        headers = {
+            "Authorization": f"Bearer {self.api_key}",
+            "Content-Type": "application/json",
+        }
+
+        for attempt in range(self.max_retries):
+            try:
+                async with aiohttp.ClientSession() as session:
+                    timeout = aiohttp.ClientTimeout(total=60)
+                    async with session.post(
+                        self.base_url, headers=headers, json=payload, timeout=timeout
+                    ) as response:
+                        if response.status == 429:
+                            # Rate limited - wait and retry
+                            delay = self.retry_delay * (2**attempt)
+                            await asyncio.sleep(delay)
+                            continue
+
+                        if response.status != 200:
+                            return None
+
+                        data = await response.json()
+                        return data["choices"][0]["message"]["content"]
+
+            except Exception as e:
+                if attempt < self.max_retries - 1:
+                    delay = self.retry_delay * (2**attempt)
+                    await asyncio.sleep(delay)
+                    continue
+
+        return None
+
+    async def generate_answer(self, query: str, sources: List[Source]) -> str:
+        """Generate answer with multi-model fallback"""
+
+        # Build prompt
+        pdf_context = self._build_context([s for s in sources if s.type.value == "pdf"])
+        web_context = self._build_context([s for s in sources if s.type.value == "web"])
+        prompt = self._build_prompt(query, pdf_context, web_context)
+
+        # Acquire queue lock (prevent burst requests)
+        await self.request_queue.acquire()
+
+        # Try each model in order
+        for model_type in self.model_order:
+            # Check cache first
+            cached = await self.cache.get(prompt, MODEL_CONFIGS[model_type].name)
+            if cached:
+                return cached
+
+            # Try this model
+            response = await self._call_groq(model_type, prompt)
+
+            if response:
+                # Cache successful response
+                await self.cache.set(prompt, MODEL_CONFIGS[model_type].name, response)
+                return response
+
+        # All models failed
+        raise Exception("All LLM models failed after retries")
+
+    def get_model_info(self) -> Dict:
+        """Get information about configured models"""
+        return {
+            "primary": MODEL_CONFIGS[ModelType.PRIMARY].name,
+            "secondary": MODEL_CONFIGS[ModelType.SECONDARY].name,
+            "fallback": MODEL_CONFIGS[ModelType.FALLBACK].name,
+            "max_retries": self.max_retries,
+            "retry_delay": self.retry_delay,
+            "cache_ttl": self.cache.ttl,
+        }
+
+    def clear_cache(self):
+        """Clear the response cache"""
+        self.cache.clear()
+
+
+# Create singleton instance
+enhanced_llm_service = EnhancedLLMService()

backend/app/services/llm_service.py

@@ -0,0 +1,104 @@
+import aiohttp
+import asyncio
+import time
+from typing import List
+from app.models.schemas import Source
+from app.config import settings
+
+
+class LLMService:
+    def __init__(self):
+        self.api_key = settings.GROQ_API_KEY
+        self.model = settings.GROQ_MODEL
+        self.base_url = "https://api.groq.com/openai/v1/chat/completions"
+        self.max_retries = 5
+        self.base_delay = 3
+
+    async def generate_answer(self, query: str, sources: List[Source]) -> str:
+        pdf_context = self._build_context([s for s in sources if s.type.value == "pdf"])
+        web_context = self._build_context([s for s in sources if s.type.value == "web"])
+
+        prompt = self._build_prompt(query, pdf_context, web_context)
+
+        headers = {
+            "Authorization": f"Bearer {self.api_key}",
+            "Content-Type": "application/json",
+        }
+
+        payload = {
+            "model": self.model,
+            "messages": [
+                {"role": "system", "content": self._get_system_prompt()},
+                {"role": "user", "content": prompt},
+            ],
+            "temperature": settings.TEMPERATURE,
+            "max_tokens": settings.MAX_TOKENS,
+        }
+
+        for attempt in range(self.max_retries):
+            try:
+                async with aiohttp.ClientSession() as session:
+                    async with session.post(
+                        self.base_url,
+                        headers=headers,
+                        json=payload,
+                        timeout=aiohttp.ClientTimeout(total=60),
+                    ) as response:
+                        if response.status == 429:
+                            delay = self.base_delay * (2**attempt)
+                            await asyncio.sleep(delay)
+                            continue
+
+                        if response.status != 200:
+                            raise Exception(f"LLM API failed: {response.status}")
+
+                        data = await response.json()
+                        answer = data["choices"][0]["message"]["content"]
+                        return answer
+
+            except Exception as e:
+                if attempt < self.max_retries - 1:
+                    delay = self.base_delay * (2**attempt)
+                    await asyncio.sleep(delay)
+                    continue
+                raise
+
+        raise Exception("LLM generation failed after retries")
+
+    def _build_context(self, sources: List[Source]) -> str:
+        if not sources:
+            return "No context available."
+
+        context_parts = []
+        for i, source in enumerate(sources, 1):
+            context_parts.append(
+                f"[Source {i}] {source.title}\n"
+                f"Reference: {source.reference}\n"
+                f"Content: {source.content}\n"
+            )
+
+        return "\n\n".join(context_parts)
+
+    def _get_system_prompt(self) -> str:
+        return """You are a precise assistant that answers questions using only the provided context.
+
+Rules:
+1. Base your answer ONLY on the provided context
+2. Cite sources using [Source N] notation
+3. If the context doesn't contain enough information, say "I don't have enough information to answer this question"
+4. Be concise and accurate
+5. Do not make assumptions or use external knowledge"""
+
+    def _build_prompt(self, query: str, pdf_context: str, web_context: str) -> str:
+        return f"""Context from Documents:
+{pdf_context}
+
+Context from Web:
+{web_context}
+
+Question: {query}
+
+Provide a comprehensive answer based on the context above. Include source citations."""
+
+
+llm_service = LLMService()

backend/app/services/pdf_processor.py

@@ -0,0 +1,82 @@
+import pypdf
+import pdfplumber
+from pathlib import Path
+from typing import List, Tuple
+from app.utils.chunking import intelligent_chunk, create_chunk_metadata
+
+
+class PDFProcessor:
+    def __init__(self):
+        self.max_file_size = 10 * 1024 * 1024
+
+    async def extract_text(self, file_path: Path) -> List[Tuple[str, int]]:
+        pages_text = []
+
+        try:
+            with pdfplumber.open(file_path) as pdf:
+                for i, page in enumerate(pdf.pages, 1):
+                    text = page.extract_text() or ""
+                    if text.strip():
+                        pages_text.append((text, i))
+        except Exception as e:
+            try:
+                with open(file_path, "rb") as file:
+                    reader = pypdf.PdfReader(file)
+                    for i, page in enumerate(reader.pages, 1):
+                        text = page.extract_text() or ""
+                        if text.strip():
+                            pages_text.append((text, i))
+            except Exception as fallback_error:
+                raise Exception(f"Failed to extract text: {fallback_error}")
+
+        return pages_text
+
+    async def process_document(self, file_path: Path, document_id: str) -> List[dict]:
+        pages_text = await self.extract_text(file_path)
+
+        full_text = "\n\n".join([text for text, _ in pages_text])
+
+        chunks = intelligent_chunk(text=full_text, chunk_size=512, overlap=50)
+
+        processed_chunks = []
+        for idx, chunk in enumerate(chunks):
+            page_num = self._find_page_number(chunk, pages_text)
+
+            chunk_data = {
+                "text": chunk,
+                "metadata": create_chunk_metadata(
+                    document_id=document_id,
+                    chunk_index=idx,
+                    page_number=page_num,
+                    total_chunks=len(chunks),
+                ),
+            }
+            processed_chunks.append(chunk_data)
+
+        return processed_chunks
+
+    def _find_page_number(self, chunk: str, pages_text: List[Tuple[str, int]]) -> int:
+        chunk_start = chunk[:50]
+
+        for text, page_num in pages_text:
+            if chunk_start in text:
+                return page_num
+
+        return 0
+
+    def validate_file(self, file_path: Path) -> bool:
+        if not file_path.exists():
+            raise Exception("File does not exist")
+
+        if file_path.stat().st_size > self.max_file_size:
+            raise Exception("File size exceeds limit")
+
+        try:
+            with open(file_path, "rb") as f:
+                pypdf.PdfReader(f)
+            return True
+        except Exception:
+            raise Exception("Invalid PDF file")
+
+
+pdf_processor = PDFProcessor()

backend/app/services/prompt_guard.py

@@ -0,0 +1,51 @@
+from typing import List, Optional
+
+
+class PromptGuardService:
+    def __init__(self):
+        self.jailbreak_patterns = [
+            "ignore previous instructions",
+            "disregard all prior",
+            "forget everything above",
+            "you are now",
+            "new role",
+            "system:",
+            "admin mode",
+        ]
+
+        self.restricted_topics = []
+
+    async def validate_input(
+        self, query: str, restrictions: Optional[List[str]] = None
+    ) -> bool:
+        query_lower = query.lower()
+
+        for pattern in self.jailbreak_patterns:
+            if pattern in query_lower:
+                raise Exception(f"Detected potential jailbreak attempt: '{pattern}'")
+
+        if restrictions:
+            for restriction in restrictions:
+                if restriction.lower() in query_lower:
+                    raise Exception(f"Query violates restriction: '{restriction}'")
+
+        if len(query) > 1000:
+            raise Exception("Query exceeds maximum length")
+
+        return True
+
+    async def sanitize_output(self, answer: str) -> str:
+        sanitized = answer.strip()
+
+        prefixes_to_remove = ["System:", "Admin:", "Debug:"]
+        for prefix in prefixes_to_remove:
+            if sanitized.startswith(prefix):
+                sanitized = sanitized[len(prefix) :].strip()
+
+        return sanitized
+
+    def set_restrictions(self, topics: List[str]):
+        self.restricted_topics = topics
+
+
+prompt_guard = PromptGuardService()

backend/app/services/retriever.py

@@ -0,0 +1,83 @@
+from typing import List, Dict, Optional
+from app.services.embeddings import embedding_service
+from app.services.vector_store import vector_store
+from app.services.web_search import web_search_service
+from app.models.schemas import Source, SourceType, QueryMode
+
+
+class RetrieverService:
+    async def retrieve(
+        self,
+        query: str,
+        mode: QueryMode,
+        top_k: int = 5,
+        document_ids: Optional[List[str]] = None,
+    ) -> List[Source]:
+        if mode == QueryMode.PDF:
+            return await self._retrieve_from_pdf(query, top_k, document_ids or [])
+
+        elif mode == QueryMode.WEB:
+            return await self._retrieve_from_web(query, top_k)
+
+        elif mode == QueryMode.HYBRID:
+            # Get PDF sources
+            pdf_sources = await self._retrieve_from_pdf(
+                query, top_k // 2, document_ids or []
+            )
+            # Get web sources
+            web_sources = await self._retrieve_from_web(query, top_k // 2)
+            # Combine and rerank
+            return self._merge_and_rerank(pdf_sources + web_sources, top_k)
+
+        elif mode == QueryMode.RESTRICTED:
+            return await self._retrieve_from_pdf(query, top_k, document_ids or [])
+
+    async def _retrieve_from_pdf(
+        self, query: str, top_k: int, document_ids: Optional[List[str]] = None
+    ) -> List[Source]:
+        query_embedding = embedding_service.embed_text(query)
+
+        results = await vector_store.search(
+            query_embedding=query_embedding, top_k=top_k, document_ids=document_ids
+        )
+
+        sources = []
+        for result in results:
+            sources.append(
+                Source(
+                    type=SourceType.PDF,
+                    content=result["text"],
+                    reference=f"Page {result['metadata']['page_number']}",
+                    title=f"Document {result['metadata']['document_id']}",
+                    relevance_score=result["similarity"],
+                )
+            )
+
+        return sources
+
+    async def _retrieve_from_web(self, query: str, top_k: int) -> List[Source]:
+        results = await web_search_service.search(query, max_results=top_k)
+
+        sources = []
+        for result in results:
+            sources.append(
+                Source(
+                    type=SourceType.WEB,
+                    content=result["snippet"],
+                    reference=result["url"],
+                    title=result["title"],
+                    relevance_score=result.get("score", 0.8),
+                )
+            )
+
+        return sources
+
+    def _merge_and_rerank(self, sources: List[Source], top_k: int) -> List[Source]:
+        sorted_sources = sorted(
+            sources, key=lambda x: x.relevance_score or 0, reverse=True
+        )
+
+        return sorted_sources[:top_k]
+
+
+retriever_service = RetrieverService()

backend/app/services/vector_store.py

@@ -0,0 +1,75 @@
+import chromadb
+from chromadb.config import Settings as ChromaSettings
+from typing import List, Dict, Optional
+from pathlib import Path
+from app.config import settings
+
+
+class VectorStore:
+    def __init__(self):
+        self.client = chromadb.PersistentClient(
+            path=str(settings.VECTOR_DB_PATH),
+            settings=ChromaSettings(anonymized_telemetry=False),
+        )
+        self.collection = self.client.get_or_create_collection(
+            name="documents", metadata={"hnsw:space": "cosine"}
+        )
+
+    async def add_chunks(self, chunks: List[dict], embeddings: List[List[float]]):
+        ids = [
+            f"{chunk['metadata']['document_id']}_chunk_{chunk['metadata']['chunk_index']}"
+            for chunk in chunks
+        ]
+
+        documents = [chunk["text"] for chunk in chunks]
+
+        metadatas = [
+            {
+                "document_id": chunk["metadata"]["document_id"],
+                "chunk_index": chunk["metadata"]["chunk_index"],
+                "page_number": chunk["metadata"]["page_number"] or 0,
+                "total_chunks": chunk["metadata"]["total_chunks"],
+            }
+            for chunk in chunks
+        ]
+
+        self.collection.add(
+            ids=ids, embeddings=embeddings, documents=documents, metadatas=metadatas
+        )
+
+    async def search(
+        self,
+        query_embedding: List[float],
+        top_k: int = 5,
+        document_ids: Optional[List[str]] = None,
+    ) -> List[Dict]:
+        where_filter = None
+        if document_ids:
+            where_filter = {"document_id": {"$in": document_ids}}
+
+        results = self.collection.query(
+            query_embeddings=[query_embedding], n_results=top_k, where=where_filter
+        )
+
+        search_results = []
+        for i in range(len(results["ids"][0])):
+            search_results.append(
+                {
+                    "text": results["documents"][0][i],
+                    "metadata": results["metadatas"][0][i],
+                    "similarity": 1 - results["distances"][0][i],
+                    "id": results["ids"][0][i],
+                }
+            )
+
+        return search_results
+
+    async def delete_document(self, document_id: str):
+        self.collection.delete(where={"document_id": document_id})
+
+    async def get_stats(self) -> Dict:
+        count = self.collection.count()
+        return {"total_chunks": count, "collection_name": self.collection.name}
+
+
+vector_store = VectorStore()

backend/app/services/web_search.py

@@ -0,0 +1,416 @@
+"""
+Unified Web Search Service
+Supports multiple search providers:
+- Tavily (AI-optimized, RAG-ready)
+- Serper (Google search)
+- Brave Search (Privacy-focused)
+- You.com (AI-ready)
+"""
+
+import aiohttp
+import asyncio
+import time
+from typing import List, Dict, Optional, Literal
+from abc import ABC, abstractmethod
+from dataclasses import dataclass
+from enum import Enum
+import json
+
+from app.config import settings
+
+
+class SearchProvider(Enum):
+    TAVILY = "tavily"
+    SERPER = "serper"
+    BRAVE = "brave"
+    YOUCOM = "youcom"
+
+
+@dataclass
+class SearchResult:
+    title: str
+    url: str
+    snippet: str
+    score: float = 0.8
+    provider: str = "unknown"
+
+
+@dataclass
+class SearchConfig:
+    provider: SearchProvider
+    api_key: str
+    max_results: int = 5
+    timeout: int = 15
+    retry_attempts: int = 3
+    retry_delay: float = 2.0
+    cache_ttl: int = 300  # 5 minutes
+
+
+class BaseSearchProvider(ABC):
+    """Abstract base class for search providers"""
+
+    def __init__(self, config: SearchConfig):
+        self.config = config
+        self._cache: Dict[str, tuple] = {}
+
+    @abstractmethod
+    async def search(self, query: str) -> List[SearchResult]:
+        pass
+
+    @abstractmethod
+    def _format_results(self, raw_data) -> List[SearchResult]:
+        pass
+
+    def _get_cache(self, query: str) -> Optional[List[SearchResult]]:
+        if query in self._cache:
+            data, timestamp = self._cache[query]
+            if time.time() - timestamp < self.config.cache_ttl:
+                return data
+            del self._cache[query]
+        return None
+
+    def _set_cache(self, query: str, results: List[SearchResult]):
+        self._cache[query] = (results, time.time())
+
+        # Clean old cache entries
+        if len(self._cache) > 100:
+            oldest = sorted(self._cache.keys(), key=lambda k: self._cache[k][1])[:10]
+            for k in oldest:
+                del self._cache[k]
+
+    async def _make_request(
+        self, url: str, params: Dict = None, headers: Dict = None, method: str = "GET"
+    ) -> Dict:
+        """Make HTTP request with retry logic"""
+
+        for attempt in range(self.config.retry_attempts):
+            try:
+                timeout = aiohttp.ClientTimeout(total=self.config.timeout)
+
+                async with aiohttp.ClientSession(timeout=timeout) as session:
+                    if method == "GET":
+                        async with session.get(
+                            url, params=params, headers=headers
+                        ) as response:
+                            if response.status == 429:
+                                await asyncio.sleep(
+                                    self.config.retry_delay * (attempt + 1)
+                                )
+                                continue
+                            if response.status != 200:
+                                return {}
+                            return await response.json()
+                    else:
+                        async with session.post(
+                            url, json=params, headers=headers
+                        ) as response:
+                            if response.status == 429:
+                                await asyncio.sleep(
+                                    self.config.retry_delay * (attempt + 1)
+                                )
+                                continue
+                            if response.status != 200:
+                                return {}
+                            return await response.json()
+
+            except Exception as e:
+                if attempt < self.config.retry_attempts - 1:
+                    await asyncio.sleep(self.config.retry_delay)
+                continue
+
+        return {}
+
+
+class TavilySearchProvider(BaseSearchProvider):
+    """Tavily AI Search - Optimized for RAG and AI applications"""
+
+    def __init__(self, api_key: str, max_results: int = 5):
+        config = SearchConfig(
+            provider=SearchProvider.TAVILY, api_key=api_key, max_results=max_results
+        )
+        super().__init__(config)
+        self.base_url = "https://api.tavily.com/search"
+
+    async def search(self, query: str) -> List[SearchResult]:
+        # Check cache first
+        cached = self._get_cache(query)
+        if cached:
+            return cached
+
+        payload = {
+            "api_key": self.config.api_key,
+            "query": query,
+            "search_depth": "advanced",
+            "max_results": self.config.max_results,
+            "include_answer": True,
+            "include_raw_content": False,
+            "include_images": False,
+        }
+
+        data = await self._make_request(self.base_url, params=payload, method="POST")
+
+        results = self._format_results(data)
+        self._set_cache(query, results)
+        return results
+
+    def _format_results(self, data: Dict) -> List[SearchResult]:
+        results = []
+
+        search_results = data.get("results", [])
+
+        for i, result in enumerate(search_results[: self.config.max_results]):
+            results.append(
+                SearchResult(
+                    title=result.get("title", ""),
+                    url=result.get("url", ""),
+                    snippet=result.get("content", ""),
+                    score=result.get("score", 0.9 - (i * 0.05)),
+                    provider="tavily",
+                )
+            )
+
+        return results
+
+
+class SerperSearchProvider(BaseSearchProvider):
+    """Serper.dev - Google Search API"""
+
+    def __init__(self, api_key: str, max_results: int = 5):
+        config = SearchConfig(
+            provider=SearchProvider.SERPER, api_key=api_key, max_results=max_results
+        )
+        super().__init__(config)
+        self.base_url = "https://serpapi.com/search"
+
+    async def search(self, query: str) -> List[SearchResult]:
+        cached = self._get_cache(query)
+        if cached:
+            return cached
+
+        params = {
+            "engine": "google",
+            "q": query,
+            "api_key": self.config.api_key,
+            "num": self.config.max_results,
+        }
+
+        data = await self._make_request(self.base_url, params=params)
+
+        results = self._format_results(data)
+        self._set_cache(query, results)
+        return results
+
+    def _format_results(self, data: Dict) -> List[SearchResult]:
+        results = []
+
+        organic_results = data.get("organic_results", [])
+
+        for i, result in enumerate(organic_results[: self.config.max_results]):
+            results.append(
+                SearchResult(
+                    title=result.get("title", ""),
+                    url=result.get("link", ""),
+                    snippet=result.get("snippet", ""),
+                    score=0.8 - (i * 0.05),
+                    provider="serper",
+                )
+            )
+
+        return results
+
+
+class BraveSearchProvider(BaseSearchProvider):
+    """Brave Search API - Privacy-focused"""
+
+    def __init__(self, api_key: str, max_results: int = 5):
+        config = SearchConfig(
+            provider=SearchProvider.BRAVE, api_key=api_key, max_results=max_results
+        )
+        super().__init__(config)
+        self.base_url = "https://api.search.brave.com/res/v1/web/search"
+
+    async def search(self, query: str) -> List[SearchResult]:
+        cached = self._get_cache(query)
+        if cached:
+            return cached
+
+        headers = {
+            "Accept": "application/json",
+            "X-Subscription-Token": self.config.api_key,
+        }
+
+        params = {"q": query, "count": self.config.max_results}
+
+        data = await self._make_request(self.base_url, params=params, headers=headers)
+
+        results = self._format_results(data)
+        self._set_cache(query, results)
+        return results
+
+    def _format_results(self, data: Dict) -> List[SearchResult]:
+        results = []
+
+        web_results = data.get("web", {}).get("results", [])
+
+        for i, result in enumerate(web_results[: self.config.max_results]):
+            results.append(
+                SearchResult(
+                    title=result.get("title", ""),
+                    url=result.get("url", ""),
+                    snippet=result.get("description", ""),
+                    score=0.85 - (i * 0.05),
+                    provider="brave",
+                )
+            )
+
+        return results
+
+
+class YouComSearchProvider(BaseSearchProvider):
+    """You.com - AI-Optimized Search"""
+
+    def __init__(self, api_key: str, max_results: int = 5):
+        config = SearchConfig(
+            provider=SearchProvider.YOUCOM, api_key=api_key, max_results=max_results
+        )
+        super().__init__(config)
+        self.base_url = "https://api.you.com/search"
+
+    async def search(self, query: str) -> List[SearchResult]:
+        cached = self._get_cache(query)
+        if cached:
+            return cached
+
+        headers = {"Authorization": f"Bearer {self.config.api_key}"}
+
+        params = {"query": query, "num": self.config.max_results}
+
+        data = await self._make_request(self.base_url, params=params, headers=headers)
+
+        results = self._format_results(data)
+        self._set_cache(query, results)
+        return results
+
+    def _format_results(self, data: Dict) -> List[SearchResult]:
+        results = []
+
+        search_results = data.get("results", [])
+
+        for i, result in enumerate(search_results[: self.config.max_results]):
+            results.append(
+                SearchResult(
+                    title=result.get("title", ""),
+                    url=result.get("url", ""),
+                    snippet=result.get("snippet", ""),
+                    score=result.get("score", 0.85 - (i * 0.05)),
+                    provider="youcom",
+                )
+            )
+
+        return results
+
+
+class WebSearchService:
+    """
+    Unified web search service with provider selection
+    """
+
+    def __init__(self):
+        # Default to Tavily (RAG-optimized)
+        self.default_provider = SearchProvider.TAVILY
+        self._providers: Dict[SearchProvider, BaseSearchProvider] = {}
+        self._initialize_providers()
+
+    def _initialize_providers(self):
+        """Initialize available search providers"""
+
+        # Tavily (Recommended for RAG)
+        tavily_key = getattr(settings, "TAVILY_API_KEY", None) or getattr(
+            settings, "SERPER_API_KEY", None
+        )
+        if tavily_key:
+            self._providers[SearchProvider.TAVILY] = TavilySearchProvider(
+                tavily_key, max_results=5
+            )
+
+        # Serper
+        serper_key = settings.SERPER_API_KEY
+        if serper_key:
+            self._providers[SearchProvider.SERPER] = SerperSearchProvider(
+                serper_key, max_results=5
+            )
+
+    def set_provider(self, provider: SearchProvider):
+        """Change the active search provider"""
+        if provider in self._providers:
+            self.default_provider = provider
+
+    async def search(
+        self,
+        query: str,
+        max_results: int = 5,
+        provider: Optional[SearchProvider] = None,
+    ) -> List[Dict]:
+        """
+        Search using specified or default provider
+
+        Args:
+            query: Search query
+            max_results: Maximum number of results
+            provider: Specific provider to use (optional)
+
+        Returns:
+            List of search results
+        """
+        search_provider = provider or self.default_provider
+
+        if search_provider not in self._providers:
+            # Fallback to any available provider
+            if self._providers:
+                search_provider = list(self._providers.keys())[0]
+            else:
+                return []
+
+        provider = self._providers[search_provider]
+        results = await provider.search(query)
+
+        # Convert to dict format
+        return [
+            {
+                "title": r.title,
+                "url": r.url,
+                "snippet": r.snippet,
+                "score": r.score,
+            }
+            for r in results
+        ]
+
+    def get_available_providers(self) -> List[str]:
+        """Get list of available providers"""
+        return [p.value for p in self._providers.keys()]
+
+    def get_current_provider(self) -> str:
+        """Get current provider"""
+        return self.default_provider.value
+
+
+# Factory function to create service
+def create_web_search_service(provider: str = "tavily") -> WebSearchService:
+    """Create web search service with specified provider"""
+    service = WebSearchService()
+
+    provider_map = {
+        "tavily": SearchProvider.TAVILY,
+        "serper": SearchProvider.SERPER,
+        "brave": SearchProvider.BRAVE,
+        "youcom": SearchProvider.YOUCOM,
+    }
+
+    if provider.lower() in provider_map:
+        service.set_provider(provider_map[provider.lower()])
+
+    return service
+
+
+# Default instance
+web_search_service = WebSearchService()

backend/app/utils/__init__.py

@@ -0,0 +1,10 @@
+from .chunking import intelligent_chunk, create_chunk_metadata
+from .rate_limiter import RateLimiter, RequestCache, RateLimitedWebSearch
+
+__all__ = [
+    "intelligent_chunk",
+    "create_chunk_metadata",
+    "RateLimiter",
+    "RequestCache",
+    "RateLimitedWebSearch",
+]

backend/app/utils/chunking.py

@@ -0,0 +1,48 @@
+import re
+from typing import List, Optional, Dict, Any
+
+
+def intelligent_chunk(text: str, chunk_size: int = 512, overlap: int = 50) -> List[str]:
+    sentences = re.split(r"(?<=[.!?])\s+", text)
+
+    chunks = []
+    current_chunk = []
+    current_length = 0
+
+    for sentence in sentences:
+        sentence_length = len(sentence.split())
+
+        if current_length + sentence_length > chunk_size and current_chunk:
+            chunks.append(" ".join(current_chunk))
+
+            overlap_sentences = (
+                current_chunk[-overlap:]
+                if len(current_chunk) > overlap
+                else current_chunk
+            )
+            current_chunk = overlap_sentences + [sentence]
+            current_length = sum(len(s.split()) for s in current_chunk)
+        else:
+            current_chunk.append(sentence)
+            current_length += sentence_length
+
+    if current_chunk:
+        chunks.append(" ".join(current_chunk))
+
+    return chunks
+
+
+def create_chunk_metadata(
+    document_id: str,
+    chunk_index: int,
+    page_number: Optional[int] = None,
+    section: Optional[str] = None,
+    total_chunks: int = 0,
+) -> Dict[str, Any]:
+    return {
+        "document_id": document_id,
+        "chunk_index": chunk_index,
+        "page_number": page_number,
+        "section": section,
+        "total_chunks": total_chunks,
+    }

backend/app/utils/rate_limiter.py

@@ -0,0 +1,199 @@
+"""
+Rate Limiting Handler for Web Search API
+
+This module provides:
+1. Exponential backoff retry logic
+2. Request caching
+3. Rate limit detection and handling
+4. Request queuing
+"""
+
+import asyncio
+import time
+from typing import List, Dict, Optional, Callable
+from dataclasses import dataclass
+from datetime import datetime, timedelta
+import threading
+
+
+@dataclass
+class RateLimitConfig:
+    max_requests_per_minute: int = 30
+    max_requests_per_hour: int = 500
+    retry_base_delay: float = 2.0
+    max_retry_attempts: int = 3
+    cache_ttl_seconds: int = 300
+
+
+class RateLimiter:
+    """Token bucket rate limiter"""
+
+    def __init__(self, config: RateLimitConfig = None):
+        self.config = config or RateLimitConfig()
+        self.tokens = self.config.max_requests_per_minute
+        self.last_update = datetime.now()
+        self.lock = threading.Lock()
+
+    def acquire(self) -> bool:
+        with self.lock:
+            now = datetime.now()
+            elapsed = (now - self.last_update).total_seconds()
+
+            # Refill tokens
+            tokens_to_add = elapsed * (self.config.max_requests_per_minute / 60)
+            self.tokens = min(
+                self.config.max_requests_per_minute, self.tokens + tokens_to_add
+            )
+
+            if self.tokens >= 1:
+                self.tokens -= 1
+                self.last_update = now
+                return True
+
+            return False
+
+    def wait_for_token(self, timeout: float = 60) -> bool:
+        start = time.time()
+        while time.time() - start < timeout:
+            if self.acquire():
+                return True
+            time.sleep(0.1)
+        return False
+
+
+class RequestCache:
+    """Simple TTL-based cache"""
+
+    def __init__(self, ttl_seconds: int = 300):
+        self.ttl = ttl_seconds
+        self._cache: Dict[str, tuple] = {}
+        self.lock = threading.Lock()
+
+    def get(self, key: str) -> Optional[List[Dict]]:
+        with self.lock:
+            if key in self._cache:
+                data, timestamp = self._cache[key]
+                if (datetime.now() - timestamp).total_seconds() < self.ttl:
+                    return data
+                del self._cache[key]
+        return None
+
+    def set(self, key: str, data: List[Dict]):
+        with self.lock:
+            self._cache[key] = (data, datetime.now())
+
+            # Clean old entries if cache is too large
+            if len(self._cache) > 100:
+                oldest = sorted(self._cache.items(), key=lambda x: x[1][1])[:10]
+                for k, _ in oldest:
+                    del self._cache[k]
+
+    def clear(self):
+        with self.lock:
+            self._cache.clear()
+
+
+class RateLimitedWebSearch:
+    """Web search with rate limiting and caching"""
+
+    def __init__(self, search_func: Callable, config: RateLimitConfig = None):
+        self.config = config or RateLimitConfig()
+        self.rate_limiter = RateLimiter(self.config)
+        self.cache = RequestCache(self.config.cache_ttl_seconds)
+        self.search_func = search_func
+
+    async def search(
+        self, query: str, max_results: int = 5, use_cache: bool = True
+    ) -> List[Dict]:
+        cache_key = f"{query}_{max_results}"
+
+        # Check cache
+        if use_cache:
+            cached = self.cache.get(cache_key)
+            if cached:
+                return cached
+
+        # Wait for rate limit
+        if not self.rate_limiter.wait_for_token():
+            return []
+
+        # Retry with exponential backoff
+        for attempt in range(self.config.max_retry_attempts):
+            try:
+                results = await self.search_func(query, max_results)
+
+                if results:
+                    if use_cache:
+                        self.cache.set(cache_key, results)
+                    return results
+
+                # If empty results, might be rate limited
+                await asyncio.sleep(self.config.retry_base_delay * (attempt + 1))
+
+            except Exception as e:
+                if attempt < self.config.max_retry_attempts - 1:
+                    await asyncio.sleep(self.config.retry_base_delay * (2**attempt))
+                continue
+
+        return []
+
+    def get_cache_stats(self) -> Dict:
+        return {
+            "cached_items": len(self.cache._cache),
+            "ttl_seconds": self.config.cache_ttl_seconds,
+            "rate_limit_per_minute": self.config.max_requests_per_minute,
+        }
+
+    def clear_cache(self):
+        self.cache.clear()
+
+
+# Example usage with Serper API
+async def serper_search(query: str, max_results: int = 5) -> List[Dict]:
+    """Example Serper API search function"""
+    import aiohttp
+
+    api_key = "92dc65b1fe92ca96ece7d0b02729f2d29f68f4fda5e31908e8d447a808e9797f"
+    url = "https://serpapi.com/search"
+
+    params = {
+        "engine": "google",
+        "q": query,
+        "api_key": api_key,
+        "num": max_results,
+    }
+
+    async with aiohttp.ClientSession() as session:
+        async with session.get(url, params=params) as response:
+            if response.status == 200:
+                data = await response.json()
+                results = data.get("organic_results", [])
+                return [
+                    {
+                        "title": r.get("title", ""),
+                        "url": r.get("link", ""),
+                        "snippet": r.get("snippet", ""),
+                        "score": 0.8,
+                    }
+                    for r in results[:max_results]
+                ]
+            return []
+
+
+# Create rate-limited instance
+rate_limited_search = RateLimitedWebSearch(serper_search)
+
+
+if __name__ == "__main__":
+
+    async def test():
+        # Test the rate-limited search
+        results = await rate_limited_search.search("Python programming", 3)
+        print(f"Found {len(results)} results")
+        print(f"Cache stats: {rate_limited_search.get_cache_stats()}")
+
+        # Test cache hit
+        results2 = await rate_limited_search.search("Python programming", 3)
+        print(f"Cache hit: {len(results2)} results")
+
+    asyncio.run(test())

backend/reproduce_query.py

@@ -0,0 +1,79 @@
+import requests
+import time
+import sys
+import os
+
+# Configuration
+BASE_URL = "http://localhost:8000/api/v1"
+UPLOAD_URL = f"{BASE_URL}/upload/"
+QUERY_URL = f"{BASE_URL}/query/"
+PDF_FILE = "test.pdf"
+
+def create_dummy_pdf():
+    from reportlab.pdfgen import canvas
+    c = canvas.Canvas(PDF_FILE)
+    c.drawString(100, 750, "This is a test PDF document for RAG system debugging.")
+    c.drawString(100, 730, "The secret code is: ALPHA-BETA-GAMMA.")
+    c.save()
+    print(f"Created dummy PDF: {PDF_FILE}")
+
+def upload_pdf():
+    print(f"Uploading {PDF_FILE}...")
+    with open(PDF_FILE, "rb") as f:
+        files = {"file": f}
+        try:
+            response = requests.post(UPLOAD_URL, files=files, timeout=10)
+            if response.status_code == 200:
+                print("Upload success:", response.json())
+                return response.json()["document_id"]
+            else:
+                print(f"Upload failed: {response.status_code} - {response.text}")
+                return None
+        except Exception as e:
+            print(f"Upload error: {e}")
+            return None
+
+def query_pdf(document_id, query="What is the secret code?"):
+    print(f"Querying: '{query}' for document {document_id}")
+    payload = {
+        "query": query,
+        "mode": "pdf",
+        "document_ids": [document_id] if document_id else [],
+        "top_k": 3
+    }
+    
+    try:
+        response = requests.post(QUERY_URL, json=payload, timeout=30)
+        print(f"Status Code: {response.status_code}")
+        print(f"Response: {response.text}")
+        
+        if response.status_code == 200:
+            data = response.json()
+            print(f"Answer: {data.get('answer')}")
+            print(f"Sources: {len(data.get('sources', []))}")
+            if data.get('answer') and "ALPHA-BETA-GAMMA" in data.get('answer'):
+                print("SUCCESS: Retrieved correct answer.")
+                return True
+            else:
+                print("FAILURE: Answer incorrect or missing.")
+                return False
+        return False
+    except Exception as e:
+        print(f"Query Error: {e}")
+        return False
+
+if __name__ == "__main__":
+    if not os.path.exists(PDF_FILE):
+        create_dummy_pdf()
+        
+    doc_id = upload_pdf()
+    if doc_id:
+        # Wait a bit for indexing if async? (Though implementation seemed synchronous await)
+        time.sleep(2)
+        success = query_pdf(doc_id)
+        if success:
+            sys.exit(0)
+        else:
+            sys.exit(1)
+    else:
+        sys.exit(1)

backend/reproduce_upload.py

@@ -0,0 +1,20 @@
+
+import requests
+import os
+
+# Create a dummy PDF file
+with open("test.pdf", "wb") as f:
+    f.write(b"%PDF-1.4\n1 0 obj\n<<\n/Type /Catalog\n/Pages 2 0 R\n>>\nendobj\n2 0 obj\n<<\n/Type /Pages\n/Kids [3 0 R]\n/Count 1\n>>\nendobj\n3 0 obj\n<<\n/Type /Page\n/Parent 2 0 R\n/MediaBox [0 0 612 792]\n/Resources <<\n/Font <<\n/F1 4 0 R\n>>\n>>\n/Contents 5 0 R\n>>\nendobj\n4 0 obj\n<<\n/Type /Font\n/Subtype /Type1\n/BaseFont /Helvetica\n>>\nendobj\n5 0 obj\n<<\n/Length 44\n>>\nstream\nBT\n/F1 24 Tf\n100 100 Td\n(Hello World) Tj\nET\nendstream\nendobj\nxref\n0 6\n0000000000 65535 f\n0000000010 00000 n\n0000000060 00000 n\n0000000117 00000 n\n0000000216 00000 n\n0000000303 00000 n\ntrailer\n<<\n/Size 6\n/Root 1 0 R\n>>\nstartxref\n397\n%%EOF")
+
+url = "http://localhost:8000/api/v1/upload/"
+files = {'file': ('test.pdf', open('test.pdf', 'rb'), 'application/pdf')}
+
+try:
+    response = requests.post(url, files=files)
+    print(f"Status Code: {response.status_code}")
+    print(f"Response: {response.text}")
+except Exception as e:
+    print(f"Error: {e}")
+finally:
+    if os.path.exists("test.pdf"):
+        os.remove("test.pdf")

backend/requirements.txt

@@ -0,0 +1,20 @@
+fastapi==0.109.0
+uvicorn[standard]==0.27.0
+python-dotenv==1.0.0
+pydantic==2.5.0
+pydantic-settings==2.1.0
+groq==0.4.0
+pypdf==3.17.0
+pdfplumber==0.11.0
+requests==2.31.0
+python-multipart==0.0.6
+aiofiles==23.2.1
+chromadb==0.4.24
+numpy<2.0.0
+sentence-transformers>=3.0.0
+aiohttp>=3.9.0
+
+# OCR Dependencies for Scanned PDFs
+pytesseract==0.3.10
+pdf2image==1.17.0
+Pillow==10.3.0

frontend/.env.example

@@ -0,0 +1 @@
+VITE_API_URL=http://localhost:8000/api/v1

frontend/.env.local

@@ -0,0 +1 @@
+VITE_API_URL=http://localhost:8001/api/v1

frontend/README.md

@@ -0,0 +1,87 @@
+# RAG System Frontend
+
+React + TypeScript frontend for the Production-Grade RAG System.
+
+## Tech Stack
+
+- **Framework**: React 18+ with TypeScript
+- **Styling**: Tailwind CSS
+- **State Management**: React Context API
+- **HTTP Client**: Axios
+- **File Upload**: react-dropzone
+- **Icons**: lucide-react
+- **Notifications**: react-hot-toast
+
+## Project Structure
+
+```
+src/
+├── components/
+│   ├── layout/
+│   │   ├── Header.tsx
+│   │   ├── Sidebar.tsx
+│   │   └── MainContent.tsx
+│   ├── documents/
+│   │   ├── FileUpload.tsx
+│   │   ├── DocumentList.tsx
+│   │   └── DocumentCard.tsx
+│   ├── query/
+│   │   ├── QueryInput.tsx
+│   │   └── ModeSelector.tsx
+│   ├── results/
+│   │   ├── ResultsDisplay.tsx
+│   │   ├── AnswerCard.tsx
+│   │   ├── ConfidenceIndicator.tsx
+│   │   ├── SourcesList.tsx
+│   │   └── SourceCard.tsx
+│   ├── common/
+│   │   └── EmptyState.tsx
+│   └── settings/
+│       └── SettingsModal.tsx
+├── services/
+│   └── api.ts
+├── hooks/
+├── context/
+│   └── AppContext.tsx
+├── types/
+│   └── index.ts
+├── App.tsx
+└── index.tsx
+```
+
+## Installation
+
+```bash
+npm install
+```
+
+## Development
+
+```bash
+npm run dev
+```
+
+## Build
+
+```bash
+npm run build
+```
+
+## Configuration
+
+Copy `.env.example` to `.env` and configure:
+
+```
+VITE_API_URL=http://localhost:8000/api/v1
+```
+
+## Features
+
+- PDF document upload with drag-and-drop
+- Document management (list, select, delete)
+- Multiple query modes (Web, PDF, Hybrid, Restricted)
+- Real-time confidence scoring
+- Source citations and attribution
+- Dark/light theme
+- Responsive design
+- Keyboard shortcuts (Enter to submit)

frontend/index.html

@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>RAG System - Production Ready</title>
+  </head>
+  <body>
+    <div id="root"></div>
+    <script type="module" src="/src/main.tsx"></script>
+  </body>
+</html>

frontend/package.json

@@ -0,0 +1,29 @@
+{
+  "name": "rag-frontend",
+  "version": "1.0.0",
+  "private": true,
+  "dependencies": {
+    "react": "^18.2.0",
+    "react-dom": "^18.2.0",
+    "axios": "^1.6.0",
+    "react-dropzone": "^14.2.0",
+    "lucide-react": "^0.294.0",
+    "react-hot-toast": "^2.4.1",
+    "clsx": "^2.0.0"
+  },
+  "devDependencies": {
+    "@types/react": "^18.2.0",
+    "@types/react-dom": "^18.2.0",
+    "@vitejs/plugin-react": "^4.2.0",
+    "typescript": "^5.3.0",
+    "vite": "^5.0.0",
+    "tailwindcss": "^3.3.0",
+    "postcss": "^8.4.0",
+    "autoprefixer": "^10.4.0"
+  },
+  "scripts": {
+    "dev": "vite",
+    "build": "tsc && vite build",
+    "preview": "vite preview"
+  }
+}

frontend/postcss.config.js

@@ -0,0 +1,6 @@
+module.exports = {
+  plugins: {
+    tailwindcss: {},
+    autoprefixer: {},
+  },
+}

frontend/src/App.tsx

@@ -0,0 +1,20 @@
+import { AppProvider } from './context/AppContext';
+import { Header } from './components/layout/Header';
+import { Sidebar } from './components/layout/Sidebar';
+import { MainContent } from './components/layout/MainContent';
+import { SettingsModal } from './components/settings/SettingsModal';
+
+function App() {
+  return (
+    <AppProvider>
+      <div className="min-h-screen bg-gray-50 dark:bg-gray-900">
+        <Header />
+        <Sidebar />
+        <MainContent />
+        <SettingsModal />
+      </div>
+    </AppProvider>
+  );
+}
+
+export default App;

frontend/src/components/common/EmptyState.tsx

@@ -0,0 +1,77 @@
+import React from 'react';
+import { 
+  Brain, 
+  FileText, 
+  Search, 
+  MessageCircle,
+  ArrowRight 
+} from 'lucide-react';
+
+export const EmptyState: React.FC = () => {
+  const features = [
+    {
+      icon: <FileText className="w-5 h-5" />,
+      title: 'Upload Documents',
+      description: 'Drag and drop PDF files to add them to your knowledge base',
+    },
+    {
+      icon: <Search className="w-5 h-5" />,
+      title: 'Smart Search',
+      description: 'Ask questions and get answers from your documents and the web',
+    },
+    {
+      icon: <Brain className="w-5 h-5" />,
+      title: 'AI Powered',
+      description: 'Powered by Groq LLM for fast, accurate responses',
+    },
+    {
+      icon: <MessageCircle className="w-5 h-5" />,
+      title: 'Source Citations',
+      description: 'Every answer includes sources so you can verify the information',
+    },
+  ];
+
+  return (
+    <div className="text-center py-16">
+      <div className="inline-flex items-center justify-center w-20 h-20 rounded-full bg-primary-100 dark:bg-primary-900/30 mb-6">
+        <Brain className="w-10 h-10 text-primary-600 dark:text-primary-400" />
+      </div>
+
+      <h2 className="text-2xl font-bold text-gray-900 dark:text-white mb-2">
+        Welcome to RAG System
+      </h2>
+      
+      <p className="text-gray-600 dark:text-gray-400 mb-8 max-w-md mx-auto">
+        Upload documents and ask questions to get AI-powered answers with source citations.
+      </p>
+
+      <div className="grid grid-cols-1 md:grid-cols-2 gap-4 max-w-2xl mx-auto">
+        {features.map((feature, index) => (
+          <div 
+            key={index}
+            className="flex items-start gap-3 p-4 bg-white dark:bg-gray-800 rounded-xl border border-gray-200 dark:border-gray-700 text-left"
+          >
+            <div className="flex-shrink-0 w-10 h-10 rounded-lg bg-primary-100 dark:bg-primary-900/30 flex items-center justify-center">
+              <div className="text-primary-600 dark:text-primary-400">
+                {feature.icon}
+              </div>
+            </div>
+            <div>
+              <h3 className="font-medium text-gray-900 dark:text-white">
+                {feature.title}
+              </h3>
+              <p className="text-sm text-gray-500 dark:text-gray-500 mt-1">
+                {feature.description}
+              </p>
+            </div>
+          </div>
+        ))}
+      </div>
+
+      <div className="mt-8 flex items-center justify-center gap-2 text-sm text-gray-500">
+        <span>Start by uploading a document</span>
+        <ArrowRight className="w-4 h-4" />
+      </div>
+    </div>
+  );
+};

frontend/src/components/documents/DocumentCard.tsx

@@ -0,0 +1,97 @@
+import React from 'react';
+import { FileText, Trash2, CheckCircle, AlertCircle } from 'lucide-react';
+import type { Document } from '../../types';
+import { useApp } from '../../context/AppContext';
+
+interface DocumentCardProps {
+  document: Document;
+}
+
+export const DocumentCard: React.FC<DocumentCardProps> = ({ document }) => {
+  const { state, dispatch, handleDeleteDocument } = useApp();
+  const isSelected = state.selectedDocuments.includes(document.id);
+
+  const handleToggle = () => {
+    dispatch({ type: 'TOGGLE_DOCUMENT_SELECTION', payload: document.id });
+  };
+
+  const handleDelete = async (e: React.MouseEvent) => {
+    e.stopPropagation();
+    await handleDeleteDocument(document.id);
+  };
+
+  const formatDate = (dateString: string) => {
+    const date = new Date(dateString);
+    return date.toLocaleDateString('en-US', {
+      month: 'short',
+      day: 'numeric',
+      year: 'numeric',
+    });
+  };
+
+  return (
+    <div
+      onClick={handleToggle}
+      className={`
+        p-3 rounded-lg border cursor-pointer transition-all duration-200
+        ${isSelected 
+          ? 'border-primary-500 bg-primary-50 dark:bg-primary-900/20' 
+          : 'border-gray-200 dark:border-gray-700 hover:border-gray-300 dark:hover:border-gray-600'
+        }
+      `}
+    >
+      <div className="flex items-start gap-3">
+        <div className={`
+          w-8 h-8 rounded-lg flex items-center justify-center flex-shrink-0
+          ${isSelected 
+            ? 'bg-primary-100 dark:bg-primary-900/40' 
+            : 'bg-gray-100 dark:bg-gray-700'
+          }
+        `}>
+          {isSelected ? (
+            <CheckCircle className="w-5 h-5 text-primary-600 dark:text-primary-400" />
+          ) : (
+            <FileText className="w-5 h-5 text-gray-500 dark:text-gray-400" />
+          )}
+        </div>
+
+        <div className="flex-1 min-w-0">
+          <p className="text-sm font-medium text-gray-900 dark:text-white truncate">
+            {document.filename}
+          </p>
+          <div className="flex items-center gap-2 mt-1">
+            <span className="text-xs text-gray-500 dark:text-gray-500">
+              {formatDate(document.uploadDate)}
+            </span>
+            <span className="text-xs px-1.5 py-0.5 bg-gray-100 dark:bg-gray-700 text-gray-600 dark:text-gray-400 rounded">
+              {document.chunkCount} chunks
+            </span>
+          </div>
+        </div>
+
+        <button
+          onClick={handleDelete}
+          className="p-1 hover:bg-red-100 dark:hover:bg-red-900/20 rounded transition-colors"
+        >
+          <Trash2 className="w-4 h-4 text-gray-400 hover:text-red-500" />
+        </button>
+      </div>
+
+      {document.status !== 'ready' && (
+        <div className="flex items-center gap-1 mt-2 text-xs text-amber-600 dark:text-amber-400">
+          {document.status === 'processing' ? (
+            <>
+              <AlertCircle className="w-3 h-3" />
+              <span>Processing...</span>
+            </>
+          ) : (
+            <>
+              <AlertCircle className="w-3 h-3" />
+              <span>Error processing</span>
+            </>
+          )}
+        </div>
+      )}
+    </div>
+  );
+};

frontend/src/components/documents/DocumentList.tsx

@@ -0,0 +1,48 @@
+import React from 'react';
+import { FileText, Search } from 'lucide-react';
+import { useApp } from '../../context/AppContext';
+import { DocumentCard } from './DocumentCard';
+
+export const DocumentList: React.FC = () => {
+  const { state } = useApp();
+
+  return (
+    <div className="space-y-4">
+      <div className="flex items-center justify-between">
+        <h3 className="text-sm font-semibold text-gray-900 dark:text-white uppercase tracking-wide">
+          Documents
+        </h3>
+        <span className="text-xs text-gray-500 dark:text-gray-500">
+          {state.documents.length} file{state.documents.length !== 1 ? 's' : ''}
+        </span>
+      </div>
+
+      <div className="relative">
+        <Search className="absolute left-3 top-1/2 -translate-y-1/2 w-4 h-4 text-gray-400" />
+        <input
+          type="text"
+          placeholder="Search documents..."
+          className="w-full pl-10 pr-4 py-2 text-sm border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-gray-700 text-gray-900 dark:text-white placeholder-gray-500 focus:outline-none focus:ring-2 focus:ring-primary-500"
+        />
+      </div>
+
+      <div className="space-y-2 max-h-96 overflow-y-auto scrollbar-thin">
+        {state.documents.length === 0 ? (
+          <div className="text-center py-8">
+            <FileText className="w-12 h-12 mx-auto text-gray-300 dark:text-gray-600 mb-3" />
+            <p className="text-sm text-gray-500 dark:text-gray-500">
+              No documents uploaded
+            </p>
+            <p className="text-xs text-gray-400 dark:text-gray-600 mt-1">
+              Upload PDFs to get started
+            </p>
+          </div>
+        ) : (
+          state.documents.map(doc => (
+            <DocumentCard key={doc.id} document={doc} />
+          ))
+        )}
+      </div>
+    </div>
+  );
+};

frontend/src/components/documents/FileUpload.tsx

@@ -0,0 +1,83 @@
+import React, { useCallback } from 'react';
+import { useDropzone } from 'react-dropzone';
+import { Upload, FileText, Loader2 } from 'lucide-react';
+import { useApp } from '../../context/AppContext';
+
+export const FileUpload: React.FC = () => {
+  const { state, handleUpload } = useApp();
+
+  const onDrop = useCallback(
+    (acceptedFiles: File[]) => {
+      const pdfFile = acceptedFiles.find(file => file.type === 'application/pdf');
+      if (pdfFile) {
+        handleUpload(pdfFile);
+      }
+    },
+    [handleUpload]
+  );
+
+  const { getRootProps, getInputProps, isDragActive } = useDropzone({
+    onDrop,
+    accept: {
+      'application/pdf': ['.pdf'],
+    },
+    maxFiles: 1,
+  });
+
+  return (
+    <div className="space-y-4">
+      <h3 className="text-sm font-semibold text-gray-900 dark:text-white uppercase tracking-wide">
+        Upload Documents
+      </h3>
+      
+      <div
+        {...getRootProps()}
+        className={`
+          border-2 border-dashed rounded-xl p-6 text-center cursor-pointer transition-all duration-200
+          ${isDragActive 
+            ? 'border-primary-500 bg-primary-50 dark:bg-primary-900/20' 
+            : 'border-gray-300 dark:border-gray-600 hover:border-primary-400 hover:bg-gray-50 dark:hover:bg-gray-700/50'
+          }
+        `}
+      >
+        <input {...getInputProps()} />
+        
+        {state.isUploading ? (
+          <div className="flex flex-col items-center gap-2">
+            <Loader2 className="w-10 h-10 text-primary-500 animate-spin" />
+            <p className="text-sm text-gray-600 dark:text-gray-400">
+              Processing... {state.uploadProgress}%
+            </p>
+            <div className="w-full max-w-xs bg-gray-200 dark:bg-gray-700 rounded-full h-2">
+              <div 
+                className="bg-primary-500 h-2 rounded-full transition-all duration-300"
+                style={{ width: `${state.uploadProgress}%` }}
+              />
+            </div>
+          </div>
+        ) : isDragActive ? (
+          <div className="flex flex-col items-center gap-2">
+            <FileText className="w-10 h-10 text-primary-500" />
+            <p className="text-sm font-medium text-primary-600 dark:text-primary-400">
+              Drop your PDF here
+            </p>
+          </div>
+        ) : (
+          <div className="flex flex-col items-center gap-2">
+            <Upload className="w-10 h-10 text-gray-400" />
+            <p className="text-sm font-medium text-gray-600 dark:text-gray-400">
+              Drag & drop a PDF
+            </p>
+            <p className="text-xs text-gray-500 dark:text-gray-500">
+              or click to browse
+            </p>
+          </div>
+        )}
+      </div>
+
+      <p className="text-xs text-gray-500 dark:text-gray-500 text-center">
+        Supports PDF files up to 10MB
+      </p>
+    </div>
+  );
+};

frontend/src/components/layout/Header.tsx

@@ -0,0 +1,56 @@
+import React from 'react';
+import { 
+  Brain, 
+  Settings, 
+  Menu,
+  Sun,
+  Moon 
+} from 'lucide-react';
+import { useApp } from '../../context/AppContext';
+import { ModeSelector } from '../query/ModeSelector';
+
+export const Header: React.FC = () => {
+  const { state, dispatch, toggleTheme } = useApp();
+
+  return (
+    <header className="fixed top-0 left-0 right-0 h-16 bg-white dark:bg-gray-800 border-b border-gray-200 dark:border-gray-700 z-50 flex items-center justify-between px-4">
+      <div className="flex items-center gap-4">
+        <button 
+          onClick={() => dispatch({ type: 'TOGGLE_SIDEBAR' })}
+          className="p-2 hover:bg-gray-100 dark:hover:bg-gray-700 rounded-lg lg:hidden"
+        >
+          <Menu className="w-5 h-5 text-gray-600 dark:text-gray-300" />
+        </button>
+        
+        <div className="flex items-center gap-2">
+          <Brain className="w-8 h-8 text-primary-600" />
+          <span className="text-xl font-bold text-gray-900 dark:text-white">
+            RAG System
+          </span>
+        </div>
+      </div>
+
+      <div className="flex items-center gap-4">
+        <ModeSelector />
+        
+        <button
+          onClick={toggleTheme}
+          className="p-2 hover:bg-gray-100 dark:hover:bg-gray-700 rounded-lg transition-colors"
+        >
+          {state.theme === 'light' ? (
+            <Moon className="w-5 h-5 text-gray-600" />
+          ) : (
+            <Sun className="w-5 h-5 text-yellow-500" />
+          )}
+        </button>
+
+        <button
+          onClick={() => dispatch({ type: 'TOGGLE_SETTINGS' })}
+          className="p-2 hover:bg-gray-100 dark:hover:bg-gray-700 rounded-lg transition-colors"
+        >
+          <Settings className="w-5 h-5 text-gray-600 dark:text-gray-300" />
+        </button>
+      </div>
+    </header>
+  );
+};

frontend/src/components/layout/MainContent.tsx

@@ -0,0 +1,25 @@
+import React from 'react';
+import { useApp } from '../../context/AppContext';
+import { QueryInput } from '../query/QueryInput';
+import { ResultsDisplay } from '../results/ResultsDisplay';
+import { EmptyState } from '../common/EmptyState';
+
+export const MainContent: React.FC = () => {
+  const { state } = useApp();
+
+  return (
+    <main className="pt-16 min-h-screen">
+      <div className={`flex-1 p-6 ${state.sidebarOpen ? 'ml-80' : 'ml-0'}`}>
+        <div className="max-w-4xl mx-auto space-y-6">
+          <QueryInput />
+
+          {state.currentAnswer ? (
+            <ResultsDisplay answer={state.currentAnswer} />
+          ) : (
+            <EmptyState />
+          )}
+        </div>
+      </div>
+    </main>
+  );
+};

frontend/src/components/layout/Sidebar.tsx

@@ -0,0 +1,19 @@
+import React from 'react';
+import { useApp } from '../../context/AppContext';
+import { FileUpload } from '../documents/FileUpload';
+import { DocumentList } from '../documents/DocumentList';
+
+export const Sidebar: React.FC = () => {
+  const { state } = useApp();
+
+  if (!state.sidebarOpen) return null;
+
+  return (
+    <aside className="fixed left-0 top-16 bottom-0 w-80 bg-white dark:bg-gray-800 border-r border-gray-200 dark:border-gray-700 flex flex-col z-40">
+      <div className="flex-1 overflow-y-auto p-4 space-y-6 scrollbar-thin">
+        <FileUpload />
+        <DocumentList />
+      </div>
+    </aside>
+  );
+};

frontend/src/components/query/ModeSelector.tsx

@@ -0,0 +1,53 @@
+import React from 'react';
+import { Globe, FileText, GitMerge, Lock } from 'lucide-react';
+import { useApp } from '../../context/AppContext';
+import type { QueryMode } from '../../types';
+
+const modes: { id: QueryMode; label: string; icon: React.ReactNode; description: string }[] = [
+  {
+    id: 'web',
+    label: 'Web Search',
+    icon: <Globe className="w-4 h-4" />,
+    description: 'Search the web for information',
+  },
+  {
+    id: 'pdf',
+    label: 'PDF Only',
+    icon: <FileText className="w-4 h-4" />,
+    description: 'Query only uploaded documents',
+  },
+  {
+    id: 'hybrid',
+    label: 'Hybrid',
+    icon: <GitMerge className="w-4 h-4" />,
+    description: 'Combine web and document search',
+  },
+  {
+    id: 'restricted',
+    label: 'Restricted',
+    icon: <Lock className="w-4 h-4" />,
+    description: 'Safe mode with content filtering',
+  },
+];
+
+export const ModeSelector: React.FC = () => {
+  const { state, dispatch } = useApp();
+
+  return (
+    <div className="flex items-center gap-1 bg-gray-100 dark:bg-gray-800 p-1 rounded-lg">
+      {modes.map(mode => (
+        <button
+          key={mode.id}
+          onClick={() => dispatch({ type: 'SET_QUERY_MODE', payload: mode.id })}
+          className={`
+            mode-tab ${state.queryMode === mode.id ? 'mode-tab-active' : 'mode-tab-inactive'}
+          `}
+          title={mode.description}
+        >
+          {mode.icon}
+          <span className="hidden sm:inline text-sm">{mode.label}</span>
+        </button>
+      ))}
+    </div>
+  );
+};

frontend/src/components/query/QueryInput.tsx

@@ -0,0 +1,125 @@
+import React, { useState, useCallback, useRef, useEffect } from 'react';
+import { Send, Sparkles, X } from 'lucide-react';
+import { useApp } from '../../context/AppContext';
+
+const sampleQueries = [
+  "What is the main topic of my documents?",
+  "Summarize the key findings",
+  "Extract important dates and events",
+];
+
+export const QueryInput: React.FC = () => {
+  const { state, handleQuery, dispatch, clearResults } = useApp();
+  const [showSamples, setShowSamples] = useState(true);
+  const textareaRef = useRef<HTMLTextAreaElement>(null);
+
+  const handleKeyDown = useCallback(
+    (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
+      if (e.key === 'Enter' && !e.shiftKey) {
+        e.preventDefault();
+        handleQuery();
+      }
+    },
+    [handleQuery]
+  );
+
+  const handleSubmit = useCallback(() => {
+    handleQuery();
+  }, [handleQuery]);
+
+  const autoResize = useCallback(() => {
+    const textarea = textareaRef.current;
+    if (textarea) {
+      textarea.style.height = 'auto';
+      textarea.style.height = `${Math.min(textarea.scrollHeight, 200)}px`;
+    }
+  }, []);
+
+  useEffect(() => {
+    autoResize();
+  }, [state.currentQuery, autoResize]);
+
+  const handleClear = () => {
+    clearResults();
+    setShowSamples(true);
+  };
+
+  return (
+    <div className="space-y-4">
+      <div className="card p-1">
+        <div className="flex items-start gap-2">
+          <textarea
+            ref={textareaRef}
+            value={state.currentQuery}
+            onChange={e => {
+              dispatch({ type: 'SET_CURRENT_QUERY', payload: e.target.value });
+              if (e.target.value && showSamples) {
+                setShowSamples(false);
+              }
+            }}
+            onKeyDown={handleKeyDown}
+            placeholder="Ask a question about your documents..."
+            className="flex-1 min-h-[120px] max-h-[200px] p-4 bg-transparent text-gray-900 dark:text-white placeholder-gray-500 resize-none focus:outline-none"
+            disabled={state.isLoading}
+          />
+
+          {state.currentQuery && (
+            <button
+              onClick={handleClear}
+              className="p-2 hover:bg-gray-100 dark:hover:bg-gray-700 rounded-lg transition-colors mt-1"
+            >
+              <X className="w-4 h-4 text-gray-400" />
+            </button>
+          )}
+        </div>
+
+        <div className="flex items-center justify-between px-4 pb-4">
+          <div className="flex items-center gap-2">
+            <Sparkles className="w-4 h-4 text-gray-400" />
+            <span className="text-xs text-gray-500 dark:text-gray-500">
+              Press Enter to submit, Shift+Enter for new line
+            </span>
+          </div>
+
+          <button
+            onClick={handleSubmit}
+            disabled={!state.currentQuery.trim() || state.isLoading}
+            className="btn-primary flex items-center gap-2"
+          >
+            {state.isLoading ? (
+              <>
+                <div className="w-4 h-4 border-2 border-white/30 border-t-white rounded-full animate-spin" />
+                <span>Processing...</span>
+              </>
+            ) : (
+              <>
+                <Send className="w-4 h-4" />
+                <span>Submit</span>
+              </>
+            )}
+          </button>
+        </div>
+      </div>
+
+      {showSamples && !state.currentQuery && (
+        <div className="flex flex-wrap gap-2">
+          <span className="text-xs text-gray-500 dark:text-gray-500 py-1">
+            Try:
+          </span>
+          {sampleQueries.map((query, index) => (
+            <button
+              key={index}
+              onClick={() => {
+                dispatch({ type: 'SET_CURRENT_QUERY', payload: query });
+                setShowSamples(false);
+              }}
+              className="text-xs px-3 py-1 bg-gray-100 dark:bg-gray-800 text-gray-600 dark:text-gray-400 rounded-full hover:bg-gray-200 dark:hover:bg-gray-700 transition-colors"
+            >
+              {query}
+            </button>
+          ))}
+        </div>
+      )}
+    </div>
+  );
+};

frontend/src/components/results/AnswerCard.tsx

@@ -0,0 +1,70 @@
+import React from 'react';
+import { Clock, Globe, FileText, GitMerge, Lock } from 'lucide-react';
+import type { Answer, QueryMode } from '../../types';
+import { ConfidenceIndicator } from './ConfidenceIndicator';
+
+interface AnswerCardProps {
+  answer: Answer;
+}
+
+const modeIcons: Record<QueryMode, React.ReactNode> = {
+  web: <Globe className="w-3 h-3" />,
+  pdf: <FileText className="w-3 h-3" />,
+  hybrid: <GitMerge className="w-3 h-3" />,
+  restricted: <Lock className="w-3 h-3" />,
+};
+
+const modeLabels: Record<QueryMode, string> = {
+  web: 'Web Search',
+  pdf: 'PDF Only',
+  hybrid: 'Hybrid',
+  restricted: 'Restricted',
+};
+
+export const AnswerCard: React.FC<AnswerCardProps> = ({ answer }) => {
+  const formatTime = (timestamp: string) => {
+    const date = new Date(timestamp);
+    return date.toLocaleTimeString('en-US', {
+      hour: 'numeric',
+      minute: '2-digit',
+    });
+  };
+
+  return (
+    <div className="card overflow-hidden">
+      <div className="border-b border-gray-200 dark:border-gray-700 p-4 bg-gray-50 dark:bg-gray-900/50">
+        <p className="text-sm text-gray-600 dark:text-gray-400 mb-2">
+          Question:
+        </p>
+        <p className="text-lg font-medium text-gray-900 dark:text-white">
+          {answer.query}
+        </p>
+      </div>
+
+      <div className="p-6 space-y-4">
+        <div className="flex items-center justify-between">
+          <div className="flex items-center gap-2">
+            {modeIcons[answer.mode]}
+            <span className="text-xs font-medium text-gray-600 dark:text-gray-400 uppercase">
+              {modeLabels[answer.mode]}
+            </span>
+          </div>
+          
+          <div className="flex items-center gap-4">
+            <ConfidenceIndicator confidence={answer.confidence} />
+            <div className="flex items-center gap-1 text-xs text-gray-500">
+              <Clock className="w-3 h-3" />
+              <span>{formatTime(answer.timestamp)}</span>
+            </div>
+          </div>
+        </div>
+
+        <div className="prose prose-gray dark:prose-invert max-w-none">
+          <p className="text-gray-800 dark:text-gray-200 leading-relaxed whitespace-pre-wrap">
+            {answer.text}
+          </p>
+        </div>
+      </div>
+    </div>
+  );
+};