update the code

README.md

@@ -77,6 +77,8 @@ Agents can intelligently:
 - **Strict multi-tenant isolation** with tenant_id filtering
 - **Intelligent text chunking** (~300 words per chunk)
 - **Vector similarity search** using cosine distance
+- **Async document ingestion** via Celery workers (PDF, DOCX, TXT support)
+- **Knowledge base management UI** for document upload and search
 
 ### 3. 🌐 Live Web Search Tool
 
@@ -121,6 +123,8 @@ Comprehensive insights for:
 - 🚨 Red-flag violation tracking
 - 🧠 Agent reasoning traces
 - 👥 Tenant activity monitoring
+- **Real-time analytics panel** in the frontend UI
+- **Async analytics processing** via Celery workers
 
 ### 7. 🏢 Multi-Tenant Isolation
 
@@ -149,16 +153,20 @@ Isolation is guaranteed via **Supabase Row-Level Security (RLS)**.
 | **Ollama / Groq** | LLM inference engine (configurable) |
 | **DuckDuckGo Search** | Web search provider |
 | **Slack / Email** | Alerting system |
+| **Celery** | Async task queue for document ingestion and analytics |
+| **Redis / RabbitMQ** | Message broker for Celery workers |
 
 ### Frontend
 
 | Technology | Purpose |
 |------------|---------|
-| **React 18** | Modern UI framework |
+| **Next.js 16** | React framework with App Router |
+| **React 19** | Modern UI framework |
 | **TailwindCSS** | Utility-first styling |
-| **Zustand** | State management |
+| **TypeScript** | Type-safe development |
 | **WebSocket** | Real-time streaming |
 | **Admin Dashboard** | Analytics & governance UI |
+| **Knowledge Base UI** | Document management interface |
 
 ---
 
@@ -245,10 +253,14 @@ IntegraChat/
 │   │   └── embeddings.py        # Sentence transformers embeddings
 │   │
 │   ├── workers/
-│   │   ├── ingestion_worker.py
-│   │   ├── analytics_worker.py
-│   │   ├── scheduler.py
-│   │   └── celeryconfig.py
+│   │   ├── ingestion_worker.py    # Celery tasks for document ingestion
+│   │   ├── analytics_worker.py     # Celery tasks for analytics processing
+│   │   ├── scheduler.py           # Scheduled task definitions
+│   │   └── celeryconfig.py        # Celery app configuration
+│   │
+│   ├── api/
+│   │   ├── ingestion/
+│   │   │   └── pdf.py             # PDF processing utilities
 │   │
 │   ├── tests/
 │   │   ├── test_agent.py
@@ -260,33 +272,20 @@ IntegraChat/
 │   └── Dockerfile
 │
 ├── frontend/
-│   ├── src/
-│   │   ├── components/
-│   │   │   ├── chat/
-│   │   │   │   ├── ChatWindow.jsx
-│   │   │   │   ├── MessageBubble.jsx
-│   │   │   │   └── ToolTraces.jsx
-│   │   │   ├── admin/
-│   │   │   │   ├── RedFlagEditor.jsx
-│   │   │   │   └── TenantManager.jsx
-│   │   │   └── analytics/
-│   │   │       ├── QueryStats.jsx
-│   │   │       ├── ToolUsageChart.jsx
-│   │   │       └── RagEffectiveness.jsx
-│   │   ├── pages/
-│   │   │   ├── ChatPage.jsx
-│   │   │   ├── AdminPage.jsx
-│   │   │   └── AnalyticsPage.jsx
-│   │   ├── store/
-│   │   │   ├── chatStore.js
-│   │   │   ├── adminStore.js
-│   │   │   └── analyticsStore.js
-│   │   ├── utils/
-│   │   │   ├── websocket.js
-│   │   │   ├── api.js
-│   │   │   └── formatters.js
-│   │   ├── App.jsx
-│   │   └── main.jsx
+│   ├── app/
+│   │   ├── page.tsx               # Main landing page
+│   │   ├── layout.tsx              # Root layout
+│   │   ├── globals.css            # Global styles
+│   │   └── knowledge-base/
+│   │       └── page.tsx           # Knowledge base management page
+│   ├── components/
+│   │   ├── chat-panel.tsx         # Chat interface component
+│   │   ├── analytics-panel.tsx    # Analytics dashboard component
+│   │   ├── knowledge-base-panel.tsx # Knowledge base search/ingest UI
+│   │   ├── ingestion-card.tsx     # Document ingestion card
+│   │   ├── hero.tsx               # Hero section
+│   │   ├── feature-grid.tsx       # Feature showcase grid
+│   │   └── footer.tsx             # Footer component
 │   ├── public/
 │   └── package.json
 │
@@ -372,6 +371,10 @@ Before you begin, ensure you have the following installed:
    OLLAMA_MODEL=llama3
    # Or use Groq instead:
    # GROQ_API_KEY=your_groq_api_key
+   
+   # Celery Configuration (for async workers)
+   CELERY_BROKER_URL=redis://localhost:6379/0
+   CELERY_RESULT_BACKEND=redis://localhost:6379/0
    ```
 
 4. **Start the RAG MCP Server**
@@ -383,7 +386,16 @@ Before you begin, ensure you have the following installed:
    - Server runs on `http://localhost:8001`
    - API docs available at `http://localhost:8001/docs`
 
-5. **Run the main API server**
+5. **Start Celery workers** (for async document ingestion and analytics)
+   ```bash
+   # Start Celery worker
+   celery -A backend.workers.celeryconfig worker --loglevel=info
+   
+   # In another terminal, start Celery beat (for scheduled tasks)
+   celery -A backend.workers.celeryconfig beat --loglevel=info
+   ```
+
+6. **Run the main API server**
    ```bash
    cd backend
    uvicorn api.main:app --reload
@@ -413,6 +425,34 @@ curl -X POST http://localhost:8001/search \
   }'
 ```
 
+### Document Ingestion via Celery Workers
+
+Documents can be ingested asynchronously using Celery workers:
+
+**Via API endpoint (triggers Celery task):**
+```bash
+curl -X POST http://localhost:8000/rag/ingest \
+  -H "Content-Type: application/json" \
+  -H "x-tenant-id: tenant123" \
+  -d '{
+    "content": "Your document text here...",
+    "doc_id": "doc_001"
+  }'
+```
+
+**Supported formats:**
+- Raw text content
+- PDF files (via file upload)
+- DOCX files (via file upload)
+- TXT files (via file upload)
+- URLs (web page content)
+
+The ingestion worker automatically:
+- Extracts text from files
+- Chunks text with configurable overlap
+- Generates embeddings using Sentence-Transformers
+- Stores chunks and embeddings in Supabase/pgvector
+
 ### Frontend Setup
 
 1. **Navigate to frontend directory**
@@ -425,10 +465,22 @@ curl -X POST http://localhost:8001/search \
    npm install
    ```
 
-3. **Start the development server**
+3. **Configure environment variables** (optional)
+   
+   Create a `.env.local` file in the frontend directory:
+   ```env
+   NEXT_PUBLIC_API_URL=http://localhost:8000
+   ```
+
+4. **Start the development server**
    ```bash
    npm run dev
    ```
+   
+   The app will be available at `http://localhost:3000` with:
+   - **Main landing page** with hero, features, and chat panel
+   - **Knowledge base page** (`/knowledge-base`) for document management
+   - **Analytics panel** showing query metrics and tool usage
 
 ### Quick Start with Docker
 
@@ -449,6 +501,8 @@ docker-compose up -d
 | 🌐 **English Web Search** | Forces English language results for better accuracy |
 | 🏢 **Production-Grade** | Multi-tenant design with strict Supabase RLS |
 | 📊 **Full Observability** | Logs, analytics, tool events, violations |
+| 📚 **Knowledge Base UI** | Complete document management interface with search and ingestion |
+| ⚡ **Async Processing** | Celery workers for scalable document ingestion and analytics |
 | 🎯 **Demo-Ready** | Perfect for enterprise presentations |
 
 ---

frontend/README.md

@@ -3,10 +3,12 @@
 Next.js 16 / React 19 app that showcases everything wired up in `backend/`.  
 It provides a polished operator console with:
 
-- Hero + feature overview describing the FastAPI + MCP stack
-- Live chat panel that POSTs to `POST /agent/message`
-- Analytics snapshot pulling from `GET /analytics/overview`
-- Knowledge ingestion pipeline summary tied to Celery workers
+- **Hero section** + feature overview describing the FastAPI + MCP stack
+- **Live chat panel** that POSTs to `POST /agent/message` for AI conversations
+- **Analytics dashboard** pulling from `GET /analytics/overview` with real-time metrics
+- **Knowledge base management** page (`/knowledge-base`) for document search and ingestion
+- **Document ingestion UI** for uploading PDF, DOCX, TXT files or raw text
+- **Feature grid** showcasing platform capabilities
 
 ## Running Locally
 
@@ -16,7 +18,7 @@ npm install
 npm run dev
 ```
 
-Visit `http://localhost:3000`.
+Visit `http://localhost:3000` for the main landing page, or `http://localhost:3000/knowledge-base` for document management.
 
 ### API configuration
 
@@ -27,6 +29,37 @@ Update `.env.local` if your backend runs elsewhere:
 NEXT_PUBLIC_API_URL=http://localhost:8000
 ```
 
+## Features
+
+### Main Landing Page (`/`)
+- **Hero section** with platform introduction
+- **Feature grid** showcasing key capabilities
+- **Chat panel** for real-time AI conversations
+- **Analytics panel** with query metrics and tool usage statistics
+- **Ingestion card** for quick document uploads
+
+### Knowledge Base Page (`/knowledge-base`)
+- **Document listing** with pagination and filtering
+- **Search interface** for semantic search across documents
+- **Document ingestion** with support for:
+  - Raw text input
+  - PDF file uploads
+  - DOCX file uploads
+  - TXT file uploads
+- **Document management** with tenant isolation
+
+### Components
+
+- `chat-panel.tsx` - Real-time chat interface with streaming responses
+- `analytics-panel.tsx` - Analytics dashboard with metrics visualization
+- `knowledge-base-panel.tsx` - Document search and ingestion component
+- `ingestion-card.tsx` - Quick document upload card
+- `hero.tsx` - Landing page hero section
+- `feature-grid.tsx` - Feature showcase grid
+- `footer.tsx` - Footer component
+
 ## Deploy
 
 Deploy like any Next.js app (Vercel, Docker, etc.). Ensure the backend endpoints are reachable from the browser and CORS is enabled (already configured in `backend/api/main.py`).
+
+**Note:** Make sure Celery workers are running in production for document ingestion and analytics processing to work properly.