Spaces:
Sleeping
Sleeping
File size: 4,613 Bytes
de852c2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 | # Multi-LLM Chatbot Backend
A modular, extensible FastAPI backend for building an AI-powered research advisor chatbot that supports:
- Multiple AI personas with configurable tone and behavior
- Dynamic switching between Gemini (cloud) and Ollama (local) LLMs
- Chat session persistence and context memory
- Document upload, chunking, and retrieval using RAG
- Rich export features (PDF, DOCX, TXT)
- User authentication and JWT-based access control
---
## Backend Architecture
```text
User Input
↓
/chat-stream → Orchestrator
↓ ↙ ↘
SessionManager ContextManager RAGManager
↓ ↓ ↓
MongoDB Token Trimming ChromaDB
↓ ↓ ↓
Persisted Chat & Doc Context → LLM (Gemini/Ollama)
```
---
## Features
- Persona-based multi-agent conversation (`Theorist`, `Pragmatist`, etc.)
- Provider switching (Gemini ↔ Ollama)
- Context-aware response routing + top-K advisor selection
- PDF, DOCX, and TXT file upload and semantic retrieval
- Developer tools: debug personas, test RAG, export sessions
- Secure authentication and session scoping
---
## Setup Instructions
### 1. Clone and Configure Environment
```bash
git clone https://github.com/yourorg/multi-llm-chatbot-backend
cd multi-llm-chatbot-backend
cp .env.example .env # already provided
```
### 2. Python Environment Setup
```bash
python -m venv venv
source venv/bin/activate # or venv\Scripts\activate on Windows
pip install -r requirements.txt
```
### 3. Run the Server
```bash
uvicorn app.main:app --reload
```
> Server will be available at: `http://localhost:8000`
---
## FastAPI Routing & Modules
| Folder | Description |
|--------|-------------|
| [`app/api`](./api_README.md) | REST API endpoints for chat, auth, RAG, exports |
| [`app/core`](./core_README.md) | Main orchestration, context windows, database logic |
| [`app/llm`](./llm_README.md) | Gemini + Ollama LLM wrappers |
| [`app/models`](./models_README.md) | Persona and user schemas |
| [`app/utils`](./utils_README.md) | File parsing, summaries, exports, vector helpers |
---
## Key Files
### `main.py`
- Loads env vars, sets up FastAPI instance with CORS and routers
- Calls `connect_to_mongo()` on startup and `close_mongo_connection()` on shutdown
- Imports and registers all routers (`auth`, `chat_sessions`, etc.)
### `.env` (Sample Vars)
```ini
# MongoDB
MONGODB_CONNECTION_STRING=mongodb://localhost:27017
MONGODB_DATABASE_NAME=neon_ai_backend
# Gemini API Key and model
GEMINI_API_KEY=... # Replace with real key
GEMINI_MODEL=gemini-2.0-flash
# Default provider
DEFAULT_PROVIDER=gemini
```
### `requirements.txt`
Includes:
- **FastAPI**, **Uvicorn**: API framework and server
- **httpx**: Async LLM request handler
- **motor**, **pymongo**: MongoDB async access
- **chromadb**, **sentence-transformers**: Vector database + embeddings
- **PyPDF2**, **docx2txt**, **reportlab**: Document parsing and PDF generation
- **passlib**, **python-jose**: Auth and security
---
## Persona Design & Context Handling
- Personas defined in `app/models/default_personas.py`
- Rich system prompts, styles, and epistemologies
- Responses routed through `ImprovedChatOrchestrator`
- Context trimmed and weighted via `ContextManager`
---
## Switching LLM Providers
You can hot-swap models via API:
```http
POST /switch-provider
{ "provider": "gemini" } | { "provider": "ollama" }
```
> Also supported: `/switch-model`, `/current-model`, `/current-provider`
---
## Document Upload + RAG
- Upload PDFs, DOCX, or TXT to sessions
- Text is extracted → chunked → embedded → stored in ChromaDB
- Queried during conversation by persona-aware `EnhancedRAGManager`
---
## Export Options
| Format | Export Endpoint |
|--------|------------------|
| PDF | `/export-chat?format=pdf` |
| DOCX | `/export-chat?format=docx` |
| TXT | `/export-chat?format=txt` |
| Summary | `/chat-summary?format=pdf` |
---
## Developer & Debug Endpoints
| Endpoint | Purpose |
|----------|---------|
| `/debug/personas` | See registered advisors and prompts |
| `/debug/ranked-personas` | View top-K advisors for context |
| `/debug/rag-status` | Run sample search to test document index |
---
## Status & Roadmap
- [x] Multi-LLM backend ready (Gemini + Ollama)
- [x] Document RAG + export system
- [x] Session-aware persona routing
- [x] JWT Auth + MongoDB user handling
- [ ] UI enhancements and persona memory
- [ ] Persona fine-tuning support (future)
---
For questions, contributions, or deployment help — feel free to reach out! |