# Negoptim AI — System Design & Architecture Negoptim AI is a **Retrieval-Augmented Generation (RAG) chatbot** designed as a local interactive demo for Users Love IT (ULiT). It provides an intelligent virtual assistant that answers business questions about commercial negotiations, procurement optimization, supplier collaboration, promotions, and assortment strategies. --- ## 1. High-Level Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ USER BROWSER │ │ localhost:3000 (Next.js client) │ └─────────────────────────┬───────────────────────────────────┘ │ HTTP Requests (proxied via Next.js rewrites) ▼ ┌─────────────────────────────────────────────────────────────┐ │ FASTAPI BACKEND (Python) │ │ localhost:8000 │ │ │ │ ┌───────────────────────────────────────────────────────┐ │ │ │ API ROUTERS │ │ │ │ /health /chat /ingest │ │ │ └───────────────────────┬───────────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────────────────────────────────────────────┐ │ │ │ RAG SERVICE │ │ │ │ (Orchestrator) │ │ │ └────┬──────────┬──────────────┬───────────────┬─────────┘ │ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ │ │ ┌─────────┐┌──────────┐┌──────────────┐┌──────────────┐ │ │ │ Intent ││Embedding ││ Vector Store ││ LLM Service │ │ │ │ Service ││ Service ││ (ChromaDB) ││ (Groq + │ │ │ │(Rule- ││(e5-small)││(cosine dist) ││ Fallbacks) │ │ │ │ based) │└──────────┘└──────────────┘└──────────────┘ │ │ └────┬────┘ │ │ │ │ ▼ │ │ │ ┌──────────────┐ │ │ │ │ Groq API LPU │ │ │ │ └──────────────┘ │ │ ▼ │ │ ┌─────────┐ │ │ │Data │ │ │ │(Struct- │ │ │ │ ured) │ │ │ └─────────┘ │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ KNOWLEDGE BASE │ │ /knowledge folder │ │ │ │ /company /guides /faqs /legal /customers │ │ (.md, .txt, .pdf files) │ └─────────────────────────────────────────────────────────────┘ ``` The system decouples the frontend and backend. Next.js functions as a reverse proxy where `/api/*` endpoint calls are forwarded to the FastAPI server running on port 8000. --- ## 2. File and Folder Structure The project directory is structured as follows: ``` NegoptimAiUlit/ ├── ARCHITECTURE.md ← This system documentation ├── README.md ← Setup and execution instructions ├── dataingestion.md ← Initial corpus summary text ├── int.md ← Integration architecture and framework notes ├── backend/ │ ├── .env ← Configuration variables (Groq keys, collection names) │ ├── requirements.txt ← Backend Python dependencies │ └── app/ │ ├── __init__.py │ ├── config.py ← Pydantic Settings management │ ├── main.py ← FastAPI initialization and middleware setups │ ├── api/ │ │ ├── __init__.py │ │ ├── chat.py ← POST /chat and DELETE /chat/{session_id} handlers │ │ ├── health.py ← Health verification GET /health │ │ └── ingestion.py ← Database ingestion initiator POST /ingest │ ├── data/ │ │ ├── __init__.py │ │ └── structured_data.py ← Hardcoded brand tables, stats, grids │ ├── models/ │ │ ├── __init__.py │ │ └── schemas.py ← Pydantic API response and request structures │ └── services/ │ ├── __init__.py │ ├── email_flow.py ← Conversational email FSM (collect → draft → confirm → send) │ ├── email_service.py ← Reusable SMTP sender (simulation mode without creds) │ ├── embedding_service.py ← SentenceTransformer multilingual-e5-small wrapper │ ├── ingestion_service.py ← Chunking, indexing, and vector insertion │ ├── intent_service.py ← Keyword-based fast classifier │ ├── language.py ← Heuristic EN/FR language detection │ ├── limits.py ← Rate limiter, TTL cache, daily counters (abuse/cost control) │ ├── llm_service.py ← Multi-provider LLM client (Groq → Cerebras → Gemini fallback) │ ├── rag_service.py ← Core orchestrator of context, intent, and LLM │ └── vector_store.py ← ChromaDB integration client ├── frontend/ │ ├── next.config.ts ← Proxy configuration (rewrites) │ ├── tailwind.config.ts ← Theme configurations and custom grid styles │ ├── package.json ← Frontend dependencies and scripts │ └── src/ │ ├── app/ │ │ ├── globals.css ← Style overrides and animations │ │ ├── layout.tsx ← Next.js root layout │ │ └── page.tsx ← Direct entry point rendering ChatInterface │ ├── components/ │ │ ├── ChatInterface.tsx ← Main layout container and text fields │ │ ├── HexAvatar.tsx ← SVG-based animated AI avatar │ │ ├── LoadingAnimation.tsx ← Node-pulsing connection indicator │ │ ├── MessageBubble.tsx ← Message rendering logic (User/Assistant) │ │ ├── SettingsPanel.tsx ← Quotas, token metrics, model fallback panel │ │ ├── SourceCard.tsx ← Citations container with toggle state │ │ ├── SuggestedQuestions.tsx ← Prompts to begin discussions │ │ ├── WelcomeSection.tsx ← Greeting layout │ │ └── rich/ │ │ ├── CompanyStatsDashboard.tsx ← Company statistics layout │ │ ├── FeatureGrid.tsx ← Interactive feature tabs │ │ ├── PartnerGrid.tsx ← Integrations/Partners display │ │ ├── RichResponseRenderer.tsx ← Router rendering the grid screens │ │ └── TeamGrid.tsx ← Company team members grid view │ ├── hooks/ │ │ └── useChat.ts ← Chat state management and request logic │ └── types/ │ └── chat.ts ← Unified TypeScript interface contracts ├── knowledge/ ← Ingestible text corpus folders │ ├── company/ │ ├── faqs/ │ ├── guides/ │ ├── legal/ │ └── customers/ └── scripts/ └── ingest.py ← Offline ingestion tool helper ``` --- ## 3. Technology Stack ### Backend Services * **FastAPI**: Async request processing with automatic documentation at `/docs`. * **ChromaDB**: Low-overhead embedded vector database configured with cosine similarity metrics. * **sentence-transformers (`intfloat/multilingual-e5-small`)**: Locally run embedding model providing high-quality representations for French and English. * **pypdf**: Extraction component for parsing PDF files in document ingestion. * **Pydantic v2**: Response serialization and boundary validation. ### Frontend Application * **Next.js 15 (App Router)**: Single-page setup with built-in reverse proxy routing. * **TypeScript**: Enforces typing contracts matching backend schemas. * **Tailwind CSS**: Custom color styling aligned with the ULiT identity. * **React Hooks**: Built-in state control for the chat window. --- ## 4. Backend Service Workflows & Components Each backend service is initialized once as a **Singleton** using standard getter factory patterns, preventing expensive initialization costs (e.g., reloading the 130MB embedding model or recreating database connections on every request). ### Service Details #### 1. [embedding_service.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/embedding_service.py) * **Responsibility**: Loads `multilingual-e5-small` locally to generate 384-dimensional vector representations. * **Prefix Strategy**: Requires task-specific prefix tokens: * Ingested passages are prefixed with `"passage: "` * Live user queries are prefixed with `"query: "` * *Rationale*: The E5 model is trained asymmetrically. Failing to apply these prefixes degrades similarity matching accuracy. #### 2. [vector_store.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/vector_store.py) * **Responsibility**: Encapsulates ChromaDB queries. Configures collection space as `hnsw:space = cosine`. * **Cosine Metrics**: Converts distance metrics returned by ChromaDB into similarity percentages using `similarity = 1.0 - distance`. #### 3. [ingestion_service.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/ingestion_service.py) * **Responsibility**: Processes documents from the `knowledge` directory, partitions them into chunks, embeds them, and upserts them. * **Chunking Strategies**: * **Markdown (`.md`)**: Splits documents logically by heading levels (`##` and `###`). Sections larger than `chunk_size` characters (700 default) are split into paragraph segments with `chunk_overlap` limits (100 default). * **Text (`.txt`) & PDF (`.pdf`)**: Paragraph-based partitioning, merging text blocks until the threshold size is met. * **Deterministic ID Generation**: * Chunks are upserted with deterministic IDs derived from the path, section, and text contents: `{safe_source_path}_{chunk_index}_{md5_hash[:8]}` * This prevents duplicate vectors when processing identical unchanged files. #### 4. [intent_service.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/intent_service.py) * **Responsibility**: A lightweight rule-based classifier that intercepts queries to detect if a specific structured dashboard view (such as team, partner list, features, stats) should be shown. * **Mechanism**: Scans queries for trigger keyword combinations (e.g., "team", "ceo", "partners") and instantly matches them to pre-defined response categories without generating LLM overhead. #### 5. [llm_service.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/llm_service.py) * **Responsibility**: Multi-provider LLM client. All providers speak the OpenAI-compatible protocol; the request chain is a flat ordered list of (provider, model) pairs. * **Fallback Strategy**: Groq models first (`llama-3.1-8b-instant` → `llama-3.3-70b-versatile` → `llama3-8b-8192`), then optional providers whose API keys are configured (Cerebras, Gemini). Falls through on HTTP 429, decommissioned-model errors, and connection failures. A UI-selected preferred model is promoted to the front of the chain. * `generate()` is the RAG entry point (injects the system prompt, knowledge context, and a language directive); `complete()` is a generic completion against an arbitrary system prompt (used by the email flow). #### 6. [rag_service.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/rag_service.py) * **Responsibility**: Acts as the central orchestrator. Binds query routing, vector search, history aggregation, and LLM text generation together. * **Turn order**: ① email flow (if active or triggered — skips RAG entirely) → ② first-turn response cache (identical opening questions served without LLM cost) → ③ intent classification → ④ embed + retrieve → ⑤ LLM generation pinned to the detected user language. * **Session hygiene**: in-memory histories are pruned after 6 h idle and capped at 1000 sessions (LRU eviction). #### 7. [email_flow.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/email_flow.py) & [email_service.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/email_service.py) * **email_flow**: deterministic finite-state machine driving email composition (demo requests + general emails): collect fields → LLM-generated draft → user review/edit → explicit confirmation → send. Sending is *structurally* impossible without a confirm action in the review stage. All prompts/messages are localized EN/FR. Send caps: per-session and global per-day. * **email_service**: dedicated reusable SMTP sender (Gmail STARTTLS). Without credentials it runs in *simulation mode* (logs + reports success). `EMAIL_OVERRIDE_TO` force-routes all mail to a test inbox. #### 8. [language.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/language.py) & [limits.py](file:///c:/Users/HP/NegoptimAiUlit/backend/app/services/limits.py) * **language**: dependency-free EN/FR detection (stopword + accent scoring). Drives response language and email draft language. * **limits**: `SlidingWindowLimiter` (per-IP chat rate limiting), `TTLCache` (first-turn response cache), `DailyCounter` (email caps). In-memory, single-instance by design. --- ## 5. Frontend & UI Visual Components The Negoptim AI user interface uses a clean light mode layout following ULiT's visual identity. ### Main Styling Specs * **Background**: Slate gray background (`#F4F6F9`) with white cards (`bg-white`). * **Borders**: Soft slate dividers (`#E8EDF2` and `#E2E8F0`). * **Accent Colors**: Custom color gradient sequence (`#C7D92F` → `#47C3A6` → `#14B7CC`). ### UI Elements and Component Tree * **[ChatInterface.tsx](file:///c:/Users/HP/NegoptimAiUlit/frontend/src/components/ChatInterface.tsx)**: Main layout controller handling text submissions, message lists, and scroll alignments. * **[SettingsPanel.tsx](file:///c:/Users/HP/NegoptimAiUlit/frontend/src/components/SettingsPanel.tsx)**: Slide-out status panel drawer that prioritizes model selection controls and live API quota/rate limits (remaining percentages, reset times), while keeping detailed token statistics collapsed. * **[WelcomeSection.tsx](file:///c:/Users/HP/NegoptimAiUlit/frontend/src/components/WelcomeSection.tsx)**: Opening splash screen featuring the brand name and an animated SVG node-mesh. * **[SuggestedQuestions.tsx](file:///c:/Users/HP/NegoptimAiUlit/frontend/src/components/SuggestedQuestions.tsx)**: Interactive trigger buttons that automatically load queries into the chat. * **[MessageBubble.tsx](file:///c:/Users/HP/NegoptimAiUlit/frontend/src/components/MessageBubble.tsx)**: Renders individual messages. Assistant responses with non-plain intents pass through the rich renderer. * **[RichResponseRenderer.tsx](file:///c:/Users/HP/NegoptimAiUlit/frontend/src/components/rich/RichResponseRenderer.tsx)**: Evaluates response types and loads specialized grid layouts: * **`company_stats`** (`CompanyStatsDashboard.tsx`): Displays statistics cards. * **`team_grid`** (`TeamGrid.tsx`): Displays team member listings with roles. * **`partner_grid`** (`PartnerGrid.tsx`): Renders partner companies. * **`feature_grid_manufacturers` / `feature_grid_retailers`** (`FeatureGrid.tsx`): Displays tabbed panels explaining different business modules. * **`email_draft`** (`EmailDraftCard.tsx`): Editable draft (subject/body) with Send/Cancel buttons — the only path that triggers an actual send. * **`email_sent`** (`EmailSentCard.tsx`): Delivery confirmation (flags simulation mode and test-override routing). --- ## 6. Key Data Flows ### Ingestion Flow 1. Contributor runs the ingestion command: `python ../scripts/ingest.py` (optionally specifying a folder). 2. `IngestionService` scans the configured directories for `.md`, `.txt`, and `.pdf` files. 3. Extracted text is segmented depending on heading structures and paragraph breaks. 4. Passages are prefixed with `"passage: "` and processed by the E5 model. 5. Embeddings are stored in the persistent ChromaDB database alongside metadata tags (e.g., path, heading). ### Query Flow ``` User Query Input ──► intent_service.py Classifies Query │ ┌───────────────────────┴───────────────────────┐ ▼ (if matches triggers) ▼ (Always runs) Return Rich Response Code embedding_service.py (prefix "query: ") & Pre-configured JSON data │ │ ▼ │ vector_store.py Queries ChromaDB │ │ │ ▼ └──────────────────────────────► rag_service.py Combines Context + History │ ▼ llm_service.py Queries Groq (Falls back on HTTP 429) │ ▼ UI Renders Bubble + Rich Dashboard Grid ``` --- ## 7. Configuration Variables Configuration values are parsed out of `backend/.env` using Pydantic Settings: * `GROQ_API_KEY`: API authentication key. * `GROQ_MODEL`: Core responding model (default: `llama-3.1-8b-instant`). * `GROQ_FALLBACK_MODELS`: List of alternate models to use when rate limits are hit. * `CHROMA_PERSIST_DIRECTORY`: Output location storing the vector database (default: `./chroma_db`). * `CHROMA_COLLECTION_NAME`: Database collection namespace. * `EMBEDDING_MODEL`: Name of the embedding model to load. * `RETRIEVAL_TOP_K`: Number of context blocks to return (default: `5`). * `CHUNK_SIZE`: Limit of character lengths per slice (default: `700`). * `CHUNK_OVERLAP`: Overlap between adjacent paragraph segments (default: `100`). * `MAX_HISTORY_TURNS`: Maximum chat history depth kept in session memory (default: `10` turns). * `CORS_ORIGINS`: Comma-separated allowed browser origins. * `CEREBRAS_API_KEY` / `GEMINI_API_KEY`: Optional free fallback LLM providers (skipped when empty). * `SMTP_USER` / `SMTP_PASSWORD` / `SMTP_HOST` / `SMTP_PORT`: Email delivery; empty credentials = simulation mode. * `EMAIL_OVERRIDE_TO`: Force-routes all outgoing mail to a test inbox (empty = real recipient). * `ADMIN_TOKEN`: Required header token for `POST /ingest`; empty = endpoint disabled. * `RATE_LIMIT_CHAT_PER_MINUTE` (default `20`/IP), `EMAIL_MAX_PER_SESSION` (`5`), `EMAIL_MAX_PER_DAY` (`50`): Abuse/cost guards. * `RESPONSE_CACHE_TTL_SECONDS` (`3600`) / `RESPONSE_CACHE_MAX_ENTRIES` (`200`): First-turn answer cache. * `SESSION_IDLE_TTL_SECONDS` (`21600`) / `MAX_SESSIONS` (`1000`): Session memory bounds. See [DEPLOYMENT.md](DEPLOYMENT.md) for local-vs-production values and the deploy workflow. --- ## 8. Rationale Behind Core Design Decisions 1. **Rule-Based Intent Classifier (`intent_service.py`)**: By routing informational questions (like team profiles or product features) through a quick, regex-based router, the application responds instantly with beautiful interactive grid cards, saving API tokens and avoiding hallucination. 2. **Groq with Fallback Chains**: Ollama runs too slowly on standard developer CPUs without dedicated GPUs. Groq's custom hardware infrastructure ensures responses return in under a second. The fallback chain makes sure rate limits do not break the demo. 3. **No Large Frameworks (LangChain / LlamaIndex)**: This RAG system is written directly in lightweight, transparent services. The entire pipeline remains easy to debug, extend, and trace. 4. **Heading-Aware Markdown Chunking**: Rather than breaking sections in the middle of sentences based on character limits, the ingestion module splits on heading structures. This keeps each document section unified and ensures context labels are accurate. --- ## 9. Development Guidelines & Best Practices 1. **Maintain Type Safety**: When updating APIs, verify that models in `backend/app/models/schemas.py` match interface structures in `frontend/src/types/chat.ts`. 2. **Preserve Service Singletons**: Instantiating services directly inside routes is an anti-pattern. Always retrieve shared singletons via the `get_*` factory methods. 3. **Run Re-Ingestion When Changing Embedding Models**: Changing the model type changes the dimensions of vectors, which will cause ChromaDB query failures if the index is not rebuilt. Run the ingestion script again to clear and rebuild the database. 4. **Keep Context Limits In Mind**: Modify the `MAX_HISTORY_TURNS` if long-context query issues arise, and always enforce task prefixes (`"query: "` / `"passage: "`) when modifying similarity query strategies.