ARCHITECTURE.md

Negoptim AI — System Design & Architecture

Negoptim AI is a Retrieval-Augmented Generation (RAG) chatbot designed as a local interactive demo for Users Love IT (ULiT). It provides an intelligent virtual assistant that answers business questions about commercial negotiations, procurement optimization, supplier collaboration, promotions, and assortment strategies.

---

1. High-Level Architecture

┌─────────────────────────────────────────────────────────────┐
│                        USER BROWSER                         │
│                  localhost:3000 (Next.js client)            │
└─────────────────────────┬───────────────────────────────────┘
                          │ HTTP Requests (proxied via Next.js rewrites)
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                    FASTAPI BACKEND (Python)                 │
│                    localhost:8000                            │
│                                                             │
│  ┌───────────────────────────────────────────────────────┐  │
│  │                     API ROUTERS                       │  │
│  │  /health            /chat             /ingest         │  │
│  └───────────────────────┬───────────────────────────────┘  │
│                          │                                  │
│                          ▼                                  │
│  ┌───────────────────────────────────────────────────────┐  │
│  │                    RAG SERVICE                        │  │
│  │                   (Orchestrator)                      │  │
│  └────┬──────────┬──────────────┬───────────────┬─────────┘  │
│       │          │              │               │            │
│       ▼          ▼              ▼               ▼            │
│  ┌─────────┐┌──────────┐┌──────────────┐┌──────────────┐    │
│  │ Intent  ││Embedding ││ Vector Store ││ LLM Service  │    │
│  │ Service ││ Service  ││  (ChromaDB)  ││ (Groq +     │    │
│  │(Rule-   ││(e5-small)││(cosine dist) ││ Fallbacks)   │    │
│  │  based) │└──────────┘└──────────────┘└──────────────┘    │
│  └────┬────┘                                    │            │
│       │                                         ▼            │
│       │                                 ┌──────────────┐    │
│       │                                 │ Groq API LPU │    │
│       │                                 └──────────────┘    │
│       ▼                                                      │
│  ┌─────────┐                                                 │
│  │Data     │                                                 │
│  │(Struct- │                                                 │
│  │  ured)  │                                                 │
│  └─────────┘                                                 │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                     KNOWLEDGE BASE                          │
│                    /knowledge folder                        │
│                                                             │
│   /company   /guides   /faqs   /legal   /customers          │
│   (.md, .txt, .pdf files)                                   │
└─────────────────────────────────────────────────────────────┘

The system decouples the frontend and backend. Next.js functions as a reverse proxy where /api/* endpoint calls are forwarded to the FastAPI server running on port 8000.

---

2. File and Folder Structure

The project directory is structured as follows:

NegoptimAiUlit/
├── ARCHITECTURE.md          ← This system documentation
├── README.md                ← Setup and execution instructions
├── dataingestion.md         ← Initial corpus summary text
├── int.md                   ← Integration architecture and framework notes
├── backend/
│   ├── .env                 ← Configuration variables (Groq keys, collection names)
│   ├── requirements.txt     ← Backend Python dependencies
│   └── app/
│       ├── __init__.py
│       ├── config.py        ← Pydantic Settings management
│       ├── main.py          ← FastAPI initialization and middleware setups
│       ├── api/
│       │   ├── __init__.py
│       │   ├── chat.py      ← POST /chat and DELETE /chat/{session_id} handlers
│       │   ├── health.py    ← Health verification GET /health
│       │   └── ingestion.py ← Database ingestion initiator POST /ingest
│       ├── data/
│       │   ├── __init__.py
│       │   └── structured_data.py ← Hardcoded brand tables, stats, grids
│       ├── models/
│       │   ├── __init__.py
│       │   └── schemas.py   ← Pydantic API response and request structures
│       └── services/
│           ├── __init__.py
│           ├── email_flow.py        ← Conversational email FSM (collect → draft → confirm → send)
│           ├── email_service.py     ← Reusable SMTP sender (simulation mode without creds)
│           ├── embedding_service.py ← SentenceTransformer multilingual-e5-small wrapper
│           ├── ingestion_service.py ← Chunking, indexing, and vector insertion
│           ├── intent_service.py    ← Keyword-based fast classifier
│           ├── language.py          ← Heuristic EN/FR language detection
│           ├── limits.py            ← Rate limiter, TTL cache, daily counters (abuse/cost control)
│           ├── llm_service.py       ← Multi-provider LLM client (Groq → Cerebras → Gemini fallback)
│           ├── rag_service.py       ← Core orchestrator of context, intent, and LLM
│           └── vector_store.py      ← ChromaDB integration client
├── frontend/
│   ├── next.config.ts       ← Proxy configuration (rewrites)
│   ├── tailwind.config.ts   ← Theme configurations and custom grid styles
│   ├── package.json         ← Frontend dependencies and scripts
│   └── src/
│       ├── app/
│       │   ├── globals.css  ← Style overrides and animations
│       │   ├── layout.tsx   ← Next.js root layout
│       │   └── page.tsx     ← Direct entry point rendering ChatInterface
│       ├── components/
│       │   ├── ChatInterface.tsx    ← Main layout container and text fields
│       │   ├── HexAvatar.tsx        ← SVG-based animated AI avatar
│       │   ├── LoadingAnimation.tsx ← Node-pulsing connection indicator
│       │   ├── MessageBubble.tsx    ← Message rendering logic (User/Assistant)
│       │   ├── SettingsPanel.tsx    ← Quotas, token metrics, model fallback panel
│       │   ├── SourceCard.tsx       ← Citations container with toggle state
│       │   ├── SuggestedQuestions.tsx ← Prompts to begin discussions
│       │   ├── WelcomeSection.tsx   ← Greeting layout
│       │   └── rich/
│       │       ├── CompanyStatsDashboard.tsx ← Company statistics layout
│       │       ├── FeatureGrid.tsx           ← Interactive feature tabs
│       │       ├── PartnerGrid.tsx           ← Integrations/Partners display
│       │       ├── RichResponseRenderer.tsx  ← Router rendering the grid screens
│       │       └── TeamGrid.tsx              ← Company team members grid view
│       ├── hooks/
│       │   └── useChat.ts   ← Chat state management and request logic
│       └── types/
│           └── chat.ts      ← Unified TypeScript interface contracts
├── knowledge/               ← Ingestible text corpus folders
│   ├── company/
│   ├── faqs/
│   ├── guides/
│   ├── legal/
│   └── customers/
└── scripts/
    └── ingest.py            ← Offline ingestion tool helper

---

3. Technology Stack

Backend Services

• FastAPI: Async request processing with automatic documentation at /docs.
• ChromaDB: Low-overhead embedded vector database configured with cosine similarity metrics.
• sentence-transformers (intfloat/multilingual-e5-small): Locally run embedding model providing high-quality representations for French and English.
• pypdf: Extraction component for parsing PDF files in document ingestion.
• Pydantic v2: Response serialization and boundary validation.

Frontend Application

• Next.js 15 (App Router): Single-page setup with built-in reverse proxy routing.
• TypeScript: Enforces typing contracts matching backend schemas.
• Tailwind CSS: Custom color styling aligned with the ULiT identity.
• React Hooks: Built-in state control for the chat window.

---

4. Backend Service Workflows & Components

Each backend service is initialized once as a Singleton using standard getter factory patterns, preventing expensive initialization costs (e.g., reloading the 130MB embedding model or recreating database connections on every request).

Service Details

1. embedding_service.py

• Responsibility: Loads multilingual-e5-small locally to generate 384-dimensional vector representations.
• Prefix Strategy: Requires task-specific prefix tokens:
  • Ingested passages are prefixed with "passage: "
  • Live user queries are prefixed with "query: "
  • Rationale: The E5 model is trained asymmetrically. Failing to apply these prefixes degrades similarity matching accuracy.

2. vector_store.py

• Responsibility: Encapsulates ChromaDB queries. Configures collection space as hnsw:space = cosine.
• Cosine Metrics: Converts distance metrics returned by ChromaDB into similarity percentages using similarity = 1.0 - distance.

3. ingestion_service.py

• Responsibility: Processes documents from the knowledge directory, partitions them into chunks, embeds them, and upserts them.
• Chunking Strategies:
  • Markdown (.md): Splits documents logically by heading levels (## and ###). Sections larger than chunk_size characters (700 default) are split into paragraph segments with chunk_overlap limits (100 default).
  • Text (.txt) & PDF (.pdf): Paragraph-based partitioning, merging text blocks until the threshold size is met.
• Deterministic ID Generation:
  • Chunks are upserted with deterministic IDs derived from the path, section, and text contents: {safe_source_path}_{chunk_index}_{md5_hash[:8]}
  • This prevents duplicate vectors when processing identical unchanged files.

4. intent_service.py

• Responsibility: A lightweight rule-based classifier that intercepts queries to detect if a specific structured dashboard view (such as team, partner list, features, stats) should be shown.
• Mechanism: Scans queries for trigger keyword combinations (e.g., "team", "ceo", "partners") and instantly matches them to pre-defined response categories without generating LLM overhead.

5. llm_service.py

• Responsibility: Multi-provider LLM client. All providers speak the OpenAI-compatible protocol; the request chain is a flat ordered list of (provider, model) pairs.
• Fallback Strategy: Groq models first (llama-3.1-8b-instant → llama-3.3-70b-versatile → llama3-8b-8192), then optional providers whose API keys are configured (Cerebras, Gemini). Falls through on HTTP 429, decommissioned-model errors, and connection failures. A UI-selected preferred model is promoted to the front of the chain.
• generate() is the RAG entry point (injects the system prompt, knowledge context, and a language directive); complete() is a generic completion against an arbitrary system prompt (used by the email flow).

6. rag_service.py

• Responsibility: Acts as the central orchestrator. Binds query routing, vector search, history aggregation, and LLM text generation together.
• Turn order: ① email flow (if active or triggered — skips RAG entirely) → ② first-turn response cache (identical opening questions served without LLM cost) → ③ intent classification → ④ embed + retrieve → ⑤ LLM generation pinned to the detected user language.
• Session hygiene: in-memory histories are pruned after 6 h idle and capped at 1000 sessions (LRU eviction).

7. email_flow.py & email_service.py

• email_flow: deterministic finite-state machine driving email composition (demo requests + general emails): collect fields → LLM-generated draft → user review/edit → explicit confirmation → send. Sending is structurally impossible without a confirm action in the review stage. All prompts/messages are localized EN/FR. Send caps: per-session and global per-day.
• email_service: dedicated reusable SMTP sender (Gmail STARTTLS). Without credentials it runs in simulation mode (logs + reports success). EMAIL_OVERRIDE_TO force-routes all mail to a test inbox.

8. language.py & limits.py

• language: dependency-free EN/FR detection (stopword + accent scoring). Drives response language and email draft language.
• limits: SlidingWindowLimiter (per-IP chat rate limiting), TTLCache (first-turn response cache), DailyCounter (email caps). In-memory, single-instance by design.

---

5. Frontend & UI Visual Components

The Negoptim AI user interface uses a clean light mode layout following ULiT's visual identity.

Main Styling Specs

• Background: Slate gray background (#F4F6F9) with white cards (bg-white).
• Borders: Soft slate dividers (#E8EDF2 and #E2E8F0).
• Accent Colors: Custom color gradient sequence (#C7D92F → #47C3A6 → #14B7CC).

UI Elements and Component Tree

• ChatInterface.tsx: Main layout controller handling text submissions, message lists, and scroll alignments.
• SettingsPanel.tsx: Slide-out status panel drawer that prioritizes model selection controls and live API quota/rate limits (remaining percentages, reset times), while keeping detailed token statistics collapsed.
• WelcomeSection.tsx: Opening splash screen featuring the brand name and an animated SVG node-mesh.
• SuggestedQuestions.tsx: Interactive trigger buttons that automatically load queries into the chat.
• MessageBubble.tsx: Renders individual messages. Assistant responses with non-plain intents pass through the rich renderer.
• RichResponseRenderer.tsx: Evaluates response types and loads specialized grid layouts:
  • company_stats (CompanyStatsDashboard.tsx): Displays statistics cards.
  • team_grid (TeamGrid.tsx): Displays team member listings with roles.
  • partner_grid (PartnerGrid.tsx): Renders partner companies.
  • feature_grid_manufacturers / feature_grid_retailers (FeatureGrid.tsx): Displays tabbed panels explaining different business modules.
  • email_draft (EmailDraftCard.tsx): Editable draft (subject/body) with Send/Cancel buttons — the only path that triggers an actual send.
  • email_sent (EmailSentCard.tsx): Delivery confirmation (flags simulation mode and test-override routing).

---

6. Key Data Flows

Ingestion Flow

1. Contributor runs the ingestion command: python ../scripts/ingest.py (optionally specifying a folder).
2. IngestionService scans the configured directories for .md, .txt, and .pdf files.
3. Extracted text is segmented depending on heading structures and paragraph breaks.
4. Passages are prefixed with "passage: " and processed by the E5 model.
5. Embeddings are stored in the persistent ChromaDB database alongside metadata tags (e.g., path, heading).

Query Flow

User Query Input  ──►  intent_service.py Classifies Query
                               │
       ┌───────────────────────┴───────────────────────┐
       ▼ (if matches triggers)                         ▼ (Always runs)
Return Rich Response Code               embedding_service.py (prefix "query: ")
& Pre-configured JSON data                             │
       │                                               ▼
       │                                vector_store.py Queries ChromaDB
       │                                               │
       │                                               ▼
       └──────────────────────────────►  rag_service.py Combines Context + History
                                                       │
                                                       ▼
                                         llm_service.py Queries Groq
                                         (Falls back on HTTP 429)
                                                       │
                                                       ▼
                                         UI Renders Bubble + Rich Dashboard Grid

---

7. Configuration Variables

Configuration values are parsed out of backend/.env using Pydantic Settings:

• GROQ_API_KEY: API authentication key.
• GROQ_MODEL: Core responding model (default: llama-3.1-8b-instant).
• GROQ_FALLBACK_MODELS: List of alternate models to use when rate limits are hit.
• CHROMA_PERSIST_DIRECTORY: Output location storing the vector database (default: ./chroma_db).
• CHROMA_COLLECTION_NAME: Database collection namespace.
• EMBEDDING_MODEL: Name of the embedding model to load.
• RETRIEVAL_TOP_K: Number of context blocks to return (default: 5).
• CHUNK_SIZE: Limit of character lengths per slice (default: 700).
• CHUNK_OVERLAP: Overlap between adjacent paragraph segments (default: 100).
• MAX_HISTORY_TURNS: Maximum chat history depth kept in session memory (default: 10 turns).
• CORS_ORIGINS: Comma-separated allowed browser origins.
• CEREBRAS_API_KEY / GEMINI_API_KEY: Optional free fallback LLM providers (skipped when empty).
• SMTP_USER / SMTP_PASSWORD / SMTP_HOST / SMTP_PORT: Email delivery; empty credentials = simulation mode.
• EMAIL_OVERRIDE_TO: Force-routes all outgoing mail to a test inbox (empty = real recipient).
• ADMIN_TOKEN: Required header token for POST /ingest; empty = endpoint disabled.
• RATE_LIMIT_CHAT_PER_MINUTE (default 20/IP), EMAIL_MAX_PER_SESSION (5), EMAIL_MAX_PER_DAY (50): Abuse/cost guards.
• RESPONSE_CACHE_TTL_SECONDS (3600) / RESPONSE_CACHE_MAX_ENTRIES (200): First-turn answer cache.
• SESSION_IDLE_TTL_SECONDS (21600) / MAX_SESSIONS (1000): Session memory bounds.

See DEPLOYMENT.md for local-vs-production values and the deploy workflow.

---

8. Rationale Behind Core Design Decisions

1. Rule-Based Intent Classifier (intent_service.py): By routing informational questions (like team profiles or product features) through a quick, regex-based router, the application responds instantly with beautiful interactive grid cards, saving API tokens and avoiding hallucination.
2. Groq with Fallback Chains: Ollama runs too slowly on standard developer CPUs without dedicated GPUs. Groq's custom hardware infrastructure ensures responses return in under a second. The fallback chain makes sure rate limits do not break the demo.
3. No Large Frameworks (LangChain / LlamaIndex): This RAG system is written directly in lightweight, transparent services. The entire pipeline remains easy to debug, extend, and trace.
4. Heading-Aware Markdown Chunking: Rather than breaking sections in the middle of sentences based on character limits, the ingestion module splits on heading structures. This keeps each document section unified and ensures context labels are accurate.

---

9. Development Guidelines & Best Practices

1. Maintain Type Safety: When updating APIs, verify that models in backend/app/models/schemas.py match interface structures in frontend/src/types/chat.ts.
2. Preserve Service Singletons: Instantiating services directly inside routes is an anti-pattern. Always retrieve shared singletons via the get_* factory methods.
3. Run Re-Ingestion When Changing Embedding Models: Changing the model type changes the dimensions of vectors, which will cause ChromaDB query failures if the index is not rebuilt. Run the ingestion script again to clear and rebuild the database.
4. Keep Context Limits In Mind: Modify the MAX_HISTORY_TURNS if long-context query issues arise, and always enforce task prefixes ("query: " / "passage: ") when modifying similarity query strategies.