ARCHITECTURE.md

CustomerCore System Architecture

This document provides a comprehensive technical overview of the CustomerCore platform's architecture, including its data flows, multi-agent orchestrator, and service topology.

---

Data Flow Diagram

flowchart TD
    subgraph Ingestion ["1. Event Ingestion"]
        API[FastAPI Gateway]
        GH[GitHub Issues API]
        SE[Synthetic Generators]
        RP[Redpanda Broker]
        
        API -->|Publish| RP
        GH -->|Webhooks| RP
        SE -->|Simulate Events| RP
    end

    subgraph Lakehouse ["2. Streaming Lakehouse"]
        SP[PySpark Structured Streaming]
        R2[(Cloudflare R2 Object Storage)]
        Duck[DuckDB / local dbt]
        
        RP -->|Bronze Stream| SP
        SP -->|PII Masking & Silver| R2
        R2 -->|dbt transformation| Duck
    end

    subgraph Modeling ["3. Feature & Model Registry"]
        FS[Feast Feature Store]
        ML[MLflow Experiment Registry]
        Models[8 Trained ML Models]
        
        Duck -->|Gold Marts| FS
        Duck -->|Train Set| ML
        ML -->|Register| Models
    end

    subgraph Routing ["4. AI & Agent Triage Flow"]
        LC[LiteLLM Routing Gateway]
        SC[Semantic Cache L1/L2/L3]
        LG[LangGraph Supervisor Graph]
        Supa[(Supabase pgvector / db)]
        
        API -->|Triage Request| LG
        LG -->|Validate Route| LC
        LC -->|Query Cache| SC
        LG -->|Store Memories| Supa
    end

    subgraph Observability ["5. Platform Observability"]
        PROM[Prometheus Metrics]
        OTEL[OTel Collector]
        GRAF[Grafana Cloud Dashboards]
        LF[Langfuse Cloud Traces]
        
        API -->|Export Metrics| PROM
        PROM --> OTEL --> GRAF
        LG -->|Traces & LLM Costs| LF
    end

    style RP fill:#FFDDDD,stroke:#CC0000,stroke-width:2px
    style R2 fill:#FFE8D6,stroke:#D4A373,stroke-width:2px
    style LG fill:#E8F0FE,stroke:#1A73E8,stroke-width:2px
    style Supa fill:#D1FAE5,stroke:#059669,stroke-width:2px
    style LF fill:#F3E8FF,stroke:#7C3AED,stroke-width:2px

---

The Nine Core Services

CustomerCore consists of nine services divided into logical layers:

1. Stream Ingestion

• Technology: Redpanda (Kafka-compatible event broker)
• Role: Direct ingestion points for four parallel topic streams: tickets, product, billing, and incidents. Custom python helper classes verify broker sockets before publishing.

2. Stream Processing

• Technology: PySpark Structured Streaming
• Role: Sub-minute micro-batch engine processing Bronze-to-Silver data. Resolves PII redaction (email, SSN, phone) using Python UDFs and guarantees strong schema enforcement before flushing to Iceberg format.

3. dbt Transform Layer

• Technology: dbt-core with dbt-duckdb
• Role: Compiles silver tables into 7 separate Gold business marts (customer health metrics, incident durations, billing tiers, etc.) to drive analytics and features.

4. Feature Store

• Technology: Feast (Feature Store)
• Role: Offline feature store manages historical training datasets; online store (Upstash Redis) provides sub-millisecond lookup latency during real-time inference.

5. ML Experiment Registry

• Technology: MLflow hosted on DagsHub
• Role: Trains, registers, and tracks 8 separate models including ticket classifiers (XGBoost), churn risk engines (LightGBM), volume forecasters (Prophet), and anomaly detectors (Isolation Forest).

6. Vector & RAG Service

• Technology: ChromaDB (Dense + BM25 Hybrid Retriever)
• Role: Holds indexed documentation and product knowledge. Realizes hybrid search with Reciprocal Rank Fusion (RRF) and reranks retrieved candidate chunks using a cross-encoder model.

7. Inference API Gateway

• Technology: FastAPI & Uvicorn
• Role: Public REST API endpoints handling triage requests, polling, SSE streaming, and health checks. Enforces strict JWT tenant authentication and rate-limiting.

8. Async Worker Service

• Technology: Celery + Redis
• Role: Manages deferred background tasks such as nightly ChromaDB backup pushes to Cloudflare R2, model cards fairness evaluations, and cache sweeps.

9. Platform Observability

• Technology: OpenTelemetry, Prometheus, Grafana Cloud, Langfuse, LangSmith, Sentry
• Role: Distributed tracking. Metrics collector publishes custom JVM/App metrics (18 signals, 5 dashboards). Langfuse parses token-level prompt performance.

---

LangGraph Multi-Agent Architecture

Triage processing uses a LangGraph supervisor orchestrating six distinct sub-agents:

               +-----------------------+
               |  LangGraph Supervisor |
               +-----------+-----------+
                           |
       +-------------------+-------------------+
       |                   |                   |
+------v-------+    +------v-------+    +------v-------+
|Classify Agent|    | Memory Agent |    |  RAG Agent   |
+------+-------+    +------+-------+    +------+-------+
       |                   |                   |
       +-------------------+-------------------+
                           |
       +-------------------+-------------------+
       |                   |                   |
+------v-------+    +------v-------+    +------v-------+
| Churn Agent  |    |Incident Agent|    |  HITL Agent  |
+--------------+    +--------------+    +--------------+

1. Classify Agent: Classifies tickets and evaluates initial priority.
2. Memory Agent: Interacts with Mem0 using Supabase pgvector to load previous tenant/customer history.
3. RAG Agent: Performs hybrid vector/keyword searches on product guides and draft responses.
4. Churn Agent: Calculates customer churn risk flags from Gold mart metrics.
5. Incident Agent: Detects ongoing service incidents and schedules escalation workflows.
6. HITL (Human-in-the-Loop) Agent: Pauses graph execution using state checkpointers if safety limits are broken, saving state for manual human review.

---

Deployment Modes

CustomerCore is designed to support three distinct operational topologies:

Mode	Target	Infrastructure	Inference Engine
Lite	Development / Testing	Docker Compose (FastAPI, Redis, Chroma)	OpenRouter LLM Cloud API
Full Local	Production Simulation	3-Node Kind Kubernetes Cluster	Local Ollama + GPU (RTX 3050 Ti)
Cloud	Production / Portfolio	Hugging Face Spaces (Docker), Cloudflare R2, Upstash Redis, Supabase DB	Cloud API / OpenRouter