docs/ARCHITECTURE.md

Architecture Overview

System Design

The Multi-Agent System uses LangGraph to orchestrate a collaborative workflow of specialized AI agents that work together to decompose and execute complex tasks.

User Request
    │
    ▼
┌─────────────────────────────────────────────────────────┐
│                    LangGraph Workflow                   │
│                                                         │
│  ┌──────────────┐    ┌──────────────┐  ┌────────────┐   │
│  │    Memory    │───▶│   Planner    │─▶│  Executor │   │
│  │  Retrieval   │    │    Agent     │  │   Agent    │   │
│  └──────────────┘    └──────────────┘  └─────┬──────┘   │
│                           ▲                   │         │
│                           │ replan            │ loop    │
│                      ┌────┴────┐        ┌─────┴──┐      │
│                      │  Critic │◀───────Executor │     │   
│                      │  Agent  │        │ Tools  │     │
│                      └────┬────┘        └────────┘     │
│                           │ approve                    │
│                           ▼                            │
│                    ┌──────────────┐                    │
│                    │    Memory    │                    │
│                    │    Store     │                    │
│                    └──────────────┘                    │
└─────────────────────────────────────────────────────────┘
    │
    ▼
  Response

Components

1. Agents

Memory Agent

• Purpose: Retrieve relevant past experiences
• Model: gpt-4o-mini
• Pattern: Two-tier retrieval (Redis hot cache + SQLite cold storage)
• Output: List of relevant past learnings

Planner Agent

• Purpose: Decompose tasks into executable steps
• Model: gpt-4
• Pattern: Function calling with structured JSON output
• Output: Ordered plan with dependencies

Executor Agent

• Purpose: Execute individual steps with available tools
• Model: gpt-4
• Pattern: Tool use with retry logic and backoff
• Output: Step results with traceability

Critic Agent

• Purpose: Evaluate task completion quality
• Model: gpt-4
• Pattern: Separate evaluator prevents self-bias
• Output: Approval or rejection with feedback

Memory Store

• Purpose: Persist learnings for future tasks
• Model: gpt-4o-mini
• Pattern: Extract semantic and episodic knowledge
• Storage: Hybrid Redis + SQLite

2. Tools

Available tools for the Executor agent:

Tool	Type	Description	Rate Limited
web_search	External	DuckDuckGo search	Yes
fetch_url	External	Read URL content	Yes
calculate	Local	Safe math evaluation	No
run_python	Sandboxed	Execute Python code	Yes

3. State Management

# Task state throughout the workflow
{
    "task_id": "uuid",
    "task": "original task",
    "status": TaskStatus.PENDING,
    "plan": [...],
    "steps_completed": 0,
    "current_step": None,
    "result": None,
    "errors": [],
    "events": [],
    "total_tokens": 0,
    "memory_context": []
}

4. Database Schema

PostgreSQL (Production)

-- Tasks table
CREATE TABLE tasks (
    id UUID PRIMARY KEY,
    task TEXT,
    status VARCHAR(50),
    result JSONB,
    created_at TIMESTAMP,
    updated_at TIMESTAMP,
    deleted_at TIMESTAMP
);

-- Memory entries
CREATE TABLE memories (
    id UUID PRIMARY KEY,
    task_id UUID REFERENCES tasks(id),
    content TEXT,
    embedding VECTOR(1536),  -- OpenAI embeddings
    memory_type VARCHAR(50),  -- episodic, semantic
    created_at TIMESTAMP
);

Redis (Caching)

Key patterns:
- task:{task_id}:state     → Current state
- task:{task_id}:status    → Quick status lookup
- memory:{task_type}       → Hot memory cache
- queue:pending            → Task queue

5. API Architecture

FastAPI Application
├── /health          → Health checks
├── /tasks           → Task management
│   ├── POST /       → Create task
│   ├── GET /{id}    → Get status
│   └── DELETE /{id} → Cancel
├── /workflows       → Workflow execution
│   ├── POST /execute → Run workflow
│   └── GET /{id}/status → Status
└── /docs            → Interactive docs

Data Flow

Task Execution Flow

1. Input: User submits task via API
2. Memory: Retrieve relevant past experience
3. Planning: Decompose into steps
4. Execution Loop:
   • Select next step
   • Choose tool/approach
   • Execute with retry logic
   • Store intermediate result
   • Check completion
5. Evaluation: Critic validates solution
6. Feedback:
   • If approved → Store learnings
   • If rejected → Replan
7. Output: Return results to user

State Transitions

PENDING
  │
  ├─▶ PLANNING (Planner agent)
  │     │
  │     ├─▶ EXECUTING (Executor agent)
  │     │     │
  │     │     ├─▶ EVALUATING (Critic agent)
  │     │     │     │
  │     │     │     ├─▶ REPLANNING (loop back)
  │     │     │     └─▶ STORING (Memory agent)
  │     │     │           │
  │     │     │           └─▶ COMPLETED
  │     │     │
  │     │     └─▶ FAILED
  │     │
  │     └─▶ FAILED
  │
  └─▶ FAILED

Performance Considerations

Latency

• First response: 2-5 seconds (planning phase)
• Per step: 1-3 seconds (execution)
• Evaluation: 1-2 seconds
• Total typical task: 30-120 seconds

Memory Usage

• Base: ~200MB
• Per concurrent task: ~50MB
• Redis memory: ~100MB default
• Database: Depends on data volume

Scaling Limits

• Requests/second: Limited by LLM API rate limits
• Concurrent tasks: ~10-100 (depends on compute)
• Database: 1M+ tasks (PostgreSQL)
• Cache: Millions of memories (Redis)

Error Handling

Retry Strategy

# Exponential backoff with jitter
max_retries = 3
base_delay = 1.0
max_delay = 32.0

delay = min(base_delay * (2 ** attempt) + random(0, 1), max_delay)

Fallback Strategies

1. Tool failure: Try alternative tool or manual approach
2. Step failure: Skip or add to error log
3. Planning failure: Use simpler, direct approach
4. Critic rejection: Auto-replan or escalate

Security Architecture

API Security

• API Key authentication (future)
• Rate limiting per user
• Request validation (Pydantic)
• CORS enabled for frontend only

Data Security

• Environment variables for secrets
• Encrypted database connections
• No sensitive data in logs
• Memory isolation between tasks

Monitoring & Observability

Metrics

• task_total - Total tasks processed
• task_duration_seconds - Execution time
• agent_calls_total - Agent invocations
• api_requests_total - API endpoints hit
• memory_hit_ratio - Cache effectiveness

Logs

• Structured JSON logging
• Trace IDs for request tracking
• Agent decision tracking
• Tool execution logs

Health Checks

• Database connectivity
• Redis connectivity
• LLM API availability
• API responsiveness

Deployment Patterns

Development

• SQLite local storage
• In-memory cache
• Console logging
• Hot reload

Production

• PostgreSQL database
• Redis cache cluster
• Centralized logging (ELK/Datadog)
• Load balancer
• Auto-scaling