walkthrough.md

Email Assistant — Full Project Walkthrough

Everything built so far, organized by component. Last updated: April 4, 2026.

---

1. Project Overview

The AI Email Assistant is an OpenEnv-compliant email triage and response environment built for the OpenEnv Hackathon. It wraps a production-grade FastAPI backend into a Gymnasium-style EmailEnv environment that AI agents can interact with, be benchmarked on, and scored against a dense reward rubric.

graph TD
    A["Agent / LLM"] -- Action --> B["EmailEnv (OpenEnv)"]
    B -- "Observations + Rewards" --> A
    B -- "Internal Calls" --> C["EmailAssistantCore"]
    C -- "AI Logic" --> D["AIEngine"]
    C -- "Persistence" --> E["SQLite Memory"]
    B -- "Task Logic" --> F["Task Handlers"]
    B -- "Scoring" --> G["Rubric Graders"]
    H["Next.js Frontend"] -- "HTTP" --> I["FastAPI Routes"]
    I -- "Delegates" --> C
    J["inference.py"] -- "OpenEnv Client" --> B

---

2. Backend — Core Application (app/)

2.1 Entry Point — main.py

• Creates a FastAPI app (v1.0.0) with:
  • CORS middleware (all origins for dev)
  • API routes from app/routes.py
  • OpenEnv environment routes from app/openenv_env/api.py
  • Optional static frontend mount at /ui
  • Custom Swagger UI at /docs with branded CSS
  • Health + root endpoints

2.2 Service Layer — core.py

The EmailAssistantCore class is the central service powering both HTTP routes and the OpenEnv environment directly. It provides:

Method	Description
fetch_emails()	Fetches unread emails via IMAP and persists them to SQLite
classify_email()	Classifies email intent (Support/Sales/Spam/General) via AIEngine
generate_reply()	Drafts a reply with tone control + thread context from SQLite memory
send_email()	Sends via SMTP and records the result

2.3 AI Engine — ai_engine.py

OpenAI API wrapper (AIEngine class) with:

• Async-friendly execution (runs sync OpenAI SDK in asyncio.to_thread)
• Retry with exponential backoff (3 attempts, 0.6s × 2^i delay)
• classify_intent(): Structured JSON prompting → {intent, confidence, reasoning}
• generate_reply(): Context-aware reply generation with tone control (formal/casual/neutral)
• Graceful fallback: Returns dummy responses when no valid API key is provided
• Auto-detects support for responses.create (newer SDK) vs chat.completions.create

2.4 Memory / Persistence — memory.py

SQLite-backed conversation memory with three tables:

Table	Purpose
emails	Stores incoming emails (indexed by from_email + subject)
generated_replies	Stores AI-generated reply drafts
sent_emails	Logs sent email status (sent/failed)

Key function: load_thread_context() — Retrieves chronological conversation history (user + assistant messages) for a given thread, enabling context-aware replies.

Auto-rebuilds corrupt/invalid SQLite databases on startup.

2.5 API Routes — routes.py

Endpoint	Method	Description
/health	GET	Health check
/emails	GET	Fetch unread emails via IMAP
/classify	POST	Classify email intent
/generate-reply	POST	Generate AI-powered reply
/send	POST	Send email via SMTP

2.6 Supporting Modules

• schemas.py: Pydantic models for EmailMessage, ClassifyRequest/Response, GenerateReplyRequest/Response, SendEmailRequest/Response, Tone, Intent types
• classifier.py: EmailClassifier wrapper around AIEngine.classify_intent()
• email_reader.py: IMAP email fetcher (unread emails from inbox)
• email_sender.py: SMTP email dispatcher
• utils.py: Logging setup, subject normalization

---

3. OpenEnv Environment (app/openenv_env/)

This is the core innovation — the Gymnasium-style environment wrapper.

3.1 Environment — env.py (591 lines)

The EmailEnv class implements Environment[Action, Observation, EmailEnvState]:

Core API:

• reset(task_id) → Returns initial Observation
• step(action) / astep(action) → Returns (Observation, Reward, done, info)
• state property → Full EmailEnvState

Action Handling:

Action Type	What It Does
classify	Classifies email intent (tool-driven or agent-provided)
prioritize	Sets priority for a specific message
reply	Drafts a reply (tool-driven or agent-provided)
send	Sends the drafted reply (dry-run or live SMTP)
skip	Skips email (optionally escalates)

Reward System — Multi-Dimensional Dense Rewards:

reward = 0.3 × classification_score + 0.3 × priority_score + 0.3 × reply_score − penalties

Penalties include: unnecessary duplicate actions, wrong send target, delay/SLA violations, escalation, errors.

Features:

• Simulated time tracking (each action costs time: classify=1min, reply=4min, etc.)
• SLA deadline enforcement with urgency scoring
• Deterministic mode (no LLM needed) + Live mode (uses EmailAssistantCore)
• Dry-run send mode for evaluation

3.2 Tasks — tasks.py

Three benchmark tasks with hardcoded email fixtures:

Task ID	Difficulty	Description	Max Steps
easy_classification	Easy	Classify a single email (expected: "Support")	4
medium_prioritization	Medium	Pick most urgent from 3 emails (invoice w/ 60min deadline)	5
hard_workflow	Hard	Full workflow: classify 2 emails → prioritize → draft → send for VIP outage	10

3.3 Models — models.py

Rich type system built on Pydantic + OpenEnv base types:

• Action (type + payload dict)
• Observation (current email, inbox summary w/ deadlines + urgency, action history, step count, time elapsed)
• InboxSummaryItem (message_id, urgency_score, deadline_minutes, predicted_intent, handled status)
• Reward / RewardComponents (classification, priority, reply scores + penalty dict)
• EmailEnvState (full state: task, time, intents, drafts, sent, history)
• StepInfo / StepResult (episode-level scoring)

3.4 Graders — graders.py

Scoring functions:

• grade_classification(predicted, expected) → 0.0 or 1.0 exact match
• grade_priority(chosen, expected) → 0.0 or 1.0
• grade_reply(body, keywords) → 0.0–1.0 based on keyword coverage

3.5 Deterministic Engine — deterministic_engine.py

Keyword-based, no-LLM classification and reply generation for reproducible evaluation. Enables running the full pipeline without an OpenAI API key.

3.6 API Router — api.py

FastAPI router exposing the OpenEnv environment at /openenv/*:

• POST /openenv/reset → Reset environment
• POST /openenv/step → Execute action
• GET /openenv/state → Get current state

---

4. Extended Server Environment (server/)

my_environment.py (427 lines)

A secondary environment (MyEmailEnvironment) extending EmailEnv with:

• Custom EmailAssistantRubric (categorize: +0.10, respond: +0.50, move_email: +1.00, skip: −0.10, etc.)
• Simulated inbox loaded from server/simulated_emails.json or programmatic fallback
• Extended action types: categorize, respond, create_folder, move_email, tag_email, schedule, archive, delete
• Inbox-zero done condition
• Full state tracking via EmailAssistantState dataclass

---

5. Client & Inference

5.1 Client — client.py

EmailAssistantEnvClient extending OpenEnv's EnvClient:

• Serializes Action → JSON for step payloads
• Parses server responses → StepResult[Observation]
• Parses state → EmailEnvState

5.2 Inference Script — inference.py

Full evaluation loop that:

1. Connects to the running environment server
2. Runs all 3 tasks (easy → medium → hard) or a specific task
3. Uses OpenAI GPT-4o-mini for action selection (with structured JSON prompting)
4. Falls back to deterministic policy when no API key is available
5. Logs step-level and episode-level metrics
6. Reports average reward across all episodes

CLI options: --base-url, --episodes, --max-steps, --model, --task

---

6. Frontend (frontend/)

6.1 Framework

Next.js app with:

• TypeScript
• Tailwind CSS + custom CSS variables
• date-fns for relative timestamps

6.2 Design System — globals.css

Premium "Elite Concierge" glassmorphic design:

Token	Light	Dark
Background	#F9F7F3 (warm off-white)	#0A0A0B (deep black)
Primary	#C49A6C (warm gold)	#E2C28B (soft gold)
Accent	#9B59B6 (purple)	#C39BD3 (lavender)
Surface	White glass (80% opacity)	Dark glass (60% opacity)

Typography: Playfair Display (headings), Inter (body), JetBrains Mono (code)

Components: .glass-card, .glass-panel, .btn-primary, .btn-outline, .btn-icon, .luxury-input, .luxury-tag

All with smooth animations, hover effects, glassmorphism (backdrop-filter blur), and custom scrollbars.

6.3 Main Page — page.tsx

Single-page Luxury Concierge email client:

• Header: "Elite Concierge — Executive Inbox" with tab navigation (Inbox / Concierge)
• Sidebar: Email list with sender, subject, body preview, relative timestamps, intent badges + confidence %
• Main Panel: Selected email details with:
  • "Engage Concierge" button → auto-classifies + drafts reply
  • AI reasoning display
  • Full email body
  • Composition area with editable draft
  • "Dispatch Securely" send button
• Toast notifications with animated pulse dot
• Decorative blur orbs for premium atmosphere

Integrates with backend via: /emails, /classify, /generate-reply, /send

---

7. Extended Backend (backend/)

A separate, more production-focused backend with:

Directory	Purpose
app/api/v1, app/api/v2	Versioned API routers
app/core/	Core business logic
app/services/	16 service modules (see below)
app/db/	Database layer
app/schemas/	Pydantic schemas
app/security/	Authentication & authorization
app/middleware/	Request middleware
app/observability/	Monitoring & logging
app/repositories/	Data access layer
app/workers/	Background workers
app/errors/	Error handling
alembic/	Database migrations

Backend Services (backend/app/services/)

Service	Description
agent_tools.py	AI agent tool definitions
attachments.py	Attachment handling
autopilot.py	Automated email processing
classifier.py	Email classification
evaluation.py	Performance evaluation
events.py	Event system
gmail_api.py	Gmail API integration
graph_api.py	Microsoft Graph API
imap_legacy.py	Legacy IMAP support
llm.py	LLM integration
niche.py	Specialized processing
prioritizer.py	Email prioritization
rag_service.py	RAG-based knowledge retrieval
safety.py	Content safety checks
smtp_send.py	SMTP sending

---

8. OpenEnv Client Package (email_assistant_env/)

Standalone Python package for interacting with the environment:

• client.py — Environment client
• models.py — Shared data models
• pyproject.toml — Package configuration
• README.md — Documentation
• server/ — Server-side environment utilities

---

9. Deployment & Configuration

9.1 Docker — Dockerfile

Multi-stage build:

1. Builder stage: Python 3.11-slim + uv for fast dependency install
2. Runtime stage: Copies installed packages, exposes port 8000
3. Health check: curl -f http://localhost:8000/health
4. Entry: uvicorn app.main:app --host 0.0.0.0 --port 8000

9.2 OpenEnv Config — openenv.yaml

Full OpenEnv-compliant configuration with:

• Hugging Face Spaces metadata (emoji: 📧, SDK: docker)
• Action space schema (ClassifyEmailAction, DraftReplyAction, RequestMoreInfoAction, EscalateAction)
• Observation space schema
• Dense reward definition with component weights
• Task definitions (3 difficulty levels)

9.3 Environment Variables

Configured via .env:

• OPENAI_API_KEY — For AI classification and reply generation
• IMAP/SMTP credentials — For live email integration
• SQLite path — For conversation memory
• Model selection — Default: gpt-4o-mini

---

10. Summary of Key Innovations

1. Multi-Dimensional Dense Reward Rubric: Beyond binary correct/incorrect — rewards classification precision (1.0), reasoning quality (0.2), tone compliance (0.3), workflow efficiency (−0.01/step), full resolution (2.0)
2. Dual Mode: Deterministic (keyword-based, no API key needed) + Live (OpenAI GPT-4o-mini)
3. SLA-Aware Time Simulation: Each action costs simulated time; deadline breaches incur penalties
4. Thread Memory: SQLite-backed conversation context for coherent multi-turn interactions
5. Premium UI: Glassmorphic "Elite Concierge" interface with luxury design tokens
6. Full Pipeline: classify → prioritize → draft → send, all within a Gymnasium-style reset/step loop