README.md

metadata

title: EmailTriage Environment Server
emoji: 📬
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
app_port: 8000
base_path: /web
tags:
  - openenv
  - rl
  - email-triage
  - multi-agent

📬 EmailTriage: An OpenEnv Reinforcement Learning Environment

A dynamic, multi-turn OpenEnv environment designed for strict, realistic email triage orchestration. This environment acts as a standard step(), reset(), and state() API that grades AI agents on how efficiently they can navigate a turbulent inbox.

This project was built explicitly to fulfill the OpenEnv Hackathon Challenge. It passes all openenv validate pre-submission testing checks and is deployed universally via a Dockerfile container on Hugging Face.

---

🎯 Core Problem Statement & Environment Logic

What does this test?

Unlike standard mini-games, this environment tests complex NLP reasoning and chronological tool-use. The agent must:

• Detect text context to differentiate between Spam/Marketing vs Escalated P1 Outages.
• Accurately trigger archive methods to clean the inbox loop without sacrificing important messages.
• Dynamically read emails, followed by executing a query_calendar sync, and finishing with an accurate draft_email containing a matched date-time string logic.

⚡ Huge Scenario Pool!

The internal environment has a massive deterministic dictionary of over 50 unique real-world emails ranging from:

• Angry CEOs making demands
• Fraudulent Nigerian Princes
• Legitimate B2B Vendor negotiations
• DataDog Server crashing alerts

---

🚀 The Three Task Difficulties

ID	Name	Initial Inbox Size	Max Steps	Dynamic Events (Mid-Episode)
easy	Quick Sort	Exactly 3	6 Iterations	❌
medium	Priority Triage	Exactly 5	10 Iterations	❌
hard	Dynamic Crisis	7 to 10	12 Iterations	✅ (See Below)

🌪️ Hard Mode Dynamic Events

If the AI engages in Hard Mode, the environment is programmed to actively combat the agent mid-episode:

1. Interrupts: Exactly on Step 3, a brand new High Priority email from the CEO is forcefully appended to the unread inbox queue.
2. Calendar Shifts: Mid-episode, an available meeting slot might spontaneously disappear, forcing the AI to re-scan its options before generating a draft scheduling email.

---

🔑 Action & Observation Schema Specifications

The Agent's Action Hook (EmailtriageAction)

The AI must reply in strictly validated JSON matching these arguments over the WebSocket:

• action_type: Strictly mapped to read, archive, query_calendar, or draft_email.
• target_email_id: Matches an Integer directly mapped to the dynamically changing Inbox observation. (Use -1 for generic commands like querying).
• draft_content: Minimum 40-character String. Scored heuristically on politeness ("thank you"), subject matching, and structural completion periods.
• proposed_slot: String matching a Date-Time object scraped from the calendar.

The Observation Space (EmailtriageObservation)

In order to prevent LLM hallucination cheating, the Observation payload strictly returns [:5] of the array (Only showing the first 5 unread items on "Screen 1"). The email Body is heavily redacted until the AI spends a turn executing the read action!

• inbox_preview: Summaries of up to 5 unread emails
• returned_emails: Contains the full string body unlocked by manual read actions
• calendar_slots: Array of currently available dates
• last_action_result: The immediate Grader string feedback returned to the AI
• reward: Floating integer normalized between [0.0, 1.0]

---

⚖️ The Grader & Reward Logic

Continuous [0, 1] rewards granting partial credit based on deterministic logic matching:

• Archiving Logic: +0.62 to +0.80 for accurately archiving Spam and Newsletters. Massive negative penalty for archiving High Priority client emails.
• Reading Logic: +0.09 to +0.25 for pulling data into focus (High Priority messages score higher).
• Drafting Workflow: Heavy reward weighting +0.55+ if the AI perfectly maps read -> query_calendar -> draft_email seamlessly. Partial deductions for proposing unavailable chronological slots or missing urgent keywords ("today").

---

🛠️ Quick Start & Setup Instructions

1. Run inference locally via Uvicorn Server

Because this project utilizes modern OpenEnv >= 0.2.2, all environment interactions are fully asynchronous.

Boot the backend HTTP server:

uvicorn server.app:app --host 0.0.0.0 --port 8000 --reload

Trigger the OpenEnv SDK baseline:

import asyncio
from openai import OpenAI
from server.EmailTriage_environment import EmailtriageEnv

async def main():
    llm = OpenAI(base_url="https://router.huggingface.co/v1")
    # Native connection without Docker bug risks locally
    env = EmailtriageEnv(base_url="http://127.0.0.1:8000")
    await env.connect()
    
    # Let the LLM play the environment
    await env.reset(options={"task_id": "hard"})
    
asyncio.run(main())

2. Evaluating for Hackathon Judges

This repository natively supports openenv validate from the root structure and natively passes the validate-submission.sh Hugging Face pipeline test.

To compile the environment into the evaluation Docker instance:

docker build -t emailtriage-env:latest -f Dockerfile .

To sync changes actively up to Hugging Face:

openenv push --repo-id YOUR_USERNAME/EmailTriage