Initial - Commit

Dockerfile

@@ -0,0 +1,45 @@
+# ── Support Ticket Triage — OpenEnv Dockerfile ─────────────────────────────
+# Targets HuggingFace Spaces (Docker SDK).
+# Build:  docker build -t ticket-triage-env .
+# Run:    docker run -p 7860:7860 ticket-triage-env
+# ---------------------------------------------------------------------------
+
+FROM python:3.11-slim
+
+LABEL org.opencontainers.image.title="Support Ticket Triage — OpenEnv"
+LABEL org.opencontainers.image.description="OpenEnv environment for AI support ticket triage"
+LABEL space_sdk="docker"
+
+# --- system deps ---
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# --- app user (HF Spaces runs as non-root) ---
+RUN useradd -m -u 1000 appuser
+WORKDIR /app
+
+# --- Python dependencies ---
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# --- Application code ---
+COPY env/ ./env/
+COPY server.py .
+COPY openenv.yaml .
+COPY inference.py .
+COPY README.md .
+
+# Transfer ownership
+RUN chown -R appuser:appuser /app
+USER appuser
+
+# --- HuggingFace Spaces expects port 7860 ---
+EXPOSE 7860
+
+# Healthcheck so HF knows when the server is ready
+HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=3 \
+    CMD curl -f http://localhost:7860/ || exit 1
+
+# --- Launch the FastAPI server ---
+CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "7860", "--workers", "2"]

README.md

@@ -0,0 +1,277 @@
+# 🎫 Support Ticket Triage — OpenEnv
+
+A real-world [OpenEnv](https://huggingface.co/openenv) environment where AI agents must
+**triage, route, and resolve customer support tickets** across three difficulty levels.
+
+This environment models a task that real support organisations handle thousands of times
+per day: reading an incoming ticket, routing it to the right team, judging its urgency,
+crafting a helpful reply, and — for hard tasks — managing a multi-turn conversation
+through to resolution.
+
+---
+
+## 🌍 Motivation
+
+Support ticket triage is:
+
+- **High-volume** — enterprise companies route millions of tickets per year
+- **High-stakes** — wrong routing costs money; slow responses lose customers
+- **Multi-step** — requires reading comprehension, classification, and generation
+- **Under-explored** in RL/agent benchmarks (most focus on code or web tasks)
+
+This environment fills a genuine gap: an OpenEnv where agents can be trained and
+evaluated on a real knowledge-worker workflow.
+
+---
+
+## 🏗️ Environment Overview
+
+| Field | Value |
+|-------|-------|
+| Action space | Discrete (7 action types) + optional text fields |
+| Observation space | Structured ticket + conversation history |
+| Reward | Shaped per-step + terminal grader score [0.0, 1.0] |
+| Episodes | Stateful, multi-step (up to 12 steps for Hard) |
+| Tasks | 3 (Easy / Medium / Hard) |
+
+---
+
+## 📋 Tasks
+
+### Task 1 — Ticket Routing *(Easy)*
+
+> Route the incoming ticket to the correct department.
+
+- **Actions required**: `ROUTE`
+- **Score**: 1.0 for correct department, 0.1 for wrong (partial), 0.0 for no attempt
+- **Departments**: `billing`, `technical_support`, `sales`, `customer_success`, `legal`
+- **Max steps**: 3
+- **Baseline score**: ~0.80
+
+### Task 2 — Full Triage *(Medium)*
+
+> Fully triage the ticket: route, set urgency, tag, and respond.
+
+| Sub-task | Weight |
+|----------|--------|
+| Correct routing | 30% |
+| Correct urgency level | 25% |
+| Relevant tags applied | 20% |
+| Informative customer response | 25% |
+
+- **Max steps**: 8
+- **Baseline score**: ~0.55
+
+### Task 3 — Full Resolution *(Hard)*
+
+> Manage a multi-turn support conversation to full resolution.
+
+| Sub-task | Weight |
+|----------|--------|
+| Correct routing | 15% |
+| Correct urgency | 10% |
+| Quality initial response | 20% |
+| Escalation decision | 20% |
+| Handle customer follow-up | 20% |
+| Close with resolution note | 15% |
+
+- **Max steps**: 12
+- **Baseline score**: ~0.40
+
+---
+
+## 🔌 API Reference
+
+The environment is served as a REST API (FastAPI).
+
+### `GET /`
+Health check. Returns environment metadata and task list.
+
+### `POST /reset`
+Start a new episode.
+
+```json
+{
+  "task_name": "route",        // "route" | "triage" | "resolve"
+  "ticket_id": "TKT-001",     // optional — omit for random
+  "seed": 42,                  // optional RNG seed
+  "session_id": "abc123"       // optional — generated if omitted
+}
+```
+
+Returns: `{ "observation": {...}, "session_id": "..." }`
+
+### `POST /step`
+Apply an action.
+
+```json
+{
+  "session_id": "abc123",
+  "action_type": "route",          // required
+  "department": "billing",          // for ROUTE
+  "response_text": "Hello...",      // for RESPOND
+  "urgency": "high",                // for SET_URGENCY
+  "tags": ["billing", "refund"],    // for TAG
+  "escalation_reason": "...",       // for ESCALATE
+  "resolution_note": "..."          // for CLOSE
+}
+```
+
+Returns: `{ "observation": {...}, "reward": {...}, "done": bool, "info": {...}, "session_id": "..." }`
+
+### `GET /state?session_id=abc123`
+Full internal state including ground truth labels (for debugging/evaluation).
+
+### `GET /tasks`
+List all tasks with metadata.
+
+---
+
+## 🎬 Action Space
+
+| action_type | Required fields | Description |
+|-------------|----------------|-------------|
+| `route` | `department` | Route ticket to a department |
+| `set_urgency` | `urgency` | Set priority level |
+| `respond` | `response_text` | Send a message to the customer |
+| `tag` | `tags` | Apply classification labels |
+| `escalate` | `escalation_reason` | Escalate with explanation |
+| `close` | `resolution_note` | Resolve and close the ticket |
+| `noop` | — | Take no action (wastes a step) |
+
+**Departments**: `billing` · `technical_support` · `sales` · `customer_success` · `legal`
+
+**Urgency levels**: `low` · `medium` · `high` · `critical`
+
+---
+
+## 👁️ Observation Space
+
+```json
+{
+  "ticket_id": "TKT-001",
+  "subject": "Double charged on my invoice",
+  "body": "Full ticket text...",
+  "sender_email": "user@example.com",
+  "sender_name": "Jane Smith",
+  "conversation_history": [
+    {"sender": "Jane Smith", "content": "...", "timestamp": "2024-01-01T12:00:00Z"}
+  ],
+  "current_department": null,
+  "current_urgency": null,
+  "tags": [],
+  "is_escalated": false,
+  "is_closed": false,
+  "step_number": 0,
+  "task_name": "route",
+  "task_description": "Route the ticket to the correct department...",
+  "available_actions": ["route", "respond", "set_urgency", "tag", "escalate", "close", "noop"]
+}
+```
+
+---
+
+## 🏆 Reward Function
+
+**Step rewards** (shaped, provide dense signal):
+- +0.30 — correct ROUTE
+- +0.20 — correct SET_URGENCY
+- +0.10×overlap — TAG matching required tags
+- +0.15×quality — RESPOND addressing key topics
+- +0.20 — justified ESCALATE
+- −0.10 — unjustified ESCALATE
+- +0.10 — CLOSE with substantive resolution note
+
+**Terminal reward** (authoritative, [0.0, 1.0]):
+Each task has a dedicated deterministic grader that computes a weighted aggregate
+of all sub-task scores. The terminal reward is returned in `info["final_grader_reward"]`.
+
+---
+
+## 🚀 Setup & Usage
+
+### Local development
+
+```bash
+# Install dependencies
+pip install -r requirements.txt
+
+# Start the environment server
+python server.py
+# → Server running at http://localhost:7860
+
+# In another terminal, run the baseline inference
+export API_BASE_URL="https://router.huggingface.co/v1"
+export MODEL_NAME="meta-llama/Llama-3.3-70B-Instruct"
+export HF_TOKEN="your_token_here"
+export ENV_BASE_URL="http://localhost:7860"
+
+python inference.py
+```
+
+### Docker
+
+```bash
+docker build -t ticket-triage-env .
+docker run -p 7860:7860 ticket-triage-env
+
+# Environment is now available at http://localhost:7860
+```
+
+### Quick API test
+
+```bash
+# Health check
+curl http://localhost:7860/
+
+# Start an episode
+curl -X POST http://localhost:7860/reset \
+  -H "Content-Type: application/json" \
+  -d '{"task_name": "route", "ticket_id": "TKT-001", "seed": 42}'
+
+# Take an action (use session_id from reset response)
+curl -X POST http://localhost:7860/step \
+  -H "Content-Type: application/json" \
+  -d '{"session_id": "<ID>", "action_type": "route", "department": "billing"}'
+```
+
+---
+
+## 📊 Baseline Scores
+
+Measured with `meta-llama/Llama-3.3-70B-Instruct` via HuggingFace Inference API,
+temperature=0.0 (greedy), seed=42:
+
+| Task | Score | Notes |
+|------|-------|-------|
+| Route (Easy) | ~0.80 | Model occasionally confuses billing ↔ customer_success |
+| Triage (Medium) | ~0.55 | Tags and urgency are hardest sub-tasks |
+| Resolve (Hard) | ~0.40 | Follow-up handling and escalation decisions are challenging |
+| **Overall** | **~0.58** | |
+
+---
+
+## 📁 Project Structure
+
+```
+ticket-triage-env/
+├── openenv.yaml          # OpenEnv metadata
+├── Dockerfile            # Container definition
+├── requirements.txt      # Python dependencies
+├── inference.py          # Baseline inference script (hackathon-required)
+├── server.py             # FastAPI HTTP server
+├── README.md             # This file
+└── env/
+    ├── __init__.py
+    ├── environment.py    # Core TicketTriageEnv class
+    ├── models.py         # Pydantic Observation/Action/Reward models
+    ├── tasks.py          # Task specifications
+    ├── graders.py        # Deterministic grader functions
+    └── data.py           # Synthetic ticket dataset with ground truth
+```
+
+---
+
+## 📜 License
+
+MIT — free to use for research and commercial applications.

data.py

@@ -0,0 +1,216 @@
+"""
+Curated synthetic support tickets with ground-truth labels.
+
+Each ticket is a dict with:
+  - ticket fields (subject, body, sender_*)
+  - ground_truth:
+      correct_department   : Department enum value
+      correct_urgency      : UrgencyLevel enum value
+      required_tags        : set of tags graders accept
+      key_response_topics  : keywords a good response must address
+      needs_escalation     : bool
+"""
+
+from env.models import Department, UrgencyLevel
+
+TICKETS = [
+    # -----------------------------------------------------------------------
+    # EASY routing tickets — one clear signal per email
+    # -----------------------------------------------------------------------
+    {
+        "ticket_id": "TKT-001",
+        "subject": "Double charged on my invoice #4821",
+        "body": (
+            "Hi, I was charged twice for my subscription this month. "
+            "My invoice number is #4821. The duplicate charge appeared on "
+            "March 15th. Please refund the extra amount ASAP. "
+            "My account email is jane.smith@example.com."
+        ),
+        "sender_email": "jane.smith@example.com",
+        "sender_name": "Jane Smith",
+        "ground_truth": {
+            "correct_department": Department.BILLING,
+            "correct_urgency": UrgencyLevel.HIGH,
+            "required_tags": {"billing", "duplicate-charge", "refund"},
+            "key_response_topics": {"refund", "invoice", "charge", "apologize"},
+            "needs_escalation": False,
+            "good_resolution_keywords": {"refund processed", "resolved", "credited"},
+        },
+    },
+    {
+        "ticket_id": "TKT-002",
+        "subject": "API returning 500 errors since yesterday",
+        "body": (
+            "Our production environment is broken. Your API has been returning "
+            "HTTP 500 errors on the /v2/users endpoint since yesterday 6pm UTC. "
+            "This is blocking our entire checkout flow. Error code: INTERNAL_500_USR. "
+            "We need this fixed urgently."
+        ),
+        "sender_email": "ops@startup.io",
+        "sender_name": "DevOps Team",
+        "ground_truth": {
+            "correct_department": Department.TECHNICAL_SUPPORT,
+            "correct_urgency": UrgencyLevel.CRITICAL,
+            "required_tags": {"api", "500-error", "production-outage"},
+            "key_response_topics": {"error", "investigating", "update", "escalate"},
+            "needs_escalation": True,
+            "good_resolution_keywords": {"resolved", "fix deployed", "restored"},
+        },
+    },
+    {
+        "ticket_id": "TKT-003",
+        "subject": "Interested in enterprise pricing for 500 seats",
+        "body": (
+            "Hello, we are evaluating your platform for our company of ~500 people. "
+            "Could you share enterprise pricing, volume discounts, and whether you "
+            "offer annual contracts? We'd also love a demo call with your team."
+        ),
+        "sender_email": "procurement@bigcorp.com",
+        "sender_name": "Sarah Johnson",
+        "ground_truth": {
+            "correct_department": Department.SALES,
+            "correct_urgency": UrgencyLevel.MEDIUM,
+            "required_tags": {"enterprise", "pricing", "demo-request"},
+            "key_response_topics": {"pricing", "demo", "enterprise", "contact"},
+            "needs_escalation": False,
+            "good_resolution_keywords": {"demo scheduled", "pricing sent", "follow-up"},
+        },
+    },
+    {
+        "ticket_id": "TKT-004",
+        "subject": "Need help setting up SSO with Okta",
+        "body": (
+            "We are trying to configure SAML SSO with Okta but keep getting "
+            "'Assertion validation failed' errors. We have followed the docs "
+            "but step 4 on the SAML config page seems outdated. Can you help?"
+        ),
+        "sender_email": "it-admin@mediumco.org",
+        "sender_name": "IT Admin",
+        "ground_truth": {
+            "correct_department": Department.TECHNICAL_SUPPORT,
+            "correct_urgency": UrgencyLevel.MEDIUM,
+            "required_tags": {"sso", "saml", "okta", "configuration"},
+            "key_response_topics": {"saml", "configuration", "steps", "guide"},
+            "needs_escalation": False,
+            "good_resolution_keywords": {"configured", "working", "resolved"},
+        },
+    },
+    {
+        "ticket_id": "TKT-005",
+        "subject": "Data retention policy — GDPR request",
+        "body": (
+            "We are undergoing a GDPR audit and need your written data retention "
+            "and deletion policy. Specifically: how long do you retain user logs, "
+            "do you have a DPA we can sign, and what is your sub-processor list?"
+        ),
+        "sender_email": "dpo@eucompany.de",
+        "sender_name": "Klaus Weber",
+        "ground_truth": {
+            "correct_department": Department.LEGAL,
+            "correct_urgency": UrgencyLevel.HIGH,
+            "required_tags": {"gdpr", "legal", "data-retention", "compliance"},
+            "key_response_topics": {"gdpr", "dpa", "data retention", "legal team"},
+            "needs_escalation": False,
+            "good_resolution_keywords": {"dpa signed", "policy sent", "compliant"},
+        },
+    },
+    # -----------------------------------------------------------------------
+    # MEDIUM tickets — ambiguous signal, multi-action required
+    # -----------------------------------------------------------------------
+    {
+        "ticket_id": "TKT-006",
+        "subject": "My account is locked and I have a board demo in 2 hours",
+        "body": (
+            "I can't log in to my account — it says 'account suspended'. "
+            "I have a critical board presentation in 2 hours where I need to "
+            "show your platform live. I'm a paying Pro subscriber (since 2021). "
+            "Please unlock immediately. This is extremely time-sensitive."
+        ),
+        "sender_email": "ceo@fastgrowth.com",
+        "sender_name": "Marcus Rivera",
+        "ground_truth": {
+            "correct_department": Department.CUSTOMER_SUCCESS,
+            "correct_urgency": UrgencyLevel.CRITICAL,
+            "required_tags": {"account-locked", "urgent", "enterprise-customer"},
+            "key_response_topics": {"unlock", "immediate", "apologize", "escalate"},
+            "needs_escalation": True,
+            "good_resolution_keywords": {"unlocked", "restored access", "resolved"},
+        },
+    },
+    {
+        "ticket_id": "TKT-007",
+        "subject": "Cancellation request + refund for annual plan",
+        "body": (
+            "I'd like to cancel my annual subscription and get a pro-rated refund "
+            "for the remaining 8 months. I'm leaving because the reporting features "
+            "don't meet our needs. Invoice #9034, paid $1,200. "
+            "Please confirm the cancellation and refund timeline."
+        ),
+        "sender_email": "finance@retailer.biz",
+        "sender_name": "Patricia Lee",
+        "ground_truth": {
+            "correct_department": Department.BILLING,
+            "correct_urgency": UrgencyLevel.MEDIUM,
+            "required_tags": {"cancellation", "refund", "annual-plan", "churn-risk"},
+            "key_response_topics": {"cancellation", "refund", "timeline", "confirm"},
+            "needs_escalation": False,
+            "good_resolution_keywords": {"cancelled", "refund issued", "confirmed"},
+        },
+    },
+    # -----------------------------------------------------------------------
+    # HARD tickets — multi-turn, escalation, full resolution required
+    # -----------------------------------------------------------------------
+    {
+        "ticket_id": "TKT-008",
+        "subject": "Data loss — all our project files are gone",
+        "body": (
+            "URGENT: All project files in workspace 'Acme-Q1' have disappeared. "
+            "Last backup shown was 3 days ago but we've been working daily. "
+            "We have a client deadline TOMORROW morning. This is catastrophic. "
+            "Account: acme@enterprise.com, workspace ID: ws-39182."
+        ),
+        "sender_email": "acme@enterprise.com",
+        "sender_name": "Acme Corp",
+        "ground_truth": {
+            "correct_department": Department.TECHNICAL_SUPPORT,
+            "correct_urgency": UrgencyLevel.CRITICAL,
+            "required_tags": {"data-loss", "critical", "enterprise", "backup"},
+            "key_response_topics": {"data", "recovery", "escalate", "urgent", "backup"},
+            "needs_escalation": True,
+            "good_resolution_keywords": {"data restored", "files recovered", "resolved"},
+            "follow_up_message": (
+                "Thank you for the quick response. We found some files are back "
+                "but project 'Acme-Q1-Sprint3' is still missing. Can you check again?"
+            ),
+            "follow_up_response_topics": {"specific project", "sprint3", "checking"},
+        },
+    },
+    {
+        "ticket_id": "TKT-009",
+        "subject": "Billing discrepancy + threat of chargeback",
+        "body": (
+            "I've been charged $299/month for the past 6 months but my contract "
+            "clearly states $199/month. Total overcharge: $600. I have the signed "
+            "contract. If this isn't resolved with a full refund within 48 hours "
+            "I will dispute all 6 charges with my bank. Account: robert@agency.co"
+        ),
+        "sender_email": "robert@agency.co",
+        "sender_name": "Robert Chen",
+        "ground_truth": {
+            "correct_department": Department.BILLING,
+            "correct_urgency": UrgencyLevel.CRITICAL,
+            "required_tags": {"billing-dispute", "chargeback-risk", "contract", "refund"},
+            "key_response_topics": {"contract", "overcharge", "refund", "investigate", "apologize"},
+            "needs_escalation": True,
+            "good_resolution_keywords": {"$600 refunded", "corrected", "resolved", "apologize"},
+            "follow_up_message": (
+                "I've been waiting 24 hours with no update on my refund. "
+                "I'm filing the chargeback now if I don't hear back."
+            ),
+            "follow_up_response_topics": {"refund status", "timeline", "processing", "urgent"},
+        },
+    },
+]
+
+# Build a lookup dict for easy access
+TICKET_LOOKUP = {t["ticket_id"]: t for t in TICKETS}

environment.py

@@ -0,0 +1,358 @@
+"""
+Support Ticket Triage — OpenEnv Environment
+============================================
+
+Implements the standard OpenEnv interface:
+  reset() -> TicketObservation
+  step(action: TicketAction) -> (TicketObservation, TicketReward, bool, dict)
+  state() -> EnvironmentState
+"""
+
+from __future__ import annotations
+
+import copy
+import random
+from datetime import datetime, timezone
+from typing import Any, Dict, List, Optional, Tuple
+
+from env.data import TICKET_LOOKUP, TICKETS
+from env.graders import GRADERS
+from env.models import (
+    ActionType,
+    Department,
+    EnvironmentState,
+    TicketAction,
+    TicketMessage,
+    TicketObservation,
+    TicketReward,
+    UrgencyLevel,
+)
+from env.tasks import ALL_TASKS, TASK_LOOKUP, TaskSpec
+
+AVAILABLE_ACTIONS = [at.value for at in ActionType]
+AVAILABLE_DEPARTMENTS = [d.value for d in Department]
+AVAILABLE_URGENCIES = [u.value for u in UrgencyLevel]
+
+
+class TicketTriageEnv:
+    """
+    OpenEnv-compliant environment for Support Ticket Triage.
+
+    Args:
+        task_name: One of "route", "triage", "resolve". Default: "route".
+        ticket_id: Pin to a specific ticket (for reproducibility). 
+                   If None, picks randomly from the task's pool.
+        seed: Optional RNG seed.
+    """
+
+    # ------------------------------------------------------------------
+    # Construction
+    # ------------------------------------------------------------------
+
+    def __init__(
+        self,
+        task_name: str = "route",
+        ticket_id: Optional[str] = None,
+        seed: Optional[int] = None,
+    ) -> None:
+        if task_name not in TASK_LOOKUP:
+            raise ValueError(
+                f"Unknown task '{task_name}'. "
+                f"Choose from: {list(TASK_LOOKUP.keys())}"
+            )
+        self._task_spec: TaskSpec = TASK_LOOKUP[task_name]
+        self._pinned_ticket_id: Optional[str] = ticket_id
+        self._rng = random.Random(seed)
+
+        # Episode state (initialised on reset)
+        self._ticket_data: Dict[str, Any] = {}
+        self._observation: Optional[TicketObservation] = None
+        self._actions_taken: List[Dict[str, Any]] = []
+        self._step_number: int = 0
+        self._done: bool = False
+        self._cumulative_reward: float = 0.0
+        self._follow_up_injected: bool = False
+
+    # ------------------------------------------------------------------
+    # OpenEnv API
+    # ------------------------------------------------------------------
+
+    def reset(self) -> TicketObservation:
+        """Reset the environment and return the initial observation."""
+        # Pick ticket
+        if self._pinned_ticket_id:
+            ticket_id = self._pinned_ticket_id
+        else:
+            ticket_id = self._rng.choice(self._task_spec.ticket_ids)
+
+        self._ticket_data = copy.deepcopy(TICKET_LOOKUP[ticket_id])
+        self._actions_taken = []
+        self._step_number = 0
+        self._done = False
+        self._cumulative_reward = 0.0
+        self._follow_up_injected = False
+
+        self._observation = TicketObservation(
+            ticket_id=ticket_id,
+            subject=self._ticket_data["subject"],
+            body=self._ticket_data["body"],
+            sender_email=self._ticket_data["sender_email"],
+            sender_name=self._ticket_data["sender_name"],
+            conversation_history=[
+                TicketMessage(
+                    sender=self._ticket_data["sender_name"],
+                    content=self._ticket_data["body"],
+                    timestamp=self._now(),
+                )
+            ],
+            current_department=None,
+            current_urgency=None,
+            tags=[],
+            is_escalated=False,
+            is_closed=False,
+            step_number=0,
+            task_name=self._task_spec.name,
+            task_description=self._task_spec.description,
+            available_actions=AVAILABLE_ACTIONS,
+        )
+        return copy.deepcopy(self._observation)
+
+    def step(
+        self, action: TicketAction
+    ) -> Tuple[TicketObservation, TicketReward, bool, Dict[str, Any]]:
+        """
+        Apply an action and return (observation, reward, done, info).
+
+        Reward is shaped per-step; the terminal reward summarises the episode.
+        """
+        if self._done:
+            raise RuntimeError("Episode is done. Call reset() to start a new one.")
+        if self._observation is None:
+            raise RuntimeError("Call reset() before step().")
+
+        self._step_number += 1
+        self._observation.step_number = self._step_number
+
+        step_reward, info = self._apply_action(action)
+        self._actions_taken.append(action.model_dump())
+
+        # Check terminal conditions
+        done = self._check_done(action)
+        self._done = done
+
+        # On terminal step: compute final episode reward from grader
+        if done:
+            final_reward = self._run_grader()
+            self._cumulative_reward += final_reward.value
+            info["final_grader_reward"] = final_reward.model_dump()
+            terminal_reward = final_reward
+        else:
+            # Mid-episode shaped reward
+            self._cumulative_reward += step_reward.value
+            terminal_reward = step_reward
+
+        # Possibly inject a follow-up message from the customer (hard task)
+        self._maybe_inject_follow_up(action)
+
+        return copy.deepcopy(self._observation), terminal_reward, done, info
+
+    def state(self) -> EnvironmentState:
+        """Return full internal state (includes hidden ground truth for graders)."""
+        if self._observation is None:
+            raise RuntimeError("Call reset() before state().")
+        return EnvironmentState(
+            observation=copy.deepcopy(self._observation),
+            ground_truth=self._ticket_data.get("ground_truth", {}),
+            cumulative_reward=self._cumulative_reward,
+            step_number=self._step_number,
+            done=self._done,
+            task_name=self._task_spec.name,
+        )
+
+    # ------------------------------------------------------------------
+    # Internal helpers
+    # ------------------------------------------------------------------
+
+    def _apply_action(
+        self, action: TicketAction
+    ) -> Tuple[TicketReward, Dict[str, Any]]:
+        """Mutate observation state and return a shaped step reward."""
+        info: Dict[str, Any] = {}
+
+        if action.action_type == ActionType.ROUTE:
+            if action.department is None:
+                return TicketReward(
+                    value=0.0,
+                    reason="ROUTE requires 'department' field.",
+                    partial_scores={},
+                ), {"error": "missing department"}
+            self._observation.current_department = action.department
+            gt_dept = self._ticket_data["ground_truth"]["correct_department"]
+            score = 0.3 if action.department == gt_dept else 0.0
+            return TicketReward(
+                value=score,
+                reason=f"Routed to {action.department.value}.",
+                partial_scores={"routing": score},
+            ), info
+
+        elif action.action_type == ActionType.SET_URGENCY:
+            if action.urgency is None:
+                return TicketReward(
+                    value=0.0,
+                    reason="SET_URGENCY requires 'urgency' field.",
+                    partial_scores={},
+                ), {"error": "missing urgency"}
+            self._observation.current_urgency = action.urgency
+            gt_urgency = self._ticket_data["ground_truth"]["correct_urgency"]
+            score = 0.2 if action.urgency == gt_urgency else 0.05
+            return TicketReward(
+                value=score,
+                reason=f"Set urgency to {action.urgency.value}.",
+                partial_scores={"urgency": score},
+            ), info
+
+        elif action.action_type == ActionType.TAG:
+            if not action.tags:
+                return TicketReward(
+                    value=0.0,
+                    reason="TAG requires non-empty 'tags' list.",
+                    partial_scores={},
+                ), {"error": "missing tags"}
+            for tag in action.tags:
+                if tag not in self._observation.tags:
+                    self._observation.tags.append(tag)
+            required = self._ticket_data["ground_truth"].get("required_tags", set())
+            overlap = len(set(self._observation.tags) & required) / max(len(required), 1)
+            return TicketReward(
+                value=round(0.1 * overlap, 4),
+                reason=f"Added tags: {action.tags}. Overlap with required: {overlap:.0%}",
+                partial_scores={"tagging": overlap},
+            ), info
+
+        elif action.action_type == ActionType.RESPOND:
+            if not action.response_text:
+                return TicketReward(
+                    value=0.0,
+                    reason="RESPOND requires 'response_text' field.",
+                    partial_scores={},
+                ), {"error": "missing response_text"}
+            self._observation.conversation_history.append(
+                TicketMessage(
+                    sender="Support Agent",
+                    content=action.response_text,
+                    timestamp=self._now(),
+                )
+            )
+            key_topics = self._ticket_data["ground_truth"].get("key_response_topics", set())
+            text_lower = action.response_text.lower()
+            found = sum(1 for kw in key_topics if kw.lower() in text_lower)
+            quality = found / max(len(key_topics), 1)
+            return TicketReward(
+                value=round(0.15 * quality, 4),
+                reason=f"Response addresses {found}/{len(key_topics)} key topics.",
+                partial_scores={"response_quality": quality},
+            ), info
+
+        elif action.action_type == ActionType.ESCALATE:
+            if not action.escalation_reason:
+                return TicketReward(
+                    value=0.0,
+                    reason="ESCALATE requires 'escalation_reason' field.",
+                    partial_scores={},
+                ), {"error": "missing escalation_reason"}
+            self._observation.is_escalated = True
+            needs = self._ticket_data["ground_truth"].get("needs_escalation", False)
+            score = 0.2 if needs else -0.1  # penalise unnecessary escalation
+            return TicketReward(
+                value=max(0.0, score),
+                reason=(
+                    "Escalated correctly." if needs
+                    else "Unnecessary escalation (penalised)."
+                ),
+                partial_scores={"escalation": score},
+            ), info
+
+        elif action.action_type == ActionType.CLOSE:
+            self._observation.is_closed = True
+            note = action.resolution_note or ""
+            score = 0.1 if len(note.split()) >= 5 else 0.0
+            return TicketReward(
+                value=score,
+                reason=f"Ticket closed. Resolution note length: {len(note.split())} words.",
+                partial_scores={"closure": score},
+            ), info
+
+        else:  # NOOP
+            return TicketReward(
+                value=0.0,
+                reason="NOOP action — no state change.",
+                partial_scores={},
+            ), info
+
+    def _check_done(self, action: TicketAction) -> bool:
+        """Episode ends on CLOSE, task-specific completion, or max_steps reached."""
+        if action.action_type == ActionType.CLOSE:
+            return True
+        # Easy task: routing is the only required action — auto-complete after ROUTE
+        if self._task_spec.name == "route" and action.action_type == ActionType.ROUTE:
+            return True
+        if self._step_number >= self._task_spec.max_steps:
+            return True
+        return False
+
+    def _maybe_inject_follow_up(self, action: TicketAction) -> None:
+        """
+        For the hard task, inject a customer follow-up after the first RESPOND.
+        Simulates a realistic multi-turn support conversation.
+        """
+        if self._task_spec.name != "resolve":
+            return
+        if self._follow_up_injected:
+            return
+        gt = self._ticket_data["ground_truth"]
+        follow_up = gt.get("follow_up_message")
+        if not follow_up:
+            return
+        respond_count = sum(
+            1 for a in self._actions_taken if a.get("action_type") == ActionType.RESPOND
+        )
+        if respond_count >= 1:
+            self._observation.conversation_history.append(
+                TicketMessage(
+                    sender=self._observation.sender_name,
+                    content=follow_up,
+                    timestamp=self._now(),
+                )
+            )
+            self._follow_up_injected = True
+
+    def _run_grader(self) -> TicketReward:
+        """Run the task's grader on the completed episode."""
+        grader_fn = GRADERS[self._task_spec.grader_name]
+        episode = {
+            "observation": self._observation,
+            "ground_truth": self._ticket_data.get("ground_truth", {}),
+            "actions_taken": self._actions_taken,
+            "step_number": self._step_number,
+        }
+        return grader_fn(episode)
+
+    @staticmethod
+    def _now() -> str:
+        return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+
+    # ------------------------------------------------------------------
+    # Class-level metadata
+    # ------------------------------------------------------------------
+
+    @classmethod
+    def list_tasks(cls) -> List[Dict[str, str]]:
+        return [
+            {
+                "name": t.name,
+                "display_name": t.display_name,
+                "difficulty": t.difficulty,
+                "max_steps": str(t.max_steps),
+            }
+            for t in ALL_TASKS
+        ]

graders.py

@@ -0,0 +1,261 @@
+"""
+Graders for the Support Ticket Triage OpenEnv environment.
+
+All graders are deterministic and produce scores in [0.0, 1.0].
+Each grader receives the full episode state and returns a TicketReward.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List
+
+from env.models import ActionType, Department, TicketReward, UrgencyLevel
+
+
+# ---------------------------------------------------------------------------
+# Helper utilities
+# ---------------------------------------------------------------------------
+
+def _keyword_overlap(text: str, keywords: set, threshold: int = 1) -> float:
+    """
+    Returns fraction of keywords found in text (case-insensitive).
+    If text contains >= threshold keywords → full credit per keyword found.
+    """
+    if not text or not keywords:
+        return 0.0
+    text_lower = text.lower()
+    found = sum(1 for kw in keywords if kw.lower() in text_lower)
+    return found / len(keywords)
+
+
+def _tag_overlap(actual_tags: List[str], required_tags: set) -> float:
+    """Fraction of required tags that appear in actual_tags."""
+    if not required_tags:
+        return 1.0
+    actual_set = {t.lower() for t in actual_tags}
+    required_lower = {t.lower() for t in required_tags}
+    found = actual_set & required_lower
+    return len(found) / len(required_lower)
+
+
+# ---------------------------------------------------------------------------
+# Route Grader (Easy task)
+# ---------------------------------------------------------------------------
+
+def route_grader(episode: Dict[str, Any]) -> TicketReward:
+    """
+    Score: 1.0 if routed to the correct department, else 0.0.
+    Small partial credit (0.1) for attempting a route at all vs. noop-ing.
+    """
+    ground_truth: Dict = episode["ground_truth"]
+    actions_taken: List[Dict] = episode.get("actions_taken", [])
+    observation = episode["observation"]
+
+    correct_dept: Department = ground_truth["correct_department"]
+    current_dept = observation.current_department
+
+    # Did the agent ever perform a ROUTE action?
+    route_actions = [a for a in actions_taken if a.get("action_type") == ActionType.ROUTE]
+
+    if not route_actions:
+        return TicketReward(
+            value=0.0,
+            reason="No ROUTE action taken.",
+            partial_scores={"routing": 0.0},
+        )
+
+    if current_dept == correct_dept:
+        return TicketReward(
+            value=1.0,
+            reason=f"Correctly routed to {correct_dept.value}.",
+            partial_scores={"routing": 1.0},
+        )
+
+    # Routed but to the wrong department
+    # Give 0.1 partial credit for at least routing (vs. doing nothing)
+    return TicketReward(
+        value=0.1,
+        reason=(
+            f"Routed to {current_dept.value if current_dept else 'unknown'} "
+            f"but correct answer is {correct_dept.value}."
+        ),
+        partial_scores={"routing": 0.1},
+    )
+
+
+# ---------------------------------------------------------------------------
+# Triage Grader (Medium task)
+# ---------------------------------------------------------------------------
+
+def triage_grader(episode: Dict[str, Any]) -> TicketReward:
+    """
+    Weighted score across four sub-tasks:
+      - Routing:   0.30
+      - Urgency:   0.25
+      - Tagging:   0.20
+      - Response:  0.25
+    """
+    ground_truth: Dict = episode["ground_truth"]
+    observation = episode["observation"]
+    actions_taken: List[Dict] = episode.get("actions_taken", [])
+
+    # --- 1. Routing (0.30) ---
+    correct_dept: Department = ground_truth["correct_department"]
+    routing_score = 1.0 if observation.current_department == correct_dept else 0.0
+
+    # --- 2. Urgency (0.25) ---
+    correct_urgency: UrgencyLevel = ground_truth["correct_urgency"]
+    urgency_score = 0.0
+    if observation.current_urgency == correct_urgency:
+        urgency_score = 1.0
+    elif observation.current_urgency is not None:
+        # Partial credit for adjacent urgency levels
+        levels = [UrgencyLevel.LOW, UrgencyLevel.MEDIUM, UrgencyLevel.HIGH, UrgencyLevel.CRITICAL]
+        correct_idx = levels.index(correct_urgency)
+        actual_idx = levels.index(observation.current_urgency)
+        diff = abs(correct_idx - actual_idx)
+        urgency_score = max(0.0, 1.0 - diff * 0.4)
+
+    # --- 3. Tagging (0.20) ---
+    required_tags: set = ground_truth.get("required_tags", set())
+    tagging_score = _tag_overlap(observation.tags, required_tags)
+
+    # --- 4. Response quality (0.25) ---
+    key_topics: set = ground_truth.get("key_response_topics", set())
+    response_texts = [
+        a.get("response_text", "") or ""
+        for a in actions_taken
+        if a.get("action_type") == ActionType.RESPOND
+    ]
+    combined_response = " ".join(response_texts)
+    response_score = min(1.0, _keyword_overlap(combined_response, key_topics) * 1.5)
+
+    # Aggregate
+    weights = {"routing": 0.30, "urgency": 0.25, "tagging": 0.20, "response": 0.25}
+    partial = {
+        "routing": routing_score,
+        "urgency": urgency_score,
+        "tagging": tagging_score,
+        "response": response_score,
+    }
+    total = sum(weights[k] * v for k, v in partial.items())
+    total = round(min(1.0, max(0.0, total)), 4)
+
+    return TicketReward(
+        value=total,
+        reason=(
+            f"Routing={'✓' if routing_score == 1 else '✗'} "
+            f"Urgency={'✓' if urgency_score == 1 else f'{urgency_score:.2f}'} "
+            f"Tags={tagging_score:.2f} Response={response_score:.2f}"
+        ),
+        partial_scores=partial,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Resolve Grader (Hard task)
+# ---------------------------------------------------------------------------
+
+def resolve_grader(episode: Dict[str, Any]) -> TicketReward:
+    """
+    Weighted score across six sub-tasks:
+      - Routing:      0.15
+      - Urgency:      0.10
+      - Initial resp: 0.20
+      - Escalation:   0.20
+      - Follow-up:    0.20
+      - Closure:      0.15
+    """
+    ground_truth: Dict = episode["ground_truth"]
+    observation = episode["observation"]
+    actions_taken: List[Dict] = episode.get("actions_taken", [])
+
+    # --- 1. Routing (0.15) ---
+    correct_dept: Department = ground_truth["correct_department"]
+    routing_score = 1.0 if observation.current_department == correct_dept else 0.0
+
+    # --- 2. Urgency (0.10) ---
+    correct_urgency: UrgencyLevel = ground_truth["correct_urgency"]
+    levels = [UrgencyLevel.LOW, UrgencyLevel.MEDIUM, UrgencyLevel.HIGH, UrgencyLevel.CRITICAL]
+    if observation.current_urgency == correct_urgency:
+        urgency_score = 1.0
+    elif observation.current_urgency is not None:
+        diff = abs(levels.index(correct_urgency) - levels.index(observation.current_urgency))
+        urgency_score = max(0.0, 1.0 - diff * 0.4)
+    else:
+        urgency_score = 0.0
+
+    # --- 3. Initial response quality (0.20) ---
+    key_topics: set = ground_truth.get("key_response_topics", set())
+    respond_actions = [a for a in actions_taken if a.get("action_type") == ActionType.RESPOND]
+    initial_response = respond_actions[0].get("response_text", "") if respond_actions else ""
+    initial_resp_score = min(1.0, _keyword_overlap(initial_response, key_topics) * 1.5)
+
+    # --- 4. Escalation (0.20) ---
+    needs_escalation: bool = ground_truth.get("needs_escalation", False)
+    escalate_actions = [a for a in actions_taken if a.get("action_type") == ActionType.ESCALATE]
+    if needs_escalation:
+        if escalate_actions:
+            reason = escalate_actions[0].get("escalation_reason", "") or ""
+            escalation_score = 0.5 + 0.5 * min(1.0, len(reason.split()) / 10.0)
+        else:
+            escalation_score = 0.0
+    else:
+        # Should NOT have escalated
+        escalation_score = 0.0 if escalate_actions else 1.0
+
+    # --- 5. Follow-up response (0.20) ---
+    follow_up_topics: set = ground_truth.get("follow_up_response_topics", set())
+    follow_up_response = respond_actions[1].get("response_text", "") if len(respond_actions) > 1 else ""
+    if follow_up_topics:
+        follow_up_score = min(1.0, _keyword_overlap(follow_up_response, follow_up_topics) * 1.5)
+    else:
+        follow_up_score = 1.0  # No follow-up expected → full credit
+
+    # --- 6. Closure quality (0.15) ---
+    close_actions = [a for a in actions_taken if a.get("action_type") == ActionType.CLOSE]
+    if not close_actions:
+        closure_score = 0.0
+    else:
+        resolution_note = close_actions[0].get("resolution_note", "") or ""
+        good_keywords: set = ground_truth.get("good_resolution_keywords", set())
+        closure_score = (
+            min(1.0, _keyword_overlap(resolution_note, good_keywords) * 1.5)
+            if good_keywords else (1.0 if resolution_note else 0.3)
+        )
+
+    weights = {
+        "routing": 0.15,
+        "urgency": 0.10,
+        "initial_response": 0.20,
+        "escalation": 0.20,
+        "follow_up": 0.20,
+        "closure": 0.15,
+    }
+    partial = {
+        "routing": routing_score,
+        "urgency": urgency_score,
+        "initial_response": initial_resp_score,
+        "escalation": escalation_score,
+        "follow_up": follow_up_score,
+        "closure": closure_score,
+    }
+    total = sum(weights[k] * v for k, v in partial.items())
+    total = round(min(1.0, max(0.0, total)), 4)
+
+    return TicketReward(
+        value=total,
+        reason=" | ".join(f"{k}={v:.2f}" for k, v in partial.items()),
+        partial_scores=partial,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Registry
+# ---------------------------------------------------------------------------
+
+GRADERS = {
+    "route_grader": route_grader,
+    "triage_grader": triage_grader,
+    "resolve_grader": resolve_grader,
+}

inference.py

@@ -0,0 +1,279 @@
+"""
+Inference Script — Support Ticket Triage OpenEnv
+=================================================
+MANDATORY environment variables:
+  API_BASE_URL   The API endpoint for the LLM.
+  MODEL_NAME     The model identifier to use for inference.
+  HF_TOKEN       Your Hugging Face / API key.
+
+This script runs the baseline agent against all 3 tasks and prints
+reproducible scores for each task and per-ticket.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import textwrap
+from typing import Any, Dict, List, Optional
+
+import requests
+from openai import OpenAI
+
+# ---------------------------------------------------------------------------
+# Configuration
+# ---------------------------------------------------------------------------
+
+API_BASE_URL: str = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
+API_KEY: str = os.getenv("HF_TOKEN") or os.getenv("API_KEY") or ""
+MODEL_NAME: str = os.getenv("MODEL_NAME", "meta-llama/Llama-3.3-70B-Instruct")
+
+# Where the environment server is running
+ENV_BASE_URL: str = os.getenv("ENV_BASE_URL", "http://localhost:7860")
+
+TEMPERATURE: float = 0.0   # Greedy for reproducibility
+MAX_TOKENS: int = 512
+MAX_STEPS: int = 10
+
+# Tickets to evaluate per task (pinned for reproducibility)
+TASK_CONFIGS = [
+    {
+        "task_name": "route",
+        "ticket_ids": ["TKT-001", "TKT-002", "TKT-003", "TKT-004", "TKT-005"],
+        "seed": 42,
+    },
+    {
+        "task_name": "triage",
+        "ticket_ids": ["TKT-006", "TKT-007"],
+        "seed": 42,
+    },
+    {
+        "task_name": "resolve",
+        "ticket_ids": ["TKT-008", "TKT-009"],
+        "seed": 42,
+    },
+]
+
+
+# ---------------------------------------------------------------------------
+# OpenAI client
+# ---------------------------------------------------------------------------
+
+client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+
+
+# ---------------------------------------------------------------------------
+# System prompt
+# ---------------------------------------------------------------------------
+
+SYSTEM_PROMPT = textwrap.dedent("""
+You are an expert customer support agent. You receive support tickets and must take
+the most appropriate action. 
+
+Reply with EXACTLY a JSON object (no markdown, no explanation):
+{
+  "action_type": "<one of: route, respond, set_urgency, tag, escalate, close, noop>",
+  "department": "<billing|technical_support|sales|customer_success|legal or null>",
+  "response_text": "<your message to the customer or null>",
+  "urgency": "<low|medium|high|critical or null>",
+  "tags": ["<tag1>", "<tag2>"] or null,
+  "escalation_reason": "<reason or null>",
+  "resolution_note": "<summary or null>"
+}
+
+Rules:
+- For ROUTE: set department, leave rest null
+- For SET_URGENCY: set urgency, leave rest null
+- For RESPOND: set response_text (empathetic, clear, actionable)
+- For TAG: set tags (relevant labels like 'billing', 'urgent', 'refund')
+- For ESCALATE: set escalation_reason (explain why escalation is needed)
+- For CLOSE: set resolution_note (what was done to resolve the ticket)
+- Think about the task description shown to you and complete all required steps
+""").strip()
+
+
+# ---------------------------------------------------------------------------
+# Environment HTTP helpers
+# ---------------------------------------------------------------------------
+
+def env_reset(task_name: str, ticket_id: str, seed: int = 42) -> Dict[str, Any]:
+    r = requests.post(f"{ENV_BASE_URL}/reset", json={
+        "task_name": task_name,
+        "ticket_id": ticket_id,
+        "seed": seed,
+    }, timeout=30)
+    r.raise_for_status()
+    return r.json()
+
+
+def env_step(session_id: str, action: Dict[str, Any]) -> Dict[str, Any]:
+    payload = {"session_id": session_id, **action}
+    r = requests.post(f"{ENV_BASE_URL}/step", json=payload, timeout=30)
+    r.raise_for_status()
+    return r.json()
+
+
+# ---------------------------------------------------------------------------
+# Agent decision logic
+# ---------------------------------------------------------------------------
+
+def observation_to_prompt(obs: Dict[str, Any]) -> str:
+    """Convert observation dict to a text prompt for the model."""
+    hist_lines = []
+    for msg in obs.get("conversation_history", []):
+        hist_lines.append(f"[{msg['sender']}]: {msg['content']}")
+
+    return textwrap.dedent(f"""
+    TASK: {obs.get('task_description', '')}
+
+    --- TICKET ---
+    Ticket ID: {obs['ticket_id']}
+    Subject: {obs['subject']}
+    From: {obs['sender_name']} <{obs['sender_email']}>
+
+    Conversation:
+    {chr(10).join(hist_lines)}
+    -------------
+
+    Current state:
+    - Department: {obs.get('current_department') or 'not set'}
+    - Urgency: {obs.get('current_urgency') or 'not set'}
+    - Tags: {obs.get('tags') or 'none'}
+    - Escalated: {obs.get('is_escalated', False)}
+    - Closed: {obs.get('is_closed', False)}
+    - Step: {obs.get('step_number', 0)}
+
+    What is your next action? Reply with the JSON object.
+    """).strip()
+
+
+def call_model(prompt: str) -> Dict[str, Any]:
+    """Call the LLM and parse its JSON action."""
+    try:
+        completion = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=[
+                {"role": "system", "content": SYSTEM_PROMPT},
+                {"role": "user", "content": prompt},
+            ],
+            temperature=TEMPERATURE,
+            max_tokens=MAX_TOKENS,
+        )
+        text = completion.choices[0].message.content or "{}"
+        # Strip markdown fences if present
+        text = text.strip()
+        if text.startswith("```"):
+            lines = text.splitlines()
+            text = "\n".join(lines[1:-1]) if len(lines) > 2 else text
+        return json.loads(text)
+    except Exception as exc:
+        print(f"  [warn] Model call failed: {exc}. Using noop.")
+        return {"action_type": "noop"}
+
+
+def clean_action(raw: Dict[str, Any]) -> Dict[str, Any]:
+    """Ensure action dict has valid fields only."""
+    valid_keys = {
+        "action_type", "department", "response_text",
+        "urgency", "tags", "escalation_reason", "resolution_note",
+    }
+    return {k: v for k, v in raw.items() if k in valid_keys and v is not None}
+
+
+# ---------------------------------------------------------------------------
+# Episode runner
+# ---------------------------------------------------------------------------
+
+def run_episode(task_name: str, ticket_id: str, seed: int = 42) -> float:
+    """Run one full episode. Returns the final reward score [0, 1]."""
+    print(f"\n  → Episode: task={task_name}, ticket={ticket_id}")
+
+    reset_resp = env_reset(task_name, ticket_id, seed)
+    session_id: str = reset_resp["session_id"]
+    obs: Dict[str, Any] = reset_resp["observation"]
+
+    final_score = 0.0
+
+    for step in range(1, MAX_STEPS + 1):
+        prompt = observation_to_prompt(obs)
+        raw_action = call_model(prompt)
+        action = clean_action(raw_action)
+
+        print(f"    Step {step}: action_type={action.get('action_type', 'noop')}", end="")
+
+        try:
+            result = env_step(session_id, action)
+        except Exception as exc:
+            print(f" [ERROR: {exc}]")
+            break
+
+        reward_val = result["reward"]["value"]
+        done = result["done"]
+        obs = result["observation"]
+
+        print(f"  reward={reward_val:.3f}  done={done}")
+
+        if done:
+            # Terminal reward from grader is the authoritative score
+            final_score = result["reward"]["value"]
+            grader_info = result["info"].get("final_grader_reward", {})
+            if grader_info:
+                print(f"    [grader] {grader_info.get('reason', '')}")
+                print(f"    [partial] {grader_info.get('partial_scores', {})}")
+            break
+    else:
+        print(f"    Max steps ({MAX_STEPS}) reached.")
+        final_score = result["reward"]["value"] if "result" in dir() else 0.0  # type: ignore[name-defined]
+
+    print(f"  ✓ Final score: {final_score:.4f}")
+    return final_score
+
+
+# ---------------------------------------------------------------------------
+# Main: run all tasks and aggregate
+# ---------------------------------------------------------------------------
+
+def main() -> None:
+    print("=" * 60)
+    print("Support Ticket Triage — Baseline Inference")
+    print(f"Model: {MODEL_NAME}")
+    print(f"Environment: {ENV_BASE_URL}")
+    print("=" * 60)
+
+    all_scores: Dict[str, List[float]] = {}
+
+    for task_cfg in TASK_CONFIGS:
+        task_name = task_cfg["task_name"]
+        ticket_ids = task_cfg["ticket_ids"]
+        seed = task_cfg["seed"]
+
+        print(f"\n{'─'*50}")
+        print(f"TASK: {task_name.upper()}")
+        print(f"{'─'*50}")
+
+        task_scores: List[float] = []
+        for tid in ticket_ids:
+            score = run_episode(task_name, tid, seed)
+            task_scores.append(score)
+
+        avg = sum(task_scores) / len(task_scores) if task_scores else 0.0
+        all_scores[task_name] = task_scores
+        print(f"\n  Task '{task_name}' average: {avg:.4f}")
+
+    # Summary
+    print(f"\n{'='*60}")
+    print("FINAL SCORES")
+    print(f"{'='*60}")
+    overall_scores = []
+    for task_name, scores in all_scores.items():
+        avg = sum(scores) / len(scores) if scores else 0.0
+        overall_scores.append(avg)
+        print(f"  {task_name:12s}: {avg:.4f}  (tickets: {[f'{s:.3f}' for s in scores]})")
+
+    grand_avg = sum(overall_scores) / len(overall_scores) if overall_scores else 0.0
+    print(f"  {'OVERALL':12s}: {grand_avg:.4f}")
+    print("=" * 60)
+
+
+if __name__ == "__main__":
+    main()

models.py

@@ -0,0 +1,144 @@
+"""
+Typed Pydantic models for the Support Ticket Triage OpenEnv environment.
+"""
+
+from __future__ import annotations
+
+from enum import Enum
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel, Field
+
+
+# ---------------------------------------------------------------------------
+# Domain enumerations
+# ---------------------------------------------------------------------------
+
+class Department(str, Enum):
+    BILLING = "billing"
+    TECHNICAL_SUPPORT = "technical_support"
+    SALES = "sales"
+    CUSTOMER_SUCCESS = "customer_success"
+    LEGAL = "legal"
+
+
+class UrgencyLevel(str, Enum):
+    LOW = "low"
+    MEDIUM = "medium"
+    HIGH = "high"
+    CRITICAL = "critical"
+
+
+class ActionType(str, Enum):
+    ROUTE = "route"           # Assign to a department
+    RESPOND = "respond"       # Send a reply to the customer
+    SET_URGENCY = "set_urgency"  # Set urgency level
+    TAG = "tag"               # Add classification tags
+    ESCALATE = "escalate"     # Escalate with a reason
+    CLOSE = "close"           # Resolve and close the ticket
+    NOOP = "noop"             # Do nothing (wastes a step)
+
+
+# ---------------------------------------------------------------------------
+# Action model
+# ---------------------------------------------------------------------------
+
+class TicketAction(BaseModel):
+    """One action the agent can take in the environment."""
+
+    action_type: ActionType = Field(
+        description="The type of action to perform."
+    )
+    department: Optional[Department] = Field(
+        default=None,
+        description="Target department (required for ROUTE action).",
+    )
+    response_text: Optional[str] = Field(
+        default=None,
+        description="Message body sent to the customer (required for RESPOND).",
+    )
+    urgency: Optional[UrgencyLevel] = Field(
+        default=None,
+        description="Urgency level (required for SET_URGENCY).",
+    )
+    tags: Optional[List[str]] = Field(
+        default=None,
+        description="Classification tags to apply (required for TAG).",
+    )
+    escalation_reason: Optional[str] = Field(
+        default=None,
+        description="Plain-text reason for escalation (required for ESCALATE).",
+    )
+    resolution_note: Optional[str] = Field(
+        default=None,
+        description="Summary of how the issue was resolved (required for CLOSE).",
+    )
+
+
+# ---------------------------------------------------------------------------
+# Observation model
+# ---------------------------------------------------------------------------
+
+class TicketMessage(BaseModel):
+    """One message in the ticket conversation thread."""
+
+    sender: str
+    content: str
+    timestamp: str
+
+
+class TicketObservation(BaseModel):
+    """Everything the agent can observe at a given step."""
+
+    # Ticket content
+    ticket_id: str
+    subject: str
+    body: str
+    sender_email: str
+    sender_name: str
+
+    # Evolving state
+    conversation_history: List[TicketMessage] = Field(default_factory=list)
+    current_department: Optional[Department] = None
+    current_urgency: Optional[UrgencyLevel] = None
+    tags: List[str] = Field(default_factory=list)
+    is_escalated: bool = False
+    is_closed: bool = False
+
+    # Episode metadata
+    step_number: int = 0
+    task_name: str = ""
+    task_description: str = ""
+    available_actions: List[str] = Field(default_factory=list)
+
+
+# ---------------------------------------------------------------------------
+# Reward model
+# ---------------------------------------------------------------------------
+
+class TicketReward(BaseModel):
+    """Structured reward with partial-credit breakdown."""
+
+    value: float = Field(ge=0.0, le=1.0, description="Aggregate reward [0, 1].")
+    reason: str = Field(description="Human-readable explanation.")
+    partial_scores: Dict[str, float] = Field(
+        default_factory=dict,
+        description="Per-criterion scores contributing to value.",
+    )
+
+
+# ---------------------------------------------------------------------------
+# State model (returned by state())
+# ---------------------------------------------------------------------------
+
+class EnvironmentState(BaseModel):
+    """Full internal state snapshot (superset of observation)."""
+
+    observation: TicketObservation
+    ground_truth: Dict[str, Any] = Field(
+        description="Hidden ground-truth labels used by graders.",
+    )
+    cumulative_reward: float = 0.0
+    step_number: int = 0
+    done: bool = False
+    task_name: str = ""

openenv.yaml

@@ -0,0 +1,115 @@
+name: support-ticket-triage
+version: "1.0.0"
+description: >
+  A real-world OpenEnv environment where AI agents must triage, route, and resolve
+  customer support tickets. Agents observe ticket content and conversation history,
+  then take actions such as routing to departments, setting urgency, responding
+  to customers, escalating issues, and closing tickets.
+
+author: "OpenEnv Hackathon Submission"
+tags:
+  - openenv
+  - customer-support
+  - triage
+  - nlp
+  - real-world
+
+# ── Task definitions ────────────────────────────────────────────────────────
+tasks:
+  - name: route
+    display_name: "Ticket Routing (Easy)"
+    difficulty: easy
+    max_steps: 3
+    success_threshold: 1.0
+    description: >
+      Route the incoming support ticket to the correct department.
+      Score 1.0 for correct routing, 0.0 for wrong department.
+    ticket_pool: [TKT-001, TKT-002, TKT-003, TKT-004, TKT-005]
+
+  - name: triage
+    display_name: "Full Triage (Medium)"
+    difficulty: medium
+    max_steps: 8
+    success_threshold: 0.7
+    description: >
+      Fully triage the ticket: route correctly (0.30), set urgency (0.25),
+      add relevant tags (0.20), and send an informative initial response (0.25).
+    ticket_pool: [TKT-006, TKT-007, TKT-001, TKT-003]
+
+  - name: resolve
+    display_name: "Full Resolution (Hard)"
+    difficulty: hard
+    max_steps: 12
+    success_threshold: 0.6
+    description: >
+      Fully resolve a complex ticket over multiple turns: route (0.15),
+      set urgency (0.10), respond initially (0.20), escalate if needed (0.20),
+      handle customer follow-up (0.20), and close with resolution note (0.15).
+    ticket_pool: [TKT-008, TKT-009]
+
+# ── Observation space ────────────────────────────────────────────────────────
+observation_space:
+  type: object
+  fields:
+    ticket_id: {type: string}
+    subject: {type: string}
+    body: {type: string}
+    sender_email: {type: string}
+    sender_name: {type: string}
+    conversation_history:
+      type: array
+      items:
+        sender: string
+        content: string
+        timestamp: string
+    current_department: {type: string, nullable: true, enum: [billing, technical_support, sales, customer_success, legal]}
+    current_urgency: {type: string, nullable: true, enum: [low, medium, high, critical]}
+    tags: {type: array, items: string}
+    is_escalated: {type: boolean}
+    is_closed: {type: boolean}
+    step_number: {type: integer}
+    task_name: {type: string}
+    task_description: {type: string}
+    available_actions: {type: array, items: string}
+
+# ── Action space ─────────────────────────────────────────────────────────────
+action_space:
+  type: object
+  fields:
+    action_type:
+      type: string
+      enum: [route, respond, set_urgency, tag, escalate, close, noop]
+    department:
+      type: string
+      nullable: true
+      enum: [billing, technical_support, sales, customer_success, legal]
+    response_text: {type: string, nullable: true}
+    urgency: {type: string, nullable: true, enum: [low, medium, high, critical]}
+    tags: {type: array, items: string, nullable: true}
+    escalation_reason: {type: string, nullable: true}
+    resolution_note: {type: string, nullable: true}
+
+# ── Reward function ──────────────────────────────────────────────────────────
+reward:
+  range: [0.0, 1.0]
+  type: shaped
+  description: >
+    Step-level shaped rewards guide the agent during the episode.
+    Terminal reward from the grader provides the authoritative episode score.
+  mid_episode: >
+    Partial credit per action: +0.3 correct routing, +0.2 correct urgency,
+    +0.1×overlap tag matching, +0.15×quality response, +0.2 justified escalation,
+    -0.1 unjustified escalation, +0.1 closure with note.
+  terminal: >
+    Task-specific weighted grader aggregates all sub-task scores into a
+    final [0.0, 1.0] score.
+
+# ── API endpoints ─────────────────────────────────────────────────────────────
+api:
+  base_url: "http://localhost:7860"
+  endpoints:
+    reset: {method: POST, path: /reset}
+    step: {method: POST, path: /step}
+    state: {method: GET, path: /state}
+    tasks: {method: GET, path: /tasks}
+    health: {method: GET, path: /}

requirements.txt

@@ -0,0 +1,7 @@
+fastapi==0.115.0
+uvicorn[standard]==0.32.0
+pydantic>=2.0,<3.0
+openai>=1.0,<2.0
+requests>=2.31,<3.0
+python-multipart==0.0.12
+httpx>=0.27,<1.0

server.py

@@ -0,0 +1,189 @@
+"""
+FastAPI server — Support Ticket Triage OpenEnv
+================================================
+
+Exposes the standard OpenEnv HTTP API:
+  GET  /           → healthcheck + metadata
+  POST /reset      → start episode, returns observation
+  POST /step       → apply action, returns (obs, reward, done, info)
+  GET  /state      → full internal state (with ground truth)
+  GET  /tasks      → list all tasks
+"""
+
+from __future__ import annotations
+
+import uuid
+from typing import Any, Dict, Optional
+
+from fastapi import FastAPI, HTTPException
+from fastapi.middleware.cors import CORSMiddleware
+from pydantic import BaseModel
+
+from env.environment import TicketTriageEnv
+from env.models import ActionType, Department, TicketAction, UrgencyLevel
+
+app = FastAPI(
+    title="Support Ticket Triage — OpenEnv",
+    description=(
+        "An OpenEnv environment where AI agents must triage, route, and resolve "
+        "customer support tickets. Real-world task with 3 difficulty levels."
+    ),
+    version="1.0.0",
+)
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+# ---------------------------------------------------------------------------
+# In-memory session store (keyed by session_id)
+# ---------------------------------------------------------------------------
+
+_sessions: Dict[str, TicketTriageEnv] = {}
+
+
+def _get_env(session_id: str) -> TicketTriageEnv:
+    env = _sessions.get(session_id)
+    if env is None:
+        raise HTTPException(status_code=404, detail=f"Session '{session_id}' not found.")
+    return env
+
+
+# ---------------------------------------------------------------------------
+# Request / response schemas
+# ---------------------------------------------------------------------------
+
+class ResetRequest(BaseModel):
+    task_name: str = "route"
+    ticket_id: Optional[str] = None
+    seed: Optional[int] = 42
+    session_id: Optional[str] = None
+
+
+class StepRequest(BaseModel):
+    session_id: str
+    action_type: str
+    department: Optional[str] = None
+    response_text: Optional[str] = None
+    urgency: Optional[str] = None
+    tags: Optional[list[str]] = None
+    escalation_reason: Optional[str] = None
+    resolution_note: Optional[str] = None
+
+
+class StepResponse(BaseModel):
+    observation: Dict[str, Any]
+    reward: Dict[str, Any]
+    done: bool
+    info: Dict[str, Any]
+    session_id: str
+
+
+# ---------------------------------------------------------------------------
+# Routes
+# ---------------------------------------------------------------------------
+
+@app.get("/")
+def healthcheck():
+    return {
+        "status": "ok",
+        "environment": "Support Ticket Triage",
+        "version": "1.0.0",
+        "tasks": TicketTriageEnv.list_tasks(),
+    }
+
+
+@app.get("/tasks")
+def list_tasks():
+    return {"tasks": TicketTriageEnv.list_tasks()}
+
+
+@app.post("/reset")
+def reset(req: ResetRequest):
+    session_id = req.session_id or str(uuid.uuid4())
+    env = TicketTriageEnv(
+        task_name=req.task_name,
+        ticket_id=req.ticket_id,
+        seed=req.seed,
+    )
+    _sessions[session_id] = env
+    obs = env.reset()
+    return {"observation": obs.model_dump(), "session_id": session_id}
+
+
+@app.post("/step", response_model=StepResponse)
+def step(req: StepRequest):
+    env = _get_env(req.session_id)
+
+    # Parse action
+    try:
+        action_type = ActionType(req.action_type)
+    except ValueError:
+        raise HTTPException(
+            status_code=422,
+            detail=f"Invalid action_type '{req.action_type}'. "
+                   f"Valid: {[a.value for a in ActionType]}",
+        )
+
+    dept = None
+    if req.department:
+        try:
+            dept = Department(req.department)
+        except ValueError:
+            raise HTTPException(
+                status_code=422,
+                detail=f"Invalid department '{req.department}'.",
+            )
+
+    urg = None
+    if req.urgency:
+        try:
+            urg = UrgencyLevel(req.urgency)
+        except ValueError:
+            raise HTTPException(
+                status_code=422,
+                detail=f"Invalid urgency '{req.urgency}'.",
+            )
+
+    action = TicketAction(
+        action_type=action_type,
+        department=dept,
+        response_text=req.response_text,
+        urgency=urg,
+        tags=req.tags,
+        escalation_reason=req.escalation_reason,
+        resolution_note=req.resolution_note,
+    )
+
+    obs, reward, done, info = env.step(action)
+
+    if done:
+        # Clean up session after episode ends
+        _sessions.pop(req.session_id, None)
+
+    return StepResponse(
+        observation=obs.model_dump(),
+        reward=reward.model_dump(),
+        done=done,
+        info=info,
+        session_id=req.session_id,
+    )
+
+
+@app.get("/state")
+def state(session_id: str):
+    env = _get_env(session_id)
+    s = env.state()
+    return s.model_dump()
+
+
+# ---------------------------------------------------------------------------
+# Entry point (for local dev)
+# ---------------------------------------------------------------------------
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host="0.0.0.0", port=7860)

tasks.py

@@ -0,0 +1,99 @@
+"""
+Task definitions for the Support Ticket Triage OpenEnv environment.
+
+Each task specifies:
+  - name & description shown to the agent
+  - which tickets belong to this task
+  - max_steps allowed
+  - grader function reference
+  - expected difficulty
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import List
+
+
+@dataclass
+class TaskSpec:
+    name: str
+    display_name: str
+    description: str
+    ticket_ids: List[str]
+    max_steps: int
+    difficulty: str          # "easy" | "medium" | "hard"
+    grader_name: str         # matches key in graders.py GRADERS dict
+    success_threshold: float  # minimum score considered "solved"
+
+
+# ---------------------------------------------------------------------------
+# Task 1 — EASY: Route to the correct department
+# ---------------------------------------------------------------------------
+TASK_ROUTE = TaskSpec(
+    name="route",
+    display_name="Ticket Routing (Easy)",
+    description=(
+        "Read the support ticket and route it to the correct department by "
+        "using the ROUTE action. You have one chance to route correctly. "
+        "Available departments: billing, technical_support, sales, "
+        "customer_success, legal. "
+        "Score 1.0 for the correct department, 0.0 for wrong."
+    ),
+    ticket_ids=["TKT-001", "TKT-002", "TKT-003", "TKT-004", "TKT-005"],
+    max_steps=3,
+    difficulty="easy",
+    grader_name="route_grader",
+    success_threshold=1.0,
+)
+
+
+# ---------------------------------------------------------------------------
+# Task 2 — MEDIUM: Route + Set Urgency + Tag + Respond
+# ---------------------------------------------------------------------------
+TASK_TRIAGE = TaskSpec(
+    name="triage",
+    display_name="Full Triage (Medium)",
+    description=(
+        "Fully triage the support ticket by completing ALL of the following: "
+        "(1) ROUTE to the correct department, "
+        "(2) SET_URGENCY to the appropriate level (low/medium/high/critical), "
+        "(3) TAG with relevant classification labels, "
+        "(4) RESPOND with a helpful initial reply to the customer. "
+        "You must close the ticket with CLOSE after completing all steps. "
+        "Partial credit is awarded for each sub-task completed correctly."
+    ),
+    ticket_ids=["TKT-006", "TKT-007", "TKT-001", "TKT-003"],
+    max_steps=8,
+    difficulty="medium",
+    grader_name="triage_grader",
+    success_threshold=0.7,
+)
+
+
+# ---------------------------------------------------------------------------
+# Task 3 — HARD: Full resolution including follow-up and escalation
+# ---------------------------------------------------------------------------
+TASK_RESOLVE = TaskSpec(
+    name="resolve",
+    display_name="Full Resolution (Hard)",
+    description=(
+        "Fully resolve a complex support ticket over multiple steps. "
+        "You must: (1) ROUTE correctly, (2) SET_URGENCY appropriately, "
+        "(3) RESPOND with an empathetic and actionable initial reply, "
+        "(4) ESCALATE with a clear reason if the ticket warrants it, "
+        "(5) Handle a follow-up message from the customer with another RESPOND, "
+        "(6) CLOSE the ticket with a resolution summary. "
+        "All steps are required for full score. Graders check response quality, "
+        "escalation decisions, and resolution completeness."
+    ),
+    ticket_ids=["TKT-008", "TKT-009"],
+    max_steps=12,
+    difficulty="hard",
+    grader_name="resolve_grader",
+    success_threshold=0.6,
+)
+
+
+ALL_TASKS: List[TaskSpec] = [TASK_ROUTE, TASK_TRIAGE, TASK_RESOLVE]
+TASK_LOOKUP = {t.name: t for t in ALL_TASKS}