README.md

metadata

title: Customer Support Ticket Management Environment
emoji: 🎧
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
app_port: 8000
base_path: /web
tags:
  - openenv
  - customer-support
  - reinforcement-learning
  - agent-training

Customer Support Ticket Management Environment

A real-world OpenEnv environment for training AI agents on customer support tasks. Agents learn to categorize tickets, assign priorities, route to appropriate teams, and draft professional responses.

Quick Start

The simplest way to use the Customer Support Env environment is through the CustomerSupportEnv class:

from customer_support_env import CustomerSupportAction, CustomerSupportEnv

try:
    # Create environment from Docker image
    customer_support_envenv = CustomerSupportEnv.from_docker_image("customer_support_env-env:latest")

    # Reset
    result = customer_support_envenv.reset()
    print(f"Reset: {result.observation.echoed_message}")

    # Send multiple messages
    messages = ["Hello, World!", "Testing echo", "Final message"]

    for msg in messages:
        result = customer_support_envenv.step(CustomerSupportAction(message=msg))
        print(f"Sent: '{msg}'")
        print(f"  → Echoed: '{result.observation.echoed_message}'")
        print(f"  → Length: {result.observation.message_length}")
        print(f"  → Reward: {result.reward}")

finally:
    # Always clean up
    customer_support_envenv.close()

That's it! The CustomerSupportEnv.from_docker_image() method handles:

• Starting the Docker container
• Waiting for the server to be ready
• Connecting to the environment
• Container cleanup when you call close()

Building the Docker Image

Before using the environment, you need to build the Docker image:

# From project root
docker build -t customer_support_env-env:latest -f server/Dockerfile .

Deploying to Hugging Face Spaces

You can easily deploy your OpenEnv environment to Hugging Face Spaces using the openenv push command:

# From the environment directory (where openenv.yaml is located)
openenv push

# Or specify options
openenv push --namespace my-org --private

The openenv push command will:

1. Validate that the directory is an OpenEnv environment (checks for openenv.yaml)
2. Prepare a custom build for Hugging Face Docker space (enables web interface)
3. Upload to Hugging Face (ensuring you're logged in)

Prerequisites

• Authenticate with Hugging Face: The command will prompt for login if not already authenticated

Options

• --directory, -d: Directory containing the OpenEnv environment (defaults to current directory)
• --repo-id, -r: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml)
• --base-image, -b: Base Docker image to use (overrides Dockerfile FROM)
• --private: Deploy the space as private (default: public)

Examples

# Push to your personal namespace (defaults to username/env-name from openenv.yaml)
openenv push

# Push to a specific repository
openenv push --repo-id my-org/my-env

# Push with a custom base image
openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest

# Push as a private space
openenv push --private

# Combine options
openenv push --repo-id my-org/my-env --base-image custom-base:latest --private

After deployment, your space will be available at: https://huggingface.co/spaces/<repo-id>

The deployed space includes:

• Web Interface at /web - Interactive UI for exploring the environment
• API Documentation at /docs - Full OpenAPI/Swagger interface
• Health Check at /health - Container health monitoring
• WebSocket at /ws - Persistent session endpoint for low-latency interactions

🎯 Environment Description

This environment simulates a realistic customer support system where an AI agent handles incoming support tickets. The agent receives ticket information along with customer history and must make decisions about:

• Categorization: Classify tickets into appropriate categories (billing, technical, account, shipping, general)
• Priority Assignment: Assign urgency levels (low, medium, high, critical)
• Team Routing: Route tickets to the right support team (tier1, tier2, billing, technical, management)
• Response Drafting: Write professional, helpful responses to customers

Why This Environment?

Customer support is a critical real-world task faced by businesses of all sizes. Effective ticket management requires:

• Understanding customer intent and context
• Recognizing urgency and priority signals
• Knowledge of organizational structure and escalation paths
• Professional communication skills
• Balancing efficiency with quality

📊 Tasks

The environment includes three tasks of increasing difficulty:

1. Easy: Ticket Classification

• Objective: Correctly categorize support tickets into one of 5 categories
• Success Threshold: 0.8
• Max Steps: 1
• Evaluation: Primarily focuses on category correctness with bonus for appropriate response length

2. Medium: Priority Assignment & Routing

• Objective: Categorize tickets, assign correct priority, AND route to appropriate team
• Success Threshold: 0.75
• Max Steps: 1
• Evaluation: Considers category (35%), priority (30%), team routing (25%), and escalation appropriateness (10%)

3. Hard: Complete Ticket Resolution

• Objective: Full ticket handling including professional response drafting
• Success Threshold: 0.70
• Max Steps: 1
• Evaluation: Comprehensive scoring including:
  • Category correctness (25%)
  • Priority correctness (20%)
  • Team routing (20%)
  • Response quality (25%)
  • Professional tone (10%)
  • Premium customer handling bonuses

🎮 Action Space

The agent takes actions with the following structure:

{
    "category": "billing" | "technical" | "account" | "shipping" | "general",
    "priority": "low" | "medium" | "high" | "critical",
    "assigned_team": "tier1" | "tier2" | "billing" | "technical" | "management",
    "response_draft": str,  # Minimum 10 characters
    "internal_notes": str | None,  # Optional
    "escalate": bool  # Whether to escalate to management
}

Action Guidelines

• Category Selection: Choose based on ticket content keywords
• Priority Assignment: Consider customer history, message urgency, and premium status
• Team Routing: Match expertise to issue type; escalate critical issues
• Response Drafting: Be professional, empathetic, and solution-focused
• Escalation: Use for critical issues or premium customers with high-priority problems

👁️ Observation Space

Each observation includes comprehensive ticket and customer information:

{
    # Ticket Metadata
    "ticket_id": str,  # e.g., "TKT-2025-123456"
    "timestamp": str,  # ISO format timestamp
    "customer_id": str,  # e.g., "CUST-12345"
    "channel": "email" | "chat" | "phone" | "social",

    # Customer Message
    "customer_message": str,  # The support request

    # Customer History
    "account_age_days": int,  # Days since account creation
    "total_tickets": int,  # Total previous tickets
    "resolved_tickets": int,  # Successfully resolved tickets
    "satisfaction_score": float,  # 0.0 - 5.0
    "is_premium": bool,  # Premium customer status
    "lifetime_value": float,  # Customer LTV in USD

    # Additional Context
    "previous_interactions": List[str],  # Previous messages in thread
    "attachments": List[str],  # Attachment filenames

    # Task Context
    "task_id": str,  # "easy", "medium", or "hard"
}

🏆 Reward Function

The environment provides detailed, shaped rewards to guide learning:

Reward Components

Component	Weight	Description
Category Correctness	0.25	Correct ticket categorization
Priority Correctness	0.20	Appropriate urgency level
Team Routing	0.25	Correct team assignment
Response Quality	0.20	Professional, relevant response
Efficiency Bonus	+0.10	All components correct
Premium Handling	+0.05	Appropriate premium customer treatment

Penalties

• Short Responses: -0.15 for responses < 20 characters
• Priority Mismatch: -0.10 for routing critical issues to tier1
• Missing Escalation: -0.05 for not escalating critical issues

Response Quality Evaluation

Response quality is assessed based on:

• Keyword relevance (40%): Addresses ticket concerns
• Professional language (30%): Uses helpful, empathetic terms
• Appropriate length (20%): 10-200 words
• Premium customer language (10%): Acknowledges value for premium customers

📈 Baseline Scores

Baseline performance using OpenAI GPT-4:

Task	Baseline Score	Success Rate	Notes
Easy	0.85	92%	Strong category classification
Medium	0.78	81%	Good routing, occasional priority errors
Hard	0.72	68%	Struggles with response quality

Running Baseline

# Set your OpenAI API key
export OPENAI_API_KEY="your-key-here"

# Run baseline inference
python baseline.py --task easy --episodes 100
python baseline.py --task medium --episodes 100
python baseline.py --task hard --episodes 100

🧪 Validation

Validate the environment conforms to OpenEnv spec:

openenv validate

🎓 Learning Challenges

Agents must learn to:

1. Parse Natural Language: Extract intent from varied customer messages
2. Contextual Reasoning: Use customer history to inform decisions
3. Urgency Detection: Identify critical issues requiring immediate attention
4. Domain Knowledge: Understand when to route to specialized teams
5. Professional Communication: Draft appropriate, empathetic responses
6. Priority Balancing: Handle premium customers appropriately without neglecting others

Advanced Usage

Connecting to an Existing Server

If you already have a Customer Support Env environment server running, you can connect directly:

from customer_support_env import CustomerSupportEnv

# Connect to existing server
customer_support_envenv = CustomerSupportEnv(base_url="<ENV_HTTP_URL_HERE>")

# Use as normal
result = customer_support_envenv.reset()
result = customer_support_envenv.step(CustomerSupportAction(message="Hello!"))

Note: When connecting to an existing server, customer_support_envenv.close() will NOT stop the server.

Using the Context Manager

The client supports context manager usage for automatic connection management:

from customer_support_env import CustomerSupportAction, CustomerSupportEnv

# Connect with context manager (auto-connects and closes)
with CustomerSupportEnv(base_url="http://localhost:8000") as env:
    result = env.reset()
    print(f"Reset: {result.observation.echoed_message}")
    # Multiple steps with low latency
    for msg in ["Hello", "World", "!"]:
        result = env.step(CustomerSupportAction(message=msg))
        print(f"Echoed: {result.observation.echoed_message}")

The client uses WebSocket connections for:

• Lower latency: No HTTP connection overhead per request
• Persistent session: Server maintains your environment state
• Efficient for episodes: Better for many sequential steps

Concurrent WebSocket Sessions

The server supports multiple concurrent WebSocket connections. To enable this, modify server/app.py to use factory mode:

# In server/app.py - use factory mode for concurrent sessions
app = create_app(
    CustomerSupportEnvironment,  # Pass class, not instance
    CustomerSupportAction,
    CustomerSupportObservation,
    max_concurrent_envs=4,  # Allow 4 concurrent sessions
)

Then multiple clients can connect simultaneously:

from customer_support_env import CustomerSupportAction, CustomerSupportEnv
from concurrent.futures import ThreadPoolExecutor

def run_episode(client_id: int):
    with CustomerSupportEnv(base_url="http://localhost:8000") as env:
        result = env.reset()
        for i in range(10):
            result = env.step(CustomerSupportAction(message=f"Client {client_id}, step {i}"))
        return client_id, result.observation.message_length

# Run 4 episodes concurrently
with ThreadPoolExecutor(max_workers=4) as executor:
    results = list(executor.map(run_episode, range(4)))

Development & Testing

Direct Environment Testing

Test the environment logic directly without starting the HTTP server:

# From the server directory
python3 server/customer_support_env_environment.py

This verifies that:

• Environment resets correctly
• Step executes actions properly
• State tracking works
• Rewards are calculated correctly

Running Locally

Run the server locally for development:

uvicorn server.app:app --reload

Project Structure

customer_support_env/
├── .dockerignore         # Docker build exclusions
├── __init__.py            # Module exports
├── README.md              # This file
├── openenv.yaml           # OpenEnv manifest
├── pyproject.toml         # Project metadata and dependencies
├── uv.lock                # Locked dependencies (generated)
├── client.py              # CustomerSupportEnv client
├── models.py              # Action and Observation models
└── server/
    ├── __init__.py        # Server module exports
    ├── customer_support_env_environment.py  # Core environment logic
    ├── app.py             # FastAPI application (HTTP + WebSocket endpoints)
    └── Dockerfile         # Container image definition