README.md

metadata

title: Customer Support OpenEnv
emoji: 👽
colorFrom: blue
colorTo: green
sdk: docker
pinned: false

🎧 Customer Support OpenEnv

OpenEnv Hackathon submission — a production-grade customer support simulation environment for training and evaluating LLM-based support agents.

---

🌍 Why This Environment?

Customer support is one of the most common enterprise AI use cases, yet no standard RL/agent benchmark exists for it.
This environment fills that gap: it provides a realistic, multi-difficulty, fully graded benchmark where agents must:

• 🏷️ Classify support tickets into the correct category
• ✍️ Respond empathetically and accurately to customer issues
• 🗣️ Hold multi-turn conversations — clarify, resolve, and close tickets

All graded automatically with clear, deterministic rubrics (0.0–1.0) and meaningful partial rewards on every turn.

---

⚙️ How It Works — Workflow

┌─────────────────────────────────────────────────────────────────────┐
│                         AGENT LOOP                                  │
│                                                                     │
│   ┌──────────┐   POST /reset   ┌─────────────────────────────────┐  │
│   │          │ ──────────────► │         FastAPI Server          │  │
│   │  Agent   │                 │         (server/app.py)         │  │
│   │ (client) │ ◄────────────── │                                 │  │
│   │          │   Observation   │  ┌──────────────────────────┐   │  │
│   │          │                 │  │   SupportEnvironment     │   │  │
│   │          │   POST /step    │  │   (environment.py)       │   │  │
│   │          │ ──────────────► │  │                          │   │  │
│   │          │   obs+reward    │  │  15 scenarios × 5 types  │   │  │
│   │          │ ◄────────────── │  │  3 graders (easy/med/hrd)│   │  │
│   └──────────┘                 │  │  Cumulative reward logic │   │  │
│                                │  └──────────────────────────┘   │  │
│   GET /state  ──────────────►  │                                 │  │
│   POST /grader ─────────────►  │  Score + turn breakdown         │  │
│   GET /tasks  ──────────────►  │  Task list + action schemas     │  │
│   POST /baseline ───────────►  │  Built-in oracle agent scores   │  │
└───────────────────────────────────────────────────────────────────-─┘

Episode lifecycle:

1. Agent calls POST /reset → gets the opening customer message
2. Agent sends replies via POST /step → gets observation + reward each turn
3. Agent calls POST /grader → gets full score breakdown with turn_scores
4. Episode ends when done: true in the observation

---

🗂️ Environment Description

Simulates a customer support system across 5 issue categories with 15 unique scenarios (3 per category):

Category	Sample Scenarios
Refund	Missing delivery refund, damaged product return, cancelled-order refund delay
Technical	App crash after update, Slack webhook error, PDF export blank
Shipping	3-week missing order, wrong-door delivery, split shipment partial arrival
Billing	Duplicate subscription charge, unauthorized upgrade charge, invoice ghost charge
Account	Password reset email missing, account suspended, email address transfer

---

📥 Action Space

Each agent step submits a SupportAction:

Field	Type	Required	Description
message	str	✅	The agent's text reply to the customer
intent	str	❌	Declared intent: "classify", "respond", "clarify", "escalate", or "close"

---

📤 Observation Space

The environment returns a SupportObservation after each step:

Field	Type	Description
conversation	List[str]	Full message history (alternating customer / agent)
customer_query	str	The latest customer message the agent must address
task_name	str	Difficulty tier: "easy", "medium", or "hard"
done	bool	Whether the episode has ended
reward	float | None	Step-level reward (0.0–1.0); None on opening observation
cumulative_reward	float	Running average reward across all turns so far
turn_scores	List[float]	Per-turn reward breakdown (useful for analysis)
info	str | None	Optional context or hints for the agent

---

📋 Tasks

🟢 Easy — Ticket Classification

Objective: Output the correct issue category for a customer message.

Scoring:

• ✅ 1.0 — Exact category in reply (refund, billing, etc.)
• 🟡 0.5 — Partial keyword match
• ❌ 0.0 — Wrong or missing category
• Max steps: 1

---

🟡 Medium — Single-Turn Response

Objective: Write a complete, empathetic reply resolving the customer's issue.

Scoring rubric:

Component	Reward
+0.20 per matching keyword (max 4)	0.00–0.80
Empathy detected (apologize/sorry/understand)	+0.10
Reply length > 80 chars	+0.10
Unnecessary escalation penalty	−0.20

All scores clamped to [0.0, 1.0]. Max steps: 1

---

🔴 Hard — Multi-Turn Conversation

Objective: Handle a 3-turn dialogue: clarify → resolve → close.

Turn	Behaviour	Max Reward
1 (clarify)	Ask a ? question (bonus if on-topic)	+0.40
2 (resolve)	Keyword + empathy + detail	+0.50
3 (close)	Polite closing phrase	+0.30

cumulative_reward = mean(turn_scores), clamped to [0.0, 1.0]. Max steps: 10

---

🎯 Reward Function Design

Signal	When	Magnitude
Correct classification	Easy, step 1	+1.0 / +0.5 / 0.0
Keyword coverage	Medium & Hard turn 2	+0.20 per keyword
Empathy language	Medium & Hard turn 2	+0.10–+0.12
Response detail (length)	Medium & Hard turn 2	+0.08–+0.10
Clarifying question	Hard turn 1	+0.30 (+0.10 bonus)
Polite close	Hard turn 3	+0.30
Unnecessary escalation	Medium	−0.20
Exceeding max steps	All tasks	Episode terminates

Key property: Every turn gives a partial signal — agents never wait until the end to learn if they did well.

---

🔌 API Endpoints

Method	Endpoint	Description
GET	/	Health check
POST	/reset	Start a new episode
POST	/step	Submit an agent action
GET	/state	Current session internal state
GET	/tasks	Task list with typed action schemas
POST	/grader	Score + turn breakdown for completed episode
POST	/baseline	Run built-in oracle agent, return average scores

---

🚀 Setup & Usage

1. Clone

git clone https://huggingface.co/spaces/sanathkumarps/customer_support_env
cd customer_support_env

2. Install

pip install -r requirements.txt

3. Run locally

uvicorn server.app:app --host 0.0.0.0 --port 7860 --reload

Visit http://localhost:7860/docs for interactive Swagger docs.

4. Run with Docker

docker build -t customer_support_env .
docker run -p 7860:7860 customer_support_env

5. Run the baseline evaluation

export GROQ_API_KEY="gsk-..."          # required
export ENV_BASE_URL="http://localhost:7860"   # optional
export GROQ_MODEL="llama-3.1-8b-instant"       # optional, default: llama-3.1-8b-instant

python run_baseline.py

Results print to console and save to baseline_scores.json.

6. Quick API test

# Health check
curl http://localhost:7860/

# Start easy episode
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_name":"easy","seed":42}'

# Submit action (replace SESSION_ID)
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"session_id":"SESSION_ID","message":"refund","intent":"classify"}'

# Get grader score
curl -X POST http://localhost:7860/grader \
  -H "Content-Type: application/json" \
  -d '{"session_id":"SESSION_ID"}'

---

📊 Baseline Scores

Task	Avg Score	Model	Episodes
Easy	0.90	llama-3.1-8b-instant	5
Medium	0.55	llama-3.1-8b-instant	5
Hard	0.40	llama-3.1-8b-instant	5

Run python run_baseline.py after starting the server to generate fresh scores.

---

📁 Project Structure

customer_support_env/
├── models.py              # Pydantic models: SupportAction, SupportObservation, SupportState
├── client.py              # OpenEnv typed client library
├── openenv.yaml           # OpenEnv metadata manifest
├── requirements.txt       # Python dependencies (groq, fastapi, uvicorn…)
├── run_baseline.py        # llama-3.1-8b-instant baseline evaluation script
├── README.md              # This file
├── Dockerfile             # Container config for Hugging Face Spaces
├── .dockerignore          # Excludes venv, __pycache__, .env from image
├── baseline_scores.json   # Auto-generated baseline results
└── server/
    ├── __init__.py        # Package marker
    ├── app.py             # FastAPI server — all 7 endpoints
    └── environment.py     # Core env: 15 scenarios, 3 graders, reward logic

---

🏗️ Architecture

models.py                  ← Pydantic contracts (Action / Observation / State)
    │
    ├── server/environment.py   ← Episode logic, 15 scenarios, 3 graders
    │       │
    │       └── Reward signals: per-turn partial rewards + cumulative score
    │
    └── server/app.py           ← FastAPI: /reset /step /state /tasks /grader /baseline
            │
            └── client.py       ← Typed client for external scripts

---

📜 License

MIT — see LICENSE.

---

Built for the OpenEnv Hackathon 🚀  |  Customer support AI benchmark that fills a real gap.