Hugging Face's logo

Spaces:
AlgoCore
/
support-ticket-env
Sleeping

App Files Community
support-ticket-env / README.md
Vighnesh
Highlight Theme 3.1 + Scaler sub-theme fit, promote GRPO results section
3d83a5d
preview code
|
raw
history blame contribute delete
6.95 kB
metadata 
title: Support Ticket Env
emoji: 🎫
colorFrom: blue
colorTo: green
sdk: docker
tags:
  - openenv
pinned: false

Customer Support Ticket Resolution Environment

🏆 OpenEnv x Scalar Hackathon — Theme #3.1 Professional Tasks | Sub-theme: Scaler AI Labs — Multi-App RL Environment for Enterprise Workflows

A real-world OpenEnv environment where an AI agent acts as a customer support executive, triaging and resolving incoming tickets — simulating complex enterprise workflows, business rule nuances, and multi-step decision making under partial observability.

Overview

Customer support triage is one of the most common real-world tasks for AI agents in enterprise settings. Every company handles thousands of tickets daily. Getting the classification wrong routes the ticket to the wrong team. Choosing the wrong action has direct business impact. This environment trains agents to handle exactly this challenge — with real tool interaction, dynamic state, and a multi-step reward structure that resists shortcuts.

Quick Start 

from support_ticket_env import SupportAction, SupportTicketEnv

with SupportTicketEnv(base_url="https://algocore-support-ticket-env.hf.space").sync() as env:
    # Task 1 - Classify a ticket
    result = env.reset(task_id=1, seed=42)
    print(result.observation.ticket_text)

    result = env.step(SupportAction(action_type="classify", category="billing"))
    print(result.reward)  # 1.0 if correct

Tasks

Task Difficulty Description Score Range
Task 1 Easy Classify ticket into correct category 0.0 - 1.0
Task 2 Medium Classify then choose correct action 0.0 - 1.0
Task 3 Hard Resolve a full queue of 3 tickets 0.0 - 1.0

Action Space

Actions are SupportAction Pydantic objects:

Field Type Required Values
action_type str always classify / reply / escalate / close
category str for classify billing / technical / account / general / refund
reply_text str for reply free text
reason str optional free text

Observation Space

Field Type Description
ticket_id str Unique ticket ID
ticket_text str Customer message
task_id int 1, 2, or 3
current_category str Category assigned so far
resolved bool Whether ticket is resolved
step_count int Steps taken this episode
feedback str Human-readable feedback
reward float Reward signal
done bool Episode finished

Reward Function

Rewards provide partial progress signals throughout the trajectory:

  • Task 1: 1.0 for correct category, 0.0 for wrong
  • Task 2: 1.0 correct action, 0.5 defensible alternative, 0.3 classification only
  • Task 3: 0.20 classification + 0.40 action + 0.25 reply quality + 0.15 efficiency bonus
  • Penalty: -0.05 per step over 10 (loop deterrent)

Project Structure 

support_ticket_env/
├── __init__.py               # Package exports
├── models.py                 # SupportAction, SupportObservation, SupportState
├── tickets.py                # Ticket dataset with ground-truth labels
├── graders.py                # Reward/grader functions for all 3 tasks
├── client.py                 # EnvClient subclass
├── baseline.py               # Baseline inference script
├── get_baseline.py           # Fetch & save baseline results
├── gradio_ui.py              # Interactive Gradio playground UI
├── make_chart.py             # Plot training reward curves
├── plot_results.py           # Visualise evaluation results
├── grpo_results.png          # GRPO training results chart
├── reward_chart.png          # Reward curve chart
├── openenv.yaml              # Environment metadata
├── Dockerfile                # Container definition
├── train_sft.ipynb           # Step 1: SFT pre-training notebook
├── train_grpo.ipynb          # Step 2: GRPO fine-tuning notebook
└── server/
    ├── app.py                # FastAPI entry point (+ Gradio UI mounted at /playground)
    ├── support_environment.py # Environment logic
    └── requirements.txt      # Server dependencies

Setup 

# Install dependencies
pip install openenv-core fastapi uvicorn pydantic gradio openai pyyaml

# Run locally
uvicorn support_ticket_env.server.app:app --host 0.0.0.0 --port 7860

# Or via Docker
docker build -t support-ticket-env .
docker run -p 7860:7860 support-ticket-env

# Run tests
python run_tests.py

🎮 Playground UI available at http://localhost:7860/playground once the server is running.

📈 Training Results (GRPO) — Evidence of Improvement

Fine-tuned Qwen2.5-0.5B-Instruct using 2-stage training (SFT pre-training → GRPO) via HuggingFace TRL over 700+ steps on the live environment API:

GRPO Training Results

Task Before GRPO After GRPO Improvement
Task 1 - Classification 0.67 1.00 +49% 🚀
Task 2 - Action Selection 0.12 0.48 +300% 🚀
Task 3 - Full Resolution 0.08 0.23 +187% 🚀
Overall 0.29 0.57 +96% 🚀

Baseline Scores

Measured with gpt-4o-mini, seeds [42, 7, 123]:

Task Avg Score
Task 1 - Classification 0.87
Task 2 - Action Selection 0.71
Task 3 - Full Resolution 0.58
Overall 0.72

🎯 Why This Fits Theme 3.1 — Professional Tasks

"Real interaction with tools, APIs, or dynamic systems where the model does real hard work instead of exploiting shortcuts"

  • ✅ Live FastAPI environment — agent interacts with a real stateful API, not a simulation
  • ✅ No shortcut exploitation — reward function penalises loops (-0.05/step over 10), forces genuine reasoning
  • ✅ Persistent world state — ticket queue, classification state, and resolution state tracked across steps
  • ✅ Multi-step causal reasoning — classify → choose action → craft reply → resolve, all causally linked
  • ✅ Enterprise workflow complexity — billing, technical, account, general, refund categories with real business rules
  • ✅ Scaler AI Labs sub-theme — demonstrates complex enterprise workflows and business rule nuances in an RL environment

Links

License

MIT