project.md

# B2B Support Triage OpenEnv Benchmark - Complete Project Overview

## 1) What this project is

This project is a complete OpenEnv simulation environment designed for **real-world B2B SaaS support triage**.

Instead of a toy game, it models a workflow support engineers actually perform:
- classify an incoming ticket
- set urgency/priority
- route to the correct team queue
- define SLA expectation
- (hard task) provide policy-compliant customer communication

It exposes the standard OpenEnv contract:
- `reset(...)`
- `step(action)`
- `state()`

with typed Pydantic models and deterministic graders.

---

## 2) Why this environment is useful

### Practical utility
This environment can evaluate agent quality on an enterprise operation where correctness, consistency, and policy compliance matter.

### Deterministic benchmarking
All tasks are fixture-driven and deterministic, so model comparisons are reproducible.

### Multi-step reasoning
Agents must make a sequence of structured decisions, not just one-shot classification.

---

## 3) High-level architecture

### Main components

1. **Environment runtime**
- File: `server/support_triage_environment.py`
- Implements environment state machine and reward shaping.

2. **Typed schemas**
- File: `models.py`
- Defines action, observation, state, and reward-breakdown models.

3. **Task fixtures**
- File: `fixtures/tasks.json`
- Defines easy/medium/hard scenarios, allowed values, answer keys, and policy hints.

4. **Grader logic**
- File: `graders.py`
- Computes normalized score in `[0,1]` using weighted criteria per task.

5. **OpenEnv API server wiring**
- File: `server/app.py`
- Binds environment + typed models into OpenEnv HTTP/WebSocket endpoints.

6. **Inference baseline**
- File: `inference.py`
- Runs all tasks using OpenAI client and emits required structured logs.

7. **Client adapter**
- File: `client.py`
- Converts typed Python actions/observations to/from wire payloads.

---

## 4) OpenEnv contract and endpoint behavior

### Manifest
- File: `openenv.yaml`
- Declares:
  - `name: b2b_support_triage_env`
  - `runtime: fastapi`
  - `app: server.app:app`
  - `port: 8000`

### Runtime endpoints
- `POST /reset`
- `POST /step`
- `GET /state`
- `GET /health`
- `GET /schema`

### Environment lifecycle

1. `reset(task_id=..., seed=...)`
- loads selected task fixture
- clears previous episode state
- returns initial observation with visible ticket + dynamic plan hints

2. `step(action)`
- validates action semantics and allowed values
- updates decision state
- computes score delta and shaped reward
- checks done conditions (`submit` or max steps)

3. `state()`
- returns current internal state object (episode info, step count, cumulative reward, history)

---

## 5) Data model design (typed)

## 5.1 Action model
File: `models.py`

`B2BSupportTriageAction` fields:
- `action_type`: enum
  - `classify`
  - `set_priority`
  - `route`
  - `draft_reply`
  - `submit`
- `ticket_id`: required for non-`submit`
- `payload`: action-specific content

`payload` supports:
- `category`
- `priority`
- `route_queue`
- `sla_minutes`
- `escalate`
- `reply_text`

### Validation rules
- non-submit requires `ticket_id`
- required fields per action:
  - `classify -> category`
  - `set_priority -> priority`
  - `route -> route_queue + sla_minutes`
  - `draft_reply -> reply_text`
  - `submit -> none`

This enforces structured behavior and prevents ambiguous free-text action control.

## 5.2 Observation model
`B2BSupportTriageObservation` includes:
- `task_id`, `step_index`, `max_steps`
- `visible_ticket`
- `current_plan` (next actions + policy hints)
- `applied_decisions`
- `last_action_error`
- `progress_score` (grader score in `[0,1]`)
- `reward_breakdown`:
  - `correctness_delta`
  - `policy_bonus`
  - `repeat_penalty`
  - `invalid_penalty`
  - `terminal_bonus`
- inherited OpenEnv fields `reward`, `done`, `metadata`

## 5.3 State model
`B2BSupportTriageState` tracks:
- episode metadata (`episode_id`, `step_count`, `task_id`, `seed`)
- `max_steps`
- `cumulative_reward`
- `applied_decisions`
- `action_history`
- `completion_flags`

---

## 6) Task system (easy -> medium -> hard)

Defined in `fixtures/tasks.json`.

## 6.1 Easy
Goal:
- correct category + priority for one billing ticket

Hidden answer key includes:
- `category=billing`
- `priority=medium`
- plus routing defaults

## 6.2 Medium
Goal:
- category + priority + queue routing + SLA

Hidden answer key includes:
- `category=billing`
- `priority=high`
- `route_queue=billing-l2`
- `sla_minutes=120`

## 6.3 Hard
Goal:
- security incident handling with escalation + compliant customer response

Hidden answer key includes:
- `category=security`
- `priority=urgent`
- `route_queue=security-incident-response`
- `sla_minutes=120`
- `escalate=true`
- required reply phrases

---

## 7) Grading mechanics

File: `graders.py`

Each task has deterministic weighted scoring.

## 7.1 Easy weighting
- category: 0.6
- priority: 0.4

## 7.2 Medium weighting
- category: 0.35
- priority: 0.25
- route_queue: 0.25
- sla_minutes: 0.15

## 7.3 Hard weighting
- category: 0.15
- priority: 0.15
- route_queue: 0.15
- sla_minutes: 0.10
- escalate: 0.20
- reply_policy phrase coverage: 0.25

The result is clamped and normalized to `[0,1]`.

### Determinism
The grader is pure:
- no network calls
- no random operations
- same input decisions always yield same score

---

## 8) Reward shaping logic

File: `server/support_triage_environment.py`

Reward is step-wise, dense enough for learning signals, and punishes bad behavior.

### Positive components
- `correctness_delta`: only when progress score increases
- `policy_bonus`: bonus for substantive policy-style draft replies
- `terminal_bonus`: `0.2 * final_score` when episode ends

### Penalty components
- `invalid_penalty`: invalid ticket/value/action
- `repeat_penalty`:
  - repeated identical action
  - contradiction penalties (changing already-set value)
  - no-progress streak penalties

### Episode boundaries
- done on `submit`
- done on max step limit
- steps after done receive penalty (`episode_already_done`)

This discourages looping and rewards incremental correctness.

---

## 9) Inference baseline design

File: `inference.py`

## 9.1 Runtime setup
Reads:
- `HF_TOKEN` (required)
- `API_BASE_URL` (default set)
- `MODEL_NAME` (default set)
- `LOCAL_IMAGE_NAME` (default set)

## 9.2 Task execution
- runs tasks in fixed order: easy, medium, hard
- fixed seeds per task for reproducibility

## 9.3 Action selection strategy
- requests JSON action from model (OpenAI client call)
- validates/coerces action
- falls back to deterministic policy if output invalid or misaligned

This keeps baseline robust and reproducible while still exercising LLM calls.

## 9.4 Required stdout format
Per task episode:
- `[START] ...`
- multiple `[STEP] ...`
- `[END] ...`

Then aggregate score summary line.

---

## 10) Deployment and packaging

## 10.1 Containerization
- Dockerfile: `server/Dockerfile`
- builds runnable image exposing uvicorn on port 8000
- includes healthcheck on `/health`

## 10.2 Python packaging
- `pyproject.toml` defines project metadata and dependencies
- installs environment package editable in image

## 10.3 OpenEnv compatibility
- `openenv validate -v` passes
- simulation API surface is compliant

---

## 11) Testing strategy

Files under `tests/` verify behavior:

- `test_environment.py`
  - reset clean state
  - invalid ticket penalty
  - hard task can reach full score
  - max-step termination behavior

- `test_graders.py`
  - perfect answer gives 1.0
  - weighted partial score check
  - deterministic hard-task reply scoring

- `test_inference_logging.py`
  - `[START]/[STEP]/[END]` formatting checks

This coverage ensures both task semantics and evaluation fidelity.

---

## 12) How to use the project locally

## 12.1 End-to-end helper
```bash
./run_all_checks.sh
```
Runs tests, validation, docker build/run smoke checks, and optional inference.

## 12.2 Inference helper (prompts for token if missing)
```bash
./run_inference.sh
```

## 12.3 Manual core commands
```bash
pytest -q
openenv validate -v
docker build -t b2b_support_triage_env-env:latest -f server/Dockerfile .
docker run --rm -p 8000:8000 b2b_support_triage_env-env:latest
```

Detailed runbook: `local_setup.md`.

---

## 13) Usability summary for your hackathon goal

What this project gives you:
- clear real-world domain
- structured and typed agent action space
- deterministic multi-level tasks
- strong reward shaping (partial progress + anti-loop penalties)
- reproducible baseline script with required logs
- Docker + OpenEnv validation support

This aligns well with your round requirements and helps reduce submission-time surprises.

---

## 14) Current limitations and next improvements

### Current limitations
- single-ticket episode scope (per task) rather than multi-ticket queue backlog
- deterministic phrase matching for hard reply policy (simple lexical strategy)
- baseline includes deterministic fallback policy (useful for reliability, less pure model-only behavior)

### Next upgrades (optional)
- multi-ticket routing episodes with queue-level capacity constraints
- richer policy engine with explicit rule graph and violation tags
- adversarial or noisy customer phrasing variants per seed
- benchmark report exporter with per-criterion analytics

---

## 15) Security and publication posture

- fixtures are synthetic
- no hardcoded credentials
- token usage is environment-variable based only
- suitable for external publication, assuming you do not commit local secret files (`.env`, shell history, etc.)