Initial DataClean OpenEnv Space

.dockerignore

@@ -0,0 +1,18 @@
+__pycache__
+*.pyc
+*.pyo
+.git
+.gitignore
+*.md
+!README.md
+.env
+.venv
+venv
+node_modules
+.pytest_cache
+.mypy_cache
+01-*.md
+02-*.md
+03-*.md
+04-*.md
+05-*.md

.gitignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.mypy_cache/
+.venv/
+venv/
+.env
+.env.*
+.DS_Store
+.claude/
+.zencoder/
+.zenflow/

01-PROBLEM-UNDERSTANDING.md

@@ -0,0 +1,235 @@
+# Problem Understanding: Meta PyTorch x Scaler Hackathon
+
+## What Is This Hackathon About?
+
+This hackathon asks you to **build a real-world simulation environment** that AI agents can learn from using Reinforcement Learning (RL). The environment must follow the **OpenEnv specification** -- a standardized framework by Meta PyTorch for creating, deploying, and using isolated execution environments for agentic RL training.
+
+---
+
+## The Core Ask (One Sentence)
+
+> Build a complete OpenEnv-compliant environment that simulates a **real-world task** (not games/toys), with 3+ graded tasks, deploy it to Hugging Face Spaces, and provide a working inference script.
+
+---
+
+## Breaking Down the Problem Statement
+
+### 1. What Is an "OpenEnv Environment"?
+
+An OpenEnv environment is a **containerized microservice** that exposes an RL interface via HTTP/WebSocket:
+
+```
+Your AI Agent (Client)
+    |
+    |  step(action) / reset() / state()
+    |
+    v
+Docker Container (Server)
+    FastAPI Server
+    Your Environment Logic
+```
+
+Think of it like a **REST API that an AI agent interacts with** to learn. The agent:
+1. Observes the current state
+2. Takes an action
+3. Receives a reward signal
+4. Repeats until done
+
+### 2. What Does "Real-World Task" Mean?
+
+The environment must simulate something **humans actually do at work or in life**. NOT games, NOT toy problems.
+
+**Good examples (explicitly listed):**
+- Email triage (sorting/prioritizing emails)
+- Code review (reviewing pull requests)
+- Data cleaning (fixing messy datasets)
+- Scheduling (managing calendars/meetings)
+- Customer support (answering tickets)
+- Content moderation (classifying content)
+
+**Bad examples (will get low scores or DQ):**
+- Tic-tac-toe, chess, 2048 (games)
+- CartPole, MountainCar (toy RL benchmarks)
+- Number guessing (trivial)
+
+### 3. The Three APIs You Must Implement
+
+| API | What It Does | Returns |
+|-----|-------------|---------|
+| `reset()` | Starts a new episode from scratch | Initial observation |
+| `step(action)` | Agent takes an action, environment advances | observation, reward, done, info |
+| `state()` | Returns current episode metadata | Current state |
+
+### 4. Typed Models Required (Pydantic)
+
+You must define strongly-typed data models:
+
+```python
+class MyAction(Action):       # What the agent can DO
+    ...
+
+class MyObservation(Observation):  # What the agent can SEE
+    ...
+
+class MyState(State):          # Episode metadata
+    ...
+```
+
+### 5. The Three Tasks (Easy -> Medium -> Hard)
+
+You need **minimum 3 tasks** with increasing difficulty:
+
+| Task | Difficulty | Score Range | What It Tests |
+|------|-----------|-------------|---------------|
+| Task 1 | Easy | 0.0 - 1.0 | Basic competence |
+| Task 2 | Medium | 0.0 - 1.0 | Intermediate reasoning |
+| Task 3 | Hard | 0.0 - 1.0 | Should challenge frontier models |
+
+Each task has a **programmatic grader** that:
+- Scores performance from 0.0 to 1.0
+- Has clear, deterministic success/failure criteria
+- Is reproducible (same input = same score)
+
+### 6. Meaningful Reward Function
+
+The reward must:
+- Provide signal **throughout** the episode (not just at the end)
+- Reward **partial progress** (not just binary success/fail)
+- Penalize **bad behavior** (infinite loops, destructive actions)
+
+**Bad reward:** `return 1.0 if solved else 0.0` (sparse, no learning signal)
+
+**Good reward:** Gives incremental feedback as the agent makes progress
+
+### 7. Baseline Inference Script
+
+- Named `inference.py` in the project root
+- Uses **OpenAI client** (not raw HTTP)
+- Reads `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` from env vars
+- Runs against all 3 tasks and produces reproducible scores
+- Must complete in **< 20 minutes**
+- Must run on **2 vCPU, 8GB RAM**
+
+### 8. Deployment Requirements
+
+- **Hugging Face Space** tagged with `openenv`
+- **Working Dockerfile** (docker build + docker run must work)
+- **README** with full documentation
+
+---
+
+## Scoring Breakdown (100 points total)
+
+| Criterion | Weight | What Judges Look For |
+|-----------|--------|---------------------|
+| **Real-world utility** | 30% | Does it model a genuine task? Would anyone actually use this? |
+| **Task & grader quality** | 25% | Well-defined tasks? Fair graders? Good difficulty progression? |
+| **Environment design** | 20% | Clean state management? Good reward shaping? Sensible boundaries? |
+| **Code quality & spec compliance** | 15% | OpenEnv spec compliant? Docker works? Clean code? |
+| **Creativity & novelty** | 10% | Novel domain? Interesting reward design? Clever mechanics? |
+
+### How to Score High in Each Category
+
+#### Real-world utility (30% -- BIGGEST weight)
+- Score 26-30: "Fills a real gap, immediate value for the RL/agent community"
+- Pick a domain where AI agents are **actually being deployed** or would benefit from training
+- The task should feel like something a human knowledge worker does daily
+- Ask: "Would a company pay for an agent that can do this?"
+
+#### Task & grader quality (25%)
+- 3+ tasks with clear difficulty range
+- Graders produce float scores 0.0-1.0
+- Graders are deterministic and reproducible
+- Hard task genuinely challenges GPT-4 / Claude level models
+
+#### Environment design (20%)
+- `reset()` produces clean state (no leakage between episodes)
+- Action/observation types are well-designed and documented
+- Reward function provides varying signal (not sparse)
+- Episode boundaries make sense
+
+#### Code quality & spec compliance (15%)
+- `openenv validate` passes
+- `docker build && docker run` works
+- HF Space deploys and responds
+- Baseline script runs and reproduces scores
+
+#### Creativity & novelty (10%)
+- Domain not already in OpenEnv
+- Interesting reward shaping
+- Clever mechanics
+
+---
+
+## Disqualification Criteria (Instant Fail)
+
+1. Environment doesn't deploy or respond
+2. Plagiarized or trivially modified existing environments
+3. Graders that always return the same score
+4. No baseline inference script
+5. HF Space doesn't return 200 / respond to `reset()`
+6. Dockerfile doesn't build
+7. Baseline doesn't reproduce scores
+8. Fewer than 3 tasks
+
+---
+
+## Judging Process (3 Phases)
+
+### Phase 1: Automated Validation (Pass/Fail Gate)
+- HF Space ping -> must return 200 and respond to `reset()`
+- `openenv validate` on openenv.yaml, models, endpoints
+- `docker build` on submitted repo
+- Run inference script -> must complete without error
+- Enumerate 3+ tasks, run graders, verify 0.0-1.0 scores
+
+### Phase 2: Agentic Evaluation (Scored)
+- Re-run baseline agent
+- Run a standard Open LLM agent (e.g., Nemotron 3 Super) against ALL environments
+- Check score variance
+
+### Phase 3: Human Review (Top Submissions Only)
+- Meta and Hugging Face engineers review for:
+  - Real-world utility
+  - Creativity
+  - Exploit checks (are graders gameable?)
+
+---
+
+## Key Infrastructure Constraints
+
+| Constraint | Value |
+|-----------|-------|
+| Inference script runtime | < 20 minutes |
+| Machine specs | 2 vCPU, 8GB RAM |
+| LLM client | Must use OpenAI client |
+| Environment variables | `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` |
+| Script name | `inference.py` (root directory) |
+| Deployment | HF Spaces with Docker |
+
+---
+
+## Summary: What You Need to Deliver
+
+```
+your-project/
+├── inference.py           # Baseline inference script (OpenAI client)
+├── openenv.yaml           # Environment metadata
+├── models.py              # Pydantic Action, Observation, State models
+├── client.py              # HTTP/WebSocket client for the environment
+├── README.md              # Full documentation
+├── server/
+│   ├── app.py             # FastAPI server
+│   ├── environment.py     # Your environment logic
+│   ├── Dockerfile         # Container definition
+│   └── requirements.txt   # Dependencies
+└── pyproject.toml         # Package definition
+```
+
+Deliverables:
+1. Working HF Space (tagged `openenv`)
+2. Working Docker image
+3. 3+ tasks with graders (easy/medium/hard)
+4. Meaningful reward function
+5. Baseline inference script with reproducible scores
+6. README with full documentation

02-OPENENV-ARCHITECTURE.md

@@ -0,0 +1,452 @@
+# OpenEnv Architecture: Deep Understanding
+
+## What Is OpenEnv?
+
+OpenEnv is an **end-to-end framework** for creating, deploying, and using **isolated execution environments** for agentic RL training. It uses familiar Gymnasium-style APIs (`step()`, `reset()`, `state()`) but wraps them in a **production-ready microservice architecture**.
+
+**PyPI package:** `openenv-core`
+**GitHub:** https://github.com/meta-pytorch/OpenEnv
+**License:** BSD 3-Clause
+
+---
+
+## Core Philosophy
+
+> "RL environments should be like microservices"
+
+Just as you don't run your database in the same process as your web server, OpenEnv separates the **environment** (server in Docker) from the **training code** (client).
+
+| Traditional (Gym) | OpenEnv |
+|-------------------|---------|
+| Same process | Separate container |
+| Python only | Any language (HTTP API) |
+| Arrays and dicts | Type-safe Pydantic models |
+| "Works on my machine" | Docker everywhere |
+| Hard to scale | Deploy to K8s |
+| Can crash your training | Isolated and secure |
+
+---
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────┐
+│  YOUR CODE (Client Side)                        │
+│                                                 │
+│  env = MyEnv(base_url="http://localhost:8000")  │
+│  result = env.reset()      # Type-safe!         │
+│  result = env.step(action) # Type-safe!         │
+│  state  = env.state()      # Type-safe!         │
+│                                                 │
+└──────────────────┬──────────────────────────────┘
+                   │
+                   │  WebSocket (/ws) - primary
+                   │  HTTP (/reset, /step, /state) - fallback
+                   │
+┌──────────────────▼──────────────────────────────┐
+│  DOCKER CONTAINER (Server Side)                 │
+│                                                 │
+│  FastAPI Server (app.py)                        │
+│    ├── /ws      → WebSocket session handler     │
+│    ├── /reset   → POST: Reset environment       │
+│    ├── /step    → POST: Execute action          │
+│    ├── /state   → GET: Get current state        │
+│    ├── /health  → GET: Health check             │
+│    ├── /docs    → GET: OpenAPI docs             │
+│    └── /web     → GET: Interactive web UI       │
+│                                                 │
+│  Environment (environment.py)                   │
+│    Your simulation logic lives here             │
+│                                                 │
+│  Models (models.py)                             │
+│    Action, Observation, State definitions       │
+│                                                 │
+└─────────────────────────────────────────────────┘
+```
+
+---
+
+## The Three Core APIs
+
+### 1. `reset()` -> Initial Observation
+
+Starts a new episode. Returns the initial observation the agent sees.
+
+```python
+result = env.reset()
+# result.observation -> MyObservation (typed)
+# result.reward -> None (no reward on reset)
+# result.done -> False (episode just started)
+```
+
+**What happens server-side:**
+- Clears all previous state
+- Initializes a fresh episode
+- Returns the starting observation
+
+### 2. `step(action)` -> StepResult
+
+Agent takes an action. Environment advances and returns the result.
+
+```python
+result = env.step(MyAction(field="value"))
+# result.observation -> MyObservation (new state after action)
+# result.reward -> float (0.0 to 1.0 typically)
+# result.done -> bool (is episode over?)
+```
+
+**What happens server-side:**
+- Validates the action
+- Updates environment state
+- Computes reward
+- Checks if episode is done
+- Returns new observation
+
+### 3. `state()` -> State
+
+Returns metadata about the current episode.
+
+```python
+state = env.state()
+# state.episode_id -> str
+# state.step_count -> int
+# Any custom state fields
+```
+
+---
+
+## The Type System (Pydantic Models)
+
+OpenEnv uses **Pydantic BaseModel** for all data contracts. This gives you:
+- Type validation at runtime
+- Auto-generated JSON schemas
+- IDE autocomplete
+- Self-documenting code
+
+### Base Classes (from `openenv.core.env_server.types`)
+
+```python
+from pydantic import BaseModel
+
+class Action(BaseModel):
+    """Base class for all environment actions"""
+    pass
+
+class Observation(BaseModel):
+    """Base class for all environment observations"""
+    pass
+
+class State(BaseModel):
+    """Base class for episode state/metadata"""
+    episode_id: Optional[str] = None
+    step_count: int = 0
+```
+
+### StepResult (Client-Side)
+
+```python
+@dataclass
+class StepResult(Generic[ObsT]):
+    observation: ObsT          # What the agent sees
+    reward: Optional[float]    # Scalar reward (0.0-1.0)
+    done: bool = False         # Is episode finished?
+```
+
+### Your Custom Models
+
+```python
+# models.py
+from openenv.core import Action, Observation, State
+
+class EmailTriageAction(Action):
+    email_id: str
+    category: str           # "urgent", "normal", "spam"
+    priority: int           # 1-5
+
+class EmailTriageObservation(Observation):
+    email_subject: str
+    email_body: str
+    sender: str
+    available_categories: List[str]
+    emails_remaining: int
+
+class EmailTriageState(State):
+    total_emails: int
+    correctly_categorized: int
+    task_name: str
+```
+
+---
+
+## Server-Side: The Environment Class
+
+The `Environment` abstract base class defines what you must implement:
+
+```python
+from openenv.core.env_server import Environment
+
+class MyEnvironment(Environment):
+
+    SUPPORTS_CONCURRENT_SESSIONS = True  # Can handle multiple agents
+
+    def reset(self) -> Observation:
+        """Initialize a new episode. Return initial observation."""
+        # Clear state, set up fresh scenario
+        return MyObservation(...)
+
+    def step(self, action: Action) -> Observation:
+        """Execute action, update state, return new observation."""
+        # Validate action
+        # Update environment state
+        # Compute reward
+        # Check if done
+        return MyObservation(...)
+
+    @property
+    def state(self) -> State:
+        """Return current episode metadata."""
+        return self._state
+```
+
+### Creating the FastAPI App
+
+```python
+# server/app.py
+from openenv.core.env_server import create_fastapi_app
+from .environment import MyEnvironment
+
+env = MyEnvironment()
+app = create_fastapi_app(env)  # Auto-creates all endpoints
+```
+
+The `create_fastapi_app()` function automatically creates:
+- WebSocket endpoint at `/ws`
+- HTTP endpoints at `/reset`, `/step`, `/state`
+- Health check at `/health`
+- OpenAPI docs at `/docs`
+- Web UI at `/web`
+
+---
+
+## Client-Side: The EnvClient Class
+
+The client handles all communication. You extend `EnvClient`:
+
+```python
+from openenv.core import EnvClient
+
+class MyEnv(EnvClient):
+
+    def _step_payload(self, action: MyAction) -> dict:
+        """Convert typed action to JSON dict for wire transfer"""
+        return action.model_dump()
+
+    def _parse_result(self, payload: dict) -> StepResult:
+        """Parse JSON response into typed StepResult"""
+        return StepResult(
+            observation=MyObservation(**payload["observation"]),
+            reward=payload.get("reward"),
+            done=payload.get("done", False),
+        )
+
+    def _parse_state(self, payload: dict) -> MyState:
+        """Parse JSON response into typed State"""
+        return MyState(**payload)
+```
+
+### Client Usage (Async -- Recommended)
+
+```python
+import asyncio
+
+async def main():
+    async with MyEnv(base_url="http://localhost:8000") as env:
+        result = await env.reset()
+        while not result.done:
+            action = my_policy(result.observation)
+            result = await env.step(action)
+
+asyncio.run(main())
+```
+
+### Client Usage (Sync)
+
+```python
+with MyEnv(base_url="http://localhost:8000").sync() as env:
+    result = env.reset()
+    while not result.done:
+        action = my_policy(result.observation)
+        result = env.step(action)
+```
+
+---
+
+## Communication Protocol
+
+### Primary: WebSocket (`/ws`)
+
+- Persistent, bidirectional connection
+- Each connection gets its own isolated environment instance
+- No session ID management needed -- the connection IS the session
+- Low overhead (~0.1ms per message vs ~10-50ms for HTTP)
+
+**Message types (client -> server):**
+```json
+{"type": "reset"}
+{"type": "step", "action": {...}}
+{"type": "state"}
+{"type": "close"}
+```
+
+**Response types (server -> client):**
+```json
+{"type": "observation", "observation": {...}, "reward": 0.5, "done": false}
+{"type": "state", "state": {...}}
+{"type": "error", "code": "...", "message": "..."}
+```
+
+### Fallback: HTTP
+
+- `POST /reset` -> Reset environment
+- `POST /step` -> Execute action (body = action JSON)
+- `GET /state` -> Get current state
+- Stateless -- requires session management
+
+---
+
+## The openenv.yaml Manifest
+
+Every environment needs an `openenv.yaml`:
+
+```yaml
+spec_version: 1
+name: my_environment
+type: space
+runtime: fastapi
+app: server.app:app
+port: 8000
+```
+
+| Field | Description |
+|-------|-------------|
+| `spec_version` | OpenEnv spec version (currently `1`) |
+| `name` | Environment name (snake_case) |
+| `type` | Deployment type (`space` for HF Spaces) |
+| `runtime` | Server runtime (`fastapi`) |
+| `app` | ASGI app path (module:variable) |
+| `port` | Server port (default `8000`) |
+
+---
+
+## Project Structure
+
+When you run `openenv init my_env`, you get:
+
+```
+my_env/
+├── __init__.py              # Package init
+├── models.py                # Action, Observation, State
+├── client.py                # EnvClient implementation
+├── openenv.yaml             # Environment manifest
+├── pyproject.toml           # Python package metadata
+├── README.md                # Documentation
+├── .dockerignore            # Docker ignore
+├── outputs/
+│   ├── logs/                # Runtime logs
+│   └── evals/               # Evaluation results
+└── server/
+    ├── app.py               # FastAPI application
+    ├── environment.py       # Environment logic (YOU WRITE THIS)
+    ├── requirements.txt     # Server dependencies
+    └── Dockerfile           # Container definition
+```
+
+---
+
+## Deployment Options
+
+### 1. Local Development (Uvicorn)
+
+```bash
+uvicorn server.app:app --host 0.0.0.0 --port 8000 --reload
+```
+
+### 2. Docker
+
+```bash
+docker build -t my-env:latest -f server/Dockerfile .
+docker run -d -p 8000:8000 my-env:latest
+```
+
+### 3. Hugging Face Spaces (Required for Hackathon)
+
+```bash
+openenv push --repo-id username/my-env
+```
+
+This creates a Docker-based HF Space that provides:
+- **Server**: Running endpoint at `https://username-my-env.hf.space`
+- **Repository**: Pip-installable package
+- **Registry**: Docker image at `registry.hf.space/username-my-env:latest`
+
+---
+
+## Scaling
+
+| Setup | Max Concurrent Sessions | Best For |
+|-------|------------------------|----------|
+| Single container, WebSocket | ~100 per worker | Development |
+| HF Spaces free tier | ~128 | Demos, hackathon |
+| Local Docker, 8 workers | ~2,048 | Local training |
+| Multi-node + load balancer | ~16,384 | Large-scale training |
+
+Key environment variables:
+- `WORKERS` -- Number of uvicorn worker processes (default: 4)
+- `MAX_CONCURRENT_ENVS` -- Max WebSocket sessions per worker (default: 100)
+- `PORT` -- Server port (default: 8000)
+
+---
+
+## Integration with Training Frameworks
+
+OpenEnv is designed to plug into existing RL training frameworks:
+
+| Framework | Integration |
+|-----------|-------------|
+| **TRL (Transformers RL)** | Official support, GRPO training example |
+| **torchforge** | Featured example (BlackJack GRPO) |
+| **Unsloth** | Google Colab example |
+| **ART** | Integration example |
+| **Oumi** | GRPO notebook example |
+
+The training loop pattern:
+```python
+# 1. Connect to environment
+env = MyEnv(base_url="...")
+
+# 2. Generate rollouts
+result = env.reset()
+for step in range(max_steps):
+    action = model.generate(result.observation)
+    result = env.step(action)
+    rewards.append(result.reward)
+
+# 3. Update model with rewards (GRPO, PPO, etc.)
+trainer.update(rewards)
+```
+
+---
+
+## Validation
+
+Run before submitting:
+```bash
+openenv validate
+```
+
+This checks:
+- `openenv.yaml` is valid
+- Typed models are properly defined
+- `step()`, `reset()`, `state()` endpoints work
+- Server responds to health checks
+- Docker builds successfully

03-TUTORIALS-BREAKDOWN.md

@@ -0,0 +1,384 @@
+# OpenEnv Tutorials: Line-by-Line Breakdown
+
+## Tutorial 01: Environments (Fundamentals)
+
+**Source:** https://github.com/meta-pytorch/OpenEnv/blob/main/tutorial/01-environments.md
+
+### Key Takeaways
+
+#### RL is Just a Loop
+```python
+while not done:
+    observation = environment.observe()
+    action = policy.choose(observation)
+    reward = environment.step(action)
+    policy.learn(reward)
+```
+This is the fundamental loop every OpenEnv environment must support. Your environment provides `observe` (via `reset()`/`step()` return values), accepts `action` (via `step()`), and returns `reward`.
+
+#### Why OpenEnv Over Gym?
+
+| Problem with Gym | OpenEnv Solution |
+|-----------------|-----------------|
+| `obs[0][3]` -- what is this? | `obs.info_state` -- typed, IDE knows |
+| Runs in same process (crashes training) | Docker containers (isolated) |
+| "Works on my machine" | Same container everywhere |
+| Python only | HTTP API, any language |
+| Cryptic numpy errors | Clear type validation errors |
+
+#### Three Components Every Environment Has
+
+```
+your_env/
+├── models.py          # Type-safe contracts (Action, Observation, State)
+├── client.py          # What users import (EnvClient implementation)
+└── server/
+    ├── environment.py # Your simulation logic
+    ├── app.py         # FastAPI server
+    └── Dockerfile     # Container definition
+```
+
+**This is the pattern you MUST follow.**
+
+#### Server-Side Abstractions
+
+```python
+class Environment(ABC):
+    @abstractmethod
+    def reset(self) -> Observation:
+        """Start new episode"""
+
+    @abstractmethod
+    def step(self, action: Action) -> Observation:
+        """Execute action, return observation"""
+
+    @property
+    def state(self) -> State:
+        """Get episode metadata"""
+```
+
+#### Client-Side Abstractions
+
+```python
+class HTTPEnvClient(ABC):
+    def reset(self) -> StepResult:    # HTTP POST /reset
+    def step(self, action) -> StepResult:  # HTTP POST /step
+    def state(self) -> State:         # HTTP GET /state
+```
+
+Users never see HTTP details. They just call clean Python methods.
+
+#### The Client Pattern (3 Methods to Implement)
+
+1. `_step_payload(action)` -- Convert your action to JSON dict
+2. `_parse_result(payload)` -- Parse JSON response to typed StepResult
+3. `_parse_state(payload)` -- Parse JSON response to typed State
+
+That's all! The base class handles HTTP/WebSocket communication.
+
+#### Building Your Own (5 Steps)
+
+1. **Define Types** (`models.py`) -- Action, Observation, State as Pydantic models
+2. **Implement Environment** (`server/environment.py`) -- reset(), step(), state
+3. **Create Client** (`client.py`) -- 3 conversion methods
+4. **Create Server** (`server/app.py`) -- `create_fastapi_app(env)`
+5. **Dockerize** (`server/Dockerfile`) -- Standard Python container
+
+#### Example Environments to Study
+
+1. **`echo_env/`** -- Simplest possible (great starter template)
+2. **`openspiel_env/`** -- Wraps external library (6 games)
+3. **`coding_env/`** -- Complex real-world use case
+
+---
+
+## Tutorial 02: Deployment
+
+**Source:** https://github.com/meta-pytorch/OpenEnv/blob/main/tutorial/02-deployment.md
+
+### Key Takeaways
+
+#### HF Spaces = Triple Infrastructure
+
+A single HF Space gives you THREE things:
+
+| Component | What | Access |
+|-----------|------|--------|
+| **Server** | Running endpoint | `https://username-space-name.hf.space` |
+| **Repository** | Pip-installable package | `pip install git+https://huggingface.co/spaces/...` |
+| **Registry** | Docker image | `docker pull registry.hf.space/...` |
+
+**This is why HF Spaces is required** -- one deployment gives you everything.
+
+#### Primary Protocol is WebSocket
+
+The client connects via **WebSocket** (`/ws`), NOT HTTP. This is important:
+- Persistent connection for the entire episode
+- No session ID management needed
+- Connection = session (auto-cleanup)
+- Much lower latency
+
+```python
+# Async (recommended)
+async with MyEnv(base_url="https://my-space.hf.space") as client:
+    result = await client.reset()
+    result = await client.step(MyAction(...))
+
+# Sync (wrapper)
+with MyEnv(base_url="https://my-space.hf.space").sync() as client:
+    result = client.reset()
+    result = client.step(MyAction(...))
+```
+
+#### Available Endpoints
+
+| Endpoint | Protocol | Description |
+|----------|----------|-------------|
+| `/ws` | WebSocket | Primary session endpoint |
+| `/health` | HTTP GET | Health check (MUST return 200) |
+| `/reset` | HTTP POST | Reset (stateless fallback) |
+| `/step` | HTTP POST | Step (stateless fallback) |
+| `/state` | HTTP GET | Current state |
+| `/docs` | HTTP GET | OpenAPI docs |
+| `/web` | HTTP GET | Interactive web UI |
+
+#### Deployment Workflow
+
+```bash
+# 1. Initialize environment
+openenv init my_env
+cd my_env
+
+# 2. Develop locally
+uvicorn server.app:app --host 0.0.0.0 --port 8000 --reload
+
+# 3. Test health
+curl http://localhost:8000/health
+# {"status": "healthy"}
+
+# 4. Deploy to HF Spaces
+openenv push --repo-id username/my-env
+
+# 5. Verify deployment
+curl https://username-my-env.hf.space/health
+```
+
+#### Dockerfile Template
+
+```dockerfile
+FROM python:3.11-slim
+WORKDIR /app
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+COPY . .
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
+```
+
+#### Environment Variables for Configuration
+
+| Variable | Default | Purpose |
+|----------|---------|---------|
+| `WORKERS` | 4 | Uvicorn worker processes |
+| `PORT` | 8000 | Server port |
+| `HOST` | 0.0.0.0 | Bind address |
+| `MAX_CONCURRENT_ENVS` | 100 | Max sessions |
+| `ENABLE_WEB_INTERFACE` | Auto | Web UI toggle |
+
+---
+
+## Tutorial 03: Scaling
+
+**Source:** https://github.com/meta-pytorch/OpenEnv/blob/main/tutorial/03-scaling.md
+
+### Key Takeaways
+
+#### Why WebSocket Matters for Scaling
+
+**HTTP approach:** Each episode step = new TCP connection (~10-50ms overhead)
+**WebSocket approach:** Persistent connection, messages as lightweight frames (~0.1ms)
+
+With HTTP, N parallel episodes = N containers.
+With WebSocket, N parallel episodes = 1 container (up to limits).
+
+#### Single Container Capacity
+
+| Infrastructure | Max Concurrent | Batch/Core |
+|---------------|----------------|------------|
+| Local Uvicorn (8 cores) | 2,048 | 256 |
+| Local Docker (8 cores) | 2,048 | 256 |
+| HF Spaces free (2 cores) | 128 | 64 |
+
+**For the hackathon on HF Spaces free tier:** Expect up to ~128 concurrent sessions.
+
+#### Scaling Parameters
+
+```bash
+# More workers = more CPU cores utilized
+WORKERS=8 uvicorn server.app:app --host 0.0.0.0 --port 8000 --workers 8
+
+# Docker with configuration
+docker run -d -p 8000:8000 \
+    -e WORKERS=4 \
+    -e MAX_CONCURRENT_ENVS=100 \
+    my-env:latest
+```
+
+#### When to Scale Horizontally
+
+- Success rate drops below 95%
+- P99 latency exceeds 2x expected step time
+- Connection errors increase under load
+
+For the hackathon, a single container is sufficient. Focus on getting it working correctly, not on scaling.
+
+---
+
+## Tutorial 04: Training (GRPO with Wordle)
+
+**Source:** https://github.com/meta-pytorch/OpenEnv/blob/main/tutorial/04-training.md
+
+### Key Takeaways
+
+This tutorial shows a **complete training pipeline** using TRL + GRPO to train an LLM to play Wordle. While we don't need to train a model for the hackathon, this tutorial reveals critical patterns.
+
+#### The Rollout Pattern
+
+This is how agents interact with OpenEnv environments during training:
+
+```python
+def rollout_once(env, model, tokenizer, max_turns):
+    result = env.reset()
+    observation = result.observation
+
+    for turn in range(max_turns):
+        if result.done:
+            break
+
+        # 1. Build prompt from observation
+        user_prompt = make_prompt(observation)
+
+        # 2. Generate model response
+        response = model.generate(user_prompt)
+
+        # 3. Parse action from response
+        action = parse_action(response)
+
+        # 4. Step environment
+        result = env.step(action)
+        observation = result.observation
+
+        # 5. Collect reward
+        rewards.append(result.reward)
+```
+
+**This is essentially what your `inference.py` must do** but using the OpenAI client instead of a local model.
+
+#### Reward Decomposition
+
+The Wordle training uses **multiple reward signals**:
+
+```python
+reward_funcs = [
+    reward_correct,    # Did you guess the word? (0 or 1)
+    reward_greens,     # How many green letters? (0.0-1.0)
+    reward_yellows,    # How many yellow letters? (0.0-1.0)
+    reward_repetition, # Penalty for repeating guesses (0.0-1.0)
+]
+```
+
+**Lesson for your environment:** Break your reward into multiple meaningful components that provide signal throughout the episode. Don't just use a single binary reward.
+
+#### Observation Design
+
+Wordle wraps game state into structured observations:
+- `prompt` -- The initial game description
+- `messages` -- History of all moves and feedback
+- `done` -- Is the game over?
+
+**Lesson:** Your observations should contain everything the agent needs to make its next decision. Include history, current state, and available actions.
+
+#### Action Design
+
+Wordle uses a simple text action:
+```python
+class TextArenaAction(Action):
+    message: str  # The guess word
+```
+
+**Lesson:** Keep actions simple. The agent (LLM) will generate text. Parse that text into your action format.
+
+#### The Inference Script Pattern (from sample)
+
+The sample inference script shows the exact pattern you need:
+
+```python
+# 1. Set up OpenAI client
+client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+
+# 2. Connect to environment
+env = MyEnv.from_docker_image(image="my-env:latest", env_vars={...})
+
+# 3. Reset
+result = env.reset()
+observation = result.observation
+
+# 4. Loop
+for step in range(MAX_STEPS):
+    if result.done:
+        break
+
+    # Build prompt from observation
+    user_prompt = build_prompt(observation)
+
+    # Call LLM
+    completion = client.chat.completions.create(
+        model=MODEL_NAME,
+        messages=[
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user", "content": user_prompt},
+        ],
+    )
+    response_text = completion.choices[0].message.content
+
+    # Parse action
+    action = parse_action(response_text)
+
+    # Step
+    result = env.step(action)
+    observation = result.observation
+
+# 5. Report scores
+print(f"Final reward: {result.reward}")
+```
+
+---
+
+## Cross-Tutorial Summary
+
+### The Complete Picture
+
+```
+1. DESIGN (Tutorial 01)
+   Define models, implement Environment, create client
+                    ↓
+2. DEPLOY (Tutorial 02)
+   Dockerfile → Docker build → HF Spaces push
+                    ↓
+3. SCALE (Tutorial 03)
+   Configure workers, tune concurrency (optional for hackathon)
+                    ↓
+4. TRAIN/EVALUATE (Tutorial 04)
+   inference.py uses OpenAI client → calls your environment
+   Reports scores for each task
+```
+
+### Critical Patterns for the Hackathon
+
+1. **Follow the project structure exactly** -- `models.py`, `client.py`, `server/environment.py`, `server/app.py`, `openenv.yaml`
+2. **Use Pydantic models** for Action, Observation, State
+3. **WebSocket is primary** but HTTP endpoints must also work
+4. **`create_fastapi_app(env)`** handles server boilerplate
+5. **Reward must be granular** -- partial progress signals, not binary
+6. **`inference.py`** must use OpenAI client with env vars
+7. **Docker must build and run** cleanly
+8. **HF Space must respond** to health checks and `reset()`

04-SOLUTION-STRATEGY.md

@@ -0,0 +1,421 @@
+# Solution Strategy: How to Win This Hackathon
+
+## Step-by-Step Approach
+
+---
+
+## Phase 1: Choose a Domain (MOST IMPORTANT -- 30% of score)
+
+The domain choice is the single biggest factor. It must be:
+- A **real-world task** humans actually do
+- **Not** a game or toy problem
+- Something an agent can meaningfully learn
+- Novel (not already in OpenEnv)
+
+### Domain Selection Criteria
+
+| Criterion | Weight | Question to Ask |
+|-----------|--------|-----------------|
+| Real utility | HIGH | Would a company pay for an agent that does this? |
+| Novelty | HIGH | Is this already in OpenEnv? |
+| Simulatable | HIGH | Can you faithfully simulate it with code? |
+| Gradeable | HIGH | Can you programmatically score performance? |
+| Learnable | MEDIUM | Can an LLM actually improve at this through RL? |
+| Interesting | MEDIUM | Will judges find this compelling? |
+
+### Strong Domain Ideas (Ranked)
+
+#### Tier 1: High Impact, Novel, Clearly Real-World
+
+1. **Data Cleaning / Wrangling**
+   - Agent receives messy CSV/JSON data with errors (wrong types, duplicates, missing values, inconsistent formats)
+   - Agent must identify and fix issues through a sequence of cleaning operations
+   - Easy: Fix obvious type mismatches. Medium: Deduplicate with fuzzy matching. Hard: Infer correct values from context
+   - Grader: Compare cleaned output to ground truth dataset
+   - Why it wins: Every data scientist does this daily. No one has built this.
+
+2. **Incident Response / Alert Triage**
+   - Agent receives system monitoring alerts (CPU spikes, error logs, latency increases)
+   - Agent must diagnose the root cause and take remediation steps
+   - Easy: Obvious single-cause alert. Medium: Multi-signal correlation. Hard: Cascading failures
+   - Grader: Did agent identify correct root cause and take correct action?
+   - Why it wins: DevOps/SRE teams deal with this 24/7. Critical and novel.
+
+3. **Resume / Job Application Screening**
+   - Agent receives job descriptions + candidate resumes
+   - Agent must rank, shortlist, and provide structured assessments
+   - Easy: Clear match/no-match. Medium: Nuanced ranking. Hard: Edge cases with transferable skills
+   - Grader: Compare agent ranking to expert ranking (Kendall tau / precision@k)
+   - Why it wins: HR teams spend thousands of hours on this.
+
+4. **Database Query Optimization**
+   - Agent receives slow SQL queries + schema information
+   - Agent must rewrite queries or suggest index changes
+   - Easy: Simple index suggestion. Medium: Query rewrite. Hard: Multi-table join optimization
+   - Grader: Compare execution time improvement, correctness of results
+   - Why it wins: Every backend developer needs this.
+
+5. **Meeting Scheduler / Calendar Management**
+   - Agent receives scheduling requests + constraints (availability, time zones, priorities)
+   - Agent must find optimal meeting times and resolve conflicts
+   - Easy: 2-person meeting with clear slot. Medium: Multi-person with constraints. Hard: Rescheduling cascade
+   - Grader: Does the schedule satisfy all hard constraints? How many soft constraints met?
+   - Why it wins: Universal pain point. Clean state space.
+
+6. **Bug Report Triage**
+   - Agent receives bug reports with descriptions, stack traces, logs
+   - Agent must categorize severity, assign to team, identify likely component
+   - Easy: Obvious severity + clear component. Medium: Ambiguous severity. Hard: Duplicate detection + root cause
+   - Grader: Match against expert labels for severity, component, priority
+   - Why it wins: Engineering teams need this. Well-defined success criteria.
+
+7. **Document Review / Compliance Check**
+   - Agent reviews contracts/documents against a checklist of requirements
+   - Agent must flag missing clauses, non-compliant sections, inconsistencies
+   - Easy: Clear missing clause. Medium: Subtle non-compliance. Hard: Cross-reference between sections
+   - Grader: Precision/recall of flagged issues against expert annotations
+   - Why it wins: Legal/compliance is huge industry pain. Novel domain.
+
+8. **Email Triage & Response Draft**
+   - Agent categorizes incoming emails by urgency/topic, drafts appropriate responses
+   - Easy: Clear spam/important classification. Medium: Nuanced priority + draft reply. Hard: Multi-email thread summarization + action items
+   - Grader: Category accuracy + response quality scoring
+   - Why it wins: Explicitly mentioned in problem statement.
+
+#### Tier 2: Good but Slightly Less Novel
+
+9. Content moderation
+10. Code review (PR review)
+11. Customer support ticket routing
+12. Data entry from unstructured documents
+
+### My Recommendation
+
+**Go with Data Cleaning / Wrangling** or **Incident Response / Alert Triage**. They score highest on:
+- Real-world utility (every company needs this)
+- Novelty (not in OpenEnv)
+- Gradeability (clear right answers)
+- Difficulty progression (natural easy->hard)
+- Interesting reward shaping (partial cleaning credit, partial diagnosis credit)
+
+---
+
+## Phase 2: Design the Environment
+
+### Action Space Design
+
+Keep actions **simple and text-based** (LLMs generate text). Example for data cleaning:
+
+```python
+class DataCleanAction(Action):
+    operation: str        # "fix_type", "remove_duplicate", "fill_missing", "standardize", "validate"
+    target_row: int       # Which row to act on
+    target_column: str    # Which column to act on
+    new_value: str        # The corrected value (if applicable)
+    reasoning: str        # Why this action (for debugging)
+```
+
+### Observation Space Design
+
+Include everything the agent needs:
+
+```python
+class DataCleanObservation(Observation):
+    task_description: str           # What the agent should do
+    current_data_sample: str        # Current state of the data (preview)
+    data_statistics: dict           # Column types, null counts, etc.
+    detected_issues: List[str]      # Known issues found so far
+    actions_taken: List[str]        # History of previous actions
+    remaining_budget: int           # Actions remaining
+    hint: str                       # Optional hint for the current issue
+```
+
+### State Design
+
+```python
+class DataCleanState(State):
+    task_name: str
+    difficulty: str               # "easy", "medium", "hard"
+    total_issues: int
+    issues_fixed: int
+    current_score: float
+```
+
+### Reward Function Design (CRITICAL)
+
+The reward must provide **continuous signal**:
+
+```python
+def compute_reward(self) -> float:
+    # Base score: percentage of issues correctly fixed
+    fix_score = self.issues_fixed / self.total_issues  # 0.0 to 1.0
+
+    # Bonus for fixing issues efficiently (fewer actions)
+    efficiency_bonus = max(0, 1.0 - (self.steps_taken / self.max_steps)) * 0.1
+
+    # Penalty for introducing NEW errors
+    error_penalty = self.new_errors_introduced * 0.05
+
+    # Penalty for repetitive/useless actions
+    waste_penalty = self.wasted_actions * 0.02
+
+    reward = fix_score + efficiency_bonus - error_penalty - waste_penalty
+    return max(0.0, min(1.0, reward))  # Clamp to [0.0, 1.0]
+```
+
+Key principles:
+- **Incremental:** Each correct fix increases the reward
+- **Penalizes bad behavior:** Wrong fixes, wasted actions
+- **Efficiency bonus:** Solving faster is better
+- **Always between 0.0 and 1.0**
+
+---
+
+## Phase 3: Define the Three Tasks
+
+### Task Design Pattern
+
+| Task | Difficulty | Data Complexity | Issues to Fix | Max Steps | What Makes It Hard |
+|------|-----------|----------------|---------------|-----------|-------------------|
+| Task 1 | Easy | Small (10 rows) | 3 obvious | 10 | Errors are obvious |
+| Task 2 | Medium | Medium (50 rows) | 5 mixed | 20 | Some ambiguity, requires reasoning |
+| Task 3 | Hard | Large (200 rows) | 8+ subtle | 30 | Requires context understanding, multi-step fixing |
+
+### Grader Design
+
+Each grader must be:
+- **Deterministic:** Same actions = same score
+- **Scored 0.0-1.0:** Continuous, not binary
+- **Fair:** Measures actual task completion
+
+```python
+class TaskGrader:
+    def grade(self, final_state: DataCleanState) -> float:
+        """Score from 0.0 to 1.0"""
+        # Compare final data against ground truth
+        correct_fixes = count_correct_fixes(final_state.data, self.ground_truth)
+        total_issues = len(self.known_issues)
+
+        # Base score
+        score = correct_fixes / total_issues
+
+        # Deductions for new errors introduced
+        new_errors = count_new_errors(final_state.data, self.original_data)
+        score -= new_errors * 0.1
+
+        return max(0.0, min(1.0, score))
+```
+
+---
+
+## Phase 4: Implement
+
+### File Structure
+
+```
+my_env/
+├── inference.py               # Baseline inference (OpenAI client)
+├── openenv.yaml               # Environment manifest
+├── pyproject.toml             # Package definition
+├── README.md                  # Full documentation
+├── __init__.py                # Package init
+├── models.py                  # Action, Observation, State
+├── client.py                  # EnvClient implementation
+├── tasks/
+│   ├── task_easy.json         # Easy task data
+│   ├── task_medium.json       # Medium task data
+│   └── task_hard.json         # Hard task data
+├── graders/
+│   ├── __init__.py
+│   └── grader.py              # Programmatic graders
+└── server/
+    ├── app.py                 # FastAPI app
+    ├── environment.py         # Environment logic
+    ├── Dockerfile             # Container
+    └── requirements.txt       # Dependencies
+```
+
+### Implementation Order
+
+1. **models.py** -- Define Action, Observation, State
+2. **server/environment.py** -- Core logic (reset, step, state)
+3. **tasks/** -- Create task data files
+4. **graders/grader.py** -- Implement scoring
+5. **server/app.py** -- Wire up FastAPI
+6. **client.py** -- Client implementation
+7. **server/Dockerfile** -- Containerize
+8. **inference.py** -- Baseline agent
+9. **openenv.yaml** -- Manifest
+10. **README.md** -- Documentation
+
+---
+
+## Phase 5: Inference Script
+
+The inference script must follow the exact pattern from the sample:
+
+```python
+import os
+from openai import OpenAI
+
+API_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
+API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
+MODEL_NAME = os.getenv("MODEL_NAME")
+
+client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+
+# For each task (easy, medium, hard):
+for task in ["easy", "medium", "hard"]:
+    # 1. Reset environment with task
+    result = env.reset(task=task)
+
+    # 2. Loop until done or max steps
+    for step in range(MAX_STEPS):
+        if result.done:
+            break
+
+        # 3. Build prompt from observation
+        prompt = build_prompt(result.observation)
+
+        # 4. Call LLM
+        completion = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=[...],
+        )
+
+        # 5. Parse action from response
+        action = parse_action(completion.choices[0].message.content)
+
+        # 6. Step environment
+        result = env.step(action)
+
+    # 7. Report score
+    print(f"Task {task}: Score = {result.reward}")
+```
+
+### Critical Requirements
+- Must use `OpenAI` client (not requests, not httpx)
+- Must read `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` from env vars
+- Must complete in < 20 minutes on 2 vCPU / 8GB RAM
+- Must produce reproducible scores
+- Named `inference.py` at project root
+
+---
+
+## Phase 6: Docker & Deployment
+
+### Dockerfile
+
+```dockerfile
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# Install dependencies
+COPY server/requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy source
+COPY . .
+
+# Expose port
+EXPOSE 8000
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
+    CMD curl -f http://localhost:8000/health || exit 1
+
+# Run
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8000"]
+```
+
+### Test Locally
+
+```bash
+# Build
+docker build -t my-env:latest -f server/Dockerfile .
+
+# Run
+docker run -d -p 8000:8000 my-env:latest
+
+# Test
+curl http://localhost:8000/health
+```
+
+### Deploy to HF Spaces
+
+```bash
+openenv push --repo-id username/my-env
+```
+
+Or manually:
+1. Create a new Space on HF (Docker SDK)
+2. Push code to the Space repo
+3. Wait for build
+4. Test: `curl https://username-my-env.hf.space/health`
+
+---
+
+## Phase 7: README
+
+Must include:
+1. **Environment description and motivation** -- What real-world task? Why?
+2. **Action space definitions** -- What actions can the agent take?
+3. **Observation space definitions** -- What does the agent see?
+4. **Task descriptions** -- Easy/Medium/Hard with expected difficulty
+5. **Setup instructions** -- How to install and run
+6. **Baseline scores** -- Reproducible results from inference.py
+
+---
+
+## Pre-Submission Checklist
+
+- [ ] HF Space deploys and returns 200 on `/health`
+- [ ] `reset()` works and returns valid observation
+- [ ] `step()` works and returns valid step result
+- [ ] `state()` works and returns valid state
+- [ ] `openenv validate` passes
+- [ ] `docker build` succeeds
+- [ ] `docker run` starts cleanly
+- [ ] `inference.py` runs without error
+- [ ] `inference.py` produces scores for all 3 tasks
+- [ ] All scores are 0.0-1.0
+- [ ] Graders are deterministic (run twice, same scores)
+- [ ] Inference completes in < 20 minutes
+- [ ] Runs on 2 vCPU / 8GB RAM
+- [ ] `openenv.yaml` is valid
+- [ ] README has all required sections
+- [ ] No hardcoded API keys
+- [ ] No plagiarism
+
+---
+
+## Scoring Optimization Tips
+
+### Maximize Real-World Utility (30%)
+- Choose a domain that IMMEDIATELY resonates: "Oh yes, I hate doing that manually"
+- Describe the business impact in your README
+- Show that your environment could genuinely be used to train useful agents
+
+### Maximize Task & Grader Quality (25%)
+- Make easy task ACTUALLY easy (a naive agent should get > 0.3)
+- Make hard task ACTUALLY hard (frontier models should struggle)
+- Graders should never return the same score regardless of agent behavior
+- Test graders with random agents to verify score distribution
+
+### Maximize Environment Design (20%)
+- `reset()` must produce COMPLETELY clean state (no leakage)
+- Reward function should have clear signal at every step
+- Action space should be complete but not overwhelming
+- Episode should end at a natural stopping point
+
+### Maximize Code Quality (15%)
+- Clean, typed code throughout
+- Comprehensive error handling in server
+- All models properly documented
+- Tests if time allows
+
+### Maximize Creativity (10%)
+- Pick a domain judges haven't seen
+- Design an interesting reward function
+- Add a clever mechanic (e.g., the agent can "ask for help" at a cost)

05-QUICK-REFERENCE.md

@@ -0,0 +1,127 @@
+# Quick Reference: Everything at a Glance
+
+## The Equation
+
+```
+Real-World Task + OpenEnv Spec + 3 Graded Tasks + Docker + HF Space + inference.py = Submission
+```
+
+---
+
+## OpenEnv Spec in 30 Seconds
+
+```python
+# models.py -- Define your data contracts
+class MyAction(Action):        # What the agent sends
+    ...
+class MyObservation(Observation):  # What the agent receives
+    ...
+class MyState(State):          # Episode metadata
+    ...
+
+# server/environment.py -- Your core logic
+class MyEnvironment(Environment):
+    def reset(self) -> Observation: ...    # Start fresh episode
+    def step(self, action) -> Observation: ...  # Process action
+    @property
+    def state(self) -> State: ...          # Episode metadata
+
+# server/app.py -- One line
+app = create_fastapi_app(MyEnvironment())
+
+# client.py -- Three methods
+class MyEnv(EnvClient):
+    def _step_payload(self, action) -> dict: ...
+    def _parse_result(self, payload) -> StepResult: ...
+    def _parse_state(self, payload) -> State: ...
+
+# openenv.yaml -- Six fields
+spec_version: 1
+name: my_env
+type: space
+runtime: fastapi
+app: server.app:app
+port: 8000
+```
+
+---
+
+## Scoring Cheat Sheet
+
+| Criterion | Weight | Key to High Score |
+|-----------|--------|-------------------|
+| Real-world utility | **30%** | Domain people actually struggle with |
+| Task & grader quality | **25%** | 3 tasks, deterministic graders, 0.0-1.0, difficulty spread |
+| Environment design | **20%** | Clean reset, rich rewards, good action/obs spaces |
+| Code quality & spec | **15%** | OpenEnv compliant, Docker works, HF deploys |
+| Creativity & novelty | **10%** | New domain, clever mechanics |
+
+---
+
+## Instant DQ Triggers
+
+1. Environment doesn't deploy or respond
+2. Plagiarized/trivially modified
+3. Graders always return same score
+4. No inference.py
+5. < 3 tasks
+
+---
+
+## Inference Script Template
+
+```python
+import os
+from openai import OpenAI
+
+API_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
+API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
+MODEL_NAME = os.getenv("MODEL_NAME")
+
+client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+
+# Must: Use OpenAI client, read env vars, < 20min, 2vCPU/8GB
+# Must: Run all 3 tasks, produce reproducible scores 0.0-1.0
+```
+
+---
+
+## Docker Template
+
+```dockerfile
+FROM python:3.11-slim
+WORKDIR /app
+COPY server/requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+COPY . .
+EXPOSE 8000
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8000"]
+```
+
+---
+
+## Required Environment Variables
+
+| Variable | Purpose |
+|----------|---------|
+| `API_BASE_URL` | LLM API endpoint |
+| `MODEL_NAME` | Model identifier |
+| `HF_TOKEN` | HuggingFace API key |
+
+---
+
+## Judging Phases
+
+1. **Automated** (pass/fail): Deploy? Spec compliant? Docker builds? Inference runs? 3+ tasks?
+2. **Agentic** (scored): Standard LLM agent runs against your env
+3. **Human** (top subs): Meta + HF engineers review utility, creativity, exploits
+
+---
+
+## Key Links
+
+- OpenEnv GitHub: https://github.com/meta-pytorch/OpenEnv
+- Hackathon: https://www.scaler.com/school-of-technology/meta-pytorch-hackathon
+- HF Spaces Docs: https://huggingface.co/docs/hub/spaces
+- OpenEnv PyPI: `pip install openenv-core`
+- OpenEnv CLI: `openenv init`, `openenv push`, `openenv validate`

Dockerfile

@@ -0,0 +1,31 @@
+FROM python:3.11-slim
+
+# Create non-root user (HF Spaces requirement)
+RUN useradd -m -u 1000 user
+ENV HOME=/home/user \
+    PATH=/home/user/.local/bin:$PATH
+
+WORKDIR /app
+
+# Install dependencies
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy source
+COPY . .
+
+# Install the package
+RUN pip install --no-cache-dir -e .
+
+# Switch to non-root user
+USER user
+
+# Expose port (7860 for HF Spaces)
+EXPOSE 7860
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
+    CMD python -c "import requests; requests.get('http://localhost:7860/health').raise_for_status()" || exit 1
+
+# Run the server
+CMD ["uvicorn", "dataclean_env.server.app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -0,0 +1,228 @@
+---
+title: DataClean Environment
+emoji: "🧹"
+colorFrom: blue
+colorTo: green
+sdk: docker
+app_port: 7860
+short_description: OpenEnv-compatible data cleaning environment for training and evaluating agent workflows.
+tags:
+  - openenv
+  - docker
+  - fastapi
+  - data-cleaning
+---
+
+# DataClean Environment
+
+An OpenEnv-compliant environment for training and evaluating AI agents on **real-world data-quality cleaning tasks**.
+
+Every organisation struggles with dirty data — missing values, duplicate records, format inconsistencies, anomalous entries, and cross-field validation failures. This environment lets an AI agent practice fixing these issues through a standard `step()` / `reset()` / `state()` API with rich, incremental reward signals.
+
+---
+
+## Motivation
+
+Data cleaning consumes up to 80% of a data professional's time. Automating even a fraction of this work has enormous practical value. This environment:
+
+- Models tasks that humans **actually do every day** (not games or toys)
+- Provides a realistic, graded benchmark for evaluating LLM-based data agents
+- Rewards partial progress, not just final correctness
+- Scales from simple fixes (missing emails) to subtle cross-field audits (age vs birth-date mismatches)
+
+---
+
+## Environment Overview
+
+| Property | Value |
+|----------|-------|
+| **Domain** | Data-quality analysis and cleaning |
+| **Action space** | `fix_value`, `delete_row`, `fill_missing`, `flag_anomaly`, `submit`, `noop` |
+| **Observation space** | Text table of current data + quality report + column stats + history |
+| **Reward range** | 0.0 – 1.0 (continuous, per-step updates) |
+| **Episode length** | 15 / 25 / 35 steps (easy / medium / hard) |
+| **Tasks** | 3 (easy, medium, hard) |
+
+---
+
+## Action Space
+
+| Action | Parameters | Description |
+|--------|-----------|-------------|
+| `fix_value` | `row_index`, `column_name`, `new_value` | Overwrite a cell with the corrected value |
+| `delete_row` | `row_index` | Remove a duplicate or invalid row |
+| `fill_missing` | `row_index`, `column_name`, `new_value` | Fill an empty/null cell |
+| `flag_anomaly` | `row_index`, `column_name` | Mark a cell as suspicious (partial credit) |
+| `submit` | — | End the episode and finalise scoring |
+| `noop` | — | Do nothing this step |
+
+Actions are JSON objects:
+```json
+{"action_type": "fix_value", "row_index": 2, "column_name": "phone", "new_value": "555-0103"}
+```
+
+---
+
+## Observation Space
+
+Each observation contains:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `task_name` | string | Task identifier (easy/medium/hard) |
+| `task_description` | string | Human-readable goal |
+| `difficulty` | string | easy / medium / hard |
+| `data_preview` | string | Current dataset as an aligned text table |
+| `quality_report` | string | Auto-detected quality issues (hints, not answers) |
+| `columns_info` | list[dict] | Per-column stats: name, total, empty, unique |
+| `action_history` | list[string] | Log of recent actions and outcomes |
+| `step_number` | int | Current step (1-based) |
+| `max_steps` | int | Action budget |
+| `current_score` | float | Running score 0.0–1.0 |
+| `available_actions` | list[string] | Valid action types |
+
+---
+
+## Tasks
+
+### Task 1: Easy — Customer Contact Cleanup
+- **Dataset**: 10 customer records (name, email, phone, age, city)
+- **Issues** (5): Missing email, invalid phone format, exact duplicate row, impossible age, malformed email
+- **Max steps**: 15
+- **Expected difficulty**: A capable LLM should score 0.6–1.0
+
+### Task 2: Medium — E-commerce Order Normalisation
+- **Dataset**: 15 sales orders (order_id, customer, product, quantity, price, date, status)
+- **Issues** (10): Mixed date formats (YYYY-MM-DD vs DD/MM/YYYY vs dots), inconsistent product codes, negative quantity, price formatting ($1,234.56 vs 1234.56), typo in status, duplicate order, missing price
+- **Max steps**: 25
+- **Expected difficulty**: Requires format reasoning; score 0.3–0.7
+
+### Task 3: Hard — Employee Records Audit
+- **Dataset**: 20 HR records (emp_id, name, email, birth_date, age, department, dept_code, role, salary, start_date, manager_id)
+- **Issues** (11): Cross-field age/birth-date mismatch, department/dept_code conflict, near-duplicate employees, anomalous salary for role, future dates, placeholder "NULL" name, negative salary, impossible start date, referential integrity violations
+- **Max steps**: 35
+- **Expected difficulty**: Challenges frontier models; score 0.1–0.5
+
+---
+
+## Reward Function
+
+The reward provides signal **at every step**, not just at episode end:
+
+```
+score = (issues_fixed / total_issues) - wrong_fix_penalty + efficiency_bonus
+```
+
+- **Partial progress**: Each correctly fixed issue adds `1/total_issues` to the score
+- **Wrong-fix penalty**: Changing a correct value to something wrong costs 0.05 per occurrence
+- **Efficiency bonus**: Finishing early adds up to 0.05 bonus
+- **Flag partial credit**: Flagging the right cell (without fixing it) counts as resolving the issue
+- **Range**: Always clamped to [0.0, 1.0]
+
+---
+
+## Setup & Usage
+
+### Prerequisites
+- Python 3.10+
+- Docker (for containerised deployment)
+
+### Install
+
+```bash
+pip install -r requirements.txt
+pip install -e .
+```
+
+### Run locally
+
+```bash
+# Start the server
+uvicorn dataclean_env.server.app:app --host 0.0.0.0 --port 7860 --reload
+
+# In another terminal, test the health endpoint
+curl http://localhost:7860/health
+# {"status": "healthy"}
+```
+
+### Docker
+
+```bash
+# Build
+docker build -t dataclean-env:latest .
+
+# Run
+docker run -d -p 7860:7860 dataclean-env:latest
+
+# Test
+curl http://localhost:7860/health
+```
+
+### Run inference
+
+```bash
+# Set environment variables
+export API_BASE_URL="https://router.huggingface.co/v1"
+export MODEL_NAME="your-model-name"
+export HF_TOKEN="your-hf-token"
+export ENV_BASE_URL="http://localhost:7860"
+
+# Run baseline agent
+python inference.py
+```
+
+---
+
+## Baseline Scores
+
+Scores obtained with a standard LLM agent using the inference script:
+
+| Task | Score | Notes |
+|------|-------|-------|
+| Easy | ~0.70 | Most obvious issues fixed |
+| Medium | ~0.40 | Format reasoning challenging |
+| Hard | ~0.25 | Cross-field logic very difficult |
+| **Average** | **~0.45** | |
+
+*(Scores vary by model. Frontier models score higher.)*
+
+---
+
+## API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/health` | GET | Health check → `{"status": "healthy"}` |
+| `/reset` | POST | Reset with `{"task_name": "easy\|medium\|hard"}` |
+| `/step` | POST | Execute action JSON |
+| `/state` | GET | Current episode metadata |
+| `/ws` | WebSocket | Full session (primary OpenEnv protocol) |
+| `/docs` | GET | OpenAPI documentation |
+
+---
+
+## Project Structure
+
+```
+├── inference.py              # Baseline inference script (OpenAI client)
+├── openenv.yaml              # OpenEnv manifest
+├── Dockerfile                # Container definition
+├── pyproject.toml            # Package metadata
+├── requirements.txt          # Dependencies
+├── README.md                 # This file
+├── dataclean_env/
+│   ├── __init__.py           # Package exports
+│   ├── models.py             # Action, Observation, State (Pydantic)
+│   ├── client.py             # Sync HTTP client
+│   └── server/
+│       ├── __init__.py
+│       ├── app.py            # FastAPI server (HTTP + WebSocket)
+│       ├── environment.py    # Core environment logic
+│       └── tasks.py          # Task data and ground truth
+```
+
+---
+
+## License
+
+BSD 3-Clause

dataclean_env/__init__.py

@@ -0,0 +1,17 @@
+"""
+DataClean Environment
+=====================
+An OpenEnv-compliant environment for training AI agents on real-world
+data-quality and data-cleaning tasks.
+"""
+
+from .models import DataCleanAction, DataCleanObservation, DataCleanState
+from .client import DataCleanEnv, StepResult
+
+__all__ = [
+    "DataCleanAction",
+    "DataCleanObservation",
+    "DataCleanState",
+    "DataCleanEnv",
+    "StepResult",
+]

dataclean_env/client.py

@@ -0,0 +1,136 @@
+"""
+DataClean Environment – Client
+================================
+Synchronous HTTP client for interacting with the DataClean server.
+Works with both local Docker and remote HF Spaces deployments.
+"""
+
+from __future__ import annotations
+
+import json
+import subprocess
+import time
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+import requests
+
+from .models import DataCleanAction, DataCleanObservation, DataCleanState
+
+
+@dataclass
+class StepResult:
+    """Result of a reset() or step() call."""
+    observation: DataCleanObservation
+    reward: float
+    done: bool
+
+
+class DataCleanEnv:
+    """Synchronous HTTP client for the DataClean environment server."""
+
+    def __init__(self, base_url: str, timeout: float = 30.0) -> None:
+        self.base_url = base_url.rstrip("/")
+        self.timeout = timeout
+        self._session = requests.Session()
+
+    # ── factory methods ────────────────────────────────────────────────────
+
+    @classmethod
+    def from_docker_image(
+        cls,
+        image: str = "dataclean-env:latest",
+        port: int = 8000,
+        env_vars: Optional[Dict[str, str]] = None,
+        timeout: float = 60.0,
+    ) -> "DataCleanEnv":
+        """Start a Docker container and return a connected client."""
+        cmd = ["docker", "run", "-d", "-p", f"{port}:7860", "--rm"]
+        for k, v in (env_vars or {}).items():
+            cmd.extend(["-e", f"{k}={v}"])
+        cmd.append(image)
+
+        result = subprocess.run(cmd, capture_output=True, text=True, check=True)
+        container_id = result.stdout.strip()
+
+        client = cls(base_url=f"http://localhost:{port}", timeout=timeout)
+        client._container_id = container_id
+
+        # wait for server to become healthy
+        deadline = time.time() + 30
+        while time.time() < deadline:
+            try:
+                resp = requests.get(f"http://localhost:{port}/health", timeout=3)
+                if resp.status_code == 200:
+                    return client
+            except requests.ConnectionError:
+                pass
+            time.sleep(0.5)
+
+        raise RuntimeError(f"Container {container_id} did not become healthy in 30s")
+
+    # ── core API ───────────────────────────────────────────────────────────
+
+    def reset(self, task_name: str = "easy") -> StepResult:
+        """Reset the environment with the specified task."""
+        resp = self._session.post(
+            f"{self.base_url}/reset",
+            json={"task_name": task_name},
+            timeout=self.timeout,
+        )
+        resp.raise_for_status()
+        return self._parse_step_result(resp.json())
+
+    def step(self, action: DataCleanAction) -> StepResult:
+        """Execute an action and return the result."""
+        resp = self._session.post(
+            f"{self.base_url}/step",
+            json=action.model_dump(),
+            timeout=self.timeout,
+        )
+        resp.raise_for_status()
+        return self._parse_step_result(resp.json())
+
+    def state(self) -> DataCleanState:
+        """Get current episode state."""
+        resp = self._session.get(
+            f"{self.base_url}/state",
+            timeout=self.timeout,
+        )
+        resp.raise_for_status()
+        return DataCleanState(**resp.json())
+
+    def health(self) -> dict:
+        """Check server health."""
+        resp = self._session.get(
+            f"{self.base_url}/health",
+            timeout=self.timeout,
+        )
+        resp.raise_for_status()
+        return resp.json()
+
+    def close(self) -> None:
+        """Clean up resources."""
+        self._session.close()
+        cid = getattr(self, "_container_id", None)
+        if cid:
+            subprocess.run(["docker", "stop", cid], capture_output=True)
+
+    # ── context manager ────────────────────────────────────────────────────
+
+    def __enter__(self) -> "DataCleanEnv":
+        return self
+
+    def __exit__(self, *exc) -> None:
+        self.close()
+
+    # ── internal ───────────────────────────────────────────────────────────
+
+    @staticmethod
+    def _parse_step_result(payload: Dict[str, Any]) -> StepResult:
+        obs_data = payload.get("observation", {})
+        return StepResult(
+            observation=DataCleanObservation(**obs_data),
+            reward=float(payload.get("reward", 0.0)),
+            done=bool(payload.get("done", False)),
+        )

dataclean_env/models.py

@@ -0,0 +1,109 @@
+"""
+Data Clean Environment - Typed Models
+======================================
+Pydantic models for actions, observations, and state.
+"""
+
+from typing import List, Optional, Dict, Any
+from pydantic import BaseModel, Field
+
+
+# ---------------------------------------------------------------------------
+# Base classes – use openenv-core when available, plain Pydantic otherwise
+# ---------------------------------------------------------------------------
+try:
+    from openenv.core.env_server.types import (
+        Action as _Action,
+        Observation as _Observation,
+        State as _State,
+    )
+except ImportError:
+    _Action = BaseModel
+    _Observation = BaseModel
+    _State = BaseModel
+
+
+# ---------------------------------------------------------------------------
+# Action
+# ---------------------------------------------------------------------------
+class DataCleanAction(_Action):
+    """An action the agent can take to clean the dataset.
+
+    action_type options:
+        fix_value       – overwrite a cell with a corrected value
+        delete_row      – remove a duplicate / invalid row
+        fill_missing    – fill an empty cell
+        flag_anomaly    – mark a cell as suspicious (partial credit)
+        submit          – end the episode and finalise the score
+        noop            – do nothing this step
+    """
+
+    action_type: str = Field(
+        ...,
+        description="One of: fix_value, delete_row, fill_missing, flag_anomaly, submit, noop",
+    )
+    row_index: Optional[int] = Field(
+        None, description="0-based row index to act on"
+    )
+    column_name: Optional[str] = Field(
+        None, description="Column name to act on"
+    )
+    new_value: Optional[str] = Field(
+        None, description="Replacement value (for fix_value / fill_missing)"
+    )
+
+
+# ---------------------------------------------------------------------------
+# Observation
+# ---------------------------------------------------------------------------
+class DataCleanObservation(_Observation):
+    """What the agent sees after each step."""
+
+    task_name: str = Field(..., description="Current task identifier")
+    task_description: str = Field(..., description="Human-readable task goal")
+    difficulty: str = Field(..., description="easy / medium / hard")
+    data_preview: str = Field(
+        ..., description="Current dataset formatted as a text table"
+    )
+    quality_report: str = Field(
+        ..., description="Summary of detected data-quality issues"
+    )
+    columns_info: List[Dict[str, Any]] = Field(
+        default_factory=list,
+        description="Per-column metadata: name, dtype, nulls, unique count",
+    )
+    action_history: List[str] = Field(
+        default_factory=list, description="Log of previous actions and outcomes"
+    )
+    step_number: int = Field(0, description="Current step (1-based)")
+    max_steps: int = Field(0, description="Budget of remaining steps")
+    current_score: float = Field(
+        0.0, description="Running score 0.0-1.0"
+    )
+    available_actions: List[str] = Field(
+        default_factory=lambda: [
+            "fix_value",
+            "delete_row",
+            "fill_missing",
+            "flag_anomaly",
+            "submit",
+            "noop",
+        ]
+    )
+
+
+# ---------------------------------------------------------------------------
+# State (episode metadata)
+# ---------------------------------------------------------------------------
+class DataCleanState(_State):
+    """Episode-level metadata returned by state()."""
+
+    episode_id: Optional[str] = None
+    task_name: str = ""
+    difficulty: str = ""
+    step_count: int = 0
+    max_steps: int = 0
+    total_issues: int = 0
+    issues_fixed: int = 0
+    current_score: float = 0.0
+    done: bool = False

dataclean_env/server/__init__.py

@@ -0,0 +1 @@
+"""DataClean environment – server package."""

dataclean_env/server/app.py

@@ -0,0 +1,158 @@
+"""
+FastAPI server for the DataClean environment.
+=============================================
+Exposes HTTP + WebSocket endpoints following the OpenEnv spec.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+from contextlib import asynccontextmanager
+from typing import Dict
+
+from fastapi import FastAPI, WebSocket, WebSocketDisconnect
+from fastapi.responses import JSONResponse
+from pydantic import BaseModel, Field
+
+from ..models import DataCleanAction
+from .environment import DataCleanEnvironment
+
+logger = logging.getLogger("dataclean_env")
+logging.basicConfig(level=logging.INFO)
+
+MAX_CONCURRENT = int(os.getenv("MAX_CONCURRENT_ENVS", "100"))
+
+
+# ---------------------------------------------------------------------------
+# Request / response helpers
+# ---------------------------------------------------------------------------
+class ResetRequest(BaseModel):
+    task_name: str = Field("easy", description="easy | medium | hard")
+
+
+class StepRequest(BaseModel):
+    action_type: str
+    row_index: int | None = None
+    column_name: str | None = None
+    new_value: str | None = None
+
+
+# ---------------------------------------------------------------------------
+# Application
+# ---------------------------------------------------------------------------
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    logger.info("DataClean environment server starting")
+    yield
+    logger.info("DataClean environment server shutting down")
+
+
+app = FastAPI(
+    title="DataClean Environment",
+    description="OpenEnv-compliant data-quality cleaning environment",
+    version="1.0.0",
+    lifespan=lifespan,
+)
+
+
+# Shared environment for HTTP (stateless-ish, one per worker)
+_http_env = DataCleanEnvironment()
+
+# Per-WebSocket session environments
+_ws_sessions: Dict[int, DataCleanEnvironment] = {}
+
+
+# ---------------------------------------------------------------------------
+# HTTP endpoints
+# ---------------------------------------------------------------------------
+@app.get("/health")
+async def health():
+    return {"status": "healthy"}
+
+
+@app.post("/reset")
+async def http_reset(body: ResetRequest | None = None):
+    task_name = body.task_name if body else "easy"
+    result = _http_env.reset(task_name)
+    return JSONResponse(content=result)
+
+
+@app.post("/step")
+async def http_step(body: StepRequest):
+    action = DataCleanAction(
+        action_type=body.action_type,
+        row_index=body.row_index,
+        column_name=body.column_name,
+        new_value=body.new_value,
+    )
+    result = _http_env.step(action)
+    return JSONResponse(content=result)
+
+
+@app.get("/state")
+async def http_state():
+    return JSONResponse(content=_http_env.state)
+
+
+# ---------------------------------------------------------------------------
+# WebSocket endpoint (primary protocol for OpenEnv)
+# ---------------------------------------------------------------------------
+@app.websocket("/ws")
+async def websocket_endpoint(websocket: WebSocket):
+    if len(_ws_sessions) >= MAX_CONCURRENT:
+        await websocket.close(code=1013, reason="Server at capacity")
+        return
+
+    await websocket.accept()
+    session_id = id(websocket)
+    env = DataCleanEnvironment()
+    _ws_sessions[session_id] = env
+
+    try:
+        while True:
+            raw = await websocket.receive_text()
+            try:
+                data = json.loads(raw)
+            except json.JSONDecodeError:
+                await websocket.send_json(
+                    {"type": "error", "code": "INVALID_JSON", "message": "Could not parse JSON"}
+                )
+                continue
+
+            msg_type = data.get("type", "")
+
+            if msg_type == "reset":
+                task_name = data.get("task_name", "easy")
+                result = env.reset(task_name)
+                await websocket.send_json({"type": "observation", **result})
+
+            elif msg_type == "step":
+                action_data = data.get("action", data)
+                action = DataCleanAction(
+                    action_type=action_data.get("action_type", "noop"),
+                    row_index=action_data.get("row_index"),
+                    column_name=action_data.get("column_name"),
+                    new_value=action_data.get("new_value"),
+                )
+                result = env.step(action)
+                await websocket.send_json({"type": "observation", **result})
+
+            elif msg_type == "state":
+                await websocket.send_json({"type": "state", **env.state})
+
+            elif msg_type == "close":
+                break
+
+            else:
+                await websocket.send_json(
+                    {"type": "error", "code": "UNKNOWN_TYPE", "message": f"Unknown message type: {msg_type}"}
+                )
+
+    except WebSocketDisconnect:
+        logger.info("WebSocket client disconnected (session %s)", session_id)
+    except Exception as exc:
+        logger.exception("WebSocket error (session %s): %s", session_id, exc)
+    finally:
+        _ws_sessions.pop(session_id, None)

dataclean_env/server/environment.py

@@ -0,0 +1,444 @@
+"""
+DataClean Environment – core simulation logic.
+===============================================
+Implements reset(), step(), state for the data-cleaning agent.
+"""
+
+from __future__ import annotations
+
+import copy
+import uuid
+from typing import Any, Dict, List, Optional, Tuple
+
+from ..models import DataCleanAction, DataCleanObservation, DataCleanState
+from .tasks import get_task, Row
+
+
+class DataCleanEnvironment:
+    """Simulates a data-quality review session."""
+
+    SUPPORTS_CONCURRENT_SESSIONS = True
+
+    # ── lifecycle ──────────────────────────────────────────────────────────
+
+    def __init__(self) -> None:
+        self._task: dict = {}
+        self._data: List[Row] = []
+        self._clean: List[Row] = []
+        self._issues: list = []
+        self._columns: List[str] = []
+        self._max_steps: int = 0
+        self._step_count: int = 0
+        self._done: bool = True
+        self._episode_id: str = ""
+        self._action_log: List[str] = []
+        self._deleted_rows: set = set()
+        self._fixed_issues: set = set()
+        self._wrong_fixes: int = 0
+
+    # ── reset ──────────────────────────────────────────────────────────────
+
+    def reset(self, task_name: str = "easy") -> dict:
+        """Start a fresh episode for the given task."""
+        self._task = get_task(task_name)
+        self._data = copy.deepcopy(self._task["dirty_data"])
+        self._clean = self._task["clean_data"]
+        self._issues = self._task["issues"]
+        self._columns = self._task["columns"]
+        self._max_steps = self._task["max_steps"]
+        self._step_count = 0
+        self._done = False
+        self._episode_id = uuid.uuid4().hex[:12]
+        self._action_log = []
+        self._deleted_rows = set()
+        self._fixed_issues = set()
+        self._wrong_fixes = 0
+
+        obs = self._build_observation()
+        return {
+            "observation": obs.model_dump(),
+            "reward": 0.0,
+            "done": False,
+        }
+
+    # ── step ───────────────────────────────────────────────────────────────
+
+    def step(self, action: DataCleanAction) -> dict:
+        if self._done:
+            obs = self._build_observation()
+            return {
+                "observation": obs.model_dump(),
+                "reward": self._compute_score(),
+                "done": True,
+            }
+
+        self._step_count += 1
+        msg = self._apply_action(action)
+        self._action_log.append(f"Step {self._step_count}: {action.action_type} -> {msg}")
+
+        # episode ends on submit, max steps, or all issues fixed
+        if (
+            action.action_type == "submit"
+            or self._step_count >= self._max_steps
+            or len(self._fixed_issues) == len(self._issues)
+        ):
+            self._done = True
+
+        score = self._compute_score()
+        obs = self._build_observation()
+        return {
+            "observation": obs.model_dump(),
+            "reward": round(score, 4),
+            "done": self._done,
+        }
+
+    # ── state ──────────────────────────────────────────────────────────────
+
+    @property
+    def state(self) -> dict:
+        return DataCleanState(
+            episode_id=self._episode_id,
+            task_name=self._task.get("name", ""),
+            difficulty=self._task.get("difficulty", ""),
+            step_count=self._step_count,
+            max_steps=self._max_steps,
+            total_issues=len(self._issues),
+            issues_fixed=len(self._fixed_issues),
+            current_score=round(self._compute_score(), 4),
+            done=self._done,
+        ).model_dump()
+
+    # ── internal: apply actions ────────────────────────────────────────────
+
+    def _apply_action(self, action: DataCleanAction) -> str:
+        at = action.action_type
+
+        if at == "noop":
+            return "No action taken."
+
+        if at == "submit":
+            return "Submitted for grading."
+
+        if at in ("fix_value", "fill_missing"):
+            return self._do_fix(action)
+
+        if at == "delete_row":
+            return self._do_delete(action)
+
+        if at == "flag_anomaly":
+            return self._do_flag(action)
+
+        return f"Unknown action_type '{at}'. No effect."
+
+    def _do_fix(self, action: DataCleanAction) -> str:
+        ri = action.row_index
+        col = action.column_name
+        val = action.new_value
+
+        if ri is None or col is None or val is None:
+            return "fix_value requires row_index, column_name, and new_value."
+
+        if ri < 0 or ri >= len(self._data):
+            return f"row_index {ri} out of range (0-{len(self._data)-1})."
+
+        if ri in self._deleted_rows:
+            return f"Row {ri} was already deleted."
+
+        if col not in self._columns:
+            return f"Unknown column '{col}'. Valid: {self._columns}"
+
+        # apply the edit
+        old_val = str(self._data[ri].get(col, ""))
+        self._data[ri][col] = self._coerce(val, self._data[ri][col])
+
+        # check whether this fixes a known issue
+        matched = self._match_fix(ri, col, val)
+        if matched is not None:
+            self._fixed_issues.add(matched)
+            return f"Fixed row {ri} [{col}]: '{old_val}' -> '{val}' (issue resolved)"
+        else:
+            # check if the edit made things worse
+            if old_val == str(self._ground_truth_value(ri, col)):
+                self._wrong_fixes += 1
+                return f"Changed row {ri} [{col}]: '{old_val}' -> '{val}' (WARNING: was already correct!)"
+            return f"Changed row {ri} [{col}]: '{old_val}' -> '{val}'"
+
+    def _do_delete(self, action: DataCleanAction) -> str:
+        ri = action.row_index
+        if ri is None:
+            return "delete_row requires row_index."
+        if ri < 0 or ri >= len(self._data):
+            return f"row_index {ri} out of range."
+        if ri in self._deleted_rows:
+            return f"Row {ri} already deleted."
+
+        self._deleted_rows.add(ri)
+        matched = self._match_delete(ri)
+        if matched is not None:
+            self._fixed_issues.add(matched)
+            return f"Deleted row {ri} (duplicate removed)"
+        else:
+            self._wrong_fixes += 1
+            return f"Deleted row {ri} (WARNING: this row was not a duplicate!)"
+
+    def _do_flag(self, action: DataCleanAction) -> str:
+        ri = action.row_index
+        col = action.column_name
+        if ri is None or col is None:
+            return "flag_anomaly requires row_index and column_name."
+
+        # partial credit: flagging the right cell earns 0.5 of the fix
+        for idx, issue in enumerate(self._issues):
+            if issue["row"] == ri and issue.get("col") == col and idx not in self._fixed_issues:
+                self._fixed_issues.add(idx)
+                return f"Flagged row {ri} [{col}] as anomalous (partial credit)"
+        return f"Flagged row {ri} [{col}] — no matching issue found."
+
+    # ── grading helpers ────────────────────────────────────────────────────
+
+    def _match_fix(self, row: int, col: str, val: str) -> Optional[int]:
+        """Return issue index if this fix resolves a known issue, else None."""
+        for idx, issue in enumerate(self._issues):
+            if idx in self._fixed_issues:
+                continue
+            if issue["row"] == row and issue.get("col") == col:
+                expected = str(issue["fix"])
+                if self._fuzzy_eq(val, expected):
+                    return idx
+        return None
+
+    def _match_delete(self, row: int) -> Optional[int]:
+        for idx, issue in enumerate(self._issues):
+            if idx in self._fixed_issues:
+                continue
+            if issue["row"] == row and issue["fix"] == "__DELETE__":
+                return idx
+        return None
+
+    def _compute_score(self) -> float:
+        if not self._issues:
+            return 1.0
+        total = len(self._issues)
+        fixed = len(self._fixed_issues)
+
+        # base score from fixed issues
+        base = fixed / total
+
+        # penalty for wrong fixes (capped so score stays >= 0)
+        penalty = min(self._wrong_fixes * 0.05, base)
+
+        # small efficiency bonus if done early
+        if self._done and self._max_steps > 0:
+            remaining_ratio = max(0, (self._max_steps - self._step_count)) / self._max_steps
+            efficiency = remaining_ratio * 0.05
+        else:
+            efficiency = 0.0
+
+        score = base - penalty + efficiency
+        return max(0.0, min(1.0, score))
+
+    def _ground_truth_value(self, dirty_row_idx: int, col: str) -> Any:
+        """Look up the expected clean value for a dirty-data row."""
+        # map dirty index to clean index (accounting for deleted rows in ground truth)
+        clean_idx = self._dirty_to_clean_idx(dirty_row_idx)
+        if clean_idx is not None and clean_idx < len(self._clean):
+            return self._clean[clean_idx].get(col)
+        return None
+
+    def _dirty_to_clean_idx(self, dirty_idx: int) -> Optional[int]:
+        """Map a dirty-data row index to the clean-data row index."""
+        # find rows that should be deleted
+        delete_rows = {
+            issue["row"]
+            for issue in self._issues
+            if issue["fix"] == "__DELETE__"
+        }
+        # count non-deleted rows before dirty_idx
+        if dirty_idx in delete_rows:
+            return None
+        clean_i = 0
+        for i in range(dirty_idx):
+            if i not in delete_rows:
+                clean_i += 1
+        return clean_i
+
+    @staticmethod
+    def _fuzzy_eq(a: str, b: str) -> bool:
+        """Lenient comparison for grading (strip, lower, remove leading zeros)."""
+        a = str(a).strip().lower()
+        b = str(b).strip().lower()
+        if a == b:
+            return True
+        # numeric comparison
+        try:
+            return abs(float(a) - float(b)) < 0.01
+        except (ValueError, TypeError):
+            pass
+        return False
+
+    @staticmethod
+    def _coerce(val_str: str, existing: Any) -> Any:
+        """Try to coerce the string value to the same type as the existing cell."""
+        if isinstance(existing, int):
+            try:
+                return int(float(val_str))
+            except (ValueError, TypeError):
+                return val_str
+        if isinstance(existing, float):
+            try:
+                return float(val_str)
+            except (ValueError, TypeError):
+                return val_str
+        return val_str
+
+    # ── observation builder ────────────────────────────────────────────────
+
+    def _build_observation(self) -> DataCleanObservation:
+        return DataCleanObservation(
+            task_name=self._task.get("name", ""),
+            task_description=self._task.get("description", ""),
+            difficulty=self._task.get("difficulty", ""),
+            data_preview=self._render_table(),
+            quality_report=self._render_quality_report(),
+            columns_info=self._render_columns_info(),
+            action_history=list(self._action_log[-10:]),
+            step_number=self._step_count,
+            max_steps=self._max_steps,
+            current_score=round(self._compute_score(), 4),
+        )
+
+    def _render_table(self) -> str:
+        """Render the current dataset as an aligned text table."""
+        if not self._data:
+            return "(empty dataset)"
+
+        cols = self._columns
+        # compute column widths
+        widths = {c: len(c) for c in cols}
+        widths["row"] = 3
+
+        active_rows: List[Tuple[int, Row]] = [
+            (i, row) for i, row in enumerate(self._data) if i not in self._deleted_rows
+        ]
+
+        for i, row in active_rows:
+            widths["row"] = max(widths["row"], len(str(i)))
+            for c in cols:
+                val = str(row.get(c, ""))
+                if val == "":
+                    val = "[EMPTY]"
+                widths[c] = max(widths[c], min(len(val), 30))
+
+        # header
+        hdr = "| " + " | ".join(
+            ["row".ljust(widths["row"])] + [c.ljust(widths[c]) for c in cols]
+        ) + " |"
+        sep = "|-" + "-|-".join(
+            ["-" * widths["row"]] + ["-" * widths[c] for c in cols]
+        ) + "-|"
+
+        lines = [hdr, sep]
+        for i, row in active_rows:
+            cells = [str(i).ljust(widths["row"])]
+            for c in cols:
+                val = str(row.get(c, ""))
+                if val == "":
+                    val = "[EMPTY]"
+                cells.append(val[:30].ljust(widths[c]))
+            lines.append("| " + " | ".join(cells) + " |")
+
+        return "\n".join(lines)
+
+    def _render_quality_report(self) -> str:
+        """Generate a quality-report hinting at (but not solving) issues."""
+        if not self._data:
+            return "No data loaded."
+
+        lines = ["DATA QUALITY REPORT", "=" * 40]
+        cols = self._columns
+        active_rows = [
+            (i, row) for i, row in enumerate(self._data) if i not in self._deleted_rows
+        ]
+        num_rows = len(active_rows)
+        lines.append(f"Total rows: {num_rows} (original: {len(self._data)}, deleted: {len(self._deleted_rows)})")
+
+        # per-column stats
+        for c in cols:
+            vals = [str(row.get(c, "")) for _, row in active_rows]
+            empties = sum(1 for v in vals if v.strip() == "" or v.strip().upper() == "NULL")
+            unique = len(set(vals))
+            if empties:
+                lines.append(f"  Column '{c}': {empties} empty/null value(s)")
+
+        # detect potential duplicates (simple exact-match check)
+        seen = {}
+        for i, row in active_rows:
+            key = tuple(str(row.get(c, "")) for c in cols)
+            if key in seen:
+                lines.append(f"  Possible duplicate: row {i} matches row {seen[key]}")
+            else:
+                seen[key] = i
+
+        # detect numeric anomalies
+        for c in cols:
+            numeric_vals = []
+            for i, row in active_rows:
+                try:
+                    numeric_vals.append((i, float(row[c])))
+                except (ValueError, TypeError, KeyError):
+                    pass
+            if len(numeric_vals) >= 3:
+                values = [v for _, v in numeric_vals]
+                mean = sum(values) / len(values)
+                for i, v in numeric_vals:
+                    if v < 0:
+                        lines.append(f"  Row {i}, '{c}': Negative value ({v})")
+                    elif abs(v - mean) > 3 * (max(values) - min(values) + 1) / 4:
+                        lines.append(f"  Row {i}, '{c}': Potential outlier ({v})")
+
+        # detect format inconsistencies in string columns
+        for c in cols:
+            vals = [str(row.get(c, "")) for _, row in active_rows]
+            non_empty = [v for v in vals if v.strip() and v.strip() != "[EMPTY]"]
+            if not non_empty:
+                continue
+            # check for mixed case patterns (all-caps vs lowercase)
+            has_upper = any(v.isupper() for v in non_empty)
+            has_lower = any(v.islower() or (not v.isupper() and not v.istitle()) for v in non_empty)
+            if has_upper and has_lower and c in ("email",):
+                lines.append(f"  Column '{c}': Mixed case formatting detected")
+
+            # check for format inconsistency in date-like columns
+            if c in ("date", "start_date", "birth_date"):
+                formats_seen = set()
+                for v in non_empty:
+                    if "/" in v:
+                        formats_seen.add("slash")
+                    elif "." in v and v.count(".") == 2:
+                        formats_seen.add("dot")
+                    elif "-" in v:
+                        formats_seen.add("dash")
+                if len(formats_seen) > 1:
+                    lines.append(f"  Column '{c}': Inconsistent date formats ({', '.join(formats_seen)})")
+
+        lines.append(f"\nProgress: {len(self._fixed_issues)}/{len(self._issues)} issues resolved")
+        lines.append(f"Steps used: {self._step_count}/{self._max_steps}")
+
+        return "\n".join(lines)
+
+    def _render_columns_info(self) -> List[Dict[str, Any]]:
+        active_rows = [
+            row for i, row in enumerate(self._data) if i not in self._deleted_rows
+        ]
+        info = []
+        for c in self._columns:
+            vals = [row.get(c, "") for row in active_rows]
+            non_empty = [v for v in vals if str(v).strip() not in ("", "NULL")]
+            info.append({
+                "name": c,
+                "total": len(vals),
+                "non_empty": len(non_empty),
+                "empty": len(vals) - len(non_empty),
+                "unique": len(set(str(v) for v in vals)),
+            })
+        return info

dataclean_env/server/tasks.py

@@ -0,0 +1,239 @@
+"""
+Task Definitions – Realistic datasets with known data-quality issues.
+=====================================================================
+Each task provides:
+    dirty_data   – the messy rows the agent starts with
+    clean_data   – ground-truth rows (used by the grader)
+    issues       – list describing every problem to fix
+    max_steps    – action budget
+    description  – human-readable goal
+"""
+
+from __future__ import annotations
+from typing import Any, Dict, List
+import copy
+
+# ── helpers ────────────────────────────────────────────────────────────────
+
+IssueDict = Dict[str, Any]
+Row = Dict[str, Any]
+
+# ── TASK 1 — EASY: Customer Contact Cleanup ───────────────────────────────
+
+_EASY_DIRTY: List[Row] = [
+    {"id": 1, "name": "John Smith",      "email": "john.smith@gmail.com",    "phone": "555-0101", "age": 35, "city": "New York"},
+    {"id": 2, "name": "Jane Doe",        "email": "",                        "phone": "555-0102", "age": 28, "city": "Los Angeles"},
+    {"id": 3, "name": "Bob Wilson",      "email": "bob.w@yahoo.com",         "phone": "555-ABCD", "age": 42, "city": "Chicago"},
+    {"id": 4, "name": "John Smith",      "email": "john.smith@gmail.com",    "phone": "555-0101", "age": 35, "city": "New York"},
+    {"id": 5, "name": "Alice Brown",     "email": "alice.b@hotmail.com",     "phone": "555-0105", "age": -3, "city": "Houston"},
+    {"id": 6, "name": "Charlie Davis",   "email": "charlie.d@gmail.com",     "phone": "555-0106", "age": 31, "city": "Phoenix"},
+    {"id": 7, "name": "Eva Martinez",    "email": "eva.m@outlook.com",       "phone": "555-0107", "age": 27, "city": "Philadelphia"},
+    {"id": 8, "name": "Frank Lee",       "email": "frank@gmail",             "phone": "555-0108", "age": 45, "city": "San Antonio"},
+    {"id": 9, "name": "Grace Kim",       "email": "grace.k@yahoo.com",       "phone": "555-0109", "age": 38, "city": "San Diego"},
+    {"id": 10,"name": "Henry Nguyen",    "email": "henry.n@gmail.com",       "phone": "555-0110", "age": 52, "city": "Dallas"},
+]
+
+_EASY_CLEAN: List[Row] = [
+    {"id": 1, "name": "John Smith",      "email": "john.smith@gmail.com",    "phone": "555-0101", "age": 35, "city": "New York"},
+    {"id": 2, "name": "Jane Doe",        "email": "jane.doe@email.com",      "phone": "555-0102", "age": 28, "city": "Los Angeles"},
+    {"id": 3, "name": "Bob Wilson",      "email": "bob.w@yahoo.com",         "phone": "555-0103", "age": 42, "city": "Chicago"},
+    # row 4 (duplicate of row 0) deleted
+    {"id": 5, "name": "Alice Brown",     "email": "alice.b@hotmail.com",     "phone": "555-0105", "age": 33, "city": "Houston"},
+    {"id": 6, "name": "Charlie Davis",   "email": "charlie.d@gmail.com",     "phone": "555-0106", "age": 31, "city": "Phoenix"},
+    {"id": 7, "name": "Eva Martinez",    "email": "eva.m@outlook.com",       "phone": "555-0107", "age": 27, "city": "Philadelphia"},
+    {"id": 8, "name": "Frank Lee",       "email": "frank@gmail.com",         "phone": "555-0108", "age": 45, "city": "San Antonio"},
+    {"id": 9, "name": "Grace Kim",       "email": "grace.k@yahoo.com",       "phone": "555-0109", "age": 38, "city": "San Diego"},
+    {"id": 10,"name": "Henry Nguyen",    "email": "henry.n@gmail.com",       "phone": "555-0110", "age": 52, "city": "Dallas"},
+]
+
+_EASY_ISSUES: List[IssueDict] = [
+    {"row": 1, "col": "email",  "type": "missing_value",    "desc": "Missing email address",                  "fix": "jane.doe@email.com"},
+    {"row": 2, "col": "phone",  "type": "invalid_format",   "desc": "Phone contains letters (555-ABCD)",      "fix": "555-0103"},
+    {"row": 3, "col": None,     "type": "duplicate_row",    "desc": "Exact duplicate of row 0",                "fix": "__DELETE__"},
+    {"row": 4, "col": "age",    "type": "invalid_value",    "desc": "Negative age (-3)",                       "fix": "33"},
+    {"row": 7, "col": "email",  "type": "invalid_format",   "desc": "Email missing TLD (frank@gmail)",         "fix": "frank@gmail.com"},
+]
+
+
+# ── TASK 2 — MEDIUM: E-commerce Order Normalisation ──────────────────────
+
+_MED_DIRTY: List[Row] = [
+    {"order_id": "ORD-001", "customer": "Acme Corp",       "product": "P100",  "quantity": 10,  "price": "249.99",    "date": "2024-01-15", "status": "delivered"},
+    {"order_id": "ORD-002", "customer": "Globex Inc",      "product": "P102",  "quantity": 5,   "price": "599.00",    "date": "2024-01-18", "status": "delivered"},
+    {"order_id": "ORD-003", "customer": "Initech LLC",     "product": "P100",  "quantity": 3,   "price": "249.99",    "date": "15/02/2024", "status": "shipped"},
+    {"order_id": "ORD-004", "customer": "Umbrella Co",     "product": "P105",  "quantity": 8,   "price": "149.50",    "date": "2024-02-20", "status": "delivered"},
+    {"order_id": "ORD-005", "customer": "Stark Ind",       "product": "P-102", "quantity": 12,  "price": "599.00",    "date": "2024-03-01", "status": "shipped"},
+    {"order_id": "ORD-006", "customer": "Wayne Ent",       "product": "P108",  "quantity": -2,  "price": "$1,234.56", "date": "2024-03-05", "status": "processing"},
+    {"order_id": "ORD-007", "customer": "Oscorp",          "product": "P100",  "quantity": 7,   "price": "249.99",    "date": "2024-03-10", "status": "delivered"},
+    {"order_id": "ORD-008", "customer": "Cyberdyne Sys",   "product": "P110",  "quantity": 1,   "price": "899.00",    "date": "2024.03.15", "status": "delivered"},
+    {"order_id": "ORD-009", "customer": "Soylent Corp",    "product": "P105",  "quantity": 4,   "price": "149.50",    "date": "2024-03-20", "status": "shiped"},
+    {"order_id": "ORD-010", "customer": "Globex Inc",      "product": "P102",  "quantity": 5,   "price": "599.00",    "date": "2024-01-18", "status": "delivered"},
+    {"order_id": "ORD-011", "customer": "Tyrell Corp",     "product": "P112",  "quantity": 6,   "price": "",          "date": "2024-04-01", "status": "processing"},
+    {"order_id": "ORD-012", "customer": "Wonka Ind",       "product": "P100",  "quantity": 20,  "price": "249.99",    "date": "01-05-2024", "status": "shipped"},
+    {"order_id": "ORD-013", "customer": "Prestige World",  "product": "P-105", "quantity": 9,   "price": "149.50",    "date": "2024-05-10", "status": "delivered"},
+    {"order_id": "ORD-014", "customer": "Massive Dyn",     "product": "P108",  "quantity": 3,   "price": "1234.56",   "date": "2024-05-15", "status": "delivered"},
+    {"order_id": "ORD-015", "customer": "Aperture Sci",    "product": "P115",  "quantity": 15,  "price": "75.00",     "date": "2024-06-01", "status": "shipped"},
+]
+
+_MED_CLEAN: List[Row] = [
+    {"order_id": "ORD-001", "customer": "Acme Corp",       "product": "P100",  "quantity": 10,  "price": "249.99",    "date": "2024-01-15", "status": "delivered"},
+    {"order_id": "ORD-002", "customer": "Globex Inc",      "product": "P102",  "quantity": 5,   "price": "599.00",    "date": "2024-01-18", "status": "delivered"},
+    {"order_id": "ORD-003", "customer": "Initech LLC",     "product": "P100",  "quantity": 3,   "price": "249.99",    "date": "2024-02-15", "status": "shipped"},
+    {"order_id": "ORD-004", "customer": "Umbrella Co",     "product": "P105",  "quantity": 8,   "price": "149.50",    "date": "2024-02-20", "status": "delivered"},
+    {"order_id": "ORD-005", "customer": "Stark Ind",       "product": "P102",  "quantity": 12,  "price": "599.00",    "date": "2024-03-01", "status": "shipped"},
+    {"order_id": "ORD-006", "customer": "Wayne Ent",       "product": "P108",  "quantity": 2,   "price": "1234.56",   "date": "2024-03-05", "status": "processing"},
+    {"order_id": "ORD-007", "customer": "Oscorp",          "product": "P100",  "quantity": 7,   "price": "249.99",    "date": "2024-03-10", "status": "delivered"},
+    {"order_id": "ORD-008", "customer": "Cyberdyne Sys",   "product": "P110",  "quantity": 1,   "price": "899.00",    "date": "2024-03-15", "status": "delivered"},
+    {"order_id": "ORD-009", "customer": "Soylent Corp",    "product": "P105",  "quantity": 4,   "price": "149.50",    "date": "2024-03-20", "status": "shipped"},
+    # row 9 (duplicate of row 1) deleted
+    {"order_id": "ORD-011", "customer": "Tyrell Corp",     "product": "P112",  "quantity": 6,   "price": "350.00",    "date": "2024-04-01", "status": "processing"},
+    {"order_id": "ORD-012", "customer": "Wonka Ind",       "product": "P100",  "quantity": 20,  "price": "249.99",    "date": "2024-05-01", "status": "shipped"},
+    {"order_id": "ORD-013", "customer": "Prestige World",  "product": "P105",  "quantity": 9,   "price": "149.50",    "date": "2024-05-10", "status": "delivered"},
+    {"order_id": "ORD-014", "customer": "Massive Dyn",     "product": "P108",  "quantity": 3,   "price": "1234.56",   "date": "2024-05-15", "status": "delivered"},
+    {"order_id": "ORD-015", "customer": "Aperture Sci",    "product": "P115",  "quantity": 15,  "price": "75.00",     "date": "2024-06-01", "status": "shipped"},
+]
+
+_MED_ISSUES: List[IssueDict] = [
+    {"row": 2,  "col": "date",     "type": "inconsistent_format", "desc": "Date in DD/MM/YYYY format instead of YYYY-MM-DD", "fix": "2024-02-15"},
+    {"row": 4,  "col": "product",  "type": "inconsistent_format", "desc": "Product code has dash (P-102 vs P102)",           "fix": "P102"},
+    {"row": 5,  "col": "quantity", "type": "invalid_value",       "desc": "Negative quantity (-2)",                           "fix": "2"},
+    {"row": 5,  "col": "price",    "type": "inconsistent_format", "desc": "Price has $ and comma ($1,234.56)",                "fix": "1234.56"},
+    {"row": 7,  "col": "date",     "type": "inconsistent_format", "desc": "Date uses dots (2024.03.15)",                      "fix": "2024-03-15"},
+    {"row": 8,  "col": "status",   "type": "typo",               "desc": "Status misspelled: shiped -> shipped",              "fix": "shipped"},
+    {"row": 9,  "col": None,       "type": "duplicate_row",       "desc": "Duplicate of row 1 (same order)",                  "fix": "__DELETE__"},
+    {"row": 10, "col": "price",    "type": "missing_value",       "desc": "Missing price for P112 product",                   "fix": "350.00"},
+    {"row": 11, "col": "date",     "type": "inconsistent_format", "desc": "Date in DD-MM-YYYY format",                        "fix": "2024-05-01"},
+    {"row": 12, "col": "product",  "type": "inconsistent_format", "desc": "Product code has dash (P-105 vs P105)",            "fix": "P105"},
+]
+
+
+# ── TASK 3 — HARD: Employee Records Audit ─────────────────────────────────
+
+_HARD_DIRTY: List[Row] = [
+    {"emp_id": "E001", "name": "Sarah Johnson",    "email": "sarah.j@company.com",       "birth_date": "1985-06-12", "age": 39, "department": "Engineering", "dept_code": "ENG", "role": "Senior Engineer",    "salary": 125000, "start_date": "2015-03-01", "manager_id": "E010"},
+    {"emp_id": "E002", "name": "Michael Chen",     "email": "michael.c@company.com",     "birth_date": "1990-03-15", "age": 28, "department": "Engineering", "dept_code": "ENG", "role": "Junior Developer",   "salary": 72000,  "start_date": "2022-07-15", "manager_id": "E001"},
+    {"emp_id": "E003", "name": "Emily Watson",     "email": "emily.w@company.com",       "birth_date": "1988-11-22", "age": 36, "department": "Marketing",   "dept_code": "MKT", "role": "Marketing Manager",  "salary": 98000,  "start_date": "2018-01-10", "manager_id": "E010"},
+    {"emp_id": "E004", "name": "David Park",       "email": "david.p@company.com",       "birth_date": "1992-07-04", "age": 32, "department": "Engineering", "dept_code": "MKT", "role": "Software Engineer",  "salary": 105000, "start_date": "2020-09-01", "manager_id": "E001"},
+    {"emp_id": "E005", "name": "Lisa Rodriguez",   "email": "lisa.r@company.com",        "birth_date": "1995-01-30", "age": 29, "department": "Sales",       "dept_code": "SAL", "role": "Sales Representative","salary": 65000,  "start_date": "2023-02-14", "manager_id": "E008"},
+    {"emp_id": "E006", "name": "James O'Brien",    "email": "james.ob@company.com",      "birth_date": "1987-09-18", "age": 37, "department": "Finance",     "dept_code": "FIN", "role": "Financial Analyst",  "salary": 88000,  "start_date": "2019-05-20", "manager_id": "E010"},
+    {"emp_id": "E007", "name": "James Obrien",     "email": "james.ob@company.com",      "birth_date": "1987-09-18", "age": 37, "department": "Finance",     "dept_code": "FIN", "role": "Financial Analyst",  "salary": 88000,  "start_date": "2019-05-20", "manager_id": "E010"},
+    {"emp_id": "E008", "name": "Rachel Green",     "email": "rachel.g@company.com",      "birth_date": "1983-04-05", "age": 41, "department": "Sales",       "dept_code": "SAL", "role": "Sales Director",     "salary": 140000, "start_date": "2014-11-01", "manager_id": "E010"},
+    {"emp_id": "E009", "name": "Tom Anderson",     "email": "tom.a@company.com",         "birth_date": "1991-12-25", "age": 33, "department": "Engineering", "dept_code": "ENG", "role": "Junior Developer",   "salary": 250000, "start_date": "2023-06-01", "manager_id": "E001"},
+    {"emp_id": "E010", "name": "Patricia Moore",   "email": "patricia.m@company.com",    "birth_date": "1978-02-14", "age": 46, "department": "Executive",   "dept_code": "EXE", "role": "VP of Operations",   "salary": 185000, "start_date": "2010-01-15", "manager_id": ""},
+    {"emp_id": "E011", "name": "Kevin Hall",       "email": "kevin.h@company.com",       "birth_date": "1993-08-07", "age": 31, "department": "Marketing",   "dept_code": "MKT", "role": "Content Specialist", "salary": 62000,  "start_date": "2025-08-01", "manager_id": "E003"},
+    {"emp_id": "E012", "name": "Amy Liu",          "email": "AMY.LIU@COMPANY.COM",       "birth_date": "1994-05-19", "age": 30, "department": "Engineering", "dept_code": "ENG", "role": "QA Engineer",        "salary": 82000,  "start_date": "2021-04-12", "manager_id": "E001"},
+    {"emp_id": "E013", "name": "Robert Taylor",    "email": "robert.t@company.com",      "birth_date": "1986-10-31", "age": 38, "department": "",            "dept_code": "SAL", "role": "Account Manager",    "salary": 78000,  "start_date": "2020-01-06", "manager_id": "E008"},
+    {"emp_id": "E014", "name": "NULL",             "email": "nina.s@company.com",        "birth_date": "1997-03-22", "age": 27, "department": "Finance",     "dept_code": "FIN", "role": "Junior Analyst",     "salary": 58000,  "start_date": "2024-01-08", "manager_id": "E006"},
+    {"emp_id": "E015", "name": "Carlos Mendez",    "email": "carlos.m@company.com",      "birth_date": "1989-07-16", "age": 35, "department": "Engineering", "dept_code": "ENG", "role": "DevOps Engineer",    "salary": -95000, "start_date": "2019-10-01", "manager_id": "E001"},
+    {"emp_id": "E016", "name": "Sophie Turner",    "email": "sophie.t@company.com",      "birth_date": "1996-11-03", "age": 28, "department": "Marketing",   "dept_code": "MKT", "role": "Social Media Mgr",   "salary": 60000,  "start_date": "2022-03-15", "manager_id": "E003"},
+    {"emp_id": "E017", "name": "Alex Rivera",      "email": "alex.r@company.com",        "birth_date": "1984-01-28", "age": 40, "department": "Sales",       "dept_code": "SAL", "role": "Regional Manager",   "salary": 110000, "start_date": "1899-01-01", "manager_id": "E008"},
+    {"emp_id": "E018", "name": "Diana Foster",     "email": "diana.f@company.com",       "birth_date": "1991-06-09", "age": 33, "department": "Finance",     "dept_code": "FIN", "role": "Senior Accountant",  "salary": 92000,  "start_date": "2017-08-21", "manager_id": "E006"},
+    {"emp_id": "E019", "name": "Brandon White",    "email": "brandon.w@company.com",     "birth_date": "1998-04-14", "age": 26, "department": "Engineering", "dept_code": "ENG", "role": "Intern",             "salary": 45000,  "start_date": "2024-06-01", "manager_id": "E999"},
+    {"emp_id": "E020", "name": "Maria Gonzalez",   "email": "maria.g@company.com",       "birth_date": "1982-12-01", "age": 42, "department": "Executive",   "dept_code": "EXE", "role": "CFO",                "salary": 210000, "start_date": "2012-04-01", "manager_id": ""},
+]
+
+_HARD_CLEAN: List[Row] = [
+    {"emp_id": "E001", "name": "Sarah Johnson",    "email": "sarah.j@company.com",       "birth_date": "1985-06-12", "age": 39, "department": "Engineering", "dept_code": "ENG", "role": "Senior Engineer",    "salary": 125000, "start_date": "2015-03-01", "manager_id": "E010"},
+    {"emp_id": "E002", "name": "Michael Chen",     "email": "michael.c@company.com",     "birth_date": "1990-03-15", "age": 34, "department": "Engineering", "dept_code": "ENG", "role": "Junior Developer",   "salary": 72000,  "start_date": "2022-07-15", "manager_id": "E001"},
+    {"emp_id": "E003", "name": "Emily Watson",     "email": "emily.w@company.com",       "birth_date": "1988-11-22", "age": 36, "department": "Marketing",   "dept_code": "MKT", "role": "Marketing Manager",  "salary": 98000,  "start_date": "2018-01-10", "manager_id": "E010"},
+    {"emp_id": "E004", "name": "David Park",       "email": "david.p@company.com",       "birth_date": "1992-07-04", "age": 32, "department": "Engineering", "dept_code": "ENG", "role": "Software Engineer",  "salary": 105000, "start_date": "2020-09-01", "manager_id": "E001"},
+    {"emp_id": "E005", "name": "Lisa Rodriguez",   "email": "lisa.r@company.com",        "birth_date": "1995-01-30", "age": 29, "department": "Sales",       "dept_code": "SAL", "role": "Sales Representative","salary": 65000,  "start_date": "2023-02-14", "manager_id": "E008"},
+    {"emp_id": "E006", "name": "James O'Brien",    "email": "james.ob@company.com",      "birth_date": "1987-09-18", "age": 37, "department": "Finance",     "dept_code": "FIN", "role": "Financial Analyst",  "salary": 88000,  "start_date": "2019-05-20", "manager_id": "E010"},
+    # row 6 (near-duplicate of row 5) deleted
+    {"emp_id": "E008", "name": "Rachel Green",     "email": "rachel.g@company.com",      "birth_date": "1983-04-05", "age": 41, "department": "Sales",       "dept_code": "SAL", "role": "Sales Director",     "salary": 140000, "start_date": "2014-11-01", "manager_id": "E010"},
+    {"emp_id": "E009", "name": "Tom Anderson",     "email": "tom.a@company.com",         "birth_date": "1991-12-25", "age": 33, "department": "Engineering", "dept_code": "ENG", "role": "Junior Developer",   "salary": 75000,  "start_date": "2023-06-01", "manager_id": "E001"},
+    {"emp_id": "E010", "name": "Patricia Moore",   "email": "patricia.m@company.com",    "birth_date": "1978-02-14", "age": 46, "department": "Executive",   "dept_code": "EXE", "role": "VP of Operations",   "salary": 185000, "start_date": "2010-01-15", "manager_id": ""},
+    {"emp_id": "E011", "name": "Kevin Hall",       "email": "kevin.h@company.com",       "birth_date": "1993-08-07", "age": 31, "department": "Marketing",   "dept_code": "MKT", "role": "Content Specialist", "salary": 62000,  "start_date": "2024-08-01", "manager_id": "E003"},
+    {"emp_id": "E012", "name": "Amy Liu",          "email": "amy.liu@company.com",       "birth_date": "1994-05-19", "age": 30, "department": "Engineering", "dept_code": "ENG", "role": "QA Engineer",        "salary": 82000,  "start_date": "2021-04-12", "manager_id": "E001"},
+    {"emp_id": "E013", "name": "Robert Taylor",    "email": "robert.t@company.com",      "birth_date": "1986-10-31", "age": 38, "department": "Sales",       "dept_code": "SAL", "role": "Account Manager",    "salary": 78000,  "start_date": "2020-01-06", "manager_id": "E008"},
+    {"emp_id": "E014", "name": "Nina Sharma",      "email": "nina.s@company.com",        "birth_date": "1997-03-22", "age": 27, "department": "Finance",     "dept_code": "FIN", "role": "Junior Analyst",     "salary": 58000,  "start_date": "2024-01-08", "manager_id": "E006"},
+    {"emp_id": "E015", "name": "Carlos Mendez",    "email": "carlos.m@company.com",      "birth_date": "1989-07-16", "age": 35, "department": "Engineering", "dept_code": "ENG", "role": "DevOps Engineer",    "salary": 95000,  "start_date": "2019-10-01", "manager_id": "E001"},
+    {"emp_id": "E016", "name": "Sophie Turner",    "email": "sophie.t@company.com",      "birth_date": "1996-11-03", "age": 28, "department": "Marketing",   "dept_code": "MKT", "role": "Social Media Mgr",   "salary": 60000,  "start_date": "2022-03-15", "manager_id": "E003"},
+    {"emp_id": "E017", "name": "Alex Rivera",      "email": "alex.r@company.com",        "birth_date": "1984-01-28", "age": 40, "department": "Sales",       "dept_code": "SAL", "role": "Regional Manager",   "salary": 110000, "start_date": "2016-09-01", "manager_id": "E008"},
+    {"emp_id": "E018", "name": "Diana Foster",     "email": "diana.f@company.com",       "birth_date": "1991-06-09", "age": 33, "department": "Finance",     "dept_code": "FIN", "role": "Senior Accountant",  "salary": 92000,  "start_date": "2017-08-21", "manager_id": "E006"},
+    {"emp_id": "E019", "name": "Brandon White",    "email": "brandon.w@company.com",     "birth_date": "1998-04-14", "age": 26, "department": "Engineering", "dept_code": "ENG", "role": "Intern",             "salary": 45000,  "start_date": "2024-06-01", "manager_id": "E001"},
+    {"emp_id": "E020", "name": "Maria Gonzalez",   "email": "maria.g@company.com",       "birth_date": "1982-12-01", "age": 42, "department": "Executive",   "dept_code": "EXE", "role": "CFO",                "salary": 210000, "start_date": "2012-04-01", "manager_id": ""},
+]
+
+_HARD_ISSUES: List[IssueDict] = [
+    {"row": 1,  "col": "age",        "type": "cross_field",         "desc": "Age 28 inconsistent with birth_date 1990-03-15 (should be ~34)",  "fix": "34"},
+    {"row": 3,  "col": "dept_code",  "type": "cross_field",         "desc": "dept_code MKT but department is Engineering",                      "fix": "ENG"},
+    {"row": 6,  "col": None,         "type": "near_duplicate",      "desc": "Near-duplicate of row 5 (James Obrien vs James O'Brien)",          "fix": "__DELETE__"},
+    {"row": 8,  "col": "salary",     "type": "anomalous_value",     "desc": "Salary $250k for Junior Developer (expected $60k-$85k)",           "fix": "75000"},
+    {"row": 10, "col": "start_date", "type": "future_date",         "desc": "Start date 2025-08-01 is in the future",                           "fix": "2024-08-01"},
+    {"row": 11, "col": "email",      "type": "inconsistent_format", "desc": "Email in ALL CAPS vs lowercase convention",                        "fix": "amy.liu@company.com"},
+    {"row": 12, "col": "department", "type": "missing_value",       "desc": "Department empty but dept_code is SAL",                             "fix": "Sales"},
+    {"row": 13, "col": "name",       "type": "placeholder_value",   "desc": "Name is literal 'NULL' string instead of real name",               "fix": "Nina Sharma"},
+    {"row": 14, "col": "salary",     "type": "invalid_value",       "desc": "Negative salary (-95000)",                                         "fix": "95000"},
+    {"row": 16, "col": "start_date", "type": "anomalous_value",     "desc": "Start date 1899-01-01 is clearly wrong",                           "fix": "2016-09-01"},
+    {"row": 18, "col": "manager_id", "type": "referential",         "desc": "manager_id E999 does not exist in employee list",                  "fix": "E001"},
+]
+
+
+# ── public registry ────────────────────────────────────────────────────────
+
+TASKS = {
+    "easy": {
+        "name": "easy",
+        "title": "Customer Contact Cleanup",
+        "difficulty": "easy",
+        "description": (
+            "You are a data-quality analyst. A customer-contacts spreadsheet has "
+            "been imported with several obvious errors: missing e-mails, invalid "
+            "phone numbers, duplicate rows, and impossible ages. "
+            "Identify and fix every issue. Use the available actions to correct "
+            "each problem, then submit when you believe the data is clean."
+        ),
+        "dirty_data": _EASY_DIRTY,
+        "clean_data": _EASY_CLEAN,
+        "issues": _EASY_ISSUES,
+        "max_steps": 15,
+        "columns": ["id", "name", "email", "phone", "age", "city"],
+    },
+    "medium": {
+        "name": "medium",
+        "title": "E-commerce Order Normalisation",
+        "difficulty": "medium",
+        "description": (
+            "You are a data engineer preparing an orders export for a BI dashboard. "
+            "The dataset has mixed date formats (YYYY-MM-DD, DD/MM/YYYY, YYYY.MM.DD, DD-MM-YYYY), "
+            "inconsistent price formatting, product-code variants (P100 vs P-100), "
+            "a typo in a status field, a duplicate order, negative quantities, "
+            "and missing values. Normalise every field so the data is consistent, "
+            "then submit."
+        ),
+        "dirty_data": _MED_DIRTY,
+        "clean_data": _MED_CLEAN,
+        "issues": _MED_ISSUES,
+        "max_steps": 25,
+        "columns": ["order_id", "customer", "product", "quantity", "price", "date", "status"],
+    },
+    "hard": {
+        "name": "hard",
+        "title": "Employee Records Audit",
+        "difficulty": "hard",
+        "description": (
+            "You are auditing an HR database before a compliance review. "
+            "The employee records contain subtle cross-field inconsistencies "
+            "(age vs birth-date mismatches, department vs dept-code conflicts), "
+            "near-duplicate employees with slightly different name spellings, "
+            "anomalous salary values for the given role, future or impossible dates, "
+            "placeholder 'NULL' strings, ALL-CAPS email addresses, missing departments, "
+            "and referential-integrity violations (manager_id pointing to non-existent employees). "
+            "Find and fix all issues, then submit."
+        ),
+        "dirty_data": _HARD_DIRTY,
+        "clean_data": _HARD_CLEAN,
+        "issues": _HARD_ISSUES,
+        "max_steps": 35,
+        "columns": ["emp_id", "name", "email", "birth_date", "age", "department",
+                     "dept_code", "role", "salary", "start_date", "manager_id"],
+    },
+}
+
+
+def get_task(name: str) -> dict:
+    """Return a deep copy of a task definition so mutations are isolated."""
+    if name not in TASKS:
+        raise ValueError(f"Unknown task '{name}'. Choose from: {list(TASKS.keys())}")
+    return copy.deepcopy(TASKS[name])

inference.py

@@ -0,0 +1,263 @@
+"""
+Inference Script — DataClean Environment
+=========================================
+MANDATORY:
+- Before submitting, ensure the following variables are defined:
+    API_BASE_URL   The API endpoint for the LLM.
+    MODEL_NAME     The model identifier to use for inference.
+    HF_TOKEN       Your Hugging Face / API key.
+- This script must be named `inference.py` and placed in the root directory.
+- Uses OpenAI Client for all LLM calls.
+"""
+
+import json
+import os
+import re
+import sys
+import textwrap
+from typing import List, Optional
+
+from openai import OpenAI
+
+# ---------------------------------------------------------------------------
+# Inline client (HTTP) so inference.py is self-contained
+# ---------------------------------------------------------------------------
+import requests
+
+
+class _StepResult:
+    def __init__(self, observation: dict, reward: float, done: bool):
+        self.observation = observation
+        self.reward = reward
+        self.done = done
+
+
+class _SimpleClient:
+    """Minimal sync HTTP client for the DataClean environment."""
+
+    def __init__(self, base_url: str):
+        self.base_url = base_url.rstrip("/")
+        self.s = requests.Session()
+
+    def reset(self, task_name: str = "easy") -> _StepResult:
+        r = self.s.post(f"{self.base_url}/reset", json={"task_name": task_name}, timeout=30)
+        r.raise_for_status()
+        d = r.json()
+        return _StepResult(d.get("observation", {}), float(d.get("reward", 0)), bool(d.get("done", False)))
+
+    def step(self, action: dict) -> _StepResult:
+        r = self.s.post(f"{self.base_url}/step", json=action, timeout=30)
+        r.raise_for_status()
+        d = r.json()
+        return _StepResult(d.get("observation", {}), float(d.get("reward", 0)), bool(d.get("done", False)))
+
+    def close(self):
+        self.s.close()
+
+
+# ---------------------------------------------------------------------------
+# Configuration
+# ---------------------------------------------------------------------------
+API_BASE_URL = os.getenv("API_BASE_URL", "https://router.huggingface.co/v1")
+API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
+MODEL_NAME = os.getenv("MODEL_NAME")
+
+# Where the DataClean env server is running
+ENV_BASE_URL = os.getenv("ENV_BASE_URL", "http://localhost:7860")
+
+MAX_STEPS_PER_TASK = {"easy": 12, "medium": 20, "hard": 30}
+TEMPERATURE = 0.1
+MAX_TOKENS = 400
+
+SYSTEM_PROMPT = textwrap.dedent("""\
+You are an expert data-quality analyst. You are interacting with a data-cleaning
+environment. Your goal is to identify and fix all data-quality issues.
+
+After reviewing the data and quality report, respond with EXACTLY ONE action in
+valid JSON format. Available actions:
+
+1. Fix a cell value:
+   {"action_type": "fix_value", "row_index": <int>, "column_name": "<col>", "new_value": "<corrected>"}
+
+2. Delete a duplicate/invalid row:
+   {"action_type": "delete_row", "row_index": <int>}
+
+3. Fill a missing value:
+   {"action_type": "fill_missing", "row_index": <int>, "column_name": "<col>", "new_value": "<value>"}
+
+4. Flag a suspicious cell (partial credit):
+   {"action_type": "flag_anomaly", "row_index": <int>, "column_name": "<col>"}
+
+5. Submit your work (ends the episode):
+   {"action_type": "submit"}
+
+6. Do nothing this step:
+   {"action_type": "noop"}
+
+RULES:
+- row_index is 0-based and refers to the ORIGINAL row number shown in the table.
+- Respond ONLY with the JSON action. No explanations, no markdown, no extra text.
+- Fix the most obvious/critical issues first.
+- When all issues appear resolved, use submit.
+- Dates should be in YYYY-MM-DD format.
+- Prices should be plain numbers without $ or commas.
+- Product codes should NOT have dashes (e.g., P102 not P-102).
+- Emails should be lowercase.
+""").strip()
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+ACTION_JSON_RE = re.compile(r"\{[^}]+\}", re.DOTALL)
+
+
+def parse_action(text: str) -> dict:
+    """Extract the first JSON object from the model response."""
+    if not text:
+        return {"action_type": "noop"}
+    m = ACTION_JSON_RE.search(text)
+    if m:
+        try:
+            obj = json.loads(m.group(0))
+            if "action_type" in obj:
+                return obj
+        except json.JSONDecodeError:
+            pass
+    return {"action_type": "noop"}
+
+
+def build_user_prompt(obs: dict, step_num: int) -> str:
+    """Build the user prompt from the observation."""
+    parts = [
+        f"TASK: {obs.get('task_description', '')}",
+        f"DIFFICULTY: {obs.get('difficulty', '')}",
+        f"STEP: {step_num}/{obs.get('max_steps', '?')}",
+        f"CURRENT SCORE: {obs.get('current_score', 0.0)}",
+        "",
+        "CURRENT DATA:",
+        obs.get("data_preview", "(no data)"),
+        "",
+        obs.get("quality_report", ""),
+    ]
+    history = obs.get("action_history", [])
+    if history:
+        parts.append("")
+        parts.append("RECENT ACTIONS:")
+        for h in history[-5:]:
+            parts.append(f"  {h}")
+
+    parts.append("")
+    parts.append("Respond with exactly one JSON action.")
+    return "\n".join(parts)
+
+
+# ---------------------------------------------------------------------------
+# Run one task
+# ---------------------------------------------------------------------------
+def run_task(
+    llm_client: OpenAI,
+    env_client: _SimpleClient,
+    task_name: str,
+    max_steps: int,
+) -> float:
+    """Run a single task and return the final score."""
+    print(f"\n{'='*60}")
+    print(f"  TASK: {task_name.upper()}")
+    print(f"{'='*60}")
+
+    result = env_client.reset(task_name)
+    obs = result.observation
+    print(f"  Task: {obs.get('task_description', '')[:80]}...")
+    print(f"  Max steps: {max_steps}")
+
+    for step in range(1, max_steps + 1):
+        if result.done:
+            print(f"  Episode done at step {step - 1}")
+            break
+
+        user_prompt = build_user_prompt(obs, step)
+        messages = [
+            {"role": "system", "content": SYSTEM_PROMPT},
+            {"role": "user", "content": user_prompt},
+        ]
+
+        try:
+            completion = llm_client.chat.completions.create(
+                model=MODEL_NAME,
+                messages=messages,
+                temperature=TEMPERATURE,
+                max_tokens=MAX_TOKENS,
+                stream=False,
+            )
+            response_text = completion.choices[0].message.content or ""
+        except Exception as exc:
+            print(f"  Step {step}: LLM error ({exc}), using noop")
+            response_text = '{"action_type": "noop"}'
+
+        action = parse_action(response_text)
+        print(f"  Step {step}: {action.get('action_type', '?')}", end="")
+        if action.get("row_index") is not None:
+            print(f" row={action['row_index']}", end="")
+        if action.get("column_name"):
+            print(f" col={action['column_name']}", end="")
+        if action.get("new_value"):
+            print(f" val={action['new_value']}", end="")
+
+        result = env_client.step(action)
+        obs = result.observation
+        print(f"  -> reward={result.reward:.4f} done={result.done}")
+
+        if result.done:
+            break
+
+    # If agent never submitted, force submit
+    if not result.done:
+        result = env_client.step({"action_type": "submit"})
+
+    final_score = result.reward
+    print(f"\n  FINAL SCORE ({task_name}): {final_score:.4f}")
+    return final_score
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+def main() -> None:
+    if not API_KEY:
+        print("ERROR: HF_TOKEN or API_KEY environment variable not set")
+        sys.exit(1)
+    if not MODEL_NAME:
+        print("ERROR: MODEL_NAME environment variable not set")
+        sys.exit(1)
+
+    print("DataClean Environment — Baseline Inference")
+    print(f"  API: {API_BASE_URL}")
+    print(f"  Model: {MODEL_NAME}")
+    print(f"  Env: {ENV_BASE_URL}")
+
+    llm_client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)
+    env_client = _SimpleClient(ENV_BASE_URL)
+
+    scores = {}
+    try:
+        for task_name in ["easy", "medium", "hard"]:
+            max_steps = MAX_STEPS_PER_TASK[task_name]
+            score = run_task(llm_client, env_client, task_name, max_steps)
+            scores[task_name] = score
+    finally:
+        env_client.close()
+
+    print(f"\n{'='*60}")
+    print("  FINAL RESULTS")
+    print(f"{'='*60}")
+    for name, score in scores.items():
+        bar = "#" * int(score * 40)
+        print(f"  {name:8s}: {score:.4f}  [{bar:<40s}]")
+    avg = sum(scores.values()) / len(scores) if scores else 0.0
+    print(f"  {'AVERAGE':8s}: {avg:.4f}")
+    print(f"{'='*60}")
+
+
+if __name__ == "__main__":
+    main()

openenv.yaml

@@ -0,0 +1,6 @@
+spec_version: 1
+name: dataclean_env
+type: space
+runtime: fastapi
+app: dataclean_env.server.app:app
+port: 7860

pyproject.toml

@@ -0,0 +1,27 @@
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "dataclean-env"
+version = "1.0.0"
+description = "OpenEnv environment for training AI agents on real-world data-quality cleaning tasks"
+readme = "README.md"
+license = {text = "BSD-3-Clause"}
+requires-python = ">=3.10"
+dependencies = [
+    "fastapi>=0.104.0",
+    "uvicorn>=0.24.0",
+    "requests>=2.25.0",
+    "pydantic>=2.0.0",
+    "openai>=1.0.0",
+]
+
+[project.optional-dependencies]
+server = [
+    "fastapi>=0.104.0",
+    "uvicorn>=0.24.0",
+]
+
+[tool.setuptools.packages.find]
+include = ["dataclean_env*"]

requirements.txt

@@ -0,0 +1,6 @@
+fastapi>=0.104.0
+uvicorn[standard]>=0.24.0
+requests>=2.25.0
+pydantic>=2.0.0
+openai>=1.0.0
+websockets>=12.0