Consolidate requirements docs and align roadmap with official submission rules

KNOWLEDGE.md

@@ -1,6 +1,6 @@
 # IT Helpdesk Ticket Routing OpenEnv - Knowledge Guide
 
-## What The Hackathon Is Looking For
+## What This Repo Needs To Prove
 
 The judges want a real-world environment that follows the OpenEnv pattern and can be understood quickly.
 
@@ -14,9 +14,9 @@ That means this repo needs:
 6. a baseline `inference.py`
 7. Docker and metadata that are easy to rerun
 
-## Why IT Helpdesk Ticket Routing Fits Well
+## Why This Domain Fits
 
-This domain is a strong fit because it is:
+IT helpdesk routing is a strong hackathon fit because it is:
 
 - realistic
 - structured
@@ -32,12 +32,12 @@ This environment simulates a short helpdesk queue where an agent routes one tick
 
 ## Judge-Facing Explanation
 
-If a judge asks why this environment is a strong submission, the concise answer is:
+If a judge asks why this environment is strong, the concise answer is:
 
 1. IT helpdesk routing is a real operational workflow with clear business value.
 2. The input is realistic free-form ticket text, but the output is typed and easy to grade deterministically.
 3. The three-task ladder creates a clean progression from basic classification to full queue routing.
-4. The repo stays judge-friendly because the vocabulary, task labels, and scoring rules are all explicit and frozen.
+4. The repo stays judge-friendly because the vocabulary, task labels, and scoring rules are explicit and frozen.
 
 ## Frozen Project Identity
 
@@ -47,6 +47,34 @@ If a judge asks why this environment is a strong submission, the concise answer
 - OpenEnv name: `it_helpdesk_ticket_routing_openenv`
 - App environment name: `it_helpdesk_ticket_routing`
 
+## Practical Mental Model
+
+```text
+inference.py
+    |
+    v
+client.py  <---->  server/app.py
+                         |
+                         v
+                server/environment.py
+                  |       |        |
+                  v       v        v
+            grader.py  reward.py  tasks.py
+                                  |
+                                  v
+                           data/dataset.json
+```
+
+The repo is a small OpenEnv stack:
+
+- `inference.py` drives episodes
+- `client.py` talks to the app
+- `server/environment.py` manages queue state and episode flow
+- `server/grader.py` scores actions
+- `server/reward.py` computes step and final reward behavior
+- `server/tasks.py` defines the task ladder and loads the dataset
+- `data/dataset.json` stores the labeled helpdesk tickets
+
 ## Frozen Runtime Vocabulary
 
 ### Fields
@@ -145,11 +173,30 @@ On each step, the environment:
 
 Returns the internal state snapshot for debugging or inspection.
 
+## Observation And State At A Glance
+
+The observation exposes:
+
+- task metadata
+- the current ticket
+- queue progress counters
+- history
+- reward and done status
+
+The state tracks:
+
+- current task
+- seed
+- queue ticket IDs
+- current ticket index
+- per-ticket scores
+- total reward
+
 ## Task Design
 
 ### Task 1: Issue Type Classification
 
-The agent only predicts:
+The agent predicts:
 
 - `issue_type`
 
@@ -257,7 +304,7 @@ It supports:
 
 ## Validation Notes
 
-The repo has now gone through two useful validation phases.
+The repo has already gone through two useful validation phases.
 
 ### April 2 consistency pass
 
@@ -273,7 +320,7 @@ What needed to agree:
 
 ### April 3 and April 4 runtime-feedback pass
 
-The first local runtime pass was then completed and surfaced a practical issue:
+The first local runtime pass surfaced one practical issue:
 
 - `data/dataset.json` was saved with a UTF-8 BOM, which caused `json.load()` to fail during environment creation on Windows
 
@@ -288,13 +335,13 @@ The local heuristic baseline completed successfully after that fix with:
 
 A merged-state rerun on the current `main` branch matched those same numbers exactly.
 
-## April 6 Repo Audit
+### April 6 repo audit
 
-An April 6 documentation and repo audit confirmed:
+An April 6 audit confirmed:
 
-- all required runtime, data, metadata, and documentation files are present in the workspace
+- all required runtime, data, metadata, and documentation files are present
 - the docs consistently describe IT helpdesk ticket routing rather than the old email-triage domain
-- the current local benchmark reference is `1.0000`, `0.8800`, `0.9400`, overall `0.9400`
+- the current local benchmark reference is still `1.0000`, `0.8800`, `0.9400`, overall `0.9400`
 - the remaining work is execution validation, not documentation cleanup
 
 ## What Still Needs Hands-On Verification
@@ -312,7 +359,9 @@ If you come back to this repo later, remember:
 
 - the domain is IT helpdesk ticket routing
 - the environment is a short queue, not a single-shot classifier
+- the architecture is a compact OpenEnv stack
+- one ticket is shown at a time
 - the agent predicts structured routing fields
-- grading is deterministic with limited partial credit
-- the inference script is the baseline player
+- the grader gives deterministic partial credit
+- `inference.py` is the baseline agent runner
 - merged-state local validation is complete, and Docker is the main remaining hands-on check

MENTAL_MODEL.md

@@ -1,173 +0,0 @@
-# IT Helpdesk Ticket Routing Mental Model
-
-This file is the practical mental model of the repo in its current form.
-
-## What The Project Is
-
-This repository is an OpenEnv environment for IT helpdesk ticket routing.
-
-The environment presents a small queue of tickets. For each ticket, the agent must decide:
-
-- issue type
-- priority
-- assignment group
-- resolution action
-
-## Main Runtime Flow
-
-```text
-inference.py
-    |
-    v
-client.py  <---->  server/app.py
-                         |
-                         v
-                server/environment.py
-                  |       |        |
-                  v       v        v
-            grader.py  reward.py  tasks.py
-                                  |
-                                  v
-                           data/dataset.json
-```
-
-## Main Files
-
-- `models.py`
-  Typed models for tickets, actions, observations, and state.
-
-- `server/environment.py`
-  Main environment engine.
-
-- `server/grader.py`
-  Deterministic partial-credit scorer.
-
-- `server/reward.py`
-  Step and trajectory reward helpers.
-
-- `server/tasks.py`
-  Task definitions and dataset loading.
-
-- `client.py`
-  Typed client used for multi-step interaction.
-
-- `inference.py`
-  Baseline runner with LLM mode and heuristic mode.
-
-## Task Ladder
-
-### Task 1
-
-- predict `issue_type`
-
-### Task 2
-
-- predict `issue_type`
-- predict `priority`
-
-### Task 3
-
-- predict `issue_type`
-- predict `priority`
-- predict `assignment_group`
-- predict `resolution_action`
-
-## Label Vocabulary
-
-### Issue types
-
-- `billing_license`
-- `identity_access`
-- `application_support`
-- `service_request`
-- `spam_phishing`
-- `general_inquiry`
-- `security_compliance`
-- `onboarding`
-- `feature_request`
-
-### Assignment groups
-
-- `license_ops`
-- `service_desk`
-- `application_team`
-- `procurement`
-- `security_team`
-- `onboarding_ops`
-
-### Resolution actions
-
-- `fulfill`
-- `escalate`
-- `assign`
-- `ignore`
-- `acknowledge`
-
-## Observation And State
-
-The observation exposes:
-
-- task metadata
-- the current ticket
-- queue progress counters
-- history
-- reward and done status
-
-The state tracks:
-
-- current task
-- seed
-- queue ticket IDs
-- current ticket index
-- per-ticket scores
-- total reward
-
-## Reward Logic
-
-- each step returns the current ticket score
-- the final reward is the average of per-ticket scores
-- a small overshoot penalty exists as a safeguard
-
-## Runtime Notes
-
-The repo has now passed both the initial local heuristic run and a merged-state rerun on the current `main` branch.
-
-Current local baseline:
-
-- Task 1: `1.0000`
-- Task 2: `0.8800`
-- Task 3: `0.9400`
-- Overall: `0.9400`
-
-The merged-state rerun matched the same baseline numbers exactly.
-
-One practical implementation note from runtime validation:
-
-- `data/dataset.json` may be saved with a UTF-8 BOM on Windows, so `server/tasks.py` intentionally loads it with `utf-8-sig`
-
-## Dataset Shape
-
-Each record includes:
-
-- `ticket_id`
-- `title`
-- `requester`
-- `description`
-- `issue_type`
-- `priority`
-- `assignment_group`
-- `resolution_action`
-- optional `ambiguity_note`
-- optional `related_ticket_id`
-
-## Short Version
-
-If coming back later, remember this:
-
-- the repo is a helpdesk ticket router
-- the architecture is a small OpenEnv stack
-- one ticket is shown at a time
-- the agent predicts structured routing fields
-- the grader gives deterministic partial credit
-- `inference.py` is the baseline agent runner
-- the local heuristic path now works end to end on the current merged repo state

PLAN.md

@@ -1,147 +0,0 @@
-# IT Helpdesk Ticket Routing OpenEnv - Project Plan
-
-## Project Goal
-
-Build a polished OpenEnv environment for IT helpdesk ticket routing that satisfies:
-
-- real-world utility
-- strong task and grader quality
-- clean environment design
-- OpenEnv spec compliance
-- reproducible baseline inference
-- Docker and Hugging Face deployment readiness
-
-## Current Product Definition
-
-The environment simulates a helpdesk queue. An agent receives one ticket at a time and predicts:
-
-- `issue_type`
-- `priority`
-- `assignment_group`
-- `resolution_action`
-
-The project keeps three tasks:
-
-1. Issue Type Classification
-2. Issue Type And Priority
-3. Full Ticket Routing
-
-## What Must Be True At Submission
-
-### Pass or fail requirements
-
-- the environment responds correctly
-- OpenEnv metadata is valid
-- `reset()`, `step()`, and `state()` work
-- there are at least 3 tasks
-- graders return scores in `[0.0, 1.0]`
-- `inference.py` runs and prints reproducible results
-- Docker builds and starts cleanly
-
-### Scored requirements
-
-- the task should clearly feel like real helpdesk work
-- the hard task should require meaningful reasoning
-- partial credit should be useful and deterministic
-- docs should be clear enough for judges to understand quickly
-
-## Core Files
-
-### Runtime
-
-- `models.py`
-- `server/environment.py`
-- `server/grader.py`
-- `server/reward.py`
-- `server/tasks.py`
-- `server/app.py`
-- `client.py`
-- `inference.py`
-
-### Data and metadata
-
-- `data/dataset.json`
-- `openenv.yaml`
-- `server/Dockerfile`
-- `pyproject.toml`
-- `requirements.txt`
-
-### Docs
-
-- `README.md`
-- `KNOWLEDGE.md`
-- `MENTAL_MODEL.md`
-
-## Technical Priorities
-
-### P0
-
-1. keep the environment behavior correct
-2. verify the task definitions and graders
-3. make the baseline script reliable
-4. confirm dataset coverage and label consistency
-
-### P1
-
-1. validate Docker
-2. validate deployment assumptions
-3. record baseline scores
-4. polish docs
-
-### P2
-
-1. strengthen ticket wording for realism
-2. expand hard-case examples if needed
-3. remove low-signal artifacts from the repo
-
-## Quality Checks To Perform
-
-### Environment
-
-- reset starts a clean episode
-- each step advances the queue correctly
-- the final step returns trajectory reward
-- state reflects the real internal status
-
-### Grader
-
-- exact matches score `1.0`
-- near misses get partial credit where intended
-- unsupported task IDs fail clearly
-- scores vary across examples
-
-### Inference
-
-- heuristic mode works without model credentials
-- LLM mode reads `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`
-- output is reproducible when the seed is fixed
-
-### Docs
-
-- no outdated domain references remain
-- team and project metadata are correct
-- setup and run instructions are accurate
-
-## Risks
-
-### Runtime risk
-
-The first local execution pass and a merged-state rerun have already completed successfully. The remaining runtime risk is Docker and clean-machine behavior, not first-pass local execution.
-
-### Benchmark risk
-
-The current merged-state local benchmark has already been recorded. The remaining benchmark risk is making sure Docker or clean-machine validation does not surface a late behavioral mismatch.
-
-### Deployment risk
-
-Docker and Hugging Face behavior should be validated before the final submission window.
-
-## Definition Of Done
-
-The project is ready when:
-
-1. the environment runs locally end to end
-2. the heuristic baseline runs successfully
-3. Docker build and run both succeed
-4. the docs are clean, current, and submission-ready
-5. the repo clearly presents Hackstreet Boys as the team

PROJECT_STATUS.md

@@ -136,7 +136,7 @@ Roopal-side work completed:
 - updated `README.md` to reflect the first local runtime pass
 - recorded the current heuristic baseline in repo docs as a working, non-final benchmark
 - updated `KNOWLEDGE.md` to distinguish consistency validation from runtime validation
-- updated `MENTAL_MODEL.md` with runtime-validated notes and the Windows BOM handling detail
+- updated the runtime mental-model notes later merged into `KNOWLEDGE.md`, including the Windows BOM handling detail
 
 Documentation fixes made from runtime feedback:
 
@@ -182,7 +182,7 @@ Roopal-side work completed:
 
 - audited required submission files and confirmed they are present in the repo
 - completed a stale-claims and outdated-wording pass across the core docs
-- updated `PLAN.md` to reflect that first-pass local execution is no longer the main runtime risk
+- updated the planning / requirements doc later consolidated into `required.md` to reflect that first-pass local execution is no longer the main runtime risk
 - left the remaining work focused on Docker and clean-machine validation rather than documentation cleanup
 
 ## Open Items

README.md

@@ -212,8 +212,7 @@ pyproject.toml
 requirements.txt
 README.md
 KNOWLEDGE.md
-PLAN.md
-MENTAL_MODEL.md
+required.md
 ROADMAP.md
 ```
 
@@ -355,7 +354,7 @@ An April 6 repo audit also confirmed that all required submission files are pres
 
 - runtime: `models.py`, `client.py`, `inference.py`, `server/app.py`, `server/environment.py`, `server/grader.py`, `server/reward.py`, `server/tasks.py`
 - data and metadata: `data/dataset.json`, `openenv.yaml`, `pyproject.toml`, `requirements.txt`, `server/Dockerfile`
-- docs and planning: `README.md`, `KNOWLEDGE.md`, `MENTAL_MODEL.md`, `PLAN.md`, `PROJECT_STATUS.md`, `ROADMAP.md`
+- docs and planning: `README.md`, `KNOWLEDGE.md`, `required.md`, `PROJECT_STATUS.md`, `ROADMAP.md`
 
 Still pending before final submission:
 

ROADMAP.md

@@ -12,9 +12,9 @@
 
 - `PROJECT_STATUS.md` is the canonical log of completed work.
 - This roadmap is the remaining execution plan from the current repo state to final submission.
-- `PLAN.md` defines the must-pass gates.
+- `required.md` is now the combined official-requirements and project-compliance file.
 - `KNOWLEDGE.md` defines the current repo truth and judge-facing explanation.
-- `analysis/comp.md`, `analysis/comp_know.md`, and `analysis/inference.md` are internal competitive notes only. Use them to prioritize work, but do not mention competitor repos in public-facing docs.
+- `analysis/comp.md` and `analysis/comp_know.md` are internal competitive notes only. Use them to prioritize work, but do not mention competitor repos in public-facing docs.
 
 ## What We Are Optimizing For
 
@@ -34,7 +34,7 @@ The highest-value wins from now to submission are:
    - do this as an audit / evidence layer, not as a late dataset merge
 
 4. **Submission readiness**
-   - satisfy every requirement from `PLAN.md` and `KNOWLEDGE.md`
+   - satisfy every requirement from `required.md` and `KNOWLEDGE.md`
    - keep the repo easy for judges to understand and rerun
 
 ## Current Repo State
@@ -57,14 +57,19 @@ The remaining work should be treated as targeted strengthening, not broad featur
 
 ## Submission Gates That Must Still Hold
 
-These come directly from `PLAN.md` and `KNOWLEDGE.md`:
+These come directly from `required.md` and `KNOWLEDGE.md`:
 
 - the environment starts correctly
 - `reset()`, `step()`, and `state()` behave correctly
 - 3 tasks exist and remain meaningfully different
 - grader scores stay in `[0.0, 1.0]`
 - `inference.py` runs reproducibly without crashing
+- `inference.py` uses the OpenAI client with `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`
+- structured stdout logs follow the official `[START]`, `[STEP]`, and `[END]` format
+- `openenv validate` passes
 - Docker builds and starts cleanly
+- HF deployment responds cleanly and reset works
+- inference stays inside the official runtime / machine envelope
 - docs and metadata are current
 - the repo is easy for judges to understand and rerun
 
@@ -80,6 +85,7 @@ These come directly from `PLAN.md` and `KNOWLEDGE.md`:
 - add only safe RL-oriented improvements
 - add external grounding evidence without changing the runtime dataset
 - finish packaging / deployment readiness
+- verify official validation constraints, not just local happy-path behavior
 
 ### Do Not Do Before Submission
 
@@ -108,7 +114,7 @@ Because we are using Codex to generate code, we should optimize for small, bound
 
 **Window:** April 3 to April 4
 
-**Goal:** eliminate the biggest competitive weakness identified in `analysis/comp.md` and `analysis/inference.md`: lack of checked-in tests.
+**Goal:** eliminate the biggest competitive weakness identified in `analysis/comp.md` and `analysis/comp_know.md`: lack of checked-in tests.
 
 ### Must produce
 
@@ -176,7 +182,7 @@ Because we are using Codex to generate code, we should optimize for small, bound
 - assignment group and resolution action remain exact
 - final episode reward stays bounded and deterministic
 
-### Safe improvement candidates from `analysis/inference.md`
+### Safe improvement candidates from `analysis/comp_know.md`
 
 - expand `ISSUE_TYPE_SIMILARITY` with only a few defensible pairs, if backed by grounding review
 - enrich `history` with:
@@ -231,14 +237,17 @@ Because we are using Codex to generate code, we should optimize for small, bound
 
 **Window:** April 6 to April 7
 
-**Goal:** close the submission-readiness gaps surfaced in `analysis/comp_know.md` and `analysis/inference.md`.
+**Goal:** close the submission-readiness gaps surfaced in `analysis/comp_know.md`.
 
 ### Must produce
 
 - Hugging Face Spaces README frontmatter
 - `.openenvignore`
+- `openenv validate` evidence
 - Docker smoke evidence on the merged branch
 - one clean-copy rerun if possible
+- structured inference logging verified against the official format
+- a practical check that inference remains inside the official runtime envelope
 
 ### Nice-to-have only if green
 
@@ -264,6 +273,7 @@ Because we are using Codex to generate code, we should optimize for small, bound
 - no runtime refactors
 - no dataset edits unless they fix a blocker
 - stop risky edits several hours before submission
+- if possible, run the official validator or the closest local equivalent before final push
 
 ## Ownership From Now Until Submission
 
@@ -276,7 +286,6 @@ Primary files:
 - `server/grader.py`
 - `README.md`
 - `KNOWLEDGE.md`
-- `MENTAL_MODEL.md`
 
 Primary responsibilities:
 
@@ -293,6 +302,7 @@ Concrete deliverables:
 - any similarity-matrix update, if justified
 - doc updates if benchmark numbers or scoring explanation change
 - README frontmatter and judge-facing clarity
+- official requirement compliance review through `required.md`
 
 ### Suyash ownership
 
@@ -326,6 +336,7 @@ Concrete deliverables:
 - `.openenvignore`
 - Docker smoke confirmation
 - clean-copy rerun if possible
+- structured inference logging compliance
 
 ### Shared responsibilities
 
@@ -335,6 +346,7 @@ Concrete deliverables:
 - use the GitHub Actions Docker smoke workflow when local Docker is blocked
 - review Codex-generated diffs before accepting them
 - freeze feature work by the end of April 7
+- do not casually change the `[START]`, `[STEP]`, `[END]` inference log format once implemented
 
 ## Date-By-Date Execution Plan
 
@@ -355,6 +367,7 @@ Suyash:
 - scaffold `tests/`
 - begin smoke tests for `reset()`, `step()`, `state()`, and deterministic seeded behavior
 - confirm how integration tests will hit the app cleanly
+- review `required.md` and identify the exact official validation items still not reflected in runtime / inference behavior
 
 Shared checkpoint:
 
@@ -377,6 +390,7 @@ Suyash:
 
 - complete smoke tests
 - add first-pass integration tests for `/health`, `/tasks`, `/reset`, and `/step`
+- begin checking how current `inference.py` differs from the official structured logging requirement
 
 Shared checkpoint:
 
@@ -400,6 +414,7 @@ Suyash:
 - add integration coverage for full seeded episode flow and `state()`
 - add a light heuristic regression path for `inference.py`
 - optionally enrich observation history if tests are already green
+- bring `inference.py` closer to official structured logging format if the change can be done safely
 
 Shared checkpoint:
 
@@ -424,6 +439,8 @@ Suyash:
 - add `.openenvignore`
 - verify Docker smoke workflow on the merged branch
 - check deployment assumptions around `app_port`, `/docs`, `/health`, `/ws`, and `/web`
+- run `openenv validate` or the closest available validation path
+- verify structured inference logging and runtime-envelope expectations
 
 Shared checkpoint:
 
@@ -439,7 +456,7 @@ Primary goal:
 
 Roopal:
 
-- final docs consistency pass across `README.md`, `KNOWLEDGE.md`, and `MENTAL_MODEL.md`
+- final docs consistency pass across `README.md` and `KNOWLEDGE.md`
 - add a short TRL / GRPO usage example only if everything else is already green
 
 Suyash:
@@ -466,6 +483,7 @@ Morning:
 - run final smoke / test slice on the submission branch
 - verify required files are present
 - verify README and metadata are current
+- run the final validation checklist from `required.md`
 
 Afternoon:
 
@@ -493,6 +511,7 @@ Do not cut these:
 3. Docker / deployment validation
 4. grounding audit evidence
 5. final benchmark sanity rerun if behavior changed
+6. official structured inference logging compliance
 
 ## Definition Of Done
 
@@ -502,9 +521,10 @@ The project is ready when:
 2. scoring is demonstrably deterministic and not fuzzy by default
 3. a grounding audit against real public support datasets exists
 4. the heuristic baseline still runs successfully
-5. Docker build and run are validated
-6. docs and metadata are current and judge-friendly
-7. the repo is frozen and submitted on time
+5. the inference path is compliant with the official log format
+6. `openenv validate` and Docker checks are validated
+7. docs and metadata are current and judge-friendly
+8. the repo is frozen and submitted on time
 
 ## Simple Rule To Remember
 

analysis/comp_know.md

@@ -1,275 +1,237 @@
-# Competition Knowledge Base — OpenEnv Hackathon
+# Competition Knowledge Base And Action Plan
 
-> Source: github.com/meta-pytorch/OpenEnv/tree/main/envs  
-> Gathered: April 4, 2026  
-> Purpose: Internal competitive intelligence — NOT for commit/push
+> Source: github.com/meta-pytorch/OpenEnv/tree/main/envs
+> Gathered: April 4, 2026
+> Purpose: Internal competitive intelligence plus action planning - NOT for commit/push
 
 ---
 
-## Full Environment Inventory (27 envs)
+## Full Environment Inventory
 
 | Env | Domain | Complexity | Reward Type | Multi-step? | MCP? |
 |-----|--------|------------|-------------|-------------|------|
 | `atari_env` | Classic games | Medium | Dense | Yes | No |
 | `browsergym_env` | Web browser automation | Very High | Task-based | Yes | No |
-| `calendar_env` | Calendar/scheduling agent | High | SQL verifier | Yes | Yes (MCP) |
+| `calendar_env` | Calendar / scheduling agent | High | SQL verifier | Yes | Yes |
 | `carla_env` | Autonomous driving sim | Very High | Dense | Yes | No |
-| `chat_env` | Conversation/tokenization | Low | Custom transform | Yes | No |
-| `chess_env` | Chess game | Medium | Win/loss | Yes | No |
+| `chat_env` | Conversation / tokenization | Low | Custom transform | Yes | No |
 | `coding_env` | Python code execution | Medium | Exit code / transform | Yes | No |
-| `connect4_env` | Connect 4 game | Low | Win/loss | Yes | No |
-| `dipg_safety_env` | Safety/policy | Medium | Unknown | Yes | No |
-| `dm_control_env` | DeepMind Control Suite | High | Dense | Yes | No |
-| `echo_env` | Reference/minimal | Minimal | Echo | No | No |
-| `finqa_env` | Financial QA (SEC 10-K) | High | Fuzzy numerical | Yes | Yes (MCP) |
-| `finrl_env` | Financial RL trading | High | Portfolio return | Yes | No |
-| `git_env` | Git operations | Medium | Task-based | Yes | No |
-| `grid_world_env` | Grid navigation | Low | Sparse | Yes | No |
-| `julia_env` | Julia code execution | Medium | Exit code | Yes | No |
-| `kernrl` | Kernel/OS operations | High | Unknown | Yes | No |
-| `maze_env` | Maze navigation | Low | Sparse | Yes | No |
-| `openapp_env` | Web app UI (BrowserGym) | Extreme | Task-based | Yes | No |
-| `openspiel_env` | Multi-agent games | High | Game outcome | Yes | No |
-| `reasoning_gym_env` | Reasoning tasks (100+ datasets) | Medium | Exact/partial | Single-step | No |
-| `repl_env` | REPL execution | Medium | Exit code | Yes | No |
-| `snake_env` | Snake game | Low | Score | Yes | No |
-| `sumo_rl_env` | Traffic simulation | High | Traffic flow | Yes | No |
-| `tbench2_env` | Terminal Bench 2 (shell tasks) | High | pytest pass/fail | Yes | No |
-| `textarena_env` | Text-based games | Medium | Game outcome | Yes | No |
-| `unity_env` | Unity 3D simulation | Very High | Task-based | Yes | No |
+| `echo_env` | Reference / minimal | Minimal | Echo | No | No |
+| `finqa_env` | Financial QA | High | Fuzzy numerical | Yes | Yes |
+| `openapp_env` | Web app UI | Extreme | Task-based | Yes | No |
+| `reasoning_gym_env` | Reasoning tasks | Medium | Exact / partial | Single-step | No |
+| `tbench2_env` | Terminal tasks | High | Pytest pass/fail | Yes | No |
 
----
-
-## Deep Dives: Most Relevant Envs
-
-### 1. `finqa_env` — Financial QA
-
-**What it does**: Agents answer complex financial questions from SEC 10-K filings using SQL tool calls.
-
-**Architecture**:
-- Subclasses `MCPEnvironment` (not plain `Environment`) — uses FastMCP with `@mcp.tool` decorators
-- Tools: `get_descriptions`, `get_table_info`, `sql_query`, `submit_answer`
-- Dataset: 290 questions from HuggingFace (`snorkelai/finqa-data`)
-- Max steps: 50 per episode
-- Reward: Binary (1.0 / 0.0) with fuzzy numerical matching (1% relative tolerance + 1.0 absolute tolerance)
-- Handles `\boxed{}` LaTeX format, percentages, fractions, thousands separators, negative parens
+This is not the full raw repo dump anymore. It is the subset that matters most for competitive positioning and late-stage prioritization.
 
-**Reward sophistication**: Very high. The `rewards.py` is ~300 lines handling multi-value answers, year-labeled pairs, percentage normalization, and both relative + absolute tolerance checks simultaneously.
-
-**Key differentiator**: MCP protocol for tool discovery. Client uses `await env.list_tools()` to discover tools at runtime. This is the most "agentic" env in the repo.
+---
 
-**Integration**: Explicitly shows TRL/GRPO integration pattern in README.
+## Most Relevant Competitor Patterns
 
----
+### `finqa_env`
 
-### 2. `coding_env` — Python Code Execution
+- strong MCP / tool-using architecture
+- larger dataset than ours
+- binary-style reward with fuzzy numerical matching
+- explicit TRL / GRPO integration story
 
-**What it does**: Executes arbitrary Python code in a sandboxed environment.
+### `coding_env`
 
-**Architecture**:
-- `PythonCodeActEnv` wraps a `PyExecutor` (sandboxed subprocess)
-- `create_safe_coding_transform()` — transform pipeline for reward computation
-- Action: `CodeAction(code: str)`
-- Observation: `CodeObservation(stdout, stderr, exit_code)`
-- State: `CodeState(episode_id, step_count, last_exit_code)`
-- Reward: computed by transform (not in step directly) — extensible pattern
+- strongest test story
+- clean transform-based reward separation
+- reference example of strong code quality and architecture hygiene
 
-**Key differentiator**: Transform-based reward. The environment itself doesn't compute reward — a pluggable `Transform` object does. This is the cleanest separation of concerns in the repo.
+### `reasoning_gym_env`
 
-**Testing**: Has both unit tests (`test_python_codeact_reset`, `test_python_codeact_rewards`) and integration tests (`test_coding_env_integration`). Most tested env in the repo.
+- broadest dataset coverage
+- configurable dataset / size pattern
+- useful deployment references for `openenv push`
 
----
+### `tbench2_env`
 
-### 3. `reasoning_gym_env` — Reasoning Tasks
+- strong agentic shell-task realism
+- binary evaluation via pytest
+- little intermediate reward signal
 
-**What it does**: Wraps the `reasoning-gym` library (100+ reasoning datasets) as a single-step OpenEnv.
+### `openapp_env`
 
-**Architecture**:
-- Single-step episodes: `reset()` gives question, `step()` gives score + done=True
-- Composite datasets: mix multiple datasets with weights
-- Dataset persistence: same dataset reused across resets until config changes
-- Supports `dataset_name`, `seed`, `size`, `dataset_specs` in `reset()` kwargs
-- Reward: 0.0–1.0 (dataset-dependent, may use partial credit)
+- highest complexity
+- multimodal / browser-based
+- difficult to beat on ambition, easier to beat on simplicity and reproducibility
 
-**Key differentiator**: Massive breadth (100+ task types in one env). The `reset()` kwargs pattern for dataset configuration is very clean. Also has `openenv push` CLI for HuggingFace Spaces deployment.
+### `calendar_env`
 
-**Scale**: uv.lock is 551KB — large dependency tree from reasoning-gym.
+- enterprise workflow flavor
+- scenario + verifier pattern
+- stronger on MCP sophistication than on reward density
 
 ---
 
-### 4. `tbench2_env` — Terminal Bench 2
+## Structural Patterns Across The Field
 
-**What it does**: Wraps Terminal-Bench-2 shell tasks. Agent executes shell commands and is evaluated by pytest.
+### Packaging
 
-**Architecture**:
-- Two modes: `local` (direct process) and `docker` (per-task container)
-- Rich action type: `exec`, `write`, `view`, `wait`, `kill`, `write_file`, `evaluate`, `close`
-- Session IDs for streaming/non-blocking processes
-- Reward: Binary (pytest pass/fail) on `evaluate` action
-- Intermediate steps: `reward=None`
+- every serious repo has `models.py`, `client.py`, `openenv.yaml`, `pyproject.toml`, `README.md`, and a `server/` package
+- Hugging Face Spaces frontmatter is standard in competitor `README.md` files
+- `.openenvignore` appears in some stronger submissions
 
-**Key differentiator**: Most realistic "agentic" shell environment. The session ID pattern for streaming processes is unique. Docker-in-Docker mode for full fidelity.
+### Reward patterns
 
----
+| Pattern | Examples | Notes |
+|---------|----------|-------|
+| Binary | `finqa_env`, `tbench2_env` | easy to verify, weaker RL signal |
+| Dense partial | ours, games | stronger RL learning signal |
+| Transform-based | `coding_env`, `chat_env` | architecturally clean |
+| SQL / verifier based | `calendar_env` | strong task verification |
 
-### 5. `openapp_env` — Web App UI
+### Testing patterns
 
-**What it does**: Wraps OpenApps (calendar, todo, messenger, maps) + BrowserGym for browser-based UI agent training.
+- many repos have little or no tests
+- `coding_env` is still the strongest example of checked-in testing
+- this makes tests a high-value differentiator for us
 
-**Architecture**:
-- Runs TWO services in Docker: OpenApps server (port 5001) + FastAPI (port 8000)
-- `start.sh` orchestrates both
-- BrowserGym for browser automation (Playwright/Chromium)
-- Docker image: ~5.7GB (includes Chromium)
-- Multimodal: screenshots + DOM observations
+### Deployment patterns
 
-**Key differentiator**: Most complex env in the repo. Multimodal (visual + text). Real browser interaction. Closest to real-world agent deployment.
+- Spaces usually expose `/web`, `/docs`, `/health`, and `/ws`
+- `openenv push` is the expected deployment workflow
+- `README` frontmatter and Docker correctness matter more than polish extras
 
 ---
 
-### 6. `calendar_env` — Calendar Scheduling
-
-**What it does**: Calendar management tasks with SQL database verification.
-
-**Architecture**:
-- MCP-based (like finqa_env)
-- Has `client_notebooks/` — Jupyter notebook for interactive evaluation
-- Has `mcp_databases/` — SQLite databases for state
-- Scenario-based: `scenario_config.json` drives task + verifiers
-- Verifiers: SQL queries that check task completion
-- Supports OpenAI, Anthropic, Google providers
+## Key Technical Observations
 
-**Key differentiator**: Scenario config pattern. Verifier-based reward (SQL queries check if the agent actually completed the task). Most "enterprise workflow" env.
+1. MCP is useful, but too big to add late.
+2. Transform-based reward is elegant, but not a deadline-critical refactor.
+3. HF Spaces frontmatter is expected and missing in our repo.
+4. `.openenvignore` is a cheap packaging win.
+5. Configurable datasets are nice, but external dataset merge is too risky late.
+6. Strong tests improve trust more than minor architectural polish.
+7. Dense, deterministic, partial-credit reward is one of our real advantages.
 
 ---
 
-### 7. `chat_env` — Chat/Tokenization
+## Actionable Inferences
 
-**What it does**: Manages conversation history + tokenization for LLM RL training.
+## Critical Missing Items
 
-**Architecture**:
-- Action: `ChatAction(tokens: torch.Tensor)` — takes raw model tokens
-- Observation: `ChatObservation(messages, tokens)` — both human-readable + model-ready
-- Transform-based reward (pluggable)
-- Dual representation: messages (human) + tokens (model)
-- No HTTP overhead option: can use directly without server
+### 1. README frontmatter for HF Spaces
 
-**Key differentiator**: Designed for direct LLM RL training loop. The only env that takes raw PyTorch tensors as actions. Pairs with GRPO/PPO training loops directly.
+This is still the cleanest obvious gap. Add it before submission.
 
----
-
-## Structural Patterns Observed Across All Envs
-
-### File Structure (canonical)
-```
-env_name/
-├── __init__.py          # exports
-├── models.py            # Action, Observation, State
-├── client.py            # EnvClient subclass
-├── openenv.yaml         # metadata
-├── pyproject.toml       # packaging
-├── README.md            # HuggingFace Space frontmatter + docs
-└── server/
-    ├── __init__.py
-    ├── app.py           # FastAPI
-    ├── environment.py   # core logic
-    └── Dockerfile
-```
+Recommended fields:
 
-### README Frontmatter (HuggingFace Spaces)
-Every env README has YAML frontmatter:
 ```yaml
 ---
-title: ...
-emoji: ...
-colorFrom: ...
-colorTo: ...
+title: IT Helpdesk Ticket Routing OpenEnv
+emoji: "ticket"
+colorFrom: blue
+colorTo: indigo
 sdk: docker
 pinned: false
-app_port: 8000
+app_port: 7860
 base_path: /web
 tags:
   - openenv
+  - helpdesk
+  - ticket-routing
+  - nlp
 ---
 ```
-This is required for HuggingFace Spaces deployment. Our README does NOT have this.
-
-### openenv.yaml — Minimal Pattern
-Most envs have very minimal `openenv.yaml` (just name + entry_point). Our yaml is the most detailed in the repo.
-
-### Dockerfile Patterns
-- Most use `openenv-base:latest` as base image (not `python:3.11-slim`)
-- Our Dockerfile uses `python:3.11-slim` directly — this is the standalone/HF Spaces pattern
-- The `openenv-base` pattern is for the monorepo CI/CD workflow
-
-### Testing
-- `coding_env`: most tested (unit + integration)
-- Most envs: no tests at all
-- Our env: no tests (matches majority)
-
-### MCP vs HTTP
-- Most envs: plain HTTP (`Environment` base class)
-- `finqa_env`, `calendar_env`: MCP (`MCPEnvironment` base class, FastMCP tools)
-- MCP envs are more "agentic" — tools are discoverable at runtime
-
-### Reward Patterns
-| Pattern | Envs | Description |
-|---------|------|-------------|
-| Binary (0/1) | finqa, tbench2, reasoning_gym | Pass/fail |
-| Dense partial | ours, chess, atari | Continuous [0,1] |
-| Transform-based | coding, chat | Pluggable reward function |
-| SQL verifier | calendar | DB state check |
-| Game outcome | chess, connect4, openspiel | Win/loss/draw |
 
----
+### 2. `.openenvignore`
+
+Cheap packaging improvement. Worth adding.
 
-## Deployment Patterns
+### 3. Verified deployment assumptions
 
-### HuggingFace Spaces
-- `openenv push` CLI command (seen in reasoning_gym README)
-- Spaces get: `/web` (UI), `/docs` (Swagger), `/health`, `/ws` (WebSocket)
-- `base_path: /web` in README frontmatter
-- Our env: missing HF Spaces frontmatter in README
+We should explicitly verify:
 
-### Docker
-- Most envs: `openenv-base:latest` (monorepo CI)
-- Standalone envs (ours, openapp): `python:3.11-slim`
-- openapp: 5.7GB image (Chromium)
-- Our image: minimal (python:3.11-slim + pip deps)
+- `app_port: 7860`
+- `/health`
+- `/docs`
+- `/ws`
+- `/web`
 
 ---
 
-## Dataset Sizes
+## High-Value Improvements That Still Make Sense
 
-| Env | Dataset Size | Source |
-|-----|-------------|--------|
-| finqa | 290 questions | HuggingFace (snorkelai/finqa-data) |
-| reasoning_gym | 100+ datasets, configurable size | reasoning-gym library |
-| calendar | SQLite DBs | Custom |
-| ours | 45 tickets | Custom (data/dataset.json) |
-| coding | N/A (generates tasks) | N/A |
-| tbench2 | Terminal-Bench-2 repo | GitHub auto-download |
+### 4. Strengthen the scorer only in grounded, tested ways
 
----
+Possible additions to `ISSUE_TYPE_SIMILARITY`:
 
-## Key Technical Observations
+- `onboarding` vs `service_request`
+- `feature_request` vs `service_request`
+- `security_compliance` vs `identity_access`
+- `billing_license` vs `identity_access`
+
+Only do this if:
+
+- the ambiguity is real
+- the change is backed by tests
+- it does not blur operationally distinct actions too much
+
+### 5. Add richer `history` if low-risk
+
+Candidate additions:
+
+- ticket title
+- predicted fields
 
-1. **MCP is the emerging pattern** for tool-using agents. finqa and calendar both use it. Our env uses plain HTTP — simpler but less "agentic."
+This can help multi-step reasoning without changing the core task.
 
-2. **Transform-based rewards** (coding_env, chat_env) are the cleanest architecture for extensible reward shaping. Our reward is hardcoded in `reward.py`.
+### 6. Add `queue_size` as an optional `reset()` kwarg
 
-3. **`openenv push` CLI** exists for HuggingFace Spaces deployment. We should use it.
+Nice RL/training flexibility, but lower priority than tests, scorer crispness, Docker, and deployment readiness.
 
-4. **README frontmatter** is required for HF Spaces. Our README is missing it.
+### 7. Add a short TRL / GRPO example to README
 
-5. **Composite/configurable datasets** (reasoning_gym) are a strong differentiator. Our dataset is fixed at 45 tickets.
+Good judge-facing signal once the repo is already green.
 
-6. **WebSocket endpoint** (`/ws`) is mentioned in reasoning_gym README as a HF Spaces feature. Our env already has `/ws` via the OpenEnv base.
+---
+
+## Improvements To Defer
+
+- MCP migration
+- transform-based reward refactor
+- major dataset expansion
+- external dataset merge into runtime
+- broad inference rewrite
+- dependency churn just for polish
+
+---
 
-7. **`uv.lock`** files appear in chat_env and reasoning_gym — reproducible dependency locking. We use `requirements.txt` only.
+## Competitive Positioning
 
-8. **`.openenvignore`** file in finqa_env — analogous to `.dockerignore` for the OpenEnv push CLI.
+### Our strengths
 
-9. **`base_path: /web`** in HF Spaces frontmatter — the web UI is at `/web`, not `/`. Our env would need this.
+1. strong real-world enterprise domain
+2. dense deterministic reward
+3. partial-credit grading that is still explainable
+4. clean 3-task difficulty ladder
+5. strong heuristic baseline
+6. compact, rerunnable environment design
+
+### Our weaknesses
+
+1. weaker checked-in test story unless we fix it
+2. missing HF Spaces frontmatter unless we fix it
+3. smaller dataset than some top competitors
+4. less ambitious architecture than the strongest simulator-style or MCP-heavy entries
+
+---
 
-10. **Episode length**: Most envs are either single-step (reasoning_gym) or unbounded (coding, tbench2). Our env is bounded (3–5 steps) — a clean middle ground.
+## Priority Action List
+
+| Priority | Action | Effort | Impact |
+|----------|--------|--------|--------|
+| P0 | Add tests and prove scorer crispness | 1-2 hrs | High |
+| P0 | Add HF Spaces frontmatter to README | 5 min | High |
+| P0 | Add `.openenvignore` | 5 min | Medium |
+| P1 | Add grounding audit against public support datasets | 1-2 hrs | High |
+| P1 | Expand similarity pairs only if grounded and tested | 20-40 min | Medium |
+| P1 | Add richer `history` if low-risk | 20 min | Medium |
+| P1 | Add TRL / GRPO README example | 30 min | High |
+| P2 | Add `queue_size` kwarg | 15 min | Low |
+| P3 | Expand dataset substantially | 2+ hrs | Medium but risky |
+| P3 | Transform-based reward refactor | 1 hr | Low |

analysis/inference.md

@@ -1,218 +0,0 @@
-# Inferences & Actionable Advantages
-
-> Based on deep analysis of all 27 OpenEnv competition entries  
-> Internal use only — NOT for commit/push
-
----
-
-## Critical Missing Items (Fix Before Submission)
-
-### 1. README HuggingFace Spaces Frontmatter — MISSING
-
-Every single env in the repo has YAML frontmatter at the top of README.md. Ours does not.
-This is required for `openenv push` and HuggingFace Spaces deployment to work correctly.
-
-**Add to top of `meta-AIHack/README.md`:**
-```yaml
----
-title: IT Helpdesk Ticket Routing OpenEnv
-emoji: 🎫
-colorFrom: blue
-colorTo: indigo
-sdk: docker
-pinned: false
-app_port: 7860
-base_path: /web
-tags:
-  - openenv
-  - helpdesk
-  - ticket-routing
-  - nlp
----
-```
-
-Note: our port is `7860` (HF Spaces default), not `8000`. Use `7860` here.
-
----
-
-### 2. `.openenvignore` File — MISSING
-
-`finqa_env` has a `.openenvignore` file (analogous to `.dockerignore` for the `openenv push` CLI).
-Without it, `openenv push` may upload unnecessary files.
-
-**Create `meta-AIHack/.openenvignore`:**
-```
-*.pyc
-__pycache__/
-.git/
-*.md
-PLAN.md
-ROADMAP.md
-MENTAL_MODEL.md
-KNOWLEDGE.md
-comp_intel/
-bugs/
-transcripts/
-```
-
----
-
-### 3. `base_path: /web` in openenv.yaml — CHECK
-
-The HF Spaces web UI is served at `/web`. The `reasoning_gym_env` README explicitly mentions:
-- Web Interface at `/web`
-- API Documentation at `/docs`
-- Health Check at `/health`
-- WebSocket at `/ws`
-
-Our `openenv.yaml` lists `/docs` in `api.endpoints` — good. But we should verify the web interface path is correct when deployed.
-
----
-
-## High-Value Improvements (Implement If Time Allows)
-
-### 4. Partial Credit Similarity Matrix — Expand
-
-Our `grader.py` has `ISSUE_TYPE_SIMILARITY` with 16 pairs and `PRIORITY_SCORES` with 10 pairs.
-
-**Observation from finqa_env**: Their reward uses both relative AND absolute tolerance simultaneously. Our grader uses a flat similarity dict.
-
-**Improvement**: Add more near-miss pairs to `ISSUE_TYPE_SIMILARITY`. Currently missing:
-- `("onboarding", "service_request")` — onboarding tickets often look like service requests
-- `("feature_request", "service_request")` — common confusion
-- `("security_compliance", "identity_access")` — MFA/SSO tickets can go either way
-- `("billing_license", "identity_access")` — license + account access overlap
-
-This directly improves the reward signal quality for RL training, which is what judges care about.
-
----
-
-### 5. Dataset Size — Expand from 45 to ~100 tickets
-
-**Observation**: finqa has 290 questions, reasoning_gym has configurable sizes up to thousands.
-Our 45 tickets is the smallest custom dataset in the repo.
-
-**Improvement**: Add 55 more tickets to reach 100. Focus on:
-- More ambiguous cases (harder for LLMs)
-- More `related_ticket_id` chains (multi-ticket threads)
-- Edge cases: tickets that span two issue types
-- More `spam_phishing` examples (currently underrepresented)
-
-This makes the benchmark more robust and harder to overfit.
-
----
-
-### 6. Transform-Based Reward (Optional Architecture Upgrade)
-
-**Observation**: `coding_env` uses a pluggable `Transform` object for reward computation instead of hardcoding it in `step()`. This is the cleanest pattern in the repo.
-
-**Improvement**: Refactor `server/reward.py` to expose a `HelpdeskRewardTransform` class that can be swapped. Low priority — our current design works fine — but it signals architectural sophistication to judges.
-
----
-
-### 7. Configurable Queue Size via `reset()` kwargs
-
-**Observation**: `reasoning_gym_env` passes `size`, `seed`, `dataset_name` as `reset()` kwargs. This makes the env much more flexible for RL training (vary episode length, vary dataset).
-
-**Improvement**: Accept `queue_size` as a `reset()` kwarg (in addition to `task_id` and `seed`):
-```python
-def reset(self, seed=None, episode_id=None, **kwargs):
-    queue_size = kwargs.get("queue_size", None)  # override QUEUE_SIZE_RANGE
-    ...
-```
-
-This lets RL trainers control episode length without modifying the env code.
-
----
-
-### 8. `uv.lock` for Reproducible Dependencies
-
-**Observation**: `chat_env` and `reasoning_gym_env` both include `uv.lock` files for fully reproducible dependency resolution.
-
-**Improvement**: Run `uv lock` in `meta-AIHack/` and commit the `uv.lock`. This signals production-quality dependency management.
-
----
-
-### 9. Explicit TRL/GRPO Integration Example in README
-
-**Observation**: `finqa_env` README explicitly shows a TRL GRPO integration snippet. This is exactly what Meta/PyTorch judges want to see — the env being used for actual RL training.
-
-**Improvement**: Add a section to our README showing how to use the env with TRL GRPO:
-```python
-# Example: Using with TRL GRPO
-from trl import GRPOTrainer
-from client import HelpdeskTicketEnvClient
-
-async def rollout_func(prompts, trainer):
-    sync_client = HelpdeskTicketEnvClient(base_url=ENV_URL).sync()
-    with sync_client:
-        result = sync_client.reset(seed=42, task_id=3)
-        # ... agent loop
-        return {"reward": final_reward, "completion": completion}
-```
-
----
-
-### 10. `history` Field — Richer Step History
-
-**Observation**: `finqa_env` passes full tool call history in observation metadata. Our `history` field currently only stores `{step, score, breakdown}`.
-
-**Improvement**: Include the ticket title and predicted fields in history so the agent can learn from its own past decisions within an episode:
-```python
-history_entry = {
-    "ticket_id": current_ticket.ticket_id,
-    "title": current_ticket.title,  # ADD THIS
-    "predicted": {k: v for k, v in action.model_dump().items() if v is not None},  # ADD THIS
-    "score": score,
-    "breakdown": breakdown,
-}
-```
-
-This gives the LLM agent richer context for multi-step reasoning.
-
----
-
-## Competitive Positioning Insights
-
-### Our Unique Strengths vs. The Field
-
-1. **Richest `openenv.yaml`**: Ours is the most detailed metadata file in the entire repo. Most envs have 3-line yaml files. Ours has tasks, evaluation, grading, reproducibility, inference config. This signals thoroughness.
-
-2. **Deterministic + Reproducible**: We explicitly set `deterministic: true` and `reproducible: true` in openenv.yaml. Only a few envs do this. Judges can rerun and get identical results.
-
-3. **Task Ladder (3 difficulty levels)**: Most envs have a single task. We have 3 explicitly difficulty-graded tasks. This is a strong differentiator for RL curriculum learning.
-
-4. **Partial Credit Grading**: Most envs use binary reward (0/1). Our grader gives partial credit for near-miss issue types and adjacent priorities. This produces a much richer reward signal for RL training.
-
-5. **Dense Reward Signal**: Every step produces a reward (not just the final step). Most envs (tbench2, finqa) only reward at the end. Dense rewards are better for RL training.
-
-6. **Heuristic Baseline**: We have a working keyword-based heuristic that achieves 0.94 overall. Most envs don't have a baseline agent. This lets judges immediately see the env working.
-
-7. **Real-World Domain**: IT helpdesk routing is a real enterprise use case. Many envs are games or synthetic tasks. Ours has immediate practical applicability.
-
-8. **Clean Episode Bounds**: 3–5 steps per episode. Not too short (single-step), not unbounded. Clean for RL training.
-
-### Our Weaknesses vs. The Field
-
-1. **No HF Spaces frontmatter** in README — fixable in 5 minutes
-2. **Smallest dataset** (45 tickets) — expandable
-3. **No MCP tools** — plain HTTP only (simpler but less "agentic")
-4. **No tests** — matches most envs, but coding_env has tests
-5. **No `uv.lock`** — minor
-6. **No `.openenvignore`** — minor
-
----
-
-## Priority Action List
-
-| Priority | Action | Effort | Impact |
-|----------|--------|--------|--------|
-| P0 | Add HF Spaces frontmatter to README | 5 min | High — required for deployment |
-| P0 | Add `.openenvignore` | 5 min | Medium — cleaner push |
-| P1 | Add TRL/GRPO example to README | 30 min | High — judges love this |
-| P1 | Expand `ISSUE_TYPE_SIMILARITY` pairs | 20 min | Medium — better reward signal |
-| P1 | Richer `history` entries (add title + predicted) | 20 min | Medium — better agent context |
-| P2 | Expand dataset to ~100 tickets | 2 hrs | Medium — more robust benchmark |
-| P2 | Add `queue_size` kwarg to `reset()` | 15 min | Low — flexibility |
-| P3 | Add `uv.lock` | 5 min | Low — polish |
-| P3 | Transform-based reward refactor | 1 hr | Low — architecture only |

required.md

@@ -0,0 +1,352 @@
+# Round 1 Requirements And Project Compliance Plan
+
+## Official Problem Statement
+
+Round 1 requires building a complete, real-world OpenEnv environment that an AI agent can learn from through the standard `step()` / `reset()` / `state()` API.
+
+### Key requirements at a glance
+
+- must simulate a real-world task, not a game or toy
+- must implement the full OpenEnv spec with typed models and `openenv.yaml`
+- must include at least 3 tasks with agent graders spanning easy -> medium -> hard
+- graders must return scores in `[0.0, 1.0]`
+- reward must provide meaningful partial-progress signal
+- must include a reproducible baseline `inference.py`
+- must deploy to Hugging Face Spaces with a working Dockerfile
+- README must include environment description, action / observation spaces, setup, usage, and baseline scores
+
+## Official Functional Requirements
+
+### Real-world task simulation
+
+The environment must simulate a task humans actually do. The official examples include:
+
+- email triage
+- code review
+- data cleaning
+- scheduling
+- customer support
+- content moderation
+
+### OpenEnv spec compliance
+
+The environment must implement the OpenEnv interface with:
+
+- typed Observation model
+- typed Action model
+- typed state model
+- `step(action)`
+- `reset()`
+- `state()`
+- `openenv.yaml`
+
+This is expected to be checked through `openenv validate`.
+
+### Minimum 3 tasks with agent graders
+
+Each task must have:
+
+- a concrete objective
+- a programmatic grader
+- score output in `[0.0, 1.0]`
+- deterministic success / failure criteria
+- clear difficulty progression from easy to hard
+
+### Meaningful reward function
+
+The reward should:
+
+- provide signal across the full trajectory
+- reward partial progress
+- penalize clearly undesirable behavior
+
+### Baseline inference script
+
+The baseline must:
+
+- use the OpenAI client for LLM calls
+- live at the project root as `inference.py`
+- produce reproducible scores
+- complete successfully across all 3 tasks
+
+## Official Non-Functional Requirements
+
+### Hugging Face Spaces
+
+- must deploy as a containerized HF Space
+- should be tagged with `openenv`
+- should respond successfully when pinged
+
+### Containerized execution
+
+- must include a working Dockerfile
+- should start cleanly with `docker build` + `docker run`
+
+### Documentation
+
+README must include:
+
+- environment description and motivation
+- action space definition
+- observation space definition
+- task descriptions with difficulty expectations
+- setup and usage instructions
+- baseline scores
+
+## Official Evaluation Criteria
+
+### Weights
+
+| Parameter | Weight | What judges look for |
+|-----------|--------|----------------------|
+| Real-world utility | 30% | Genuine practical task and value |
+| Task & grader quality | 25% | Clear objectives, fair graders, real progression |
+| Environment design | 20% | Clean state, sensible API, good reward shaping |
+| Code quality & spec compliance | 15% | OpenEnv compliance, structure, typing, tests, Docker |
+| Creativity & novelty | 10% | Original domain, mechanics, reward ideas |
+
+### Phase 1: Automated validation
+
+Pass / fail gate:
+
+- HF Space deploys
+- OpenEnv spec compliance
+- Dockerfile builds
+- baseline reproduces
+- 3+ tasks with graders
+
+### Phase 2: Agentic evaluation
+
+Scored:
+
+- baseline agent rerun
+- standard Open LLM agent run against the environment
+- score variance check
+
+### Phase 3: Human review
+
+Top submissions are reviewed by Meta and Hugging Face engineers for:
+
+- real-world utility
+- creativity
+- exploit resistance
+
+## Official Disqualification Criteria
+
+- environment does not deploy or respond
+- plagiarized or trivially modified existing environment
+- graders always return the same score
+- no baseline inference script
+
+## Official Pre-Submission Checklist
+
+All of these must pass:
+
+- HF Space deploys and responds
+- automated ping to the Space URL returns `200`
+- reset path works on the deployed environment
+- `openenv validate` passes
+- Dockerfile builds
+- baseline inference completes and produces scores
+- 3+ tasks with graders are present and score in `[0.0, 1.0]`
+
+## Mandatory Additional Instructions
+
+### Required inference environment variables
+
+- `API_BASE_URL`
+- `MODEL_NAME`
+- `HF_TOKEN`
+
+The official text also mentions `OPENAI_API_KEY` in one place, but the more specific submission instructions above consistently emphasize `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`. We should follow the later, more specific instruction while continuing to use the OpenAI client.
+
+### Inference script constraints
+
+- script must be named `inference.py`
+- it must live in the project root
+- all LLM calls must use the OpenAI client
+- stdout logs must strictly follow the `[START]`, `[STEP]`, and `[END]` format from the official sample
+
+### Infra restrictions
+
+- inference runtime should stay under 20 minutes
+- env and inference should run on a machine with `vcpu=2` and `memory=8gb`
+
+### Validator
+
+- run the official pre-submission validation script before final submission if possible
+
+---
+
+## Project Compliance Plan
+
+## Project Goal
+
+Build a polished OpenEnv environment for IT helpdesk ticket routing that satisfies:
+
+- real-world utility
+- strong task and grader quality
+- clean environment design
+- OpenEnv spec compliance
+- reproducible baseline inference
+- Docker and Hugging Face deployment readiness
+
+## Current Product Definition
+
+The environment simulates a helpdesk queue. An agent receives one ticket at a time and predicts:
+
+- `issue_type`
+- `priority`
+- `assignment_group`
+- `resolution_action`
+
+The project keeps three tasks:
+
+1. Issue Type Classification
+2. Issue Type And Priority
+3. Full Ticket Routing
+
+## What Must Be True At Submission
+
+### Pass / fail requirements
+
+- the environment responds correctly
+- OpenEnv metadata is valid
+- `reset()`, `step()`, and `state()` work
+- there are at least 3 tasks
+- graders return scores in `[0.0, 1.0]`
+- `inference.py` runs and prints reproducible results
+- `inference.py` uses the OpenAI client and required env vars
+- structured stdout logging matches the official format
+- `openenv validate` passes
+- Docker builds and starts cleanly
+- HF Space responds and reset works
+
+### Scored requirements
+
+- the task clearly feels like real helpdesk work
+- the hard task requires meaningful reasoning
+- partial credit is useful and deterministic
+- docs are clear enough for judges to understand quickly
+- reward is informative over the trajectory, not only at the end
+
+## Core Files
+
+### Runtime
+
+- `models.py`
+- `server/environment.py`
+- `server/grader.py`
+- `server/reward.py`
+- `server/tasks.py`
+- `server/app.py`
+- `client.py`
+- `inference.py`
+
+### Data and metadata
+
+- `data/dataset.json`
+- `openenv.yaml`
+- `server/Dockerfile`
+- `pyproject.toml`
+- `requirements.txt`
+
+### Docs
+
+- `README.md`
+- `KNOWLEDGE.md`
+- `required.md`
+
+## Technical Priorities
+
+### P0
+
+1. keep environment behavior correct
+2. verify task definitions and graders
+3. make the baseline script reliable and compliant with official logging format
+4. confirm dataset coverage and label consistency
+5. validate the official submission gates, not just local behavior
+
+### P1
+
+1. validate Docker
+2. validate deployment assumptions
+3. record baseline scores
+4. polish docs
+5. verify the runtime envelope and structured inference logs
+
+### P2
+
+1. strengthen ticket wording for realism
+2. expand hard-case examples if needed
+3. remove low-signal artifacts from the repo
+
+## Quality Checks To Perform
+
+### Environment
+
+- reset starts a clean episode
+- each step advances the queue correctly
+- the final step returns trajectory reward
+- state reflects the real internal status
+- episode boundaries are sensible
+
+### Grader
+
+- exact matches score `1.0`
+- near misses get partial credit where intended
+- unsupported task IDs fail clearly
+- scores vary across examples
+- graders do not collapse to constant scores
+
+### Inference
+
+- heuristic mode works without model credentials
+- LLM mode reads `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`
+- uses the OpenAI client
+- stdout follows `[START]`, `[STEP]`, and `[END]`
+- output is reproducible when the seed is fixed
+- runtime stays below the official time budget
+
+### Deployment and validation
+
+- `openenv validate` passes
+- Docker build succeeds
+- Docker run succeeds
+- HF ping / reset behavior works
+- official validator script is run if practical
+
+### Docs
+
+- no outdated domain references remain
+- team and project metadata are correct
+- setup and run instructions are accurate
+- README reflects the current inference and deployment path
+
+## Risks
+
+### Runtime risk
+
+The first local execution pass and merged-state rerun have already succeeded. The remaining runtime risk is Docker, clean-machine behavior, and official-validator-style behavior, not first-pass local execution.
+
+### Benchmark risk
+
+The current local benchmark is already recorded. Remaining benchmark risk is whether deployment / validation changes expose a mismatch late.
+
+### Deployment risk
+
+Docker, HF Spaces, `openenv validate`, and structured inference logging should be verified before the final submission window closes.
+
+## Definition Of Done
+
+The project is ready when:
+
+1. the environment runs locally end to end
+2. unit, smoke, and integration tests cover the critical paths
+3. the heuristic baseline runs successfully
+4. the inference script is compliant with the official logging format
+5. `openenv validate` passes
+6. Docker build and run both succeed
+7. HF deployment checks succeed or are as close to verified as possible before submission
+8. the docs are clean, current, and submission-ready
+9. the repo clearly presents Hackstreet Boys as the team