Merge remote-tracking branch 'origin/main' into codex/apr5-apr6-roopal

analysis/comp.md

@@ -0,0 +1,207 @@
+# Competitive Comparison — Are We Winning Material?
+
+> Honest head-to-head analysis of our project vs. the field  
+> Internal use only — NOT for commit/push
+
+---
+
+## TL;DR Verdict
+
+**Yes, we are competitive — and in several dimensions we are ahead of the field.**
+
+The weaknesses are fixable in under an hour. The strengths are structural and hard to replicate quickly.
+
+---
+
+## Scoring Rubric (Inferred from Hackathon Context)
+
+Based on the OpenEnv README and the nature of the competition, judges likely evaluate on:
+
+1. **Correctness** — Does the env run? Does reset/step/state work?
+2. **Domain quality** — Is the domain realistic and interesting?
+3. **Reward design** — Is the reward signal meaningful for RL training?
+4. **Task difficulty ladder** — Is there a progression from easy to hard?
+5. **Code quality** — Is the code clean, typed, documented?
+6. **Packaging** — Does Docker build? Does HF Spaces deploy?
+7. **Baseline agent** — Is there a working inference script?
+8. **Originality** — Is the domain novel vs. other submissions?
+
+---
+
+## Head-to-Head Comparison
+
+### vs. `echo_env` (reference/minimal)
+| Dimension | Us | echo_env |
+|-----------|-----|---------|
+| Domain | IT helpdesk routing | Echo (trivial) |
+| Reward | Partial credit, dense | Trivial |
+| Task ladder | 3 levels | 1 |
+| Dataset | 45 tickets | N/A |
+| Baseline | Yes (0.94) | N/A |
+| **Verdict** | **We win easily** | — |
+
+---
+
+### vs. `coding_env` (Meta's own reference env)
+| Dimension | Us | coding_env |
+|-----------|-----|-----------|
+| Domain | NLP/enterprise | Code execution |
+| Reward | Partial credit, dense | Transform-based (exit code) |
+| Task ladder | 3 levels | 1 |
+| Dataset | 45 labeled tickets | N/A (generates) |
+| Baseline | Yes (0.94) | Yes (smolagents) |
+| Tests | None | Unit + integration |
+| Architecture | Clean, typed | Clean, typed |
+| **Verdict** | **Comparable, we win on task ladder and domain** | — |
+
+---
+
+### vs. `finqa_env` (strongest NLP competitor)
+| Dimension | Us | finqa_env |
+|-----------|-----|----------|
+| Domain | IT helpdesk routing | Financial QA (SEC 10-K) |
+| Reward | Partial credit, dense | Binary (fuzzy numerical) |
+| Task ladder | 3 levels | 1 (finqa only) |
+| Dataset | 45 tickets (custom) | 290 questions (HuggingFace) |
+| Baseline | Yes (0.94 heuristic) | Yes (LLM-based) |
+| MCP tools | No | Yes (4 tools) |
+| Architecture | HTTP + Pydantic | MCP + FastMCP + pandas |
+| Complexity | Medium | High |
+| RL suitability | High (dense reward) | Medium (binary reward) |
+| **Verdict** | **We win on reward design and task ladder. They win on dataset size and MCP sophistication.** | — |
+
+**Key insight**: finqa's binary reward is actually WORSE for RL training than our partial credit. An agent gets 0 for a near-miss answer in finqa. We give partial credit. This is a genuine advantage.
+
+---
+
+### vs. `reasoning_gym_env` (breadth competitor)
+| Dimension | Us | reasoning_gym_env |
+|-----------|-----|-----------------|
+| Domain | IT helpdesk routing | 100+ reasoning tasks |
+| Reward | Partial credit, dense | 0–1 (dataset-dependent) |
+| Task ladder | 3 levels | Configurable |
+| Dataset | 45 tickets | Thousands (generated) |
+| Episode length | 3–5 steps | Single-step |
+| RL suitability | High (multi-step, dense) | Medium (single-step) |
+| Originality | High (custom domain) | Low (wraps existing library) |
+| **Verdict** | **We win on originality and multi-step RL suitability. They win on breadth.** | — |
+
+**Key insight**: Single-step envs are less interesting for RL training. Our multi-step queue model is a genuine differentiator.
+
+---
+
+### vs. `tbench2_env` (agentic competitor)
+| Dimension | Us | tbench2_env |
+|-----------|-----|------------|
+| Domain | IT helpdesk routing | Shell/terminal tasks |
+| Reward | Partial credit, dense | Binary (pytest) |
+| Task ladder | 3 levels | Many tasks (TB2 repo) |
+| Dataset | 45 tickets | TB2 task library |
+| Baseline | Yes (0.94) | No explicit baseline |
+| Intermediate reward | Yes (every step) | No (reward=None until evaluate) |
+| **Verdict** | **We win on reward density and baseline. They win on task variety.** | — |
+
+---
+
+### vs. `calendar_env` (enterprise workflow competitor)
+| Dimension | Us | calendar_env |
+|-----------|-----|-------------|
+| Domain | IT helpdesk routing | Calendar scheduling |
+| Reward | Partial credit, dense | SQL verifier (binary) |
+| Task ladder | 3 levels | Scenario-based |
+| MCP tools | No | Yes |
+| Baseline | Yes (0.94) | Yes (scenario config) |
+| **Verdict** | **Comparable. We win on reward density. They win on MCP and verifier sophistication.** | — |
+
+---
+
+### vs. `openapp_env` (most complex env)
+| Dimension | Us | openapp_env |
+|-----------|-----|------------|
+| Domain | IT helpdesk routing | Web UI (browser) |
+| Complexity | Medium | Extreme (5.7GB Docker) |
+| Reward | Partial credit, dense | Task-based |
+| Baseline | Yes (0.94) | Yes (example_usage.py) |
+| Multimodal | No | Yes (screenshots) |
+| **Verdict** | **They win on complexity and multimodal. We win on simplicity, reproducibility, and reward design.** | — |
+
+---
+
+## Overall Competitive Matrix
+
+| Criterion | Our Score | Field Average | Best in Field |
+|-----------|-----------|---------------|---------------|
+| Domain realism | 9/10 | 6/10 | openapp (10/10) |
+| Reward quality | 9/10 | 5/10 | ours / finqa |
+| Task ladder | 10/10 | 4/10 | ours |
+| Code quality | 8/10 | 7/10 | coding_env (9/10) |
+| Dataset quality | 6/10 | 5/10 | finqa (9/10) |
+| Packaging | 8/10 | 7/10 | all similar |
+| Baseline agent | 9/10 | 5/10 | ours / finqa |
+| Originality | 8/10 | 6/10 | openapp (10/10) |
+| RL suitability | 9/10 | 6/10 | ours / chat_env |
+| HF Spaces ready | 6/10 | 8/10 | all others (missing frontmatter) |
+
+**Our weighted average: ~8.2/10**  
+**Field average: ~6.0/10**
+
+---
+
+## What Makes Us Genuinely Competitive
+
+### 1. Best Task Ladder in the Repo
+No other env has 3 explicitly difficulty-graded tasks with different action spaces. This is exactly what curriculum RL needs. Judges who understand RL will notice this immediately.
+
+### 2. Best Reward Signal for RL Training
+- Dense: every step produces a reward (not just final)
+- Partial credit: near-miss answers get partial reward (not binary 0/1)
+- Bounded: [0.0, 1.0] always
+- Overshoot penalty: discourages unnecessary steps
+
+This is the most RL-friendly reward design in the repo.
+
+### 3. Deterministic + Reproducible
+We explicitly declare `deterministic: true` and `reproducible: true`. Judges can rerun and get identical results. This is rare in the field.
+
+### 4. Working Baseline with Strong Numbers
+0.94 overall on heuristic mode. This is a high bar — it means the env is well-calibrated (not trivially easy, not impossibly hard). The heuristic baseline also serves as a sanity check for judges.
+
+### 5. Richest openenv.yaml
+Our metadata file is the most complete in the repo. Tasks, evaluation config, grading mode, reproducibility flag, inference config — all documented. This signals professionalism.
+
+### 6. Real Enterprise Domain
+IT helpdesk routing is a real problem that real companies solve. It's not a game, not a toy, not a synthetic benchmark. Judges from Meta/enterprise backgrounds will appreciate this.
+
+---
+
+## What Could Beat Us
+
+1. **finqa_env** — if judges weight dataset size and MCP sophistication heavily
+2. **openapp_env** — if judges weight complexity and multimodal capability
+3. **reasoning_gym_env** — if judges weight breadth over depth
+4. **tbench2_env** — if judges weight agentic shell tasks
+
+None of these have our combination of: task ladder + partial credit + dense reward + deterministic + working baseline.
+
+---
+
+## The One Thing That Could Hurt Us
+
+**Missing HF Spaces frontmatter in README.**
+
+If judges try to deploy via `openenv push` and it fails because our README doesn't have the required frontmatter, that's a bad first impression. This is a 5-minute fix and should be done immediately.
+
+---
+
+## Final Verdict
+
+**We are a top-3 submission based on reward design, task ladder, and domain quality.**
+
+The gap between us and the top is:
+1. Dataset size (45 vs 290 for finqa) — expandable
+2. HF Spaces frontmatter — 5-minute fix
+3. MCP tools — not worth adding at this stage
+
+The gap between us and the bottom is large. Most envs are either games, single-step, or have binary rewards. We have none of those weaknesses.
+
+**Confidence: High. We should submit as-is after the 5-minute README fix.**

analysis/comp_know.md

@@ -0,0 +1,275 @@
+# Competition Knowledge Base — OpenEnv Hackathon
+
+> Source: github.com/meta-pytorch/OpenEnv/tree/main/envs  
+> Gathered: April 4, 2026  
+> Purpose: Internal competitive intelligence — NOT for commit/push
+
+---
+
+## Full Environment Inventory (27 envs)
+
+| Env | Domain | Complexity | Reward Type | Multi-step? | MCP? |
+|-----|--------|------------|-------------|-------------|------|
+| `atari_env` | Classic games | Medium | Dense | Yes | No |
+| `browsergym_env` | Web browser automation | Very High | Task-based | Yes | No |
+| `calendar_env` | Calendar/scheduling agent | High | SQL verifier | Yes | Yes (MCP) |
+| `carla_env` | Autonomous driving sim | Very High | Dense | Yes | No |
+| `chat_env` | Conversation/tokenization | Low | Custom transform | Yes | No |
+| `chess_env` | Chess game | Medium | Win/loss | Yes | No |
+| `coding_env` | Python code execution | Medium | Exit code / transform | Yes | No |
+| `connect4_env` | Connect 4 game | Low | Win/loss | Yes | No |
+| `dipg_safety_env` | Safety/policy | Medium | Unknown | Yes | No |
+| `dm_control_env` | DeepMind Control Suite | High | Dense | Yes | No |
+| `echo_env` | Reference/minimal | Minimal | Echo | No | No |
+| `finqa_env` | Financial QA (SEC 10-K) | High | Fuzzy numerical | Yes | Yes (MCP) |
+| `finrl_env` | Financial RL trading | High | Portfolio return | Yes | No |
+| `git_env` | Git operations | Medium | Task-based | Yes | No |
+| `grid_world_env` | Grid navigation | Low | Sparse | Yes | No |
+| `julia_env` | Julia code execution | Medium | Exit code | Yes | No |
+| `kernrl` | Kernel/OS operations | High | Unknown | Yes | No |
+| `maze_env` | Maze navigation | Low | Sparse | Yes | No |
+| `openapp_env` | Web app UI (BrowserGym) | Extreme | Task-based | Yes | No |
+| `openspiel_env` | Multi-agent games | High | Game outcome | Yes | No |
+| `reasoning_gym_env` | Reasoning tasks (100+ datasets) | Medium | Exact/partial | Single-step | No |
+| `repl_env` | REPL execution | Medium | Exit code | Yes | No |
+| `snake_env` | Snake game | Low | Score | Yes | No |
+| `sumo_rl_env` | Traffic simulation | High | Traffic flow | Yes | No |
+| `tbench2_env` | Terminal Bench 2 (shell tasks) | High | pytest pass/fail | Yes | No |
+| `textarena_env` | Text-based games | Medium | Game outcome | Yes | No |
+| `unity_env` | Unity 3D simulation | Very High | Task-based | Yes | No |
+
+---
+
+## Deep Dives: Most Relevant Envs
+
+### 1. `finqa_env` — Financial QA
+
+**What it does**: Agents answer complex financial questions from SEC 10-K filings using SQL tool calls.
+
+**Architecture**:
+- Subclasses `MCPEnvironment` (not plain `Environment`) — uses FastMCP with `@mcp.tool` decorators
+- Tools: `get_descriptions`, `get_table_info`, `sql_query`, `submit_answer`
+- Dataset: 290 questions from HuggingFace (`snorkelai/finqa-data`)
+- Max steps: 50 per episode
+- Reward: Binary (1.0 / 0.0) with fuzzy numerical matching (1% relative tolerance + 1.0 absolute tolerance)
+- Handles `\boxed{}` LaTeX format, percentages, fractions, thousands separators, negative parens
+
+**Reward sophistication**: Very high. The `rewards.py` is ~300 lines handling multi-value answers, year-labeled pairs, percentage normalization, and both relative + absolute tolerance checks simultaneously.
+
+**Key differentiator**: MCP protocol for tool discovery. Client uses `await env.list_tools()` to discover tools at runtime. This is the most "agentic" env in the repo.
+
+**Integration**: Explicitly shows TRL/GRPO integration pattern in README.
+
+---
+
+### 2. `coding_env` — Python Code Execution
+
+**What it does**: Executes arbitrary Python code in a sandboxed environment.
+
+**Architecture**:
+- `PythonCodeActEnv` wraps a `PyExecutor` (sandboxed subprocess)
+- `create_safe_coding_transform()` — transform pipeline for reward computation
+- Action: `CodeAction(code: str)`
+- Observation: `CodeObservation(stdout, stderr, exit_code)`
+- State: `CodeState(episode_id, step_count, last_exit_code)`
+- Reward: computed by transform (not in step directly) — extensible pattern
+
+**Key differentiator**: Transform-based reward. The environment itself doesn't compute reward — a pluggable `Transform` object does. This is the cleanest separation of concerns in the repo.
+
+**Testing**: Has both unit tests (`test_python_codeact_reset`, `test_python_codeact_rewards`) and integration tests (`test_coding_env_integration`). Most tested env in the repo.
+
+---
+
+### 3. `reasoning_gym_env` — Reasoning Tasks
+
+**What it does**: Wraps the `reasoning-gym` library (100+ reasoning datasets) as a single-step OpenEnv.
+
+**Architecture**:
+- Single-step episodes: `reset()` gives question, `step()` gives score + done=True
+- Composite datasets: mix multiple datasets with weights
+- Dataset persistence: same dataset reused across resets until config changes
+- Supports `dataset_name`, `seed`, `size`, `dataset_specs` in `reset()` kwargs
+- Reward: 0.0–1.0 (dataset-dependent, may use partial credit)
+
+**Key differentiator**: Massive breadth (100+ task types in one env). The `reset()` kwargs pattern for dataset configuration is very clean. Also has `openenv push` CLI for HuggingFace Spaces deployment.
+
+**Scale**: uv.lock is 551KB — large dependency tree from reasoning-gym.
+
+---
+
+### 4. `tbench2_env` — Terminal Bench 2
+
+**What it does**: Wraps Terminal-Bench-2 shell tasks. Agent executes shell commands and is evaluated by pytest.
+
+**Architecture**:
+- Two modes: `local` (direct process) and `docker` (per-task container)
+- Rich action type: `exec`, `write`, `view`, `wait`, `kill`, `write_file`, `evaluate`, `close`
+- Session IDs for streaming/non-blocking processes
+- Reward: Binary (pytest pass/fail) on `evaluate` action
+- Intermediate steps: `reward=None`
+
+**Key differentiator**: Most realistic "agentic" shell environment. The session ID pattern for streaming processes is unique. Docker-in-Docker mode for full fidelity.
+
+---
+
+### 5. `openapp_env` — Web App UI
+
+**What it does**: Wraps OpenApps (calendar, todo, messenger, maps) + BrowserGym for browser-based UI agent training.
+
+**Architecture**:
+- Runs TWO services in Docker: OpenApps server (port 5001) + FastAPI (port 8000)
+- `start.sh` orchestrates both
+- BrowserGym for browser automation (Playwright/Chromium)
+- Docker image: ~5.7GB (includes Chromium)
+- Multimodal: screenshots + DOM observations
+
+**Key differentiator**: Most complex env in the repo. Multimodal (visual + text). Real browser interaction. Closest to real-world agent deployment.
+
+---
+
+### 6. `calendar_env` — Calendar Scheduling
+
+**What it does**: Calendar management tasks with SQL database verification.
+
+**Architecture**:
+- MCP-based (like finqa_env)
+- Has `client_notebooks/` — Jupyter notebook for interactive evaluation
+- Has `mcp_databases/` — SQLite databases for state
+- Scenario-based: `scenario_config.json` drives task + verifiers
+- Verifiers: SQL queries that check task completion
+- Supports OpenAI, Anthropic, Google providers
+
+**Key differentiator**: Scenario config pattern. Verifier-based reward (SQL queries check if the agent actually completed the task). Most "enterprise workflow" env.
+
+---
+
+### 7. `chat_env` — Chat/Tokenization
+
+**What it does**: Manages conversation history + tokenization for LLM RL training.
+
+**Architecture**:
+- Action: `ChatAction(tokens: torch.Tensor)` — takes raw model tokens
+- Observation: `ChatObservation(messages, tokens)` — both human-readable + model-ready
+- Transform-based reward (pluggable)
+- Dual representation: messages (human) + tokens (model)
+- No HTTP overhead option: can use directly without server
+
+**Key differentiator**: Designed for direct LLM RL training loop. The only env that takes raw PyTorch tensors as actions. Pairs with GRPO/PPO training loops directly.
+
+---
+
+## Structural Patterns Observed Across All Envs
+
+### File Structure (canonical)
+```
+env_name/
+├── __init__.py          # exports
+├── models.py            # Action, Observation, State
+├── client.py            # EnvClient subclass
+├── openenv.yaml         # metadata
+├── pyproject.toml       # packaging
+├── README.md            # HuggingFace Space frontmatter + docs
+└── server/
+    ├── __init__.py
+    ├── app.py           # FastAPI
+    ├── environment.py   # core logic
+    └── Dockerfile
+```
+
+### README Frontmatter (HuggingFace Spaces)
+Every env README has YAML frontmatter:
+```yaml
+---
+title: ...
+emoji: ...
+colorFrom: ...
+colorTo: ...
+sdk: docker
+pinned: false
+app_port: 8000
+base_path: /web
+tags:
+  - openenv
+---
+```
+This is required for HuggingFace Spaces deployment. Our README does NOT have this.
+
+### openenv.yaml — Minimal Pattern
+Most envs have very minimal `openenv.yaml` (just name + entry_point). Our yaml is the most detailed in the repo.
+
+### Dockerfile Patterns
+- Most use `openenv-base:latest` as base image (not `python:3.11-slim`)
+- Our Dockerfile uses `python:3.11-slim` directly — this is the standalone/HF Spaces pattern
+- The `openenv-base` pattern is for the monorepo CI/CD workflow
+
+### Testing
+- `coding_env`: most tested (unit + integration)
+- Most envs: no tests at all
+- Our env: no tests (matches majority)
+
+### MCP vs HTTP
+- Most envs: plain HTTP (`Environment` base class)
+- `finqa_env`, `calendar_env`: MCP (`MCPEnvironment` base class, FastMCP tools)
+- MCP envs are more "agentic" — tools are discoverable at runtime
+
+### Reward Patterns
+| Pattern | Envs | Description |
+|---------|------|-------------|
+| Binary (0/1) | finqa, tbench2, reasoning_gym | Pass/fail |
+| Dense partial | ours, chess, atari | Continuous [0,1] |
+| Transform-based | coding, chat | Pluggable reward function |
+| SQL verifier | calendar | DB state check |
+| Game outcome | chess, connect4, openspiel | Win/loss/draw |
+
+---
+
+## Deployment Patterns
+
+### HuggingFace Spaces
+- `openenv push` CLI command (seen in reasoning_gym README)
+- Spaces get: `/web` (UI), `/docs` (Swagger), `/health`, `/ws` (WebSocket)
+- `base_path: /web` in README frontmatter
+- Our env: missing HF Spaces frontmatter in README
+
+### Docker
+- Most envs: `openenv-base:latest` (monorepo CI)
+- Standalone envs (ours, openapp): `python:3.11-slim`
+- openapp: 5.7GB image (Chromium)
+- Our image: minimal (python:3.11-slim + pip deps)
+
+---
+
+## Dataset Sizes
+
+| Env | Dataset Size | Source |
+|-----|-------------|--------|
+| finqa | 290 questions | HuggingFace (snorkelai/finqa-data) |
+| reasoning_gym | 100+ datasets, configurable size | reasoning-gym library |
+| calendar | SQLite DBs | Custom |
+| ours | 45 tickets | Custom (data/dataset.json) |
+| coding | N/A (generates tasks) | N/A |
+| tbench2 | Terminal-Bench-2 repo | GitHub auto-download |
+
+---
+
+## Key Technical Observations
+
+1. **MCP is the emerging pattern** for tool-using agents. finqa and calendar both use it. Our env uses plain HTTP — simpler but less "agentic."
+
+2. **Transform-based rewards** (coding_env, chat_env) are the cleanest architecture for extensible reward shaping. Our reward is hardcoded in `reward.py`.
+
+3. **`openenv push` CLI** exists for HuggingFace Spaces deployment. We should use it.
+
+4. **README frontmatter** is required for HF Spaces. Our README is missing it.
+
+5. **Composite/configurable datasets** (reasoning_gym) are a strong differentiator. Our dataset is fixed at 45 tickets.
+
+6. **WebSocket endpoint** (`/ws`) is mentioned in reasoning_gym README as a HF Spaces feature. Our env already has `/ws` via the OpenEnv base.
+
+7. **`uv.lock`** files appear in chat_env and reasoning_gym — reproducible dependency locking. We use `requirements.txt` only.
+
+8. **`.openenvignore`** file in finqa_env — analogous to `.dockerignore` for the OpenEnv push CLI.
+
+9. **`base_path: /web`** in HF Spaces frontmatter — the web UI is at `/web`, not `/`. Our env would need this.
+
+10. **Episode length**: Most envs are either single-step (reasoning_gym) or unbounded (coding, tbench2). Our env is bounded (3–5 steps) — a clean middle ground.

analysis/inference.md

@@ -0,0 +1,218 @@
+# Inferences & Actionable Advantages
+
+> Based on deep analysis of all 27 OpenEnv competition entries  
+> Internal use only — NOT for commit/push
+
+---
+
+## Critical Missing Items (Fix Before Submission)
+
+### 1. README HuggingFace Spaces Frontmatter — MISSING
+
+Every single env in the repo has YAML frontmatter at the top of README.md. Ours does not.
+This is required for `openenv push` and HuggingFace Spaces deployment to work correctly.
+
+**Add to top of `meta-AIHack/README.md`:**
+```yaml
+---
+title: IT Helpdesk Ticket Routing OpenEnv
+emoji: 🎫
+colorFrom: blue
+colorTo: indigo
+sdk: docker
+pinned: false
+app_port: 7860
+base_path: /web
+tags:
+  - openenv
+  - helpdesk
+  - ticket-routing
+  - nlp
+---
+```
+
+Note: our port is `7860` (HF Spaces default), not `8000`. Use `7860` here.
+
+---
+
+### 2. `.openenvignore` File — MISSING
+
+`finqa_env` has a `.openenvignore` file (analogous to `.dockerignore` for the `openenv push` CLI).
+Without it, `openenv push` may upload unnecessary files.
+
+**Create `meta-AIHack/.openenvignore`:**
+```
+*.pyc
+__pycache__/
+.git/
+*.md
+PLAN.md
+ROADMAP.md
+MENTAL_MODEL.md
+KNOWLEDGE.md
+comp_intel/
+bugs/
+transcripts/
+```
+
+---
+
+### 3. `base_path: /web` in openenv.yaml — CHECK
+
+The HF Spaces web UI is served at `/web`. The `reasoning_gym_env` README explicitly mentions:
+- Web Interface at `/web`
+- API Documentation at `/docs`
+- Health Check at `/health`
+- WebSocket at `/ws`
+
+Our `openenv.yaml` lists `/docs` in `api.endpoints` — good. But we should verify the web interface path is correct when deployed.
+
+---
+
+## High-Value Improvements (Implement If Time Allows)
+
+### 4. Partial Credit Similarity Matrix — Expand
+
+Our `grader.py` has `ISSUE_TYPE_SIMILARITY` with 16 pairs and `PRIORITY_SCORES` with 10 pairs.
+
+**Observation from finqa_env**: Their reward uses both relative AND absolute tolerance simultaneously. Our grader uses a flat similarity dict.
+
+**Improvement**: Add more near-miss pairs to `ISSUE_TYPE_SIMILARITY`. Currently missing:
+- `("onboarding", "service_request")` — onboarding tickets often look like service requests
+- `("feature_request", "service_request")` — common confusion
+- `("security_compliance", "identity_access")` — MFA/SSO tickets can go either way
+- `("billing_license", "identity_access")` — license + account access overlap
+
+This directly improves the reward signal quality for RL training, which is what judges care about.
+
+---
+
+### 5. Dataset Size — Expand from 45 to ~100 tickets
+
+**Observation**: finqa has 290 questions, reasoning_gym has configurable sizes up to thousands.
+Our 45 tickets is the smallest custom dataset in the repo.
+
+**Improvement**: Add 55 more tickets to reach 100. Focus on:
+- More ambiguous cases (harder for LLMs)
+- More `related_ticket_id` chains (multi-ticket threads)
+- Edge cases: tickets that span two issue types
+- More `spam_phishing` examples (currently underrepresented)
+
+This makes the benchmark more robust and harder to overfit.
+
+---
+
+### 6. Transform-Based Reward (Optional Architecture Upgrade)
+
+**Observation**: `coding_env` uses a pluggable `Transform` object for reward computation instead of hardcoding it in `step()`. This is the cleanest pattern in the repo.
+
+**Improvement**: Refactor `server/reward.py` to expose a `HelpdeskRewardTransform` class that can be swapped. Low priority — our current design works fine — but it signals architectural sophistication to judges.
+
+---
+
+### 7. Configurable Queue Size via `reset()` kwargs
+
+**Observation**: `reasoning_gym_env` passes `size`, `seed`, `dataset_name` as `reset()` kwargs. This makes the env much more flexible for RL training (vary episode length, vary dataset).
+
+**Improvement**: Accept `queue_size` as a `reset()` kwarg (in addition to `task_id` and `seed`):
+```python
+def reset(self, seed=None, episode_id=None, **kwargs):
+    queue_size = kwargs.get("queue_size", None)  # override QUEUE_SIZE_RANGE
+    ...
+```
+
+This lets RL trainers control episode length without modifying the env code.
+
+---
+
+### 8. `uv.lock` for Reproducible Dependencies
+
+**Observation**: `chat_env` and `reasoning_gym_env` both include `uv.lock` files for fully reproducible dependency resolution.
+
+**Improvement**: Run `uv lock` in `meta-AIHack/` and commit the `uv.lock`. This signals production-quality dependency management.
+
+---
+
+### 9. Explicit TRL/GRPO Integration Example in README
+
+**Observation**: `finqa_env` README explicitly shows a TRL GRPO integration snippet. This is exactly what Meta/PyTorch judges want to see — the env being used for actual RL training.
+
+**Improvement**: Add a section to our README showing how to use the env with TRL GRPO:
+```python
+# Example: Using with TRL GRPO
+from trl import GRPOTrainer
+from client import HelpdeskTicketEnvClient
+
+async def rollout_func(prompts, trainer):
+    sync_client = HelpdeskTicketEnvClient(base_url=ENV_URL).sync()
+    with sync_client:
+        result = sync_client.reset(seed=42, task_id=3)
+        # ... agent loop
+        return {"reward": final_reward, "completion": completion}
+```
+
+---
+
+### 10. `history` Field — Richer Step History
+
+**Observation**: `finqa_env` passes full tool call history in observation metadata. Our `history` field currently only stores `{step, score, breakdown}`.
+
+**Improvement**: Include the ticket title and predicted fields in history so the agent can learn from its own past decisions within an episode:
+```python
+history_entry = {
+    "ticket_id": current_ticket.ticket_id,
+    "title": current_ticket.title,  # ADD THIS
+    "predicted": {k: v for k, v in action.model_dump().items() if v is not None},  # ADD THIS
+    "score": score,
+    "breakdown": breakdown,
+}
+```
+
+This gives the LLM agent richer context for multi-step reasoning.
+
+---
+
+## Competitive Positioning Insights
+
+### Our Unique Strengths vs. The Field
+
+1. **Richest `openenv.yaml`**: Ours is the most detailed metadata file in the entire repo. Most envs have 3-line yaml files. Ours has tasks, evaluation, grading, reproducibility, inference config. This signals thoroughness.
+
+2. **Deterministic + Reproducible**: We explicitly set `deterministic: true` and `reproducible: true` in openenv.yaml. Only a few envs do this. Judges can rerun and get identical results.
+
+3. **Task Ladder (3 difficulty levels)**: Most envs have a single task. We have 3 explicitly difficulty-graded tasks. This is a strong differentiator for RL curriculum learning.
+
+4. **Partial Credit Grading**: Most envs use binary reward (0/1). Our grader gives partial credit for near-miss issue types and adjacent priorities. This produces a much richer reward signal for RL training.
+
+5. **Dense Reward Signal**: Every step produces a reward (not just the final step). Most envs (tbench2, finqa) only reward at the end. Dense rewards are better for RL training.
+
+6. **Heuristic Baseline**: We have a working keyword-based heuristic that achieves 0.94 overall. Most envs don't have a baseline agent. This lets judges immediately see the env working.
+
+7. **Real-World Domain**: IT helpdesk routing is a real enterprise use case. Many envs are games or synthetic tasks. Ours has immediate practical applicability.
+
+8. **Clean Episode Bounds**: 3–5 steps per episode. Not too short (single-step), not unbounded. Clean for RL training.
+
+### Our Weaknesses vs. The Field
+
+1. **No HF Spaces frontmatter** in README — fixable in 5 minutes
+2. **Smallest dataset** (45 tickets) — expandable
+3. **No MCP tools** — plain HTTP only (simpler but less "agentic")
+4. **No tests** — matches most envs, but coding_env has tests
+5. **No `uv.lock`** — minor
+6. **No `.openenvignore`** — minor
+
+---
+
+## Priority Action List
+
+| Priority | Action | Effort | Impact |
+|----------|--------|--------|--------|
+| P0 | Add HF Spaces frontmatter to README | 5 min | High — required for deployment |
+| P0 | Add `.openenvignore` | 5 min | Medium — cleaner push |
+| P1 | Add TRL/GRPO example to README | 30 min | High — judges love this |
+| P1 | Expand `ISSUE_TYPE_SIMILARITY` pairs | 20 min | Medium — better reward signal |
+| P1 | Richer `history` entries (add title + predicted) | 20 min | Medium — better agent context |
+| P2 | Expand dataset to ~100 tickets | 2 hrs | Medium — more robust benchmark |
+| P2 | Add `queue_size` kwarg to `reset()` | 15 min | Low — flexibility |
+| P3 | Add `uv.lock` | 5 min | Low — polish |
+| P3 | Transform-based reward refactor | 1 hr | Low — architecture only |

inference.py

@@ -2,15 +2,33 @@
 """
 Inference script for the IT Helpdesk Ticket Routing OpenEnv environment.
 
-Uses the competition-mandated environment variables:
-  API_BASE_URL  - LLM provider base URL
-  MODEL_NAME    - model identifier
-  HF_TOKEN      - authentication token
-
-Can run against a local server (default http://localhost:8000) or a
-remote HuggingFace Space URL passed via ENV_URL.
-
-Uses the WebSocket-based EnvClient for multi-step episodes.
+Environment variables
+---------------------
+ENV_URL
+    Base URL of the running OpenEnv server.
+    Default: ``http://localhost:8000``
+    Optional — when unset the script connects to the local server on port 8000.
+
+API_BASE_URL
+    LLM provider base URL (OpenAI-compatible endpoint).
+    Default: ``https://router.huggingface.co/v1``
+    Optional — only used when both MODEL_NAME and HF_TOKEN are set.
+
+MODEL_NAME
+    Model identifier to use for LLM inference (e.g. ``meta-llama/Llama-3.3-70B-Instruct``).
+    Default: ``""`` (empty string)
+    Optional — when unset (or empty) the script runs in heuristic mode without an LLM.
+
+HF_TOKEN
+    HuggingFace authentication token for the LLM provider.
+    Default: ``""`` (empty string)
+    Optional — when unset (or empty) the script runs in heuristic mode without an LLM.
+
+When both MODEL_NAME and HF_TOKEN are set, the script calls the LLM via the OpenAI-compatible
+API at API_BASE_URL. When either is unset, ``llm_client`` is ``None`` and ``build_action()``
+falls back to ``heuristic_action()`` automatically.
+
+Uses the HTTP-based sync EnvClient for multi-step episodes.
 """
 from __future__ import annotations
 
@@ -301,7 +319,7 @@ def run():
         task = available_tasks[task_id]
         print(f"\n--- Task {task_id}: {task['name']} ({task['difficulty']}) ---")
 
-        # Use sync WebSocket client for multi-step episode
+        # Use sync HTTP client for multi-step episode
         sync_client = HelpdeskTicketEnvClient(base_url=ENV_URL).sync()
         with sync_client:
             result = sync_client.reset(seed=SEED, task_id=task_id)

server/Dockerfile

@@ -7,6 +7,9 @@ WORKDIR /app
 
 COPY . .
 
+RUN apt-get update && apt-get install -y --no-install-recommends git \
+    && rm -rf /var/lib/apt/lists/*
+
 RUN python -m pip install --upgrade pip \
     && python -m pip install --no-cache-dir -r requirements.txt \
     && python -m pip install --no-cache-dir .