v1 code review env

README.md

@@ -1,255 +1,152 @@
 ---
-title: Code Review Env Environment Server
-emoji: 🎬
-colorFrom: red
-colorTo: gray
+title: Code Review Environment
+emoji: 🧠
+colorFrom: blue
+colorTo: purple
 sdk: docker
 pinned: false
 app_port: 8000
 base_path: /web
 tags:
   - openenv
+  - reinforcement-learning
+  - code-review
 ---
 
-# Code Review Env Environment
+# Code Review Environment (`code-review-env`)
 
-A simple test environment that echoes back messages. Perfect for testing the env APIs as well as demonstrating environment usage patterns.
+This OpenEnv environment simulates a real software engineering task: reviewing buggy Python code and identifying security and logic issues with fixed taxonomy tags.
 
-## Quick Start
+## Why this environment
 
-The simplest way to use the Code Review Env environment is through the `CodeReviewEnv` class:
+- Real-world utility: PR/code review is a common and valuable engineering workflow.
+- RL-friendly: structured action space with dense rewards and deterministic grader.
+- Progressive tasks: easy → medium → hard.
 
-```python
-from code_review_env import CodeReviewAction, CodeReviewEnv
+## Action space
 
-try:
-    # Create environment from Docker image
-    code_review_envenv = CodeReviewEnv.from_docker_image("code_review_env-env:latest")
+`ReviewAction` (`models.py`):
 
-    # Reset
-    result = code_review_envenv.reset()
-    print(f"Reset: {result.observation.echoed_message}")
+- `review_comment` (`str`): human-readable explanation
+- `issues_found` (`list[str]`): tags from `ISSUE_TAXONOMY`
+- `severity` (`low|medium|high|critical`)
 
-    # Send multiple messages
-    messages = ["Hello, World!", "Testing echo", "Final message"]
+## Observation space
 
-    for msg in messages:
-        result = code_review_envenv.step(CodeReviewAction(message=msg))
-        print(f"Sent: '{msg}'")
-        print(f"  → Echoed: '{result.observation.echoed_message}'")
-        print(f"  → Length: {result.observation.message_length}")
-        print(f"  → Reward: {result.reward}")
+`ReviewObservation` (`models.py`):
 
-finally:
-    # Always clean up
-    code_review_envenv.close()
-```
-
-That's it! The `CodeReviewEnv.from_docker_image()` method handles:
-- Starting the Docker container
-- Waiting for the server to be ready
-- Connecting to the environment
-- Container cleanup when you call `close()`
-
-## Building the Docker Image
-
-Before using the environment, you need to build the Docker image:
-
-```bash
-# From project root
-docker build -t code_review_env-env:latest -f server/Dockerfile .
-```
+- `task_id`, `file_name`, `task_description`
+- `code_snippet` to review
+- `feedback` after each step
+- `step_number`, `reward`, `done`, `metadata`
 
-## Deploying to Hugging Face Spaces
+## Tasks
 
-You can easily deploy your OpenEnv environment to Hugging Face Spaces using the `openenv push` command:
+- `task_easy`: `null_pointer`, `missing_return`
+- `task_medium`: `sql_injection`, `hardcoded_secret`
+- `task_hard`: `race_condition`, `improper_error_handling`, `timing_attack`
 
-```bash
-# From the environment directory (where openenv.yaml is located)
-openenv push
-
-# Or specify options
-openenv push --namespace my-org --private
-```
+Defined in `server/tasks.py`.
 
-The `openenv push` command will:
-1. Validate that the directory is an OpenEnv environment (checks for `openenv.yaml`)
-2. Prepare a custom build for Hugging Face Docker space (enables web interface)
-3. Upload to Hugging Face (ensuring you're logged in)
+## Reward and grading
 
-### Prerequisites
+Implemented in `server/graders.py`:
 
-- Authenticate with Hugging Face: The command will prompt for login if not already authenticated
+- `base_score = |correctly_found| / |planted_issues|`
+- `quality_bonus = +0.05` per correctly found issue with comment keywords
+- `precision_penalty = -0.1` per false-positive issue
+- final score clamped to `[0.0, 1.0]`
 
-### Options
+## Required endpoints
 
-- `--directory`, `-d`: Directory containing the OpenEnv environment (defaults to current directory)
-- `--repo-id`, `-r`: Repository ID in format 'username/repo-name' (defaults to 'username/env-name' from openenv.yaml)
-- `--base-image`, `-b`: Base Docker image to use (overrides Dockerfile FROM)
-- `--private`: Deploy the space as private (default: public)
+- `GET /tasks`
+- `POST /grader`
+- `POST /baseline`
+- plus OpenEnv core endpoints (`/reset`, `/step`, `/state`, `/health`, `/ws`)
 
-### Examples
+## Local setup
 
 ```bash
-# Push to your personal namespace (defaults to username/env-name from openenv.yaml)
-openenv push
-
-# Push to a specific repository
-openenv push --repo-id my-org/my-env
-
-# Push with a custom base image
-openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest
-
-# Push as a private space
-openenv push --private
-
-# Combine options
-openenv push --repo-id my-org/my-env --base-image custom-base:latest --private
+cd /home/manvith/OpenEnv/code_review_env
+pip install -e .
+pip install -r server/requirements.txt
+uvicorn server.app:app --host 0.0.0.0 --port 8000
 ```
 
-After deployment, your space will be available at:
-`https://huggingface.co/spaces/<repo-id>`
-
-The deployed space includes:
-- **Web Interface** at `/web` - Interactive UI for exploring the environment
-- **API Documentation** at `/docs` - Full OpenAPI/Swagger interface
-- **Health Check** at `/health` - Container health monitoring
-- **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions
-
-## Environment Details
-
-### Action
-**CodeReviewAction**: Contains a single field
-- `message` (str) - The message to echo back
-
-### Observation
-**CodeReviewObservation**: Contains the echo response and metadata
-- `echoed_message` (str) - The message echoed back
-- `message_length` (int) - Length of the message
-- `reward` (float) - Reward based on message length (length × 0.1)
-- `done` (bool) - Always False for echo environment
-- `metadata` (dict) - Additional info like step count
-
-### Reward
-The reward is calculated as: `message_length × 0.1`
-- "Hi" → reward: 0.2
-- "Hello, World!" → reward: 1.3
-- Empty message → reward: 0.0
-
-## Advanced Usage
+## Manual API smoke test
 
-### Connecting to an Existing Server
-
-If you already have a Code Review Env environment server running, you can connect directly:
-
-```python
-from code_review_env import CodeReviewEnv
-
-# Connect to existing server
-code_review_envenv = CodeReviewEnv(base_url="<ENV_HTTP_URL_HERE>")
-
-# Use as normal
-result = code_review_envenv.reset()
-result = code_review_envenv.step(CodeReviewAction(message="Hello!"))
+```bash
+curl http://localhost:8000/health
+curl http://localhost:8000/tasks
+curl -X POST http://localhost:8000/baseline
+curl -X POST http://localhost:8000/grader \
+  -H "Content-Type: application/json" \
+  -d '{"task_id":"task_easy","issues_found":["null_pointer"],"review_comment":"missing null check"}'
 ```
 
-Note: When connecting to an existing server, `code_review_envenv.close()` will NOT stop the server.
+## Baseline inference script
 
-### Using the Context Manager
+`inference.py` supports two modes:
 
-The client supports context manager usage for automatic connection management:
+1. LLM mode (if `OPENAI_API_KEY` is set)
+2. Rule-based fallback (no key required)
 
-```python
-from code_review_env import CodeReviewAction, CodeReviewEnv
-
-# Connect with context manager (auto-connects and closes)
-with CodeReviewEnv(base_url="http://localhost:8000") as env:
-    result = env.reset()
-    print(f"Reset: {result.observation.echoed_message}")
-    # Multiple steps with low latency
-    for msg in ["Hello", "World", "!"]:
-        result = env.step(CodeReviewAction(message=msg))
-        print(f"Echoed: {result.observation.echoed_message}")
-```
-
-The client uses WebSocket connections for:
-- **Lower latency**: No HTTP connection overhead per request
-- **Persistent session**: Server maintains your environment state
-- **Efficient for episodes**: Better for many sequential steps
-
-### Concurrent WebSocket Sessions
+```bash
+# Local fallback mode
+python inference.py
 
-The server supports multiple concurrent WebSocket connections. To enable this,
-modify `server/app.py` to use factory mode:
+# Local LLM mode
+OPENAI_API_KEY=sk-... OPENAI_MODEL=gpt-4o-mini python inference.py
 
-```python
-# In server/app.py - use factory mode for concurrent sessions
-app = create_app(
-    CodeReviewEnvironment,  # Pass class, not instance
-    CodeReviewAction,
-    CodeReviewObservation,
-    max_concurrent_envs=4,  # Allow 4 concurrent sessions
-)
+# Against HF Space
+ENV_URL=https://<your-space>.hf.space python inference.py
 ```
 
-Then multiple clients can connect simultaneously:
+Example output shape:
 
-```python
-from code_review_env import CodeReviewAction, CodeReviewEnv
-from concurrent.futures import ThreadPoolExecutor
-
-def run_episode(client_id: int):
-    with CodeReviewEnv(base_url="http://localhost:8000") as env:
-        result = env.reset()
-        for i in range(10):
-            result = env.step(CodeReviewAction(message=f"Client {client_id}, step {i}"))
-        return client_id, result.observation.message_length
-
-# Run 4 episodes concurrently
-with ThreadPoolExecutor(max_workers=4) as executor:
-    results = list(executor.map(run_episode, range(4)))
+```json
+{
+  "task_easy": {"score": 0.85, "issues_found": ["null_pointer", "missing_return"]},
+  "task_medium": {"score": 0.7, "issues_found": ["sql_injection", "hardcoded_secret"]},
+  "task_hard": {"score": 0.5, "issues_found": ["race_condition", "improper_error_handling"]}
+}
 ```
 
-## Development & Testing
-
-### Direct Environment Testing
-
-Test the environment logic directly without starting the HTTP server:
+## Docker
 
 ```bash
-# From the server directory
-python3 server/code_review_env_environment.py
+docker build -t code-review-env:latest -f server/Dockerfile .
+docker run -p 8000:8000 code-review-env:latest
 ```
 
-This verifies that:
-- Environment resets correctly
-- Step executes actions properly
-- State tracking works
-- Rewards are calculated correctly
-
-### Running Locally
-
-Run the server locally for development:
+## Deploy to Hugging Face Space
 
 ```bash
-uvicorn server.app:app --reload
+openenv push --repo-id YOUR_USERNAME/code-review-env
 ```
 
-## Project Structure
+## Submission links (both required)
 
-```
+1. GitHub repository URL
+2. Hugging Face Space URL
+
+## Folder layout
+
+```text
 code_review_env/
-├── .dockerignore         # Docker build exclusions
-├── __init__.py            # Module exports
-├── README.md              # This file
-├── openenv.yaml           # OpenEnv manifest
-├── pyproject.toml         # Project metadata and dependencies
-├── uv.lock                # Locked dependencies (generated)
-├── client.py              # CodeReviewEnv client
-├── models.py              # Action and Observation models
+├── __init__.py
+├── client.py
+├── inference.py
+├── models.py
+├── openenv.yaml
+├── pyproject.toml
+├── README.md
 └── server/
-    ├── __init__.py        # Server module exports
-    ├── code_review_env_environment.py  # Core environment logic
-    ├── app.py             # FastAPI application (HTTP + WebSocket endpoints)
-    └── Dockerfile         # Container image definition
+    ├── __init__.py
+    ├── app.py
+    ├── code_review_env_environment.py
+    ├── graders.py
+    ├── tasks.py
+    ├── requirements.txt
+    └── Dockerfile
 ```

__init__.py

@@ -7,10 +7,23 @@
 """Code Review Env Environment."""
 
 from .client import CodeReviewEnv
-from .models import CodeReviewAction, CodeReviewObservation
+from .models import (
+    ISSUE_TAXONOMY,
+    CodeReviewAction,
+    CodeReviewObservation,
+    CodeReviewState,
+    ReviewAction,
+    ReviewObservation,
+    ReviewState,
+)
 
 __all__ = [
+    "ISSUE_TAXONOMY",
+    "ReviewAction",
+    "ReviewObservation",
+    "ReviewState",
     "CodeReviewAction",
     "CodeReviewObservation",
+    "CodeReviewState",
     "CodeReviewEnv",
 ]

client.py

@@ -4,96 +4,53 @@
 # This source code is licensed under the BSD-style license found in the
 # LICENSE file in the root directory of this source tree.
 
-"""Code Review Env Environment Client."""
+"""Client for the Code Review OpenEnv environment."""
 
-from typing import Dict
+from typing import Any
 
 from openenv.core import EnvClient
 from openenv.core.client_types import StepResult
-from openenv.core.env_server.types import State
 
-from .models import CodeReviewAction, CodeReviewObservation
+try:
+    from .models import ReviewAction, ReviewObservation, ReviewState
+except ImportError:
+    from models import ReviewAction, ReviewObservation, ReviewState
 
 
-class CodeReviewEnv(
-    EnvClient[CodeReviewAction, CodeReviewObservation, State]
-):
-    """
-    Client for the Code Review Env Environment.
+class CodeReviewEnv(EnvClient[ReviewAction, ReviewObservation, ReviewState]):
+    """WebSocket client for interacting with the code review environment."""
 
-    This client maintains a persistent WebSocket connection to the environment server,
-    enabling efficient multi-step interactions with lower latency.
-    Each client instance has its own dedicated environment session on the server.
-
-    Example:
-        >>> # Connect to a running server
-        >>> with CodeReviewEnv(base_url="http://localhost:8000") as client:
-        ...     result = client.reset()
-        ...     print(result.observation.echoed_message)
-        ...
-        ...     result = client.step(CodeReviewAction(message="Hello!"))
-        ...     print(result.observation.echoed_message)
-
-    Example with Docker:
-        >>> # Automatically start container and connect
-        >>> client = CodeReviewEnv.from_docker_image("code_review_env-env:latest")
-        >>> try:
-        ...     result = client.reset()
-        ...     result = client.step(CodeReviewAction(message="Test"))
-        ... finally:
-        ...     client.close()
-    """
-
-    def _step_payload(self, action: CodeReviewAction) -> Dict:
-        """
-        Convert CodeReviewAction to JSON payload for step message.
-
-        Args:
-            action: CodeReviewAction instance
-
-        Returns:
-            Dictionary representation suitable for JSON encoding
-        """
+    def _step_payload(self, action: ReviewAction) -> dict[str, Any]:
         return {
-            "message": action.message,
+            "review_comment": action.review_comment,
+            "issues_found": action.issues_found,
+            "severity": action.severity,
         }
 
-    def _parse_result(self, payload: Dict) -> StepResult[CodeReviewObservation]:
-        """
-        Parse server response into StepResult[CodeReviewObservation].
-
-        Args:
-            payload: JSON response data from server
-
-        Returns:
-            StepResult with CodeReviewObservation
-        """
+    def _parse_result(self, payload: dict[str, Any]) -> StepResult[ReviewObservation]:
         obs_data = payload.get("observation", {})
-        observation = CodeReviewObservation(
-            echoed_message=obs_data.get("echoed_message", ""),
-            message_length=obs_data.get("message_length", 0),
+        observation = ReviewObservation(
+            task_id=obs_data.get("task_id", "task_easy"),
+            file_name=obs_data.get("file_name", ""),
+            task_description=obs_data.get("task_description", ""),
+            code_snippet=obs_data.get("code_snippet", ""),
+            feedback=obs_data.get("feedback", ""),
+            step_number=obs_data.get("step_number", 0),
+            available_issue_tags=obs_data.get("available_issue_tags", []),
             done=payload.get("done", False),
             reward=payload.get("reward"),
             metadata=obs_data.get("metadata", {}),
         )
-
         return StepResult(
             observation=observation,
             reward=payload.get("reward"),
             done=payload.get("done", False),
         )
 
-    def _parse_state(self, payload: Dict) -> State:
-        """
-        Parse server response into State object.
-
-        Args:
-            payload: JSON response from state request
-
-        Returns:
-            State object with episode_id and step_count
-        """
-        return State(
-            episode_id=payload.get("episode_id"),
+    def _parse_state(self, payload: dict[str, Any]) -> ReviewState:
+        return ReviewState(
+            episode_id=payload.get("episode_id", ""),
             step_count=payload.get("step_count", 0),
+            current_task_id=payload.get("current_task_id", "task_easy"),
+            max_steps=payload.get("max_steps", 3),
         )

inference.py

@@ -2,18 +2,25 @@ import json
 import os
 import re
 import sys
+from collections.abc import Callable
 from typing import Any
 
-import httpx
 from openai import OpenAI
 
+try:
+    from .client import CodeReviewEnv
+    from .models import ReviewAction
+except ImportError:
+    from client import CodeReviewEnv
+    from models import ReviewAction
+
 
 TASK_IDS = ["task_easy", "task_medium", "task_hard"]
 DEFAULT_ENV_URL = "http://localhost:8000"
 DEFAULT_MODEL = "gpt-4o-mini"
 
 
-DETECTION_RULES: dict[str, callable] = {
+DETECTION_RULES: dict[str, Callable[[str], bool]] = {
     "null_pointer": lambda code: ".get(" in code or "= None" in code,
     "missing_return": lambda code: "# todo: return" in code.lower(),
     "sql_injection": lambda code: (
@@ -134,16 +141,14 @@ def run_baseline() -> dict[str, dict[str, Any]]:
 
     results: dict[str, dict[str, Any]] = {}
 
-    with httpx.Client(timeout=30.0) as client:
+    with CodeReviewEnv(base_url=env_url).sync() as env:
         for task_id in TASK_IDS:
-            reset_response = client.post(f"{env_url}/reset", json={"task_id": task_id})
-            reset_response.raise_for_status()
-            reset_payload = reset_response.json()
+            reset_result = env.reset(task_id=task_id)
+            observation = reset_result.observation
 
-            observation = reset_payload.get("observation", {})
-            code_snippet = str(observation.get("code_snippet", ""))
-            file_name = str(observation.get("file_name", ""))
-            task_description = str(observation.get("task_description", ""))
+            code_snippet = observation.code_snippet
+            file_name = observation.file_name
+            task_description = observation.task_description
 
             action_payload: dict[str, Any]
             if openai_client:
@@ -161,11 +166,8 @@ def run_baseline() -> dict[str, dict[str, Any]]:
             else:
                 action_payload = build_rule_action(code_snippet)
 
-            step_response = client.post(f"{env_url}/step", json={"action": action_payload})
-            step_response.raise_for_status()
-            step_payload = step_response.json()
-
-            score = float(step_payload.get("reward") or 0.0)
+            step_result = env.step(ReviewAction.model_validate(action_payload))
+            score = float(step_result.reward or 0.0)
             results[task_id] = {
                 "score": score,
                 "issues_found": action_payload.get("issues_found", []),

openenv.yaml

@@ -1,5 +1,17 @@
 spec_version: 1
-name: code_review_env
+name: code-review-env
+version: "1.0.0"
+description: >
+	A code review agent environment where an AI agent reads buggy Python code
+	and learns to identify security vulnerabilities, logic errors, and code smells.
+	Simulates the real-world software engineering task of pull request review.
+author: YOUR_HF_USERNAME
+tasks:
+	- task_easy
+	- task_medium
+	- task_hard
+sdk: gradio
+sdk_version: "4.0"
 type: space
 runtime: fastapi
 app: server.app:app

server/Dockerfile

@@ -4,77 +4,19 @@
 # This source code is licensed under the BSD-style license found in the
 # LICENSE file in the root directory of this source tree.
 
-# Multi-stage build using openenv-base
-# This Dockerfile is flexible and works for both:
-# - In-repo environments (with local OpenEnv sources)
-# - Standalone environments (with openenv from PyPI/Git)
-# The build script (openenv build) handles context detection and sets appropriate build args.
-
 ARG BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest
-FROM ${BASE_IMAGE} AS builder
-
-WORKDIR /app
-
-# Ensure git is available (required for installing dependencies from VCS)
-RUN apt-get update && \
-    apt-get install -y --no-install-recommends git && \
-    rm -rf /var/lib/apt/lists/*
-
-# Build argument to control whether we're building standalone or in-repo
-ARG BUILD_MODE=in-repo
-ARG ENV_NAME=code_review_env
-
-# Copy environment code (always at root of build context)
-COPY . /app/env
-
-# For in-repo builds, openenv is already vendored in the build context
-# For standalone builds, openenv will be installed via pyproject.toml
-WORKDIR /app/env
-
-# Ensure uv is available (for local builds where base image lacks it)
-RUN if ! command -v uv >/dev/null 2>&1; then \
-        curl -LsSf https://astral.sh/uv/install.sh | sh && \
-        mv /root/.local/bin/uv /usr/local/bin/uv && \
-        mv /root/.local/bin/uvx /usr/local/bin/uvx; \
-    fi
-    
-# Install dependencies using uv sync
-# If uv.lock exists, use it; otherwise resolve on the fly
-RUN --mount=type=cache,target=/root/.cache/uv \
-    if [ -f uv.lock ]; then \
-        uv sync --frozen --no-install-project --no-editable; \
-    else \
-        uv sync --no-install-project --no-editable; \
-    fi
-
-RUN --mount=type=cache,target=/root/.cache/uv \
-    if [ -f uv.lock ]; then \
-        uv sync --frozen --no-editable; \
-    else \
-        uv sync --no-editable; \
-    fi
-
-# Final runtime stage
 FROM ${BASE_IMAGE}
 
 WORKDIR /app
 
-# Copy the virtual environment from builder
-COPY --from=builder /app/env/.venv /app/.venv
-
-# Copy the environment code
-COPY --from=builder /app/env /app/env
-
-# Set PATH to use the virtual environment
-ENV PATH="/app/.venv/bin:$PATH"
+COPY server/requirements.txt /tmp/requirements.txt
+RUN pip install --no-cache-dir -r /tmp/requirements.txt
 
-# Set PYTHONPATH so imports work correctly
-ENV PYTHONPATH="/app/env:$PYTHONPATH"
+COPY . /app/
 
 # Health check
 HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
     CMD curl -f http://localhost:8000/health || exit 1
 
-# Run the FastAPI server
-# The module path is constructed to work with the /app/env structure
-CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]
+EXPOSE 8000
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8000"]

server/__init__.py

@@ -7,5 +7,7 @@
 """Code Review Env environment server components."""
 
 from .code_review_env_environment import CodeReviewEnvironment
+from .graders import grade_review
+from .tasks import TASKS, Task, get_task
 
-__all__ = ["CodeReviewEnvironment"]
+__all__ = ["CodeReviewEnvironment", "Task", "TASKS", "get_task", "grade_review"]

server/app.py

@@ -22,7 +22,7 @@ try:
     from .code_review_env_environment import CodeReviewEnvironment
     from .graders import grade_review
     from .tasks import TASKS, Task, get_task
-except ModuleNotFoundError:
+except ImportError:
     from models import ReviewAction, ReviewObservation
     from server.code_review_env_environment import CodeReviewEnvironment
     from server.graders import grade_review

server/requirements.txt

@@ -1,6 +1,7 @@
-openenv[core]>=0.2.0
+openenv-core>=0.2.0
 fastapi>=0.115.0
 uvicorn>=0.24.0
+pydantic>=2.0
 openai>=1.0
 httpx>=0.24.0
 

server/tasks.py

@@ -1,6 +1,9 @@
 from dataclasses import dataclass
 
-from ..models import ISSUE_TAXONOMY
+try:
+    from ..models import ISSUE_TAXONOMY
+except ImportError:
+    from models import ISSUE_TAXONOMY
 
 
 @dataclass(frozen=True)