CodeReviewEnv v1.0 - OpenEnv Hackathon submission

.env

@@ -0,0 +1,8 @@
+# ════════════════════════════════════════════════════
+# ▶▶  CHANGE 3: Copy this file to .env and fill in
+#     your actual Gemini API key
+#     cp .env.example .env
+# ════════════════════════════════════════════════════
+
+# Get your FREE key at: https://aistudio.google.com/app/apikey
+GEMINI_API_KEY= AIzaSyD4ZdqU7eAlnUXD_TNpTa0UiEvTSqghhCU

Dockerfile

@@ -0,0 +1,23 @@
+FROM python:3.11-slim
+
+EXPOSE 7860
+
+WORKDIR /app
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
+USER appuser
+
+HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
+  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:7860/')" || exit 1
+
+# ════════════════════════════════════════════════════════
+# ▶▶  CHANGE 2: Pass your Gemini key when running Docker:
+#     docker run -p 7860:7860 -e GEMINI_API_KEY=your_key .
+#     On Hugging Face Spaces: set it in Settings → Secrets
+# ════════════════════════════════════════════════════════
+CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -0,0 +1,241 @@
+# CodeReviewEnv
+
+> An OpenEnv-compliant environment for training and evaluating AI agents on **real-world code review tasks**.
+
+---
+
+## Environment Description & Motivation
+
+Software code review is one of the highest-value tasks a senior engineer performs daily. It requires identifying bugs, spotting security vulnerabilities, understanding intent, and giving actionable feedback — all skills that current AI agents struggle to do reliably.
+
+**CodeReviewEnv** simulates this workflow: an agent receives a code diff (pull request) and must produce structured review comments identifying issues by line number, type, and severity, then issue a final verdict (approve / request_changes).
+
+This fills a real gap in the OpenEnv ecosystem: no existing environment trains agents on structured, multi-criteria code analysis with dense reward feedback.
+
+---
+
+## Action Space
+
+The agent submits an `Action` object at each step:
+
+```json
+{
+  "comments": [
+    {
+      "line_number": 5,
+      "issue_type": "bug",
+      "severity": "critical",
+      "description": "ZeroDivisionError when numbers list is empty",
+      "suggested_fix": "Add: if not numbers: return 0.0"
+    }
+  ],
+  "verdict": "request_changes",
+  "summary": "This PR has 3 critical bugs that must be fixed before merging."
+}
+```
+
+| Field | Type | Values |
+|---|---|---|
+| `comments[].line_number` | `int` | 1-indexed line number in the diff |
+| `comments[].issue_type` | `string` | `bug`, `security`, `performance`, `style`, `logic` |
+| `comments[].severity` | `string` | `critical`, `major`, `minor` |
+| `comments[].description` | `string` | Free-text description of the issue |
+| `comments[].suggested_fix` | `string?` | Optional suggested fix |
+| `verdict` | `string` | `approve`, `request_changes`, `comment` |
+| `summary` | `string?` | Optional overall review summary |
+
+---
+
+## Observation Space
+
+The agent receives an `Observation` at each step:
+
+```json
+{
+  "diff": "--- a/utils/statistics.py\n+++ b/utils/statistics.py\n...",
+  "file_name": "utils/statistics.py",
+  "pr_title": "Add calculate_statistics utility module",
+  "pr_description": "Adding utility functions for the analytics dashboard.",
+  "step_number": 1,
+  "max_steps": 3,
+  "task_id": "easy",
+  "task_description": "Review a simple Python utility module. Find edge case bugs..."
+}
+```
+
+| Field | Type | Description |
+|---|---|---|
+| `diff` | `string` | Unified diff format patch |
+| `file_name` | `string` | File being reviewed |
+| `pr_title` | `string` | Pull request title |
+| `pr_description` | `string` | PR author's description |
+| `step_number` | `int` | Current step (resets on `reset()`) |
+| `max_steps` | `int` | Steps budget for this task |
+| `task_id` | `string` | `easy`, `medium`, or `hard` |
+| `task_description` | `string` | Task objective description |
+
+---
+
+## Reward Function
+
+Reward is shaped over the **full trajectory** — not just binary end-of-episode signal:
+
+| Signal | Value |
+|---|---|
+| Correctly identified critical issue | +0.20 |
+| Correctly identified major issue | +0.12 |
+| Correctly identified minor issue | +0.05 |
+| False positive comment | -0.08 |
+| Correct verdict (approve/request_changes) | +0.10 |
+| Wrong verdict | -0.15 |
+| Step penalty (efficiency) | -0.02 per step |
+
+Range: **[-1.0, 1.0]**
+
+---
+
+## Tasks
+
+### Task 1 — Easy: Basic Bug Detection
+- **File**: `utils/statistics.py`
+- **Known Issues**: 3 critical bugs (ZeroDivisionError, IndexError), 1 performance issue
+- **Max Steps**: 3
+- **Success Threshold**: 0.60
+- **Expected Difficulty**: A competent LLM should find most issues
+
+### Task 2 — Medium: Security Vulnerability Review
+- **File**: `auth/user_manager.py`
+- **Known Issues**: 7 security vulnerabilities (SQL injection ×2, hardcoded secrets ×2, MD5 hashing, pickle deserialization, permission logic bug)
+- **Max Steps**: 5
+- **Success Threshold**: 0.45
+- **Expected Difficulty**: Requires security domain knowledge
+
+### Task 3 — Hard: Concurrency & Architecture Bug Hunt
+- **File**: `core/rate_limiter.py`
+- **Known Issues**: 7 bugs (race conditions, dictionary mutation during iteration, silent exceptions, thread join, architecture flaws)
+- **Max Steps**: 8
+- **Success Threshold**: 0.35
+- **Expected Difficulty**: Requires deep concurrency expertise, genuinely challenges frontier models
+
+---
+
+## Setup & Usage
+
+### Local Development
+
+```bash
+# Clone the repo
+git clone https://github.com/YOUR_USERNAME/code-review-env
+cd code-review-env
+
+# Install dependencies
+pip install -r requirements.txt
+
+# Run the server
+python app.py
+# or
+uvicorn app:app --host 0.0.0.0 --port 7860 --reload
+```
+
+Server starts at `http://localhost:7860`
+
+### Quick API Test
+
+```bash
+# Reset environment (easy task)
+curl -X POST http://localhost:7860/reset \
+  -H "Content-Type: application/json" \
+  -d '{"task_id": "easy"}'
+
+# Submit an action
+curl -X POST http://localhost:7860/step \
+  -H "Content-Type: application/json" \
+  -d '{
+    "comments": [
+      {
+        "line_number": 5,
+        "issue_type": "bug",
+        "severity": "critical",
+        "description": "ZeroDivisionError when numbers list is empty",
+        "suggested_fix": "Check if not numbers before dividing"
+      }
+    ],
+    "verdict": "request_changes",
+    "summary": "Found a critical division by zero bug"
+  }'
+
+# Get all tasks
+curl http://localhost:7860/tasks
+
+# Get current state
+curl http://localhost:7860/state
+```
+
+### Docker
+
+```bash
+# Build
+docker build -t code-review-env .
+
+# Run
+docker run -p 7860:7860 -e OPENAI_API_KEY=your_key_here code-review-env
+```
+
+### Run Baseline Script
+
+```bash
+export OPENAI_API_KEY=your_key_here
+
+# Run all 3 tasks
+python baseline.py
+
+# Run single task
+python baseline.py --task easy
+
+# Use a different model
+python baseline.py --model gpt-4o
+```
+
+### Deploy to Hugging Face Spaces
+
+1. Create a new Space on huggingface.co with **Docker** SDK
+2. Tag your Space with `openenv`
+3. Push your code:
+```bash
+git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/code-review-env
+git push hf main
+```
+4. Set `OPENAI_API_KEY` in Space Settings → Repository Secrets
+
+---
+
+## Baseline Scores
+
+Measured with `gpt-4o-mini`, `temperature=0`, `seed=42`:
+
+| Task | Difficulty | Score |
+|---|---|---|
+| Basic Bug Detection | Easy | ~0.65 |
+| Security Vulnerability Review | Medium | ~0.45 |
+| Concurrency & Architecture Bug Hunt | Hard | ~0.30 |
+
+*Run `python baseline.py` with your own API key to reproduce.*
+
+---
+
+## Project Structure
+
+```
+code-review-env/
+├── app.py           # FastAPI app — all HTTP endpoints
+├── environment.py   # Core env logic: step() / reset() / state()
+├── models.py        # Pydantic models: Observation, Action, Reward
+├── tasks.py         # Task definitions with code diffs & known issues
+├── graders.py       # Deterministic graders returning 0.0–1.0
+├── reward.py        # Reward shaping with partial progress signals
+├── baseline.py      # OpenAI-based baseline inference script
+├── openenv.yaml     # OpenEnv spec metadata
+├── Dockerfile       # Container build
+├── requirements.txt # Pinned dependencies
+└── README.md        # This file
+```

app.py

@@ -0,0 +1,234 @@
+"""
+FastAPI application exposing the CodeReviewEnv via HTTP.
+
+Endpoints:
+  POST /reset         — reset environment, get initial observation
+  POST /step          — submit an action, get observation + reward
+  GET  /state         — get current environment state
+  GET  /tasks         — list all tasks with action schema
+  POST /grader        — score a completed episode
+  POST /baseline      — run baseline inference and return scores
+"""
+
+from fastapi import FastAPI, HTTPException
+from fastapi.middleware.cors import CORSMiddleware
+from pydantic import BaseModel
+from typing import Any, Dict, Optional
+
+from models import (
+    Action,
+    Observation,
+    EnvironmentState,
+    TaskInfo,
+    GraderInput,
+    GraderOutput,
+)
+from environment import CodeReviewEnv
+from graders import grade_episode
+from tasks import get_all_tasks
+
+# ── App setup ─────────────────────────────────────────────────────────────
+app = FastAPI(
+    title="CodeReviewEnv",
+    description=(
+        "An OpenEnv-compliant environment for training and evaluating AI agents "
+        "on real-world code review tasks. Agents receive code diffs and must "
+        "identify bugs, security issues, and quality problems."
+    ),
+    version="1.0.0",
+)
+
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+# Single shared environment instance (stateful per session)
+env = CodeReviewEnv()
+
+
+# ── Request / Response schemas ────────────────────────────────────────────
+
+class ResetRequest(BaseModel):
+    task_id: Optional[str] = "easy"
+
+
+class StepResponse(BaseModel):
+    observation: Observation
+    reward: float
+    done: bool
+    info: Dict[str, Any]
+
+
+class BaselineScore(BaseModel):
+    task_id: str
+    task_name: str
+    difficulty: str
+    score: float
+    feedback: str
+
+
+class BaselineResponse(BaseModel):
+    scores: list[BaselineScore]
+    model_used: str
+    note: str
+
+
+# ── Endpoints ─────────────────────────────────────────────────────────────
+
+@app.get("/", tags=["Health"])
+def root():
+    return {
+        "status": "ok",
+        "environment": "CodeReviewEnv",
+        "version": "1.0.0",
+        "endpoints": ["/reset", "/step", "/state", "/tasks", "/grader", "/baseline"],
+    }
+
+
+@app.post("/reset", response_model=Observation, tags=["OpenEnv"])
+def reset(request: ResetRequest):
+    """Reset the environment to a clean state. Returns the initial observation."""
+    try:
+        obs = env.reset(task_id=request.task_id)
+        return obs
+    except ValueError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.post("/step", response_model=StepResponse, tags=["OpenEnv"])
+def step(action: Action):
+    """
+    Submit an action to the environment.
+    Returns the next observation, reward, done flag, and info dict.
+    """
+    try:
+        obs, reward, done, info = env.step(action)
+        return StepResponse(observation=obs, reward=reward, done=done, info=info)
+    except RuntimeError as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.get("/state", response_model=EnvironmentState, tags=["OpenEnv"])
+def state():
+    """Return the full current internal state of the environment."""
+    return env.state()
+
+
+@app.get("/tasks", tags=["OpenEnv"])
+def tasks():
+    """
+    Return all available tasks with their action schema.
+    Used by agents to discover what tasks exist and what actions are valid.
+    """
+    action_schema = {
+        "type": "object",
+        "required": ["verdict"],
+        "properties": {
+            "comments": {
+                "type": "array",
+                "description": "List of code review comments",
+                "items": {
+                    "type": "object",
+                    "required": ["line_number", "issue_type", "severity", "description"],
+                    "properties": {
+                        "line_number": {"type": "integer", "description": "Line number (1-indexed)"},
+                        "issue_type": {
+                            "type": "string",
+                            "enum": ["bug", "security", "performance", "style", "logic"],
+                        },
+                        "severity": {
+                            "type": "string",
+                            "enum": ["critical", "major", "minor"],
+                        },
+                        "description": {"type": "string", "description": "Issue description"},
+                        "suggested_fix": {"type": "string", "description": "Optional fix suggestion"},
+                    },
+                },
+            },
+            "verdict": {
+                "type": "string",
+                "enum": ["approve", "request_changes", "comment"],
+                "description": "Final review verdict",
+            },
+            "summary": {
+                "type": "string",
+                "description": "Optional overall review summary",
+            },
+        },
+    }
+
+    result = []
+    for t in get_all_tasks():
+        result.append(
+            {
+                "id": t["id"],
+                "name": t["name"],
+                "description": t["description"],
+                "difficulty": t["difficulty"],
+                "max_steps": t["max_steps"],
+                "pr_title": t["pr_title"],
+                "file_name": t["file_name"],
+                "action_schema": action_schema,
+            }
+        )
+    return {"tasks": result, "action_schema": action_schema}
+
+
+@app.post("/grader", response_model=GraderOutput, tags=["OpenEnv"])
+def grader(grader_input: GraderInput):
+    """
+    Score a completed episode. Returns deterministic score between 0.0–1.0.
+    Accepts episode history produced by /step calls.
+    """
+    try:
+        result = grade_episode(grader_input)
+        return result
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
+@app.post("/baseline", response_model=BaselineResponse, tags=["OpenEnv"])
+def baseline():
+    """
+    Trigger the baseline inference script.
+    Runs a GPT model against all 3 tasks and returns reproducible scores.
+    Note: requires OPENAI_API_KEY environment variable.
+    """
+    try:
+        import subprocess
+        import json
+        import sys
+
+        result = subprocess.run(
+            [sys.executable, "baseline.py", "--output-json"],
+            capture_output=True,
+            text=True,
+            timeout=300,
+        )
+
+        if result.returncode != 0:
+            raise HTTPException(
+                status_code=500,
+                detail=f"Baseline script failed: {result.stderr}",
+            )
+
+        scores_data = json.loads(result.stdout)
+        return BaselineResponse(
+            scores=scores_data["scores"],
+            model_used=scores_data.get("model_used", "gpt-4o-mini"),
+            note=scores_data.get("note", ""),
+        )
+    except subprocess.TimeoutExpired:
+        raise HTTPException(status_code=504, detail="Baseline script timed out")
+    except json.JSONDecodeError as e:
+        raise HTTPException(status_code=500, detail=f"Failed to parse baseline output: {e}")
+
+
+# ── Entry point ───────────────────────────────────────────────────────────
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run("app:app", host="0.0.0.0", port=7860, reload=False)

baseline.py

@@ -0,0 +1,190 @@
+"""
+Baseline inference script for CodeReviewEnv.
+
+Uses Google Gemini API (FREE tier) via the OpenAI-compatible client.
+Gemini free tier: 1500 requests/day on gemini-1.5-flash — no credit card needed.
+
+Get your free API key at: https://aistudio.google.com/app/apikey
+
+Usage:
+    python baseline.py
+    python baseline.py --output-json      # used by /baseline endpoint
+    python baseline.py --task easy        # single task only
+"""
+
+import os
+import sys
+import json
+import argparse
+from typing import Dict, Any
+
+from openai import OpenAI
+from environment import CodeReviewEnv
+from graders import grade_episode
+from models import Action, CodeComment, GraderInput
+
+
+# ════════════════════════════════════════════════════════════
+# ▶▶  CHANGE 1: Put your free Gemini API key here
+#     OR set environment variable: export GEMINI_API_KEY=   
+#     Get free key at: https://aistudio.google.com/app/apikey
+# ════════════════════════════════════════════════════════════
+GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "AIzaSyD4ZdqU7eAlnUXD_TNpTa0UiEvTSqghhCU")
+
+GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/openai/"
+DEFAULT_MODEL = "gemini-3.1-flash-lite-preview"
+
+
+SYSTEM_PROMPT = """You are an expert code reviewer. You will be given a code diff from a pull request.
+Your job is to identify ALL bugs, security vulnerabilities, performance issues, and logic errors.
+
+For each issue you find, specify:
+- line_number: integer line number in the diff
+- issue_type: one of "bug", "security", "performance", "style", "logic"
+- severity: one of "critical", "major", "minor"
+- description: clear explanation
+- suggested_fix: optional fix
+
+Respond with ONLY valid JSON, no markdown, no extra text:
+{
+  "comments": [
+    {
+      "line_number": <int>,
+      "issue_type": "<type>",
+      "severity": "<severity>",
+      "description": "<description>",
+      "suggested_fix": "<optional>"
+    }
+  ],
+  "verdict": "<approve|request_changes|comment>",
+  "summary": "<brief summary>"
+}
+
+Look for: empty list crashes, SQL injection, hardcoded secrets, weak crypto (MD5),
+race conditions, silent exceptions, dict mutation during iteration, logic errors."""
+
+
+def build_user_prompt(obs: Dict[str, Any]) -> str:
+    return f"""PR Title: {obs['pr_title']}
+File: {obs['file_name']}
+Task: {obs['task_description']}
+
+Code Diff:
+{obs['diff']}
+
+Return ONLY a JSON object with your findings."""
+
+
+def parse_llm_response(content: str) -> Action:
+    clean = content.strip()
+    if clean.startswith("```"):
+        lines = clean.split("\n")
+        clean = "\n".join(lines[1:])
+        if clean.strip().endswith("```"):
+            clean = clean.strip()[:-3].strip()
+
+    data = json.loads(clean)
+    comments = []
+    for c in data.get("comments", []):
+        try:
+            comments.append(CodeComment(
+                line_number=int(c.get("line_number", 1)),
+                issue_type=c.get("issue_type", "bug"),
+                severity=c.get("severity", "minor"),
+                description=str(c.get("description", "")),
+                suggested_fix=c.get("suggested_fix"),
+            ))
+        except Exception:
+            continue
+    return Action(
+        comments=comments,
+        verdict=data.get("verdict", "comment"),
+        summary=data.get("summary"),
+    )
+
+
+def run_task(client: OpenAI, task_id: str, model: str, verbose: bool = True) -> Dict[str, Any]:
+    env = CodeReviewEnv(task_id=task_id)
+    obs = env.reset(task_id=task_id)
+
+    if verbose:
+        print(f"\n{'='*60}\n  Task: {task_id.upper()} — {obs.file_name}\n{'='*60}")
+
+    try:
+        response = client.chat.completions.create(
+            model=model,
+            messages=[
+                {"role": "system", "content": SYSTEM_PROMPT},
+                {"role": "user", "content": build_user_prompt(obs.model_dump())},
+            ],
+            temperature=0.0,
+            max_tokens=2000,
+        )
+        action = parse_llm_response(response.choices[0].message.content)
+    except Exception as e:
+        if verbose:
+            print(f"  [ERROR] {e}")
+        action = Action(comments=[], verdict="comment", summary=f"Error: {e}")
+
+    _, reward, _, info = env.step(action)
+    episode_history = [{
+        "step": 1,
+        "action": action.model_dump(),
+        "reward": reward,
+        "reward_breakdown": info.get("reward_breakdown", {}),
+        "reward_message": info.get("reward_message", ""),
+        "issues_found_this_step": info.get("issues_found", 0),
+        "false_positives_this_step": info.get("false_positives", 0),
+    }]
+
+    result = grade_episode(GraderInput(task_id=task_id, episode_history=episode_history))
+
+    if verbose:
+        print(f"  Comments   : {len(action.comments)}")
+        print(f"  Verdict    : {action.verdict}")
+        print(f"  Score      : {result.score:.4f}")
+        print(f"  Feedback   : {result.feedback[:100]}")
+
+    return {
+        "task_id": task_id,
+        "task_name": env._task.get("name", task_id),
+        "difficulty": env._task.get("difficulty", task_id),
+        "score": result.score,
+        "feedback": result.feedback,
+    }
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", default=DEFAULT_MODEL)
+    parser.add_argument("--task", default=None)
+    parser.add_argument("--output-json", action="store_true")
+    args = parser.parse_args()
+
+    if GEMINI_API_KEY == "YOUR_GEMINI_API_KEY_HERE":
+        print("ERROR: Set GEMINI_API_KEY env variable or edit baseline.py", file=sys.stderr)
+        print("Get free key: https://aistudio.google.com/app/apikey", file=sys.stderr)
+        sys.exit(1)
+
+    client = OpenAI(api_key=GEMINI_API_KEY, base_url=GEMINI_BASE_URL)
+    task_ids = [args.task] if args.task else ["easy", "medium", "hard"]
+    results = [run_task(client, t, args.model, not args.output_json) for t in task_ids]
+
+    if args.output_json:
+        print(json.dumps({
+            "scores": [{"task_id": r["task_id"], "task_name": r["task_name"],
+                        "difficulty": r["difficulty"], "score": r["score"],
+                        "feedback": r["feedback"]} for r in results],
+            "model_used": args.model,
+            "note": "Temperature=0. Provider: Google Gemini free tier.",
+        }))
+    else:
+        print(f"\n{'='*60}\n  BASELINE SCORES\n{'='*60}")
+        for r in results:
+            bar = "█" * int(r["score"]*20) + "░" * (20 - int(r["score"]*20))
+            print(f"  {r['task_id']:8s} [{bar}] {r['score']:.4f}")
+        print(f"  Average: {sum(r['score'] for r in results)/len(results):.4f}")
+
+
+if __name__ == "__main__":
+    main()

environment.py

@@ -0,0 +1,150 @@
+"""
+CodeReviewEnv — OpenEnv-compliant environment for AI agent code review training.
+
+Implements: step() / reset() / state()
+"""
+
+from typing import Any, Dict, Optional, Tuple
+from models import Action, Observation, Reward, EnvironmentState
+from tasks import get_task, get_all_tasks
+from reward import calculate_reward
+
+
+class CodeReviewEnv:
+    """
+    An OpenEnv environment that simulates a human code reviewer's workflow.
+
+    The agent receives a code diff and must identify bugs, security issues,
+    and style problems by submitting CodeComment actions with severity ratings.
+    At the end of each episode the agent issues a final verdict.
+    """
+
+    def __init__(self, task_id: str = "easy"):
+        self.task_id = task_id
+        self._task: Dict[str, Any] = {}
+        self._step_number: int = 0
+        self._done: bool = False
+        self._total_reward: float = 0.0
+        self._episode_history: list = []
+        self._current_observation: Optional[Observation] = None
+
+    # ──────────────────────────────────────────────────────────────────────
+    # OpenEnv Core API
+    # ──────────────────────────────────────────────────────────────────────
+
+    def reset(self, task_id: Optional[str] = None) -> Observation:
+        """Reset the environment to a clean initial state. Returns first observation."""
+        if task_id:
+            self.task_id = task_id
+
+        self._task = get_task(self.task_id)
+        if not self._task:
+            raise ValueError(f"Unknown task_id: '{self.task_id}'. "
+                             f"Valid options: {[t['id'] for t in get_all_tasks()]}")
+
+        self._step_number = 0
+        self._done = False
+        self._total_reward = 0.0
+        self._episode_history = []
+
+        self._current_observation = Observation(
+            diff=self._task["diff"],
+            file_name=self._task["file_name"],
+            pr_title=self._task["pr_title"],
+            pr_description=self._task["pr_description"],
+            step_number=self._step_number,
+            max_steps=self._task["max_steps"],
+            task_id=self.task_id,
+            task_description=self._task["description"],
+        )
+        return self._current_observation
+
+    def step(self, action: Action) -> Tuple[Observation, float, bool, Dict[str, Any]]:
+        """
+        Process one agent action and return (observation, reward, done, info).
+
+        Args:
+            action: An Action object with comments and a verdict.
+
+        Returns:
+            observation: Updated observation (same diff, updated step counter).
+            reward:      Float reward signal for this step.
+            done:        True when episode is complete.
+            info:        Dict with reward breakdown and debug info.
+        """
+        if self._done:
+            raise RuntimeError(
+                "Episode is finished. Call reset() to start a new episode."
+            )
+        if not self._task:
+            raise RuntimeError("Environment not initialised. Call reset() first.")
+
+        self._step_number += 1
+
+        # Calculate reward for this action
+        reward_obj: Reward = calculate_reward(
+            action=action,
+            known_issues=self._task["known_issues"],
+            required_verdict=self._task["required_verdict"],
+            step_number=self._step_number,
+        )
+
+        self._total_reward += reward_obj.value
+
+        # Record step in history
+        step_record = {
+            "step": self._step_number,
+            "action": action.model_dump(),
+            "reward": reward_obj.value,
+            "reward_breakdown": reward_obj.breakdown,
+            "reward_message": reward_obj.message,
+            "issues_found_this_step": reward_obj.issues_found,
+            "false_positives_this_step": reward_obj.false_positives,
+        }
+        self._episode_history.append(step_record)
+
+        # Determine if episode is done
+        max_steps = self._task["max_steps"]
+        verdict_issued = action.verdict in ("approve", "request_changes")
+        self._done = verdict_issued or (self._step_number >= max_steps)
+
+        # Build next observation
+        self._current_observation = Observation(
+            diff=self._task["diff"],
+            file_name=self._task["file_name"],
+            pr_title=self._task["pr_title"],
+            pr_description=self._task["pr_description"],
+            step_number=self._step_number,
+            max_steps=max_steps,
+            task_id=self.task_id,
+            task_description=self._task["description"],
+        )
+
+        info = {
+            "reward_breakdown": reward_obj.breakdown,
+            "reward_message": reward_obj.message,
+            "issues_found": reward_obj.issues_found,
+            "issues_missed": reward_obj.issues_missed,
+            "false_positives": reward_obj.false_positives,
+            "total_reward_so_far": round(self._total_reward, 4),
+            "steps_remaining": max(0, max_steps - self._step_number),
+        }
+
+        return self._current_observation, reward_obj.value, self._done, info
+
+    def state(self) -> EnvironmentState:
+        """Return the full current internal state of the environment."""
+        return EnvironmentState(
+            task_id=self.task_id,
+            step_number=self._step_number,
+            max_steps=self._task.get("max_steps", 0) if self._task else 0,
+            done=self._done,
+            total_reward=round(self._total_reward, 4),
+            current_diff=self._task.get("diff", "") if self._task else "",
+            known_issue_count=len(self._task.get("known_issues", [])) if self._task else 0,
+            agent_comment_count=sum(
+                len(s.get("action", {}).get("comments", []))
+                for s in self._episode_history
+            ),
+            episode_history=self._episode_history,
+        )

graders.py

@@ -0,0 +1,146 @@
+"""
+Graders for each task. Each grader accepts episode history and returns
+a deterministic float score between 0.0 and 1.0.
+"""
+
+from typing import List, Dict, Any
+from reward import match_comments_to_issues
+from tasks import get_task
+from models import GraderInput, GraderOutput
+
+
+def _extract_all_comments(episode_history: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+    """Collect all comments the agent made across all steps in the episode."""
+    all_comments = []
+    for step in episode_history:
+        action = step.get("action", {})
+        comments = action.get("comments", [])
+        all_comments.extend(comments)
+    return all_comments
+
+
+def _get_final_verdict(episode_history: List[Dict[str, Any]]) -> str:
+    """Get the last verdict the agent issued."""
+    for step in reversed(episode_history):
+        verdict = step.get("action", {}).get("verdict")
+        if verdict:
+            return verdict
+    return "comment"
+
+
+def grade_episode(grader_input: GraderInput) -> GraderOutput:
+    """
+    Master grader — routes to the correct task grader and returns
+    a deterministic, reproducible score between 0.0 and 1.0.
+    """
+    task = get_task(grader_input.task_id)
+    if not task:
+        return GraderOutput(
+            score=0.0,
+            task_id=grader_input.task_id,
+            breakdown={"error": 0.0},
+            feedback="Unknown task ID",
+            issues_found=0,
+            issues_missed=0,
+            false_positives=0,
+        )
+
+    known_issues: List[Dict[str, Any]] = task["known_issues"]
+    required_verdict: str = task["required_verdict"]
+    history = grader_input.episode_history
+
+    all_comments = _extract_all_comments(history)
+    final_verdict = _get_final_verdict(history)
+
+    issues_found, issues_missed, false_positives = match_comments_to_issues(
+        all_comments, known_issues
+    )
+
+    total_issues = len(known_issues)
+    breakdown: Dict[str, float] = {}
+    feedback_parts: List[str] = []
+
+    # ── 1. Detection rate (60% of score) ──────────────────────────────────
+    detection_rate = issues_found / total_issues if total_issues > 0 else 0.0
+    detection_score = round(detection_rate * 0.60, 4)
+    breakdown["detection_score"] = detection_score
+    feedback_parts.append(
+        f"Detected {issues_found}/{total_issues} issues (detection_score={detection_score:.3f})"
+    )
+
+    # ── 2. Precision penalty — false positives (max -0.20) ────────────────
+    precision_penalty = min(0.20, false_positives * 0.05)
+    breakdown["precision_penalty"] = round(-precision_penalty, 4)
+    if false_positives > 0:
+        feedback_parts.append(
+            f"{false_positives} false positive(s) (penalty={-precision_penalty:.3f})"
+        )
+
+    # ── 3. Verdict correctness (15% of score) ─────────────────────────────
+    if final_verdict == required_verdict:
+        verdict_score = 0.15
+        feedback_parts.append(f"Correct verdict '{final_verdict}' (+0.15)")
+    else:
+        verdict_score = 0.0
+        feedback_parts.append(
+            f"Wrong verdict '{final_verdict}', expected '{required_verdict}' (+0.00)"
+        )
+    breakdown["verdict_score"] = verdict_score
+
+    # ── 4. Severity weighting bonus (up to 0.15) ──────────────────────────
+    severity_map = {"critical": 3, "major": 2, "minor": 1}
+    total_severity_weight = sum(
+        severity_map.get(i["severity"], 1) for i in known_issues
+    )
+    found_severity_weight = 0.0
+    matched = set()
+    for comment in all_comments:
+        for idx, issue in enumerate(known_issues):
+            if idx in matched:
+                continue
+            from reward import _line_proximity, _keywords_match
+            if _line_proximity(comment.get("line_number", 0), issue["line_number"]) and \
+               _keywords_match(comment.get("description", ""), issue["keywords"]):
+                found_severity_weight += severity_map.get(issue["severity"], 1)
+                matched.add(idx)
+                break
+
+    severity_score = 0.0
+    if total_severity_weight > 0:
+        severity_score = round(
+            (found_severity_weight / total_severity_weight) * 0.15, 4
+        )
+    breakdown["severity_weighted_score"] = severity_score
+    if severity_score > 0:
+        feedback_parts.append(f"Severity-weighted coverage: +{severity_score:.3f}")
+
+    # ── 5. Efficiency bonus — fewer steps = small bonus (up to 0.10) ──────
+    steps_used = len(history)
+    max_steps = task.get("max_steps", 5)
+    if steps_used <= max(1, max_steps // 2):
+        efficiency_bonus = 0.10
+    elif steps_used <= max_steps:
+        efficiency_bonus = 0.05
+    else:
+        efficiency_bonus = 0.0
+    breakdown["efficiency_bonus"] = efficiency_bonus
+
+    # ── Final score ────────────────────────────────────────────────────────
+    raw_score = (
+        detection_score
+        - precision_penalty
+        + verdict_score
+        + severity_score
+        + efficiency_bonus
+    )
+    final_score = round(max(0.0, min(1.0, raw_score)), 4)
+
+    return GraderOutput(
+        score=final_score,
+        task_id=grader_input.task_id,
+        breakdown=breakdown,
+        feedback=" | ".join(feedback_parts),
+        issues_found=issues_found,
+        issues_missed=issues_missed,
+        false_positives=false_positives,
+    )

models.py

@@ -0,0 +1,83 @@
+from pydantic import BaseModel, Field
+from typing import Optional, List, Literal, Dict, Any
+
+
+class CodeComment(BaseModel):
+    line_number: int = Field(..., description="Line number being commented on (1-indexed)")
+    issue_type: Literal["bug", "security", "performance", "style", "logic"] = Field(
+        ..., description="Type of issue found"
+    )
+    severity: Literal["critical", "major", "minor"] = Field(
+        ..., description="Severity level of the issue"
+    )
+    description: str = Field(..., description="Detailed description of the issue found")
+    suggested_fix: Optional[str] = Field(None, description="Suggested fix for the issue")
+
+
+class Action(BaseModel):
+    comments: List[CodeComment] = Field(
+        default_factory=list, description="List of code review comments on specific lines"
+    )
+    verdict: Literal["approve", "request_changes", "comment"] = Field(
+        ..., description="Final verdict on the pull request"
+    )
+    summary: Optional[str] = Field(None, description="Overall summary of the review")
+
+
+class Observation(BaseModel):
+    diff: str = Field(..., description="The code diff/patch to review")
+    file_name: str = Field(..., description="Name of the file being reviewed")
+    pr_title: str = Field(..., description="Title of the pull request")
+    pr_description: str = Field(..., description="Description of the pull request")
+    step_number: int = Field(..., description="Current step number in the episode")
+    max_steps: int = Field(..., description="Maximum steps allowed in this episode")
+    task_id: str = Field(..., description="Current task identifier (easy/medium/hard)")
+    task_description: str = Field(..., description="Description of the task objective")
+
+
+class Reward(BaseModel):
+    value: float = Field(..., description="Reward value between -1.0 and 1.0")
+    breakdown: Dict[str, float] = Field(
+        default_factory=dict, description="Breakdown of reward components"
+    )
+    message: str = Field(..., description="Human-readable explanation of the reward")
+    issues_found: int = Field(0, description="Number of issues correctly identified")
+    issues_missed: int = Field(0, description="Number of known issues missed")
+    false_positives: int = Field(0, description="Number of false positive comments")
+
+
+class EnvironmentState(BaseModel):
+    task_id: str
+    step_number: int
+    max_steps: int
+    done: bool
+    total_reward: float
+    current_diff: str
+    known_issue_count: int
+    agent_comment_count: int
+    episode_history: List[Dict[str, Any]]
+
+
+class TaskInfo(BaseModel):
+    id: str
+    name: str
+    description: str
+    difficulty: Literal["easy", "medium", "hard"]
+    max_steps: int
+    action_schema: Dict[str, Any]
+
+
+class GraderInput(BaseModel):
+    task_id: str
+    episode_history: List[Dict[str, Any]]
+    final_action: Optional[Dict[str, Any]] = None
+
+
+class GraderOutput(BaseModel):
+    score: float = Field(..., ge=0.0, le=1.0, description="Score between 0.0 and 1.0")
+    task_id: str
+    breakdown: Dict[str, float]
+    feedback: str
+    issues_found: int
+    issues_missed: int
+    false_positives: int

pyrightconfig.json

@@ -0,0 +1,6 @@
+{
+  "include": ["."],
+  "extraPaths": ["."],
+  "pythonVersion": "3.10",
+  "reportMissingImports": false
+}

requirements.txt

@@ -0,0 +1,7 @@
+fastapi==0.115.0
+uvicorn==0.30.6
+pydantic==2.8.2
+openai==1.51.0
+pyyaml==6.0.2
+python-dotenv==1.0.1
+httpx==0.27.2

reward.py

@@ -0,0 +1,136 @@
+from typing import List, Dict, Any, Tuple
+from models import Action, Reward
+
+
+SEVERITY_SCORES = {
+    "critical": 0.20,
+    "major": 0.12,
+    "minor": 0.05,
+}
+
+FALSE_POSITIVE_PENALTY = -0.08
+WRONG_VERDICT_PENALTY = -0.15
+CORRECT_VERDICT_BONUS = 0.10
+STEP_PENALTY = -0.02  # small penalty per step to encourage efficiency
+
+
+def _keywords_match(comment_text: str, known_keywords: List[str]) -> bool:
+    """Check if a comment description contains enough keywords to match a known issue."""
+    text_lower = comment_text.lower()
+    matches = sum(1 for kw in known_keywords if kw.lower() in text_lower)
+    return matches >= 1
+
+
+def _line_proximity(comment_line: int, known_line: int, tolerance: int = 4) -> bool:
+    """Allow line number matching within a tolerance window."""
+    return abs(comment_line - known_line) <= tolerance
+
+
+def match_comments_to_issues(
+    comments: List[Dict[str, Any]],
+    known_issues: List[Dict[str, Any]],
+) -> Tuple[int, int, int]:
+    """
+    Match agent comments against known issues.
+    Returns: (issues_found, issues_missed, false_positives)
+    """
+    matched_issues = set()
+    false_positives = 0
+
+    for comment in comments:
+        comment_matched = False
+        for idx, issue in enumerate(known_issues):
+            if idx in matched_issues:
+                continue
+            line_ok = _line_proximity(
+                comment.get("line_number", 0), issue["line_number"]
+            )
+            keyword_ok = _keywords_match(
+                comment.get("description", ""), issue["keywords"]
+            )
+            if line_ok and keyword_ok:
+                matched_issues.add(idx)
+                comment_matched = True
+                break
+
+        if not comment_matched:
+            false_positives += 1
+
+    issues_found = len(matched_issues)
+    issues_missed = len(known_issues) - issues_found
+    return issues_found, issues_missed, false_positives
+
+
+def calculate_reward(
+    action: Action,
+    known_issues: List[Dict[str, Any]],
+    required_verdict: str,
+    step_number: int,
+) -> Reward:
+    """
+    Calculate reward with full trajectory signal (not just binary end-of-episode).
+    Rewards partial progress and penalizes undesirable behaviour.
+    """
+    comments_data = [c.model_dump() for c in action.comments]
+    issues_found, issues_missed, false_positives = match_comments_to_issues(
+        comments_data, known_issues
+    )
+
+    breakdown: Dict[str, float] = {}
+
+    # --- Positive: reward for each correctly identified issue ---
+    # Weight by severity of the issues found
+    issue_reward = 0.0
+    matched_issues = set()
+    for comment in comments_data:
+        for idx, issue in enumerate(known_issues):
+            if idx in matched_issues:
+                continue
+            if _line_proximity(comment.get("line_number", 0), issue["line_number"]) and \
+               _keywords_match(comment.get("description", ""), issue["keywords"]):
+                severity_bonus = SEVERITY_SCORES.get(issue["severity"], 0.05)
+                issue_reward += severity_bonus
+                matched_issues.add(idx)
+                break
+
+    breakdown["issue_detection"] = round(issue_reward, 4)
+
+    # --- Negative: penalty for false positives ---
+    fp_penalty = false_positives * FALSE_POSITIVE_PENALTY
+    breakdown["false_positive_penalty"] = round(fp_penalty, 4)
+
+    # --- Verdict correctness ---
+    if action.verdict == required_verdict:
+        breakdown["correct_verdict"] = CORRECT_VERDICT_BONUS
+    else:
+        breakdown["wrong_verdict"] = WRONG_VERDICT_PENALTY
+
+    # --- Small step efficiency penalty ---
+    step_pen = step_number * STEP_PENALTY
+    breakdown["step_penalty"] = round(step_pen, 4)
+
+    total = sum(breakdown.values())
+    total = round(max(-1.0, min(1.0, total)), 4)
+
+    message_parts = []
+    if issues_found > 0:
+        message_parts.append(f"Found {issues_found}/{len(known_issues)} known issues (+{issue_reward:.2f})")
+    if issues_missed > 0:
+        message_parts.append(f"Missed {issues_missed} issue(s)")
+    if false_positives > 0:
+        message_parts.append(f"{false_positives} false positive(s) ({fp_penalty:.2f})")
+    if action.verdict == required_verdict:
+        message_parts.append(f"Correct verdict '{action.verdict}' (+{CORRECT_VERDICT_BONUS})")
+    else:
+        message_parts.append(
+            f"Wrong verdict '{action.verdict}' (expected '{required_verdict}') ({WRONG_VERDICT_PENALTY})"
+        )
+
+    return Reward(
+        value=total,
+        breakdown=breakdown,
+        message=" | ".join(message_parts) if message_parts else "No reward signal",
+        issues_found=issues_found,
+        issues_missed=issues_missed,
+        false_positives=false_positives,
+    )

run_test.py

@@ -0,0 +1,47 @@
+"""
+Quick diagnostic: test all imports and run baseline for 'easy' task.
+"""
+import sys
+import traceback
+
+print("Python:", sys.version)
+print("Testing imports...")
+
+try:
+    from openai import OpenAI
+    print("[OK] openai")
+except ImportError as e:
+    print(f"[FAIL] openai: {e}")
+
+try:
+    from environment import CodeReviewEnv
+    print("[OK] environment")
+except Exception as e:
+    print(f"[FAIL] environment: {e}")
+    traceback.print_exc()
+
+try:
+    from graders import grade_episode
+    print("[OK] graders")
+except Exception as e:
+    print(f"[FAIL] graders: {e}")
+    traceback.print_exc()
+
+try:
+    from models import Action, CodeComment, GraderInput
+    print("[OK] models")
+except Exception as e:
+    print(f"[FAIL] models: {e}")
+    traceback.print_exc()
+
+print("\nAll import checks done.")
+
+# Now try running the actual baseline
+try:
+    import baseline
+    # Override sys.argv to simulate --task easy
+    sys.argv = ["baseline.py", "--task", "easy"]
+    baseline.main()
+except Exception as e:
+    print(f"\n[ERROR in main]: {e}")
+    traceback.print_exc()

tasks.py

@@ -0,0 +1,395 @@
+from typing import Dict, Any, List
+
+TASKS: Dict[str, Dict[str, Any]] = {
+    "easy": {
+        "id": "easy",
+        "name": "Basic Bug Detection",
+        "description": (
+            "Review a simple Python utility module. Find edge case bugs and "
+            "performance issues. The code looks functional at first glance but "
+            "has several critical and minor issues."
+        ),
+        "difficulty": "easy",
+        "max_steps": 3,
+        "pr_title": "Add calculate_statistics utility module",
+        "pr_description": (
+            "Adding utility functions for calculating statistics on lists of numbers. "
+            "Used by the analytics dashboard."
+        ),
+        "file_name": "utils/statistics.py",
+        "diff": """\
+--- a/utils/statistics.py
++++ b/utils/statistics.py
+@@ -0,0 +1,30 @@
++def calculate_average(numbers):
++    total = 0
++    for num in numbers:
++        total += num
++    return total / len(numbers)
++
++def calculate_max(numbers):
++    max_val = numbers[0]
++    for num in numbers:
++        if num > max_val:
++            max_val = num
++    return max_val
++
++def get_percentage(value, total):
++    return (value / total) * 100
++
++def find_duplicates(items):
++    seen = []
++    duplicates = []
++    for item in items:
++        if item in seen:
++            duplicates.append(item)
++        seen.append(item)
++    return duplicates
++
++def safe_divide(a, b):
++    if b != 0:
++        return a / b
++    else:
++        return 0
+""",
+        "known_issues": [
+            {
+                "line_number": 5,
+                "issue_type": "bug",
+                "severity": "critical",
+                "description": "ZeroDivisionError when numbers list is empty — len(numbers) returns 0",
+                "keywords": ["zero", "division", "empty", "len", "zerodivision", "divide by zero"],
+            },
+            {
+                "line_number": 8,
+                "issue_type": "bug",
+                "severity": "critical",
+                "description": "IndexError when numbers list is empty — numbers[0] raises IndexError",
+                "keywords": ["index", "indexerror", "empty", "list", "numbers[0]", "first element"],
+            },
+            {
+                "line_number": 14,
+                "issue_type": "bug",
+                "severity": "critical",
+                "description": "ZeroDivisionError in get_percentage when total is 0",
+                "keywords": ["zero", "division", "total", "percentage", "zerodivision"],
+            },
+            {
+                "line_number": 17,
+                "issue_type": "performance",
+                "severity": "minor",
+                "description": "Using list for 'seen' causes O(n^2) complexity — use a set for O(1) lookups",
+                "keywords": ["set", "performance", "o(n)", "o(n^2)", "lookup", "efficiency", "complexity"],
+            },
+        ],
+        "required_verdict": "request_changes",
+        "success_threshold": 0.6,
+    },
+
+    "medium": {
+        "id": "medium",
+        "name": "Security Vulnerability Review",
+        "description": (
+            "Review a user authentication module. Identify security vulnerabilities "
+            "including SQL injection, hardcoded credentials, weak cryptography, and "
+            "logic errors in permission checking."
+        ),
+        "difficulty": "medium",
+        "max_steps": 5,
+        "pr_title": "Add user authentication and database access layer",
+        "pr_description": (
+            "Implements user login, password reset, and role-based permission checks. "
+            "Connects to PostgreSQL for user data."
+        ),
+        "file_name": "auth/user_manager.py",
+        "diff": """\
+--- a/auth/user_manager.py
++++ b/auth/user_manager.py
+@@ -0,0 +1,52 @@
++import hashlib
++
++DB_PASSWORD = "admin123"
++API_SECRET = "supersecret_key_do_not_share"
++
++def get_user(user_id):
++    query = f"SELECT * FROM users WHERE id = {user_id}"
++    return execute_query(query)
++
++def login(username, password):
++    hashed = hashlib.md5(password.encode()).hexdigest()
++    query = f"SELECT * FROM users WHERE username = '{username}' AND password = '{hashed}'"
++    user = execute_query(query)
++    if user:
++        return {"status": "success", "user": user}
++    return {"status": "failed"}
++
++def reset_password(email, new_password):
++    if len(new_password) > 6:
++        query = f"UPDATE users SET password = '{new_password}' WHERE email = '{email}'"
++        execute_query(query)
++        return True
++    return False
++
++def delete_user(user_id):
++    execute_query(f"DELETE FROM users WHERE id = {user_id}")
++    return True
++
++def serialize_user(user_data):
++    import pickle
++    return pickle.dumps(user_data)
++
++def deserialize_user(data):
++    import pickle
++    return pickle.loads(data)
++
++def log_action(user, action):
++    log_entry = f"[{action}] User: {user}"
++    print(log_entry)
++
++def check_permission(user_role, required_role):
++    roles = ["user", "moderator", "admin"]
++    return roles.index(user_role) == roles.index(required_role)
+""",
+        "known_issues": [
+            {
+                "line_number": 3,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "Hardcoded database password in source code — use environment variables",
+                "keywords": ["hardcoded", "password", "credential", "environment variable", "env", "secret"],
+            },
+            {
+                "line_number": 4,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "Hardcoded API secret key in source code — use environment variables",
+                "keywords": ["hardcoded", "secret", "api key", "credential", "environment variable", "env"],
+            },
+            {
+                "line_number": 7,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "SQL injection — user_id interpolated directly into query, use parameterized queries",
+                "keywords": ["sql injection", "parameterized", "sanitize", "f-string", "interpolat", "injection"],
+            },
+            {
+                "line_number": 11,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "MD5 is cryptographically broken for password hashing — use bcrypt or argon2",
+                "keywords": ["md5", "bcrypt", "argon2", "hash", "cryptograph", "broken", "weak"],
+            },
+            {
+                "line_number": 12,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "SQL injection in login — username interpolated into query string",
+                "keywords": ["sql injection", "parameterized", "username", "injection", "interpolat"],
+            },
+            {
+                "line_number": 20,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "Password stored in plaintext — must be hashed before storage",
+                "keywords": ["plaintext", "hash", "password", "bcrypt", "plain text", "unhashed"],
+            },
+            {
+                "line_number": 33,
+                "issue_type": "security",
+                "severity": "critical",
+                "description": "pickle.loads() on untrusted data allows arbitrary code execution — use JSON instead",
+                "keywords": ["pickle", "arbitrary code", "unsafe", "deserialization", "json", "remote code"],
+            },
+            {
+                "line_number": 41,
+                "issue_type": "bug",
+                "severity": "major",
+                "description": "Permission check uses == instead of >= — admin cannot access user-level resources",
+                "keywords": ["permission", ">=", "role", "hierarchy", "comparison", "greater", "equal"],
+            },
+        ],
+        "required_verdict": "request_changes",
+        "success_threshold": 0.45,
+    },
+
+    "hard": {
+        "id": "hard",
+        "name": "Concurrency & Architecture Bug Hunt",
+        "description": (
+            "Review a distributed rate limiter and async task queue. Find subtle "
+            "concurrency bugs, race conditions, silent exception swallowing, "
+            "mutation bugs, and architectural issues that only manifest under load."
+        ),
+        "difficulty": "hard",
+        "max_steps": 8,
+        "pr_title": "Implement distributed rate limiter and async task queue",
+        "pr_description": (
+            "Adds a rate limiter and background task queue for handling "
+            "high-throughput job processing with retry logic."
+        ),
+        "file_name": "core/rate_limiter.py",
+        "diff": """\
+--- a/core/rate_limiter.py
++++ b/core/rate_limiter.py
+@@ -0,0 +1,82 @@
++import time
++import threading
++from collections import defaultdict
++
++class RateLimiter:
++    def __init__(self, max_requests, window_seconds):
++        self.max_requests = max_requests
++        self.window_seconds = window_seconds
++        self.requests = defaultdict(list)
++        self.lock = threading.Lock()
++
++    def is_allowed(self, user_id):
++        now = time.time()
++        window_start = now - self.window_seconds
++        with self.lock:
++            self.requests[user_id] = [
++                t for t in self.requests[user_id] if t > window_start
++            ]
++            if len(self.requests[user_id]) < self.max_requests:
++                self.requests[user_id].append(now)
++                return True
++            return False
++
++class TaskQueue:
++    def __init__(self, max_workers=4):
++        self.queue = []
++        self.max_workers = max_workers
++        self.workers = []
++        self.running = False
++
++    def add_task(self, task_fn, *args):
++        self.queue.append((task_fn, args))
++
++    def _worker(self):
++        while self.running:
++            if self.queue:
++                task_fn, args = self.queue.pop(0)
++                try:
++                    task_fn(*args)
++                except Exception:
++                    pass
++            time.sleep(0.01)
++
++    def start(self):
++        self.running = True
++        for _ in range(self.max_workers):
++            t = threading.Thread(target=self._worker)
++            t.start()
++            self.workers.append(t)
++
++    def stop(self):
++        self.running = False
++
++class RetryQueue:
++    def __init__(self, max_retries=3):
++        self.max_retries = max_retries
++        self.failed_tasks = {}
++        self.retry_counts = defaultdict(int)
++
++    def add_failed(self, task_id, task_fn):
++        self.failed_tasks[task_id] = task_fn
++
++    def retry_all(self):
++        for task_id, task_fn in self.failed_tasks.items():
++            if self.retry_counts[task_id] < self.max_retries:
++                try:
++                    task_fn()
++                    del self.failed_tasks[task_id]
++                except Exception:
++                    self.retry_counts[task_id] += 1
++
++    def get_stats(self):
++        return {
++            "pending": len(self.failed_tasks),
++            "total_retries": sum(self.retry_counts.values())
++        }
++
++def process_batch(items, batch_size=100):
++    results = []
++    for i in range(0, len(items), batch_size):
++        batch = items[i:i+batch_size]
++        result = process_items(batch)
++        results.append(result)
++    return results
++
++def merge_configs(base_config, override_config):
++    merged = base_config
++    for key, value in override_config.items():
++        merged[key] = value
++    return merged
+""",
+        "known_issues": [
+            {
+                "line_number": 9,
+                "issue_type": "bug",
+                "severity": "critical",
+                "description": "RateLimiter uses in-memory storage — not actually distributed, state not shared across processes or instances",
+                "keywords": ["distributed", "in-memory", "shared", "redis", "instance", "process", "not distributed"],
+            },
+            {
+                "line_number": 25,
+                "issue_type": "bug",
+                "severity": "critical",
+                "description": "TaskQueue.queue is a plain list with no lock — race condition when multiple workers call queue.pop(0) simultaneously",
+                "keywords": ["race condition", "thread", "lock", "synchronization", "queue", "concurrent", "list", "pop"],
+            },
+            {
+                "line_number": 39,
+                "issue_type": "bug",
+                "severity": "major",
+                "description": "Exceptions silently swallowed with bare except pass — failed tasks are lost with no logging or retry",
+                "keywords": ["exception", "silent", "swallow", "pass", "log", "lost", "bare except", "suppress"],
+            },
+            {
+                "line_number": 49,
+                "issue_type": "bug",
+                "severity": "major",
+                "description": "stop() sets running=False but never calls thread.join() — threads may still execute after stop() returns",
+                "keywords": ["join", "thread", "stop", "shutdown", "daemon", "join()", "running"],
+            },
+            {
+                "line_number": 63,
+                "issue_type": "bug",
+                "severity": "critical",
+                "description": "retry_all() deletes from failed_tasks while iterating over it — RuntimeError: dictionary changed size during iteration",
+                "keywords": ["dictionary", "iteration", "modify", "runtimeerror", "list()", "copy", "iterate", "delete"],
+            },
+            {
+                "line_number": 73,
+                "issue_type": "bug",
+                "severity": "major",
+                "description": "process_batch uses append instead of extend — returns list of batch-results not individual results, wrong data shape",
+                "keywords": ["extend", "append", "batch", "result", "granularity", "list", "shape"],
+            },
+            {
+                "line_number": 79,
+                "issue_type": "bug",
+                "severity": "major",
+                "description": "merge_configs mutates base_config in-place — merged = base_config is a reference, not a copy; use base_config.copy()",
+                "keywords": ["copy", "reference", "mutate", "shallow copy", "dict", "base_config", "in-place"],
+            },
+            {
+                "line_number": 25,
+                "issue_type": "performance",
+                "severity": "minor",
+                "description": "Use collections.deque instead of list for O(1) popleft() vs O(n) pop(0)",
+                "keywords": ["deque", "o(1)", "performance", "popleft", "list", "collections"],
+            },
+        ],
+        "required_verdict": "request_changes",
+        "success_threshold": 0.35,
+    },
+}
+
+
+def get_task(task_id: str) -> Dict[str, Any]:
+    return TASKS.get(task_id, {})
+
+
+def get_all_tasks() -> List[Dict[str, Any]]:
+    return list(TASKS.values())