README.md

---
title: AI Code Review OpenEnv
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: docker
app_file: inference.py
pinned: false
---

# 🔍 AI Code Review — OpenEnv RL Environment

> Train and evaluate AI agents on real-world code review using structured RL environments.

A **real-world reinforcement learning environment** where AI agents learn structured code review:
**identify bugs → reason about them → fix them correctly**.

Built for the **Meta × Hugging Face × OpenEnv × Scaler Hackathon 2026 (Round 1 Submission)**.

---

## 🚀 Why This Matters

Most AI coding tools are built for *generation*, not *review*.
But in real-world software engineering, that's the wrong priority:

* Code review is the last line of defence against production bugs
* Bugs often slip through because reviewers miss the *why*, not the *what*
* Understanding why code is wrong matters more than just patching it

👉 This environment turns **code review into a measurable, learnable RL problem** — with structure, reward signal, and reproducibility built in from the ground up.

---

## 🧠 What Makes This Environment Strong

| Property                  | Detail                                                 |
| ------------------------- | ------------------------------------------------------ |
| ✅ Real-world task         | Code review, not a toy problem                         |
| ✅ Two-phase reasoning     | Agent must identify before it can fix                  |
| ✅ Deterministic grading   | No LLM judge, no randomness                            |
| ✅ Reward shaping          | Partial progress is rewarded at every step             |
| ✅ Reproducible evaluation | Same input always produces same score                  |
| ✅ Spec compliant          | Fully compatible with OpenEnv + hackathon requirements |

---

## 🏗️ Environment Design

Implements the full OpenEnv interface — three methods, no more:

```python
obs      = env.reset()       # Start a new episode
result   = env.step(action)  # Execute one action → {observation, reward, done, info}
metadata = env.state         # Inspect episode state
```

---

## 📥 Observation Space

| Field        | Description                         |
| ------------ | ----------------------------------- |
| `code`       | Buggy JavaScript code to review     |
| `task`       | Task description + current phase    |
| `history`    | Log of all prior steps this episode |
| `task_id`    | Task number (1, 2, or 3)            |
| `language`   | `javascript`                        |
| `difficulty` | `easy` / `medium` / `hard`          |
| `category`   | `syntax` / `logic` / `performance`  |

---

## 🎮 Action Space

| Field         | Values                        |
| ------------- | ----------------------------- |
| `action_type` | `"identify"` or `"fix"`       |
| `content`     | Free-text explanation or code |

---

## 🔁 Episode Flow

```
identify → fix → identify → fix → identify → fix
   task 1            task 2            task 3
```

One episode covers all 3 tasks in order.

---

## 🧪 Tasks

### 🟢 Task 1 — Syntax Error (Easy)

```javascript
// buggy
function add(a, b {
  return a + b;
}

// fixed
function add(a, b) {
  return a + b;
}
```

---

### 🟡 Task 2 — Logic Bug (Medium)

```javascript
// buggy
function isEven(n) {
  return n % 2 === 1;
}

// fixed
function isEven(n) {
  return n % 2 === 0;
}
```

---

### 🔴 Task 3 — Performance Issue (Hard)

```javascript
// buggy
for (let i = 0; i < arr.length; i++) {
  console.log(arr[i]);
}

// fixed
arr.forEach(item => {
  console.log(item);
});
```

---

## 🎯 Reward System

### ✔ Deterministic & Reproducible

No randomness. Same input → same score.

---

### 🧩 Identify Reward

```
score = matched_keywords / total_keywords
```

---

### 🛠️ Fix Reward

| Condition             | Score |
| --------------------- | ----- |
| Exact match           | 1.0   |
| Whitespace-normalised | 0.9   |
| ≥ 80% token overlap   | 0.6   |
| ≥ 50% token overlap   | 0.3   |
| Below 50%             | 0.0   |

---

### 🎁 Identify Bonus

+0.1 bonus if identify ≥ 0.4
(final reward clamped to 1.0)

---

### ⚠️ Edge Case Handling

| Situation              | Effect          |
| ---------------------- | --------------- |
| Skipped identify phase | fix score × 0.5 |
| Unknown `action_type`  | 0.0             |
| Repeated identify      | 0.0             |

---

### 📊 Max Reward Per Episode

| Scope        | Max Reward |
| ------------ | ---------- |
| Per step     | 1.0        |
| Per task     | ~2.0       |
| Full episode | ~6.0       |

---

## 🤖 Inference Pipeline

```
1. Read HF_TOKEN, API_BASE_URL, MODEL_NAME
2. Initialise OpenAI client + environment
3. Reset environment
4. Identify → Fix for each task
5. Log results in required format
```

---

## 🧾 Log Format (Strict)

```
[START]
[STEP]
[STEP]
...
[END]
```

---

## 📊 Example Output

```
[START] task=code-review env=ai-code-review-env model=gpt-4.1-mini
[STEP] step=1 action=identify:task1 reward=0.71 done=false error=null
[STEP] step=2 action=fix:task1 reward=1.00 done=false error=null
[STEP] step=3 action=identify:task2 reward=0.60 done=false error=null
[STEP] step=4 action=fix:task2 reward=1.00 done=false error=null
[STEP] step=5 action=identify:task3 reward=0.45 done=false error=null
[STEP] step=6 action=fix:task3 reward=1.00 done=true error=null
[END] success=true steps=6 rewards=0.71,1.00,0.60,1.00,0.45,1.00
```

---

## 📌 Why This Stands Out

* Real-world problem (not synthetic)
* Deterministic evaluation (fair + reproducible)
* Reward shaping (better learning signal)
* Two-phase reasoning (prevents shortcutting)

---

## ⚙️ Setup

```bash
pip install -r requirements.txt
export HF_TOKEN=your_token
python inference.py
```

---

## 📁 Project Structure

```
code-review-env/
├── inference.py
├── openenv.yaml
├── Dockerfile
├── README.md
└── env/
```

---

## 🏆 Evaluation Alignment

| Criteria           | Covered |
| ------------------ | ------- |
| Real-world utility | ✅       |
| Task design        | ✅       |
| Grader quality     | ✅       |
| Environment design | ✅       |
| Spec compliance    | ✅       |

---

## 📄 License

MIT License
Built for Meta × Hugging Face × OpenEnv × Scaler Hackathon 2026