---
title: CodeReviewEnv
emoji: 🛡️
colorFrom: yellow
colorTo: red
sdk: docker
pinned: false
tags:
  - openenv
---

# CodeReviewEnv

An RL environment for vulnerability triage, built on real CVE data from the NVD.

The idea: most RL envs are toy problems (gridworld, cartpole, etc). We wanted something closer to what devs actually deal with — triaging security patches across a codebase with limited time and attention.

The agent gets a stream of files from a real CVE patch and has to decide: **flag** this file for human review, or **skip** it. There's a fixed review budget so you can't just flag everything.

* **HF Space**: https://huggingface.co/spaces/lucid987654/code-review-env
* **GitHub**: https://github.com/subwaycookiecrunch/Meta-project

---

### Data

1715 files across 65 CVEs scraped from actual GitHub vulnerability patches. Each file has four features extracted from the commit history: churn, complexity, TODO count, and recency.

### Rewards

Asymmetric on purpose — missing a real bug is worse than wasting a review slot on a clean file.

| Outcome | Reward | Why |
|---------|--------|-----|
| True Positive | +1.0 | found a real bug |
| True Negative | +0.8 | correctly skipped clean file |
| False Positive | -0.4 | wasted budget on safe file |
| False Negative | -0.2 | missed a bug |
| Over-budget flag | -0.5 | budget is a hard limit |

### Tasks

Three difficulty levels based on repo size:

- **easy**: ≤15 files, generous budget
- **medium**: 16-29 files
- **hard**: 30+ files, tight budget — agent really has to pick its spots

Grading is F1 score (precision × recall), always in [0, 1].

---

### Observation fields

Each step gives you:
- `file_path`, `file_index`, `total_files`, `files_remaining`
- `churn_score`, `complexity_score`, `todo_score`, `recency_score`
- `review_budget`, `files_flagged`
- `difficulty_level`, `cve_id`, `repo_name`
- terminal: `precision`, `recall`, `f1_score`, `true_positives`, `false_positives`, etc.

Action is just `{"decision": "flag"}` or `{"decision": "skip"}`.

---

## Running it

**Install:**
```bash
pip install openenv-core openai
```

**Docker:**
```bash
docker build -t codereviewenv .
docker run -p 7860:7860 codereviewenv
```

**Inference:**
```bash
export HF_TOKEN="your_token"
python inference.py
```

---

## Agents

### LLM baseline (`inference.py`)

Sends the file stats to Qwen2.5-Coder-32B via the HF inference API and asks it to flag or skip. Runs all three difficulty levels.

Rough zero-shot numbers:

| Difficulty | F1 | Precision | Recall |
|---|---|---|---|
| Easy | ~0.15 | ~0.12 | ~0.25 |
| Medium | ~0.10 | ~0.08 | ~0.18 |
| Hard | ~0.08 | ~0.06 | ~0.15 |

Lots of room to improve — the LLM has no training signal, it's just guessing from feature names.

```bash
export HF_TOKEN="your_token"
python inference.py
```

### PyTorch agent (`train_pytorch_agent.py`)

REINFORCE with a 3-layer MLP. Takes the 6 observation features as input, outputs flag/skip probabilities. Trains directly against the env reward signal.

```bash
pip install torch
python train_pytorch_agent.py
```

## File layout

- `Dockerfile` + `openenv.yaml` — deployment config
- `inference.py` — LLM baseline (hackathon submission script)
- `train_pytorch_agent.py` — pytorch RL agent
- `models.py` — pydantic action/observation/state types
- `server/environment.py` — core env logic + reward math
- `data/` — the CVE dataset

MIT License