README.md

---
title: CodeDebugger
emoji: 🐛
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
---

# 🐛 CodeDebugger

**An RL Environment for LLM-Powered Python Code Debugging**

*Meta + Scalar OpenEnv Hackathon 2026*

---

## 🎯 What Is This?

CodeDebugger is an OpenEnv-compliant reinforcement learning environment where an LLM agent:

1. **Sees** buggy Python code + error messages
2. **Proposes** fixes step by step
3. **Gets rewarded** based on how many test cases pass after each fix

The agent learns to debug code through trial and error, with an 8-component reward function and anti-reward-hacking safeguards.

---

## 🏗️ Architecture

```
┌─────────────────────────────────────────────────────────┐
│                    Orchestrator                          │
│  ┌──────────────┐    ┌──────────────┐    ┌───────────┐ │
│  │  Bug Dataset  │───▶│  Environment │───▶│  Executor │ │
│  │  (30 problems)│    │  (OpenEnv)   │    │ (Sandbox) │ │
│  └──────────────┘    └──────┬───────┘    └───────────┘ │
│                             │                           │
│              ┌──────────────┼──────────────┐           │
│              ▼              ▼              ▼           │
│       ┌───────────┐  ┌──────────┐  ┌───────────────┐  │
│       │  LLM Fixer│  │  Reward  │  │  Anti-Hacking │  │
│       │  (3-tier) │  │ (8 comp) │  │  Critic (4 ck)│  │
│       └───────────┘  └──────────┘  └───────────────┘  │
└─────────────────────────────────────────────────────────┘
```

---

## 📦 Project Structure

```
codedebugger/
├── env/
│   ├── codedebugger_env.py     # OpenEnv environment + FastAPI
│   └── executor.py             # Sandboxed code runner (subprocess)
├── agents/
│   ├── fixer.py                # LLM fixer agent (3-tier: Groq → simplified → heuristic)
│   ├── reward.py               # 8-component reward function
│   └── critic.py               # 4 anti-hacking checks
├── data/
│   └── bug_dataset.py          # 30 buggy problems + test cases
├── training/
│   ├── train_grpo.py           # GRPO training script (Unsloth + TRL)
│   ├── run_baseline.py         # Baseline evaluation
│   └── train_colab.ipynb       # Google Colab notebook
├── utils/
│   └── prompts.py              # All LLM prompts
├── outputs/                    # Results saved here
├── app.py                      # Streamlit UI (3 tabs)
├── orchestrator.py             # Main debug loop controller
├── openenv.yaml                # OpenEnv registry manifest
├── Dockerfile                  # HF Spaces deployment
├── requirements.txt
└── README.md
```

---

## 🚀 Quick Start

### 1. Install Dependencies

```bash
cd codedebugger
pip install -r requirements.txt
```

### 2. Set API Key

```bash
export GROQ_API_KEY="your-key-here"
```

### 3. Run Baselines

```bash
python training/run_baseline.py
```

### 4. Launch Streamlit UI

```bash
streamlit run app.py
```

### 5. Start FastAPI Server

```bash
uvicorn env.codedebugger_env:app --host 0.0.0.0 --port 8000
```

Then interact via HTTP:

```bash
# Health check
curl http://localhost:8000/health

# Reset with a problem dict
curl -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"id": "bug_001", "difficulty": "easy", ...}'

# Step with fixed code
curl -X POST http://localhost:8000/step \
  -H "Content-Type: application/json" \
  -d '{"code": "def get_last(lst):\n    return lst[-1]"}'

# Render current state
curl http://localhost:8000/render
```

---

## 🎯 Reward Function (8 Components)

| Component | Range | Description |
|-----------|-------|-------------|
| **test_score** | 0–50 | Proportional to test cases passed, +10 bonus for 100% |
| **fix_quality** | 0–20 | Rewards minimal, targeted changes (1–3 lines = 20 pts) |
| **format_score** | 0–15 | Valid AST, preserved function names, consistent style |
| **speed_score** | 0–10 | Rewards solving in fewer iterations (10/7/4 for iter 1/2/3) |
| **safety_score** | 0–10 | No forbidden imports or dangerous patterns |
| **improvement_score** | 0–15 | Rewards passing more tests than previous iteration |
| **anti_hack_penalty** | ≤ 0 | Negative penalty for detected cheating |
| **process_bonus** | 0–3 | Bonus for fix addressing the root cause |

**Total range: 0–120** (clamped to ≥ 0)

---

## 🛡️ Anti-Reward-Hacking (4 Checks)

| Check | What It Detects | Penalty |
|-------|----------------|---------|
| **C1: Hardcoded Returns** | If-chains returning literal test answers (≥2 matches, excludes constants) | −30 |
| **C2: Function Deletion** | Removal or renaming of the original function signature | −25 |
| **C3: Trivial Pass** | Suspiciously fast execution (all pass in <0.01s) | −20 |
| **C4: Code Gutting** | Submitted code is <30% of original length, or trivial (`pass`/`return None`) | −35 |

---

## 📊 Dataset

30 buggy Python problems across 3 difficulty levels:

- **Easy** (10 problems): off-by-one, missing return, wrong operator, wrong method, etc.
- **Medium** (10 problems): wrong base case, wrong sort order, missing edge case, wrong range, etc.
- **Hard** (10 problems): binary search, DP transitions, graph traversal, backtracking, etc.

Each problem includes:
- Buggy code with a single injected bug
- Known correct solution
- 4 test cases
- Error type classification
- Hint for debugging

---

## 🏋️ Training with GRPO

### Local (requires GPU)

```bash
python training/train_grpo.py
```

- **Model:** `unsloth/Llama-3.2-1B-Instruct` (4-bit quantized)
- **Method:** GRPO via TRL + Unsloth
- **Steps:** 100 (configurable)
- **GPU:** Tesla T4 or better

### Google Colab

Open `training/train_colab.ipynb` in Colab with a T4 GPU.

---

## 📈 Results

### Baseline (Groq LLM — llama-3.1-8b-instant)

| Metric | Baseline | After GRPO Training |
|--------|----------|---------------------|
| Solved | 25/30 (83%) | 23/30 (77%) |
| Peak Reward | 88 | 100 |
| Avg Reward | 75.2 | ~65 (100 steps) |
| RL Evidence | Starting point | Reward 76→100 in epoch 1 |

### By Difficulty

| Difficulty | Baseline Solved | Baseline Avg Reward |
|------------|-----------------|---------------------|
| Easy | 9/10 | 79.2 |
| Medium | 8/10 | 67.9 |
| Hard | 8/10 | 78.6 |

---

## 🐳 Deploy to Hugging Face Spaces

```bash
# Build and test locally
docker build -t codedebugger .
docker run -p 7860:7860 codedebugger

# Push to HF Spaces
# 1. Create a Space on huggingface.co (Docker type)
# 2. Push this repo to the Space's git remote
```

---

## 📄 License

MIT

---

*Built with ❤️ for the Meta + Scalar OpenEnv Hackathon 2026*