README copy.md

# DebugOps: AI Incident Response Environment

## Overview

**DebugOps** is a reinforcement-learning environment that simulates real-world production incident response. An AI agent acts as an on-call Site Reliability Engineer (SRE), diagnosing system failures from noisy logs and degraded metrics, then executing the correct multi-step remediation sequence.

The environment models real DevOps/SRE workflows from cloud infrastructure and distributed systems — unlike toy environments, fixes require correct *ordered* sequences and the agent must separate signal from noise in log data.

Built with the [OpenEnv](https://github.com/raun/openenv-course) framework for the Meta × PyTorch Hackathon.

---

## Environment Description

At each step the agent receives an **observation** containing:
- **services** — per-service health (api, db, cache)
- **logs** — system log lines, some of which are noisy red herrings
- **metrics** — latency (ms), error_rate (0–1), cpu (%)
- **time_step** — elapsed steps

The agent must identify the hidden root cause and execute the correct multi-step fix sequence before the episode times out. System metrics degrade every step, creating real urgency.

---

### Observation Space

| Field | Type | Description |
|---|---|---|
| `services` | `Dict[str, str]` | Service health: `healthy` or `degraded` |
| `logs` | `List[str]` | System logs (may include noise/red herrings) |
| `metrics.latency` | `float` | Current system latency (ms) |
| `metrics.error_rate` | `float` | Error rate (0.0–1.0) |
| `metrics.cpu` | `float` | CPU utilisation (%) |
| `time_step` | `int` | Steps elapsed in this episode |

---

### Action Space (discrete, 5 actions)

| Action | Description |
|---|---|
| `restart_api` | Restart the API service |
| `restart_db` | Restart the database service |
| `restart_cache` | Restart the cache service |
| `scale_up` | Add compute capacity |
| `noop` | Take no action |

---

### Root Causes & Fix Sequences

| Root Cause | Fix Sequence | Affected Services |
|---|---|---|
| `api_timeout` | `scale_up → restart_api` | api |
| `db_connection_leak` | `restart_db → scale_up` | db |
| `cache_miss_storm` | `restart_cache → scale_up` | cache |
| `memory_leak` | `restart_api → restart_db` | api, db |

Actions must be performed **in order** — wrong steps degrade metrics further.

---

### Reward Function

```
reward =
  +150   full resolution bonus
  + 30   correct intermediate fix step
  - 15   wrong action (no progress)
  -0.04  × latency (ms) per step
  - 25   × error_rate per step
  -  2   time penalty per step (escalates after step 10)
```

The shaped reward provides dense signal throughout the episode, not just at termination.

---

## Tasks

| Task | Module | Description | Max Steps | Difficulty |
|---|---|---|---|---|
| Simple | `tasks.task_simple` | Single-service failure | 15 | Low |
| Multi-Service | `tasks.task_multi_service` | Two services degrade simultaneously | 12 | Medium |
| Critical | `tasks.task_critical` | Memory-leak with misleading logs + SLA penalties | 10 | High |

---

## Setup

```bash
# Clone the repo
git clone <your-repo-url>
cd debugops

# Install dependencies
pip install -r requirements.txt
```

---

## Running Inference (LLM Agent)

```bash
python inference.py
```

Expected output:

```
[START] task=simple env=debugops model=Qwen/Qwen2.5-72B-Instruct
[STEP] step=0 action=scale_up reward=-18.4 done=false error=null
[STEP] step=1 action=restart_api reward=145.3 done=true error=null
[END] success=true steps=2 score=0.881 rewards=-18.4,145.3
```

If `HF_TOKEN` is not set, the heuristic fallback agent is used automatically — no API key required.

---

## Running the Baseline Agent

```bash
python app.py
```

---

## Grader

```bash
python -m grader.grader
```

The grader scores each episode in `[0.0, 1.0]` using a weighted formula:

| Component | Weight | Description |
|---|---|---|
| Resolution | 50% | Was the incident resolved? |
| Efficiency | 30% | How quickly was it resolved? |
| Quality | 20% | Normalised average reward |

---

## Pre-submission Validation

```bash
python scripts/validate_submission.py
# or
bash scripts/validate-submission.sh
```

---

## Docker

```bash
# Build
docker build -t debugops-env .

# Run (heuristic agent, no key required)
docker run debugops-env

# Run with LLM agent
docker run -e HF_TOKEN=your_token \
           -e API_BASE_URL=https://router.huggingface.co/v1 \
           -e MODEL_NAME=Qwen/Qwen2.5-72B-Instruct \
           debugops-env
```

---

## Project Structure

```
.
├── inference.py              ← Main LLM inference script (required entry point)
├── app.py                    ← Lightweight local runner (baseline agent)
├── Dockerfile
├── requirements.txt
├── openenv.yaml              ← OpenEnv spec
├── README.md
├── env/
│   ├── environment.py        ← DebugEnv class (reset/step/state)
│   ├── dynamics.py           ← State transition logic
│   ├── reward.py             ← Shaped reward function
│   └── incident_generator.py ← Random incident sampling
├── tasks/
│   ├── task_simple.py
│   ├── task_multi_service.py
│   └── task_critical.py
├── agent/
│   └── baseline.py           ← Heuristic baseline agent
├── grader/
│   └── grader.py             ← Episode evaluator → score in [0, 1]
└── scripts/
    ├── validate_submission.py
    └── validate-submission.sh
```

---

## Design Decisions

**Why multi-step fix sequences?** Single-action fixes are trivially solved by keyword matching. Requiring ordered sequences forces the agent to model state transitions, not just classify root causes.

**Why noisy logs?** Real production systems always emit irrelevant log lines. An agent that cannot filter noise will be unreliable.

**Why escalating time penalties?** Incident SLAs are real constraints — an agent that solves the issue in 10 steps is materially worse than one that solves it in 2.

**Why shaped rewards?** Sparse rewards (terminal-only) are notoriously hard to learn from. Continuous metric penalties and partial-progress bonuses provide useful gradient signal at every step.