README.md

---
title: ConfigDebugEnv
colorFrom: red
colorTo: yellow
sdk: docker
app_port: 7860
tags:
  - openenv
  - devops
  - configuration
  - debugging
  - reinforcement-learning
  - multi-task-rl
pinned: false
---

# ConfigDebugEnv: Multi-Task RL Environment for Configuration Debugging

> **Challenge**: Train AI agents to autonomously debug broken configuration files across 7 real-world DevOps formats. This is a **multi-step sequential decision-making problem** requiring iterative reasoning and progressive understanding.

## Why This Matters

**The Problem**: Configuration errors cause ~40% of production incidents and are among the hardest to debug manually because they require domain expertise across multiple technologies (JSON, YAML, Docker, Kubernetes, nginx, etc.).

**Why RL?**: Static rule-based fixes fail because:
- Bugs interact (fixing one reveals another)
- Context matters (nginx syntax differs from Kubernetes)
- Validation is semantic (type checker would pass some errors)

**ConfigDebugEnv's Solution**: Train agents with **partial rewards** to discover multi-step fixes:
- Agent attempts fix → gets 0.4 reward + error guidance
- Agent iterates → gets 0.7 reward + next error guidance
- Agent completes → gets 1.0 reward + advances to next task

This mirrors real-world debugging where solutions emerge through iteration, not revelation.

---

## Environment Design

### Action Space
```
ConfigDebugAction.fixed_config: str  # Corrected configuration
```

### Observation Space
```
{
  "broken_config": str,           # Current broken config
  "file_type": str,               # Format: json, yaml, dockerfile, etc
  "error_message": str,           # Specific guidance (e.g., "replicas must be int")
  "task_id": str,                 # Current task ID
  "task_description": str,        # Human-readable task
  "difficulty": str,              # easy, medium, hard, very_hard
  "num_bugs": int,                # Total bugs in this config
  "bugs_found_so_far": int,       # Bugs fixed in this attempt
  "previous_reward": float,       # Last reward: [0.0, 1.0]
}
```

### Reward Structure (Sequential Decision Making)

Each task has **3 progressive levels**:

| Level | Example (Kubernetes) | Reward | Guides Next Attempt |
|-------|----------------------|--------|---------------------|
| L1    | Fix replicas type    | 0.4    | "containerPort must be int" |
| L2    | Fix port type too    | 0.7    | "cpu must include unit (m)" |
| L3    | Fix all bugs         | 1.0    | ✅ Task complete, advance |

**Why this works**: Agents learn to read error messages, make targeted fixes, and build on partial success—exactly like human debugging.

**⚠️ Important**: Hard tasks (Kubernetes, Nginx) **require sequential reasoning and cannot be solved in a single step**. Perfect first-attempt solutions will not occur; agents must iterate based on error guidance.

---

## The 7 Tasks (Progressive Complexity)

### Benchmark Quality Design
Each task is carefully crafted with **realistic, interdependent bugs** that require progressive debugging:

| Task | Format | Difficulty | Bugs | Description | Requires Multi-Step Fix? |
|------|--------|------------|------|-------------|------------------------|
| **task1_json** | JSON | Medium | 3 | Microservice config: missing comma, env structure bug, volumes structure bug | ✓ Yes |
| **task2_yaml** | YAML | Medium | 3 | CI/CD pipeline: indentation error, env array→object, missing job timeouts | ✓ Yes |
| **task3_dockerfile** | Dockerfile | Medium | 3 | Multi-stage build: base image, build args, runtime setup | ✓ Progressive |
| **task4_compose** | Docker-Compose | Medium | 4 | Service mesh: compose syntax, volumes, service networking | ✓ Progressive |
| **task5_k8s** | Kubernetes | Hard | 3 | Deployment manifest: type errors, missing fields, configuration validation | ✓ Yes |
| **task6_github_actions** | GitHub Actions | Hard | 5 | Workflow automation: YAML syntax, job dependencies, environment configuration | ✓ Yes |
| **task7_nginx** | Nginx config | Very Hard | 3 | Reverse proxy: syntax (semicolons), protocol prefix, routing headers | ✓ Yes |

### Grading Philosophy
Graders use **progressive, dependency-aware validation**:
- **Level 1**: Syntax pass/fail (foundational)
- **Level 2**: Structure validation (builds on syntax pass)
- **Level 3**: Semantic correctness (builds on structure pass)

Rewards are **emergent from fixes**, not hand-tuned. Example for task1_json:
- Syntax error only: 0.05 (penalty state)
- Syntax fixed: +0.3 → 0.35
- Structure fixed: +0.25 → 0.60
- All semantics fixed: +0.35 → 0.95 ✅

---

## The 7 Tasks (Original Overview)

| Task | Format | Difficulty | Bugs | Key Challenge |

---

## Multi-Task Learning

The environment progresses through all 7 tasks sequentially:
1. Agent learns from task1 → task2 → ... → task7
2. Each task builds on previous knowledge
3. Harder tasks should show better reasoning (agents see more diverse error types)

---

## API Endpoints

### Core OpenEnv Endpoints
- `POST /reset` - Reset environment to task1
- `POST /step` - Submit action (fixed config) → get reward + next observation
- `GET /observation` - Get current observation
- `GET /state` - Get full environment state

### Utility Endpoints
- `GET /metadata` - Environment specification (auto-generated by OpenEnv)
- `GET /info` - Service info
- `GET /health` - Health check
- `GET /tasks` - List all tasks with metadata

---

## Example Session

```python
from server.config_debug_environment import ConfigDebugEnvironment
from server.models import ConfigDebugAction

env = ConfigDebugEnvironment()
obs = env.reset()  # Start at task1_json

# Attempt 1: Agent tries initial fix
action = ConfigDebugAction(fixed_config='{"key": "value"}')
obs = env.step(action)
print(f"Reward: {obs.reward}")        # → 0.4 (partial credit)
print(f"Error: {obs.error_message}")  # → Guides next attempt

# Attempt 2: Agent learns and improves
action = ConfigDebugAction(fixed_config='{"key": "value", "number": 42}')
obs = env.step(action)
print(f"Reward: {obs.reward}")        # → 0.7
print(f"Error: {obs.error_message}")  # → Final hint

# Attempt 3: Agent completes
action = ConfigDebugAction(fixed_config='{"key": "value", "number": 42, "enabled": true}')
obs = env.step(action)
print(f"Reward: {obs.reward}")        # → 1.0 ✅
print(f"Done: {obs.done}")            # → False (more tasks remain)
obs = obs.task_id                     # → task2_yaml
```

---

## Key Strengths

✅ **True RL Problem**: Partial rewards with sequential decision-making  
✅ **Multi-Step Reasoning**: Errors cascade; fixes must be iterative  
✅ **Domain-Diverse**: 7 config formats = varied error types  
✅ **Scalable Difficulty**: Easy tasks build foundation for hard tasks  
✅ **Real-World Relevance**: Configuration bugs are a major DevOps pain point  

---

## Technical Stack

- **Framework**: OpenEnv (FastAPI-based)
- **Language**: Python 3.12+
- **Graders**: Domain-specific validators (YAML, JSON, Docker, K8s, nginx)
- **Deployment**: Docker + Hugging Face Spaces

---

## Getting Started

```bash
# Install dependencies
pip install -r requirements.txt

# Run tests
python test_env.py

# Start server
uvicorn server.app:app --reload

# Access at http://localhost:8000
```

---

## Evaluation Criteria

- ✅ **Correct Task Progression** - All 7 tasks with proper graders
- ✅ **Multi-Step Learning** - Agents show iterative improvement  
- ✅ **Error Guidance** - Clear feedback directs next fix
- ✅ **Reward Semantics** - Partial credit enables intermediate learning

---

## For Judges

This project demonstrates **advanced RL environment design**:

1. **Problem Formulation**: Configuration debugging is inherently sequential and iterative
2. **Reward Engineering**: Partial rewards guide exploration; not just binary pass/fail
3. **Task Curriculum**: 7 tasks from easy→hard with consistent structure
4. **Error Pedagogy**: Error messages teach agents what to fix next
5. **Scalability**: Framework generalizes to any text-based debugging task

ConfigDebugEnv isn't just a benchmark—it's a **learning tool** that teaches agents to think like human debuggers.

---

**Built for**: Meta PyTorch OpenEnv Hackathon x SST  
**Version**: 1.0.0
- GET /docs - Swagger API docs

## Setup

Run locally:

    pip install fastapi uvicorn pydantic pyyaml httpx openai gradio
    uvicorn server.env:app --host 0.0.0.0 --port 7860

Run with Docker:

    docker build -t config-debug-env .
    docker run -p 7860:7860 config-debug-env

## Baseline Results

Qwen/Qwen2.5-72B-Instruct: 7/7 tasks, score = 1.000