README.md

# CI/CD Pipeline Debugger Environment (OpenEnv)

## 1. Project Goal

This repository implements an AI training and evaluation environment where an agent learns to debug broken CI/CD pipelines automatically.

The environment targets real-world DevOps failure patterns, including:

- YAML syntax and structure issues
- Incorrect build/test commands (for example, npm tset -> npm test)
- Dependency and setup failures
- Multi-stage pipeline execution errors

This is designed as an RL-style interaction loop:

Observe -> Think -> Act -> Get Reward -> Repeat

## 2. Why This Matters

CI/CD failures are common, repetitive, and often multi-step to resolve. This project turns that workflow into a structured learning environment where agents:

- Read failure context
- Reason about root causes
- Propose and apply fixes
- Get shaped rewards for robust behavior

## 3. System Architecture

High-level flow:

Agent (LLM) -> Action -> Environment.step() -> Reward/Evaluation -> Next step

Core integration path:

Model -> Action -> Environment.step() -> RewardCalculator

RewardCalculator integrates:

- DeterministicGrader
- LLMJudge
- HiddenTestRunner
- AntiHackingDetector

## 4. Core Modules

### 4.1 Quality Judge

- File: env/graders/llm_judge.py
- Purpose: quality-aware scoring of fixes
- Output keys: correctness, minimalism, quality (all in [0,1])
- Guarantees:
	- strict JSON parsing attempt
	- robust fallback parsing for messy output
	- no-crash behavior (safe zero scores on failure)

### 4.2 Deterministic Grader

- File: env/graders/deterministic.py
- Purpose: reproducible correctness scoring (0-1)
- Checks:
	- YAML validity
	- command and fix correctness
	- similarity and issue resolution
- Rules:
	- deterministic only
	- same input, same score

### 4.3 Anti-Hacking Detector

- File: env/anti_hacking.py
- Purpose: detect reward-hacking and shortcut behavior
- Penalty detectors:
	- stage skipping (if: false, when: never)
	- fake success (echo tests passed, unsafe exit 0 patterns)
	- pipeline breakage between versions
	- excessive edits
	- timeout abuse via too many steps

### 4.4 Hidden Tests

- File: env/hidden_tests.py
- Purpose: test fix robustness, not just exact-match overfitting
- Method:
	- deterministic variant generation (OS, versions, env shifts)
	- evaluate pass rate across variants

### 4.5 Reward Shaping

- File: env/rewards.py
- Purpose: step-level learning signal
- Components:
	- progress rewards (logs, analysis, fix proposal)
	- execution rewards (pipeline run, tests pass)
	- quality rewards (deterministic + hidden tests + LLM judge)
	- anti-hacking penalties

## 5. Inference and Evaluation

### 5.1 Prompt and Model Layers

- inference/prompts.py: stable prompt templates and fallback action heuristics
- inference/model_wrapper.py: OpenAI-client action generation, candidate generation, and safe fallback

### 5.2 Metrics and Artifacts

- inference/metrics.py: reward, success-rate, and failure reason tracking
- inference/visualize.py: reward curve and metrics artifact export

### 5.3 Submission-Critical Runtime

- File: inference.py (root)
- Responsibilities:
	- initialize model and environment
	- run step loop
	- calculate rewards
	- emit strict stdout contract
	- always emit END line

Required output format:

- [START] task=... env=... model=...
- [STEP] step=<n> action=... reward=0.00 done=<true|false> error=<msg|null>
- [END] success=<true|false> steps=<n> rewards=<r1,r2,...>

Rules enforced:

- single-line logs only
- reward values with 2 decimals
- lowercase booleans
- no extra runtime log noise

## 6. Task Coverage

The project includes 13 CI-fix tasks spanning:

- easy: syntax and typo fixes
- medium: dependency/env/cache/permissions issues
- hard: matrix logic, conditional flow, orchestration-level failures

## 7. Setup

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

Environment variables:

```bash
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
export HF_TOKEN="<your_token>"
export LOCAL_IMAGE_NAME="<your_env_image_name>"
```

## 8. Run Inference

Offline/local mode:

```bash
python inference.py --offline --force-local-env --max-steps 8 --policy-mode imp --trajectories 4
```

Model-backed mode:

```bash
python inference.py --max-steps 8 --policy-mode imp --trajectories 4
```

Policy modes:

- sft: deterministic heuristic policy
- direct: single model action per step
- imp: multi-candidate generation and ranking

## 9. Tests

Run all tests:

```bash
python -m unittest discover -s tests -v
```

Coverage includes:

- LLM judge
- deterministic grader
- anti-hacking detectors
- hidden tests
- reward system
- end-to-end inference output format

## 10. Validation and Submission

OpenEnv validation:

```bash
openenv validate
```

Pre-submission script:

```bash
./validate-submission.sh <your_hf_space_url>
```

Docker run:

```bash
docker build -t cicd-debugger-env .
docker run --rm -e OFFLINE_INFERENCE=1 cicd-debugger-env
```

## 11. One-line Presentation Summary

We built an OpenEnv-compliant reinforcement learning environment where AI agents learn to debug real CI/CD pipelines using multi-step reasoning, hybrid grading, anti-hacking safeguards, and robust reward shaping.