docs: hackathon-ready README with training evidence and 4-stage cascade

README.md

@@ -11,95 +11,140 @@ tags:
   - ml
   - data-engineering
   - debugging
+  - reinforcement-learning
 ---
 
-# DataEngEnv — ML Data Pipeline Debugger
+# 🔧 PipelineOps Arena — ML Data Pipeline Debugger
 
 ![OpenEnv](https://img.shields.io/badge/OpenEnv-Compliant-blue)
 ![Docker](https://img.shields.io/badge/Docker-Ready-success)
 ![HF Spaces](https://img.shields.io/badge/HuggingFace-Spaces-yellow)
+![Stages](https://img.shields.io/badge/Stages-4-purple)
+![Baseline Score](https://img.shields.io/badge/Baseline-1.00-brightgreen)
 
 > *It's 2 AM. The model ships to production in 6 hours. The pipeline is broken.*
 > *Can your AI agent fix it in time?*
 
 ---
 
-## What is DataEngEnv?
+## 📦 Submission Links
 
-DataEngEnv is a real-world OpenEnv environment where an AI agent plays the role of a **frantic Data Engineer debugging a broken ML pipeline before a model goes to production.**
+| Resource | Link |
+|---|---|
+| 🌐 **Live Environment** | [https://huggingface.co/spaces/CoBeDigger/DataEngEnv](https://huggingface.co/spaces/CoBeDigger/DataEngEnv) |
+| 🤖 **Trained Model** | [CoBeDigger/pipelineops-arena-llama3-grpo](https://huggingface.co/CoBeDigger/pipelineops-arena-llama3-grpo) |
+| 📈 **Reward Curve** | [View on Model Repo](https://huggingface.co/CoBeDigger/pipelineops-arena-llama3-grpo/blob/main/reward_curve.png) |
+| 📓 **Training Notebook** | *(Kaggle notebook link)* |
 
-Every data team has been here: a schema change breaks the feature script, dirty data crashes the scaler, or a subtle data leakage bug silently inflates model accuracy to 98% — and nobody notices until it's too late.
+---
+
+## What is PipelineOps Arena?
+
+PipelineOps Arena is a **4-stage cascading OpenEnv environment** where an AI agent plays the role of a **frantic Data Engineer debugging a broken ML pipeline before a model goes to production.**
 
-DataEngEnv turns these real failure modes into a structured learning environment. Agents must **inspect data, read error logs, patch broken code, and verify fixes** — exactly the workflow a human engineer would follow. No games. No toys. Just the debugging tasks that happen every day at every ML company.
+Unlike simple game environments, this tests real-world engineering skills: schema debugging, feature scaling, data leakage detection, and fairness auditing — the exact workflow a human ML engineer follows every day.
 
-**Why this matters:** Training agents to debug data pipelines has direct real-world value. A capable pipeline-debugging agent could save ML teams hours of manual work on every model release cycle.
+### Why This Environment Is Different
+
+- **4-stage cascade**: Each stage must be solved to unlock the next
+- **Loop mechanic**: Failing Stage 4 sends the agent back to Stage 1 with a penalty — forcing genuine long-horizon planning
+- **Actor bots**: Stage 4 includes a simulated MLOps reviewer that provides fairness feedback
+- **Deterministic grading**: Every reward comes from objective, code-based evaluation — not LLM-as-judge
 
 ---
 
-## Environment Overview
+## 🏗️ The 4-Stage Pipeline
 
-The agent receives a **broken Python ML pipeline** and must fix it before running out of steps (max 20). At each step the agent chooses one action, receives an observation, and earns a partial reward for meaningful progress.
+```
+Stage 1          Stage 2          Stage 3          Stage 4
+Data Repair  →  Training Fix  →  Eval Fix     →  Deploy Gate
+(Schema Bug)    (Divergence)     (Leakage)       (Fairness)
+   🟢 Easy       🟡 Medium       🔴 Hard         🔴 Hard
+     │               │               │               │
+     └─── Fix ───────┘─── Fix ───────┘─── Fix ───────┘
+                                                      │
+                                              ┌───────┘
+                                              │ FAIL?
+                                              ▼
+                                         Back to Stage 1
+                                         (with penalty)
+```
 
-### Observation Space
+### Stage 1 — Data Repair 🟢
 
-| Field | Type | Description |
-|---|---|---|
-| `task_id` | `int` | Which task is currently running (1, 2, or 3) |
-| `step_number` | `int` | Current step (max 20) |
-| `script_content` | `str` | The current state of the pipeline script |
-| `last_run_output` | `str` | stdout from the last `run_script` action |
-| `last_run_error` | `str \| null` | stderr / exception if the script crashed |
-| `data_preview` | `str` | DataFrame shape, columns, nulls, sample rows, stats |
-| `schema_info` | `dict` | Column names, dtypes, null counts |
-| `done` | `bool` | True when submitted or steps exhausted |
+**Bug:** Column `age_years` was renamed to `age` in source data. Script crashes with `KeyError`. Also has ~15% NaN values.
 
-### Action Space
+**Fix:** Rename column reference + `dropna()`
 
-| Action | Payload | What it does |
-|---|---|---|
-| `inspect_data` | `{}` | Reveals full dataset preview — columns, nulls, outliers, sample rows |
-| `check_schema` | `{}` | Compares expected vs actual column names |
-| `run_script` | `{}` | Executes current script in a 10s sandboxed subprocess |
-| `edit_script` | `{"old": str, "new": str}` | Patches the script with a string replacement |
-| `submit` | `{}` | Submits the fix to the grader — ends the episode |
+### Stage 2 — Training Monitor 🟡
 
----
+**Bug:** `MLPClassifier` trains on unscaled features → loss diverges, model fails to converge.
+
+**Fix:** Add `StandardScaler` before the classifier.
 
-## The 3 Tasks
+### Stage 3 — Eval Validation 🔴
 
-### Task 1 — Column Rename Bug 🟢 Easy
+**Bug:** `scaler.fit_transform()` runs on the **entire dataset before** `train_test_split()`. The test set has seen training statistics. 98% accuracy is a lie.
 
-**The scenario:** A source CSV was updated and the column `age_years` was renamed to `age`. The pipeline script still references `age_years` and crashes instantly with a `KeyError`.
+**Fix:** Move `scaler.fit()` to after the split — fit only on training data.
 
-**What the agent must do:** Read the error log, inspect the data schema, identify the column mismatch, patch the one broken line.
+### Stage 4 — Deploy Gate 🔴
 
-**Why it's easy:** The fix is surgical — one string replacement. A capable agent should solve this in 3–5 steps.
+**Bug:** Model passes accuracy checks but has demographic bias. The MLOps bot rejects deployment.
 
-**Grader:** Did the script run without error and print an accuracy value?
+**Fix:** Add `class_weight='balanced'` to `LogisticRegression` to ensure fairness across groups.
 
 ---
 
-### Task 2 — Dirty Data 🟡 Medium
+## 📊 Training Results
 
-**The scenario:** The training dataset has ~15% NaN values across 3 columns AND extreme salary outliers (`9,999,999.0`). The `StandardScaler` crashes with `ValueError: Input contains NaN` before training even starts.
+We fine-tuned **LLaMA 3.1 8B** using SFT warmup followed by GRPO (Group Relative Policy Optimization) via HuggingFace TRL.
 
-**What the agent must do:** Detect the nulls and outliers via `inspect_data`, write cleaning logic (`dropna`/`fillna` for NaNs, `clip`/`quantile` for outliers), and get the training script to complete successfully.
+| Metric | Value |
+|---|---|
+| First 10 episodes avg | **0.15** |
+| Last 10 episodes avg | **0.52** |
+| Improvement | **+0.37** |
+| Peak score | **1.00** |
+| Total episodes | 30 |
 
-**Why it's medium:** There's no single-line fix. The agent must reason about the data, choose a cleaning strategy, and implement it correctly.
+The reward curve shows a clear upward learning trend — the agent progresses from random exploration (near-zero reward) to consistently completing multiple pipeline stages.
 
-**Grader:** Did training complete without crashing and print an accuracy score?
+### Baseline Agent Performance
 
----
+| Stage | Task | Score | Steps |
+|---|---|---|---|
+| Stage 1 | Data Repair | **1.00** | 4 |
+| Stage 2 | Training Monitor | **1.00** | 4 |
+| Stage 3 | Eval Validation | **1.00** | 4 |
+| Stage 4 | Deploy Gate | **1.00** | 6 |
+| **Total** | **All 4 Stages** | **1.00** | **18** |
 
-### Task 3 — Data Leakage 🔴 Hard
+---
 
-**The scenario:** The pipeline runs perfectly. No errors. It prints `Accuracy: 0.98`. But the model is cheating — `StandardScaler.fit_transform()` was called on the **entire dataset** before `train_test_split()`. The scaler has seen the test data. The 0.98 accuracy is a lie.
+## Observation Space
 
-**What the agent must do:** Notice that suspiciously high accuracy, diagnose the data leakage (scaler fit before split), fix the pipeline so the scaler only sees training data, and achieve ≥ 0.75 accuracy on a **server-side held-out test set** the agent never sees.
+| Field | Type | Description |
+|---|---|---|
+| `current_stage` | `int` | Current pipeline stage (1-4) |
+| `step_number` | `int` | Current step (max 20) |
+| `script_content` | `str` | The current state of the pipeline script |
+| `last_run_output` | `str` | stdout from the last `run_script` action |
+| `last_run_error` | `str \| null` | stderr / exception if the script crashed |
+| `data_preview` | `str` | DataFrame shape, columns, nulls, sample rows |
+| `schema_info` | `dict` | Column names, dtypes, null counts |
+| `done` | `bool` | True when all stages complete or steps exhausted |
 
-**Why it's hard:** There's no error message. No crash. The agent must reason about *correctness*, not just syntax. This is the hardest class of ML bug to catch.
+## Action Space
 
-**Grader:** Does the fixed script move `scaler.fit()` after `train_test_split()`? Does it achieve ≥ 0.75 accuracy on the held-out set?
+| Action | Payload | What it does |
+|---|---|---|
+| `inspect_data` | `{}` | Reveals full dataset preview — columns, nulls, outliers |
+| `check_schema` | `{}` | Compares expected vs actual column names |
+| `run_script` | `{}` | Executes current script in a 10s sandboxed subprocess |
+| `edit_script` | `{"old": str, "new": str}` | Patches the script with a string replacement |
+| `query_actor` | `{}` | Consult the MLOps bot for deployment feedback |
+| `submit` | `{}` | Submits the fix to the grader — advances to next stage |
 
 ---
 
@@ -107,42 +152,46 @@ The agent receives a **broken Python ML pipeline** and must fix it before runnin
 
 DataEngEnv provides **partial credit throughout the episode** — not just a binary win/lose at the end.
 
-| Task | Action | Reward |
+| Stage | Action | Reward |
 |---|---|---|
-| Task 1 | First `inspect_data` or `check_schema` | +0.1 |
-| Task 1 | `edit_script` removes `age_years` | +0.3 |
-| Task 1 | `run_script` with no error | +0.2 |
-| Task 2 | `inspect_data` reveals null counts | +0.15 |
-| Task 2 | `inspect_data` reveals outliers | +0.15 |
-| Task 2 | `edit_script` adds `dropna`/`fillna` | +0.3 |
-| Task 2 | `edit_script` adds `clip`/`quantile` | +0.2 |
-| Task 3 | Any `inspect_data` action | +0.1 |
-| Task 3 | `run_script` (sees suspicious accuracy) | +0.2 |
-| Task 3 | `edit_script` moves `fit` after `split` | +0.3 |
+| Stage 1 | First `inspect_data` or `check_schema` | +0.1 |
+| Stage 1 | `edit_script` fixes column name | +0.3 |
+| Stage 1 | `run_script` with no error | +0.2 |
+| Stage 2 | `inspect_data` reveals divergence | +0.15 |
+| Stage 2 | `edit_script` adds StandardScaler | +0.3 |
+| Stage 3 | `run_script` (sees suspicious accuracy) | +0.2 |
+| Stage 3 | `edit_script` moves fit after split | +0.3 |
+| Stage 4 | `query_actor` gets fairness feedback | +0.1 |
+| Stage 4 | `edit_script` adds class_weight balanced | +0.3 |
 | All | `submit` with correct fix | **grader score** |
-| Task 3 | `submit` without editing anything | -0.2 |
 
-Rewards are cumulative and capped at 1.0 before the terminal step.
-The terminal reward is always the grader score (0.0–1.0).
+Rewards are cumulative and capped at 1.0 per stage. The terminal reward is the grader score (0.0–1.0).
 
 ---
 
-## Baseline Scores
+## 🚀 Quick Start
 
-*Evaluated using `llama-3.1-8b-instant` via Groq API*
+### Use the Live API
 
-| Task | Difficulty | Score | Steps |
-|------|-----------|-------|-------|
-| Task 1 — Column Rename Bug | 🟢 Easy | **1.00** | 3 |
-| Task 2 — Dirty Data | 🟡 Medium | **1.00** | 15 |
-| Task 3 — Data Leakage | 🔴 Hard | **1.00** | 15 |
-| **Average** | — | **1.00** | — |
+```bash
+# Health check
+curl https://cobedigger-dataengenv.hf.space/health
 
----
+# Reset environment (start fresh episode)
+curl -X POST https://cobedigger-dataengenv.hf.space/reset \
+  -H "Content-Type: application/json" -d '{}'
 
-## Setup & Usage
+# Take an action
+curl -X POST https://cobedigger-dataengenv.hf.space/step \
+  -H "Content-Type: application/json" \
+  -d '{"action_type": "inspect_data", "payload": {}}'
+
+# Check pipeline progress
+curl https://cobedigger-dataengenv.hf.space/pipeline_status
+```
+
+### Run Locally
 
-### Run locally
 ```bash
 git clone https://huggingface.co/spaces/CoBeDigger/DataEngEnv
 cd DataEngEnv
@@ -151,78 +200,85 @@ uvicorn app.main:app --host 0.0.0.0 --port 8000
 ```
 
 ### Run with Docker
+
 ```bash
 docker build -t dataengenv .
 docker run -p 7860:7860 dataengenv
 ```
 
-### Environment variables
-```bash
-OPENAI_API_KEY=your_key    # for baseline script (OpenAI)
-GROQ_API_KEY=your_key      # for baseline script (Groq alternative)
-DATAENGENV_URL=http://localhost:8000  # baseline target URL
-```
+### Train Your Own Agent
 
----
+```python
+import requests
 
-## API Reference
-```bash
-# Health check
-curl https://cobedigger-dataengenv.hf.space/health
-
-# List all tasks
-curl https://cobedigger-dataengenv.hf.space/tasks
-
-# Start an episode
-curl -X POST https://cobedigger-dataengenv.hf.space/reset \
-  -H "Content-Type: application/json" -d '{"task_id": 1}'
-
-# Take a step
-curl -X POST https://cobedigger-dataengenv.hf.space/step \
-  -H "Content-Type: application/json" \
-  -d '{"action_type": "inspect_data", "payload": {}}'
+ENV_URL = "https://cobedigger-dataengenv.hf.space"
 
-# Get current state
-curl https://cobedigger-dataengenv.hf.space/state
+# Reset
+obs = requests.post(f"{ENV_URL}/reset").json()
 
-# Grade current episode
-curl -X POST https://cobedigger-dataengenv.hf.space/grader \
-  -H "Content-Type: application/json" -d '{"task_id": 1}'
+# Agent loop
+for step in range(20):
+    action = your_agent.decide(obs)
+    result = requests.post(f"{ENV_URL}/step", json=action).json()
+    obs = result["observation"]
+    reward = result["reward"]
+    
+    if obs["done"]:
+        break
 
-# Run full baseline
-curl https://cobedigger-dataengenv.hf.space/baseline
+# Check final score
+status = requests.get(f"{ENV_URL}/pipeline_status").json()
+print(f"Score: {status['episode_score']}")
 ```
 
 ---
 
 ## Project Structure
+
 ```
-dataengenv/
+PipelineOps-Arena/
 ├── app/
 │   ├── main.py              ← FastAPI app, all endpoints
 │   ├── models.py            ← Pydantic v2 typed models
 │   ├── environment.py       ← Core env: reset(), step(), state()
 │   ├── reward.py            ← Partial reward shaping engine
 │   ├── tasks/
-│   │   ├── task1_easy.py    ��� Column rename scenario
-│   │   ├── task2_medium.py  ← Dirty data scenario
-│   │   └── task3_hard.py    ← Data leakage scenario
-│   └── graders/
-│       ├── grader1.py       ← Easy task grader
-│       ├── grader2.py       ← Medium task grader
-│       └── grader3.py       ← Hard task grader
+│   │   ├── stage1_data_repair.py     ← Column rename + NaN scenario
+│   │   ├── stage2_training_monitor.py ← Feature scaling scenario
+│   │   ├── stage3_eval_validation.py  ← Data leakage scenario
+│   │   └── stage4_deploy_gate.py      ← Fairness gate scenario
+│   ├── graders/
+│   │   ├── grader_stage1.py  ← Schema fix grader
+│   │   ├── grader_stage2.py  ← Training convergence grader
+│   │   ├── grader_stage3.py  ← Leakage detection grader
+│   │   └── grader_stage4.py  ← Fairness threshold grader
+│   └── actors/
+│       ├── mlops_bot.py      ← Simulated MLOps reviewer
+│       └── code_reviewer.py  ← Code review actor
 ├── baseline/
-│   └── run_baseline.py      ← LLM agent inference script
-├── openenv.yaml             ← OpenEnv spec metadata
-├── Dockerfile               ← HF Spaces deployment
+│   └── run_baseline.py       ← Expert baseline agent
+├── gradio_app.py             ← Premium Gradio dashboard UI
+├── openenv.yaml              ← OpenEnv spec metadata
+├── Dockerfile                ← HF Spaces deployment
+├── pyproject.toml            ← Package configuration
 └── requirements.txt
 ```
 
 ---
 
+## Environment Variables
+
+```bash
+OPENAI_API_KEY=your_key    # for actor bots (optional)
+GROQ_API_KEY=your_key      # for baseline script (optional)
+```
+
+---
+
 ## Built By
 
 **CobeDivers — Jey Pranav, Kavin Krish, Hari**
-Meta AI × Hugging Face OpenEnv Hackathon
 
-*Live environment: https://huggingface.co/spaces/CoBeDigger/DataEngEnv*
+Meta AI × Hugging Face OpenEnv Hackathon 2026
+
+*Live environment: [https://huggingface.co/spaces/CoBeDigger/DataEngEnv](https://huggingface.co/spaces/CoBeDigger/DataEngEnv)*

openenv.yaml

@@ -1,20 +1,21 @@
 name: DataEngEnv
-version: "1.0.0"
-description: "ML Data Pipeline Debugger — AI agent fixes broken ML pipelines before production"
-tags: [openenv, data-engineering, machine-learning, debugging, real-world]
+version: "2.0.0"
+description: "PipelineOps Arena — 4-stage cascading ML pipeline debugger with fairness gate"
+tags: [openenv, data-engineering, machine-learning, debugging, reinforcement-learning, fairness]
 tasks:
-  - {id: 1, name: column_rename_bug, difficulty: easy}
-  - {id: 2, name: dirty_data, difficulty: medium}
-  - {id: 3, name: data_leakage, difficulty: hard}
+  - {id: 1, name: data_repair, difficulty: easy, description: "Fix column rename bug + NaN handling"}
+  - {id: 2, name: training_monitor, difficulty: medium, description: "Add StandardScaler for MLP convergence"}
+  - {id: 3, name: eval_validation, difficulty: hard, description: "Fix data leakage — scaler after split"}
+  - {id: 4, name: deploy_gate, difficulty: hard, description: "Fix model fairness with class_weight balanced"}
 endpoints:
   reset: POST /reset
   step: POST /step
   state: GET /state
   tasks: GET /tasks
   grader: POST /grader
-  baseline: GET /baseline
+  pipeline_status: GET /pipeline_status
 observation_space:
-  task_id: int
+  current_stage: int
   step_number: int
   script_content: str
   last_run_output: str
@@ -27,6 +28,9 @@ action_space:
   - {type: check_schema, payload: {}}
   - {type: run_script, payload: {}}
   - {type: edit_script, payload: {old: str, new: str}}
+  - {type: query_actor, payload: {}}
   - {type: submit, payload: {}}
 reward_range: [0.0, 1.0]
 max_steps: 20
+cascade: true
+loop_on_failure: true