Initial release: OpenEnv Flow Debugger

README.md

@@ -0,0 +1,130 @@
+# OpenEnv Flow Debugger (Just a Simple Version for Now!)
+
+This project is a small, easy-to-use debugging tool built with OpenEnv. It's inspired by those tricky real-world problems we hit in tools like Power Automate.
+
+Our environment focuses on a super common issue: those annoying '400 BadRequest' errors that pop up when a condition in your automation flow has a syntax mistake.
+
+The main idea here isn't to build a perfect smart agent right away. Instead, we want to create a clear, realistic, and expandable way to test and improve how agents fix bugs.
+
+---
+
+## What You Need to Do
+
+Imagine you have a Power Automate Flow that just failed.
+
+It failed because of an "HTTP 400 BadRequest" error.
+This error happened in a "Condition" step.
+And the condition expression has a tiny syntax error.
+
+Your job as the agent is to fix that broken condition expression so the flow can run perfectly.
+
+Each time you play (each "episode"), it's like facing a real-life debugging puzzle that automation engineers deal with all the time.
+
+---
+
+## What You See (Observation Space)
+
+At each step, you'll get some info in a JSON-like format. It includes:
+
+-   `case_id`: A unique ID for this specific problem.
+-   `run_status`: Tells you if the flow is still 'Failed' or 'Succeeded'.
+-   `failed_step`: Which step caused the problem.
+-   `error`: Details about the error, like the code and a message.
+-   `steps`: A list of all the steps in the flow, showing their inputs and outputs.
+-   `attempts_left`: How many more tries you have to fix it.
+
+**Example observation (kept simple):**
+
+```
+case_id: CASE_001
+run_status: Failed
+failed_step: Condition_Check
+error: code=400, message=BadRequest, details=InvalidTemplate: The expression is invalid
+steps:
+- Compose_Ext (Succeeded, outputs: xlsx)
+- Condition_Check (Failed, expression: @equals(outputs('Compose_Ext'),'xlsx')
+attempts_left: 3
+```
+
+---
+
+## What You Can Do (Action Space - Just Starting!)
+
+Right now, in this simple version, you can only do one type of action.
+
+You can submit a `patch_step` action. This action targets the `Condition_Check` step and updates its `inputs.expression` field.
+
+**Example action:**
+
+```
+action = patch_step
+step = Condition_Check
+field = inputs.expression
+value = @equals(outputs('Compose_Ext'),'xlsx')
+```
+
+For now, your fix needs to be an *exact* match to what's expected for it to count as correct.
+
+---
+
+## How You Get Graded (Reward Function)
+
+Our scoring system is pretty straightforward:
+
+-   **+1.0** if you successfully fix the flow.
+-   **-0.1** for trying an incorrect fix (but you still have tries left).
+-   **-0.2** if you run out of tries without fixing it.
+
+The game (episode) ends when the flow is fixed, or when you run out of chances.
+
+---
+
+## The Problems (Dataset)
+
+The specific bugs we're trying to fix are stored in JSON files here:
+
+`flow_debugger_env/data/cases.json`
+
+Each problem includes the messed-up flow state, error details, and a hidden 'gold_fix' (the right answer) that the environment uses to check your work. You, the agent, never see this 'gold_fix'.
+
+---
+
+## How to Run the Example
+
+Just run the `demo.py` file from the main project folder like this:
+
+`python demo.py`
+
+The demo will pick a random bug, use a basic rule-based agent to try and fix the condition expression, and then show you how it went.
+
+---
+
+## What This Can't Do Yet (Limitations)
+
+This simple version is kept small on purpose:
+
+-   It only deals with syntax errors in Condition expressions.
+-   It doesn't actually run real Power Automate flows.
+-   It doesn't connect to any outside services or APIs.
+-   It's not doing fancy AI learning (like reinforcement learning) yet.
+
+Keeping things simple means it's fast, predictable, and easy for us to build on later.
+
+---
+
+## What's Next?
+
+We could add more cool stuff later, like:
+
+-   Figuring out errors in 'filter array' settings.
+-   Dealing with 'null' values or wrong data types.
+-   Fixing multiple steps at once.
+-   Using smarter, AI-powered agents.
+-   Training AI using special tools like TRL or Unsloth.
+-   Adding 'Green Agent' wrappers.
+
+---
+
+## Why We Made This
+
+Debugging Power Automate is a real headache for many, and it's a big deal. This environment turns those everyday automation failures into a structured task for agents and a useful testbed for learning and experimenting with OpenEnv.

demo.py

@@ -0,0 +1,41 @@
+import re
+from flow_debugger_env.env import FlowDebugEnv
+
+def rule_based_agent(obs):
+    condition_step = next(s for s in obs["steps"] if s["name"] == "Condition_Check")
+    expr = condition_step["inputs"]["expression"]
+
+    fixed = expr
+    fixed = fixed.replace("@equal(", "@equals(")
+    fixed = re.sub(r",\s*xlsx\s*\)", r",'xlsx')", fixed)
+    fixed = re.sub(r"\)\s*'xlsx'\s*\)", r"),'xlsx')", fixed)
+
+    if fixed.count("(") > fixed.count(")"):
+        fixed = fixed + (")" * (fixed.count("(") - fixed.count(")")))
+
+    while fixed.endswith("))") and fixed.count(")") > fixed.count("("):
+        fixed = fixed[:-1]
+
+    return {
+        "action": "patch_step",
+        "step": "Condition_Check",
+        "field": "inputs.expression",
+        "value": fixed
+    }
+
+def main():
+    env = FlowDebugEnv.from_json("flow_debugger_env/data/cases.json", max_attempts=3, seed=42)
+    obs = env.reset()
+    done = False
+    total = 0.0
+
+    while not done:
+        action = rule_based_agent(obs)
+        result = env.step(action)
+        obs, reward, done, info = result.obs, result.reward, result.done, result.info
+        total += reward
+
+    print("Finished:", info, "total_reward:", total)
+
+if __name__ == "__main__":
+    main()

flow_debugger_env/data/cases.json

@@ -0,0 +1,152 @@
+[
+  {
+    "case_id": "CASE_001",
+    "failed_step": "Condition_Check",
+    "error": {
+      "code": 400,
+      "message": "BadRequest",
+      "details": "InvalidTemplate: The expression is invalid (missing closing parenthesis)."
+    },
+    "steps": [
+      {
+        "name": "Compose_Ext",
+        "type": "compose",
+        "status": "Succeeded",
+        "outputs": "xlsx"
+      },
+      {
+        "name": "Condition_Check",
+        "type": "condition",
+        "status": "Failed",
+        "inputs": {
+          "expression": "@equals(outputs('Compose_Ext'),'xlsx'"
+        }
+      }
+    ],
+    "gold_fix": {
+      "step": "Condition_Check",
+      "field": "inputs.expression",
+      "value": "@equals(outputs('Compose_Ext'),'xlsx')"
+    }
+  },
+  {
+    "case_id": "CASE_002",
+    "failed_step": "Condition_Check",
+    "error": {
+      "code": 400,
+      "message": "BadRequest",
+      "details": "InvalidTemplate: The expression is invalid (extra closing parenthesis)."
+    },
+    "steps": [
+      {
+        "name": "Compose_Ext",
+        "type": "compose",
+        "status": "Succeeded",
+        "outputs": "xlsx"
+      },
+      {
+        "name": "Condition_Check",
+        "type": "condition",
+        "status": "Failed",
+        "inputs": {
+          "expression": "@equals(outputs('Compose_Ext'),'xlsx'))"
+        }
+      }
+    ],
+    "gold_fix": {
+      "step": "Condition_Check",
+      "field": "inputs.expression",
+      "value": "@equals(outputs('Compose_Ext'),'xlsx')"
+    }
+  },
+  {
+    "case_id": "CASE_003",
+    "failed_step": "Condition_Check",
+    "error": {
+      "code": 400,
+      "message": "BadRequest",
+      "details": "InvalidTemplate: The expression is invalid (missing quotes around string literal)."
+    },
+    "steps": [
+      {
+        "name": "Compose_Ext",
+        "type": "compose",
+        "status": "Succeeded",
+        "outputs": "xlsx"
+      },
+      {
+        "name": "Condition_Check",
+        "type": "condition",
+        "status": "Failed",
+        "inputs": {
+          "expression": "@equals(outputs('Compose_Ext'),xlsx)"
+        }
+      }
+    ],
+    "gold_fix": {
+      "step": "Condition_Check",
+      "field": "inputs.expression",
+      "value": "@equals(outputs('Compose_Ext'),'xlsx')"
+    }
+  },
+  {
+    "case_id": "CASE_004",
+    "failed_step": "Condition_Check",
+    "error": {
+      "code": 400,
+      "message": "BadRequest",
+      "details": "InvalidTemplate: Unknown function 'equal' (typo)."
+    },
+    "steps": [
+      {
+        "name": "Compose_Ext",
+        "type": "compose",
+        "status": "Succeeded",
+        "outputs": "xlsx"
+      },
+      {
+        "name": "Condition_Check",
+        "type": "condition",
+        "status": "Failed",
+        "inputs": {
+          "expression": "@equal(outputs('Compose_Ext'),'xlsx')"
+        }
+      }
+    ],
+    "gold_fix": {
+      "step": "Condition_Check",
+      "field": "inputs.expression",
+      "value": "@equals(outputs('Compose_Ext'),'xlsx')"
+    }
+  },
+  {
+    "case_id": "CASE_005",
+    "failed_step": "Condition_Check",
+    "error": {
+      "code": 400,
+      "message": "BadRequest",
+      "details": "InvalidTemplate: The expression is invalid (missing comma between args)."
+    },
+    "steps": [
+      {
+        "name": "Compose_Ext",
+        "type": "compose",
+        "status": "Succeeded",
+        "outputs": "xlsx"
+      },
+      {
+        "name": "Condition_Check",
+        "type": "condition",
+        "status": "Failed",
+        "inputs": {
+          "expression": "@equals(outputs('Compose_Ext') 'xlsx')"
+        }
+      }
+    ],
+    "gold_fix": {
+      "step": "Condition_Check",
+      "field": "inputs.expression",
+      "value": "@equals(outputs('Compose_Ext'),'xlsx')"
+    }
+  }
+]

flow_debugger_env/env.py

@@ -0,0 +1,117 @@
+import copy
+import json
+import random
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Dict, Optional, List
+
+
+@dataclass
+class StepResult:
+    obs: Dict[str, Any]
+    reward: float
+    done: bool
+    info: Dict[str, Any]
+
+
+class FlowDebugEnv:
+    """
+    This is a simple environment made for OpenEnv.
+    Here's what it does:
+    - It gives you information in text or JSON.
+    - You can only do one thing: fix the 'inputs.expression' in a 'Condition_Check' step.
+    - If your fix is exactly right, you win!
+    """
+    def __init__(self, cases: List[Dict[str, Any]], max_attempts: int = 3, seed: Optional[int] = None):
+        self.cases = cases
+        self.max_attempts = max_attempts
+        self.rng = random.Random(seed)
+        self.current_case: Optional[Dict[str, Any]] = None
+        self.attempts_left = max_attempts
+
+    @classmethod
+    def from_json(cls, cases_json_path: str, max_attempts: int = 3, seed: Optional[int] = None):
+        path = Path(cases_json_path)
+        with open(path, "r", encoding="utf-8") as f:
+            cases = json.load(f)
+        return cls(cases=cases, max_attempts=max_attempts, seed=seed)
+
+    def reset(self) -> Dict[str, Any]:
+        self.current_case = copy.deepcopy(self.rng.choice(self.cases))
+        self.attempts_left = self.max_attempts
+        return self._make_observation()
+
+    def step(self, action: Dict[str, Any]) -> StepResult:
+        if self.current_case is None:
+            raise RuntimeError("Call reset() before step().")
+
+        self.attempts_left -= 1
+
+        if action.get("action") != "patch_step":
+            return self._invalid_action("Unsupported action type")
+
+        step_name = action.get("step")
+        field = action.get("field")
+        value = action.get("value")
+
+        patched_ok = self._apply_patch(step_name, field, value)
+        if not patched_ok:
+            return self._invalid_action("Patch failed (step/field not found)")
+
+        gold = self.current_case["gold_fix"]
+        solved = (step_name == gold["step"] and field == gold["field"] and value == gold["value"])
+
+        if solved:
+            self._mark_success()
+            obs = self._make_observation(run_status="Succeeded", error=None, failed_step=None)
+            return StepResult(obs=obs, reward=1.0, done=True,
+                              info={"result": "success", "case_id": self.current_case["case_id"]})
+
+        if self.attempts_left <= 0:
+            obs = self._make_observation()
+            return StepResult(obs=obs, reward=-0.2, done=True,
+                              info={"result": "out_of_attempts", "case_id": self.current_case["case_id"]})
+
+        obs = self._make_observation()
+        return StepResult(obs=obs, reward=-0.1, done=False,
+                          info={"result": "still_failed", "case_id": self.current_case["case_id"]})
+
+    # --------- helpers ----------
+    def _apply_patch(self, step_name: str, field: str, value: str) -> bool:
+        for step in self.current_case["steps"]:
+            if step["name"] == step_name:
+                if field == "inputs.expression":
+                    step.setdefault("inputs", {})
+                    step["inputs"]["expression"] = value
+                    return True
+        return False
+
+    def _mark_success(self):
+        for step in self.current_case["steps"]:
+            step["status"] = "Succeeded"
+
+    def _make_observation(self, run_status="Failed", error="keep", failed_step="keep"):
+        if error == "keep":
+            err_obj = self.current_case["error"]
+        else:
+            err_obj = error
+
+        if failed_step == "keep":
+            failed = self.current_case["failed_step"]
+        else:
+            failed = failed_step
+
+        return {
+            "case_id": self.current_case["case_id"],
+            "run_status": run_status,
+            "failed_step": failed,
+            "error": err_obj,
+            "steps": self.current_case["steps"],
+            "attempts_left": self.attempts_left
+        }
+
+    def _invalid_action(self, msg: str) -> StepResult:
+        obs = self._make_observation()
+        done = (self.attempts_left <= 0)
+        return StepResult(obs=obs, reward=-0.1, done=done,
+                          info={"result": "invalid_action", "message": msg, "case_id": self.current_case["case_id"]})