Enhance code generation environment with diverse coder strategies and tiered adversarial testing

Implement multiple coder strategies (bubble sort, selection sort with bug, and an improving coder) and a tiered breaker agent with progressively harder test cases. Add comprehensive logging for rewards and episode metrics. Improve environment state clarity and coach memory with human-readable lessons.

Replit-Commit-Author: Agent
Replit-Commit-Session-Id: a7518b1f-70c7-4487-82d2-42195935723e
Replit-Commit-Checkpoint-Type: full_checkpoint
Replit-Commit-Event-Id: 6f92db1c-7ebb-4a38-b6ed-3dc81054bda2
Replit-Helium-Checkpoint-Created: true

FORGE-v4/agents.py

@@ -0,0 +1,263 @@
+# agents.py
+# Coder strategies and tiered Breaker agent for FORGE-v4.
+#
+# Coder strategies:
+#   weak_coder_v1     — bubble sort (O(n²), slow on large arrays)
+#   weak_coder_v2     — selection sort with a subtle bug on negatives
+#   improving_coder   — picks strategy based on episode count
+#
+# Breaker agent:
+#   BreakerAgent      — tiered adversarial test case generator
+
+import random
+from typing import Any
+from config import (
+    ARRAY_VALUE_RANGE,
+    MAX_ARRAY_SIZE,
+    BREAKER_TIER_UNLOCK_RATE,
+    BREAKER_TIER3_MIN_EPISODE,
+    BREAKER_TIER4_MIN_EPISODE,
+    IMPROVING_CODER_TIER1_UNTIL,
+    IMPROVING_CODER_TIER2_UNTIL,
+)
+
+
+# ══════════════════════════════════════════════
+#  CODER STRATEGIES
+# ══════════════════════════════════════════════
+
+# Each strategy returns a Python source string that defines solution(arr).
+
+WEAK_CODER_V1_CODE = '''
+def solution(arr):
+    """Bubble sort — O(n²), fails slowly on large arrays."""
+    a = list(arr)
+    n = len(a)
+    for i in range(n):
+        for j in range(n - i - 1):
+            if a[j] > a[j + 1]:
+                a[j], a[j + 1] = a[j + 1], a[j]
+    return a
+'''
+
+WEAK_CODER_V2_CODE = '''
+def solution(arr):
+    """
+    Selection sort — correct for positive-only arrays.
+    Bug: uses abs() comparison, so negatives can end up out of order.
+    """
+    a = list(arr)
+    n = len(a)
+    for i in range(n):
+        min_idx = i
+        for j in range(i + 1, n):
+            # BUG: comparing absolute values breaks negative ordering
+            if abs(a[j]) < abs(a[min_idx]):
+                min_idx = j
+        a[i], a[min_idx] = a[min_idx], a[i]
+    return a
+'''
+
+IMPROVING_CODER_TEMPLATE = '''
+def solution(arr):
+    """
+    Improving coder — strategy selected by episode {episode}.
+    Episode <= {tier1_until}: bubble sort (weakest)
+    Episode <= {tier2_until}: selection sort (medium)
+    Episode >  {tier2_until}: built-in sorted (strongest)
+    """
+    episode = {episode}
+    a = list(arr)
+
+    if episode <= {tier1_until}:
+        # Bubble sort
+        n = len(a)
+        for i in range(n):
+            for j in range(n - i - 1):
+                if a[j] > a[j + 1]:
+                    a[j], a[j + 1] = a[j + 1], a[j]
+        return a
+    elif episode <= {tier2_until}:
+        # Selection sort with abs() bug
+        n = len(a)
+        for i in range(n):
+            min_idx = i
+            for j in range(i + 1, n):
+                if abs(a[j]) < abs(a[min_idx]):
+                    min_idx = j
+            a[i], a[min_idx] = a[min_idx], a[i]
+        return a
+    else:
+        # Strong solution
+        return sorted(a)
+'''
+
+
+def get_coder_code(version: str, episode: int = 1) -> str:
+    """
+    Return the Python source code for the given coder version.
+
+    Args:
+        version: "weak_coder_v1" | "weak_coder_v2" | "improving_coder"
+        episode: current episode number (used by improving_coder)
+    """
+    if version == "weak_coder_v1":
+        return WEAK_CODER_V1_CODE
+
+    if version == "weak_coder_v2":
+        return WEAK_CODER_V2_CODE
+
+    if version == "improving_coder":
+        return IMPROVING_CODER_TEMPLATE.format(
+            episode=episode,
+            tier1_until=IMPROVING_CODER_TIER1_UNTIL,
+            tier2_until=IMPROVING_CODER_TIER2_UNTIL,
+        )
+
+    raise ValueError(f"Unknown coder version: {version!r}")
+
+
+def coder_version_label(version: str, episode: int) -> str:
+    """Human-readable label for what strategy the coder is using this episode."""
+    if version == "weak_coder_v1":
+        return "weak_coder_v1 (bubble sort)"
+    if version == "weak_coder_v2":
+        return "weak_coder_v2 (selection sort / abs-bug)"
+    if version == "improving_coder":
+        if episode <= IMPROVING_CODER_TIER1_UNTIL:
+            return f"improving_coder → bubble sort  (ep {episode} ≤ {IMPROVING_CODER_TIER1_UNTIL})"
+        if episode <= IMPROVING_CODER_TIER2_UNTIL:
+            return f"improving_coder → selection sort (ep {episode} ≤ {IMPROVING_CODER_TIER2_UNTIL})"
+        return f"improving_coder → sorted()  (ep {episode} > {IMPROVING_CODER_TIER2_UNTIL})"
+    return version
+
+
+# ══════════════════════════════════════════════
+#  TIERED BREAKER AGENT
+# ══════════════════════════════════════════════
+
+# Test case banks per tier
+_TIER1_CASES: list[list[int]] = [
+    [],
+    [1],
+    [2, 1],
+    [3, 2, 1],
+    [1, 2, 3],
+]
+
+_TIER2_CASES: list[list[int]] = [
+    [1, 1, 1, 1],                               # all duplicates
+    [2, 2, 1, 1, 3, 3],                         # duplicate pairs
+    [-5, -1, -3, -7, -2],                       # all negatives
+    [-3, 0, 3, -1, 1],                          # mixed sign
+    [1, 2, 3, 4, 5],                            # already sorted
+    [5, 4, 3, 2, 1],                            # reverse sorted
+    [0, 0, 0],                                  # all zeros
+]
+
+_TIER3_CASES: list[list[int]] = [
+    list(range(MAX_ARRAY_SIZE, 0, -1)),                      # full reverse
+    [random.choice([1, 2]) for _ in range(MAX_ARRAY_SIZE)], # heavy duplicates
+    [random.randint(-100, 100) for _ in range(MAX_ARRAY_SIZE)],  # large random
+    [0] * MAX_ARRAY_SIZE,                                    # all zeros, large
+    list(range(MAX_ARRAY_SIZE)),                             # sorted ascending, large
+]
+
+_TIER4_CASES: list[list[int]] = [
+    [-100, 100],                                             # boundary values only
+    [100, 100, 100, -100, -100, -100],                      # boundary duplicates
+    [-100] * 10 + [100] * 10,                               # boundary mixed
+    list(range(-10, 11)),                                    # full range small
+    [random.randint(-100, 100) for _ in range(MAX_ARRAY_SIZE)],  # stress random
+]
+
+
+class BreakerAgent:
+    """
+    Adversarial test-case generator with four tiers of difficulty.
+
+    Tier unlocking rules:
+        Tier 2 → always available from episode 1
+        Tier 3 → unlocks when break_rate >= BREAKER_TIER_UNLOCK_RATE
+                 AND episode >= BREAKER_TIER3_MIN_EPISODE
+        Tier 4 → unlocks when at tier 3 AND episode >= BREAKER_TIER4_MIN_EPISODE
+
+    The agent samples cases from all unlocked tiers, weighted toward the
+    current (highest) tier for maximum adversarial pressure.
+    """
+
+    def __init__(self) -> None:
+        self.current_tier: int = 1
+        self._recent_break_rates: list[float] = []
+
+    def update_tier(self, break_rate: float, episode: int) -> None:
+        """
+        Update the current tier based on recent performance and episode count.
+
+        Args:
+            break_rate: Breaker's break_rate from the last step.
+            episode:    Current episode number.
+        """
+        self._recent_break_rates.append(break_rate)
+        # Use rolling window of last 3 steps to smooth noise
+        recent = self._recent_break_rates[-3:]
+        avg_break = sum(recent) / len(recent)
+
+        if self.current_tier == 1 and avg_break >= BREAKER_TIER_UNLOCK_RATE:
+            self.current_tier = 2
+
+        if self.current_tier == 2 and (
+            avg_break >= BREAKER_TIER_UNLOCK_RATE
+            and episode >= BREAKER_TIER3_MIN_EPISODE
+        ):
+            self.current_tier = 3
+
+        if self.current_tier == 3 and episode >= BREAKER_TIER4_MIN_EPISODE:
+            self.current_tier = 4
+
+    def get_tests(self, n_per_tier: int = 2) -> list[dict[str, Any]]:
+        """
+        Return adversarial test cases sampled from all unlocked tiers,
+        with extra weight on the current highest tier.
+
+        Args:
+            n_per_tier: Number of cases to sample from each unlocked tier.
+
+        Returns:
+            List of {"input": [...], "expected_output": [...]} dicts.
+        """
+        pools: list[tuple[int, list[list[int]]]] = [
+            (1, _TIER1_CASES),
+            (2, _TIER2_CASES),
+            (3, _TIER3_CASES),
+            (4, _TIER4_CASES),
+        ]
+
+        selected: list[list[int]] = []
+        for tier_num, pool in pools:
+            if tier_num > self.current_tier:
+                break
+            # Sample more from the highest tier
+            k = n_per_tier * 2 if tier_num == self.current_tier else n_per_tier
+            k = min(k, len(pool))
+            selected.extend(random.sample(pool, k))
+
+        # Remove duplicates (by converting to tuple for hashability)
+        seen: set[tuple[int, ...]] = set()
+        unique: list[list[int]] = []
+        for arr in selected:
+            key = tuple(arr)
+            if key not in seen:
+                seen.add(key)
+                unique.append(arr)
+
+        return [
+            {"input": arr, "expected_output": sorted(arr)}
+            for arr in unique
+        ]
+
+    @property
+    def tier_name(self) -> str:
+        """Human-readable tier label."""
+        from config import BREAKER_TIER_NAMES
+        return BREAKER_TIER_NAMES.get(self.current_tier, f"Tier-{self.current_tier}")

FORGE-v4/app.py

@@ -1,84 +1,205 @@
 # app.py
 # Main runner script for FORGE-v4.
-# Runs a minimal CLI demo of one sample episode.
+# Runs one demo episode with the improving_coder and tiered BreakerAgent,
+# then prints a structured results report.
 
 import sys
 import json
+
 from env import FORGEEnv
 from memory import CoachMemory
-from trainer import default_coder_policy, default_breaker_policy
+from agents import get_coder_code, coder_version_label, BreakerAgent
+from logger import log_episode, update_summary, print_log_paths
 from config import STEPS_PER_EPISODE
 
 
-def run_demo_episode() -> None:
+# ──────────────────────────────────────────────
+# Demo configuration
+# ──────────────────────────────────────────────
+DEFAULT_CODER_VERSION = "improving_coder"
+
+
+def run_demo_episode(coder_version: str = DEFAULT_CODER_VERSION) -> None:
     """
-    Execute a single demo episode and print the results to stdout.
+    Execute one demo episode and print a rich results report.
+
+    Args:
+        coder_version: Which coder strategy to use.
+            "weak_coder_v1" | "weak_coder_v2" | "improving_coder"
     """
-    print("=" * 60)
-    print("  FORGE-v4  |  Adversarial Code Generation Environment")
-    print("=" * 60)
+    _banner()
 
-    # Initialise coach memory and environment
     memory = CoachMemory()
+    memory.clear()          # Start fresh for the demo run
     env = FORGEEnv(memory=memory)
-
-    # Reset to start the episode
     state = env.reset()
 
-    print(f"\n[Episode {state['episode']}]  Task prompt:\n")
-    print(state["task_prompt"])
+    episode = state["episode"]
+    print(f"\n{'─'*60}")
+    print(f"  Task ID  : {state['task_id']}")
+    print(f"  Episode  : {episode}")
+    print(f"  Coder    : {coder_version_label(coder_version, episode)}")
+    print(f"  Breaker  : {env.breaker.tier_name}  (starts here, tiers up during run)")
+    print(f"{'─'*60}")
+    print(f"\n  Problem:\n")
+    print(f"  {state['problem_description']}")
     print()
 
-    for step in range(1, STEPS_PER_EPISODE + 1):
-        print(f"── Step {step}/{STEPS_PER_EPISODE} " + "─" * 40)
+    # ── Accumulators ──────────────────────────────────────────────────────
+    ep_coder_rewards:   list[float] = []
+    ep_breaker_rewards: list[float] = []
+    ep_pass_rates:      list[float] = []
+    ep_fail_counts:     list[int]   = []
+    ep_error_counts:    list[int]   = []
+    ep_timeout_counts:  list[int]   = []
+    ep_break_rates:     list[float] = []
 
-        # Agents produce their actions (placeholder policies for the demo)
-        coder_code    = default_coder_policy(state)
-        breaker_tests = default_breaker_policy(state)
-
-        action = {
-            "coder_code":    coder_code,
-            "breaker_tests": breaker_tests,
-        }
+    for step_num in range(1, STEPS_PER_EPISODE + 1):
+        # Build coder action
+        code   = get_coder_code(coder_version, episode=episode)
+        action = {"coder_code": code, "coder_version": coder_version}
 
         result = env.step(action)
+        state  = result["state"]
 
         cr = result["coder_reward"]
         br = result["breaker_reward"]
-
+        info = result["info"]
+
+        # Accumulate
+        ep_coder_rewards.append(cr["total_reward"])
+        ep_breaker_rewards.append(br["total_reward"])
+        ep_pass_rates.append(cr["pass_rate"])
+        ep_fail_counts.append(cr["fail_count"])
+        ep_error_counts.append(cr["error_count"])
+        ep_timeout_counts.append(cr["error_count"])
+        ep_break_rates.append(br["break_rate"])
+
+        # Per-step print
+        print(f"  ── Step {step_num}/{STEPS_PER_EPISODE}  [breaker: {info['breaker_tier_name']}]")
         print(
-            f"  Coder   → pass_rate: {cr['pass_rate']:.2f}  "
+            f"     Coder   → pass_rate: {cr['pass_rate']:.2f}  "
             f"| passes: {cr['pass_count']}  "
             f"| fails: {cr['fail_count']}  "
             f"| errors: {cr['error_count']}  "
             f"| reward: {cr['total_reward']:+.2f}"
         )
         print(
-            f"  Breaker → break_rate: {br['break_rate']:.2f}  "
+            f"     Breaker → break_rate: {br['break_rate']:.2f}  "
             f"| breaks: {br['breaks']}  "
-            f"| passes: {br['passes']}  "
+            f"| no-break: {br['passes']}  "
             f"| reward: {br['total_reward']:+.2f}"
         )
+        if state.get("recent_breaker_case") is not None:
+            print(f"     Recent adversarial input: {state['recent_breaker_case']}")
+        print()
 
         if result["done"]:
             break
 
-    print("\n" + "=" * 60)
-    print("  Episode complete.  Coach memory summary:")
-    print(json.dumps(memory.summary(), indent=2))
-    print("=" * 60)
+    # ── Episode log ───���───────────────────────────────────────────────────
+    def avg(lst: list) -> float:
+        return round(sum(lst) / len(lst), 4) if lst else 0.0
+
+    log_episode(
+        episode=episode,
+        coder_version=coder_version,
+        breaker_tier=env.breaker.current_tier,
+        avg_coder_reward=avg(ep_coder_rewards),
+        avg_breaker_reward=avg(ep_breaker_rewards),
+        avg_pass_rate=avg(ep_pass_rates),
+        total_fail_count=sum(ep_fail_counts),
+        total_error_count=sum(ep_error_counts),
+        total_timeout_count=sum(ep_timeout_counts),
+        avg_break_rate=avg(ep_break_rates),
+        steps=env.step_count,
+    )
+
+    update_summary(
+        total_episodes=1,
+        coder_version=coder_version,
+        final_breaker_tier=env.breaker.current_tier,
+        all_coder_rewards=ep_coder_rewards,
+        all_breaker_rewards=ep_breaker_rewards,
+        all_pass_rates=ep_pass_rates,
+        all_break_rates=ep_break_rates,
+        coach_memory_summary=memory.summary(),
+    )
+
+    # ── Final report ──────────────────────────────────────────────────────
+    print(f"{'═'*60}")
+    print("  EPISODE SUMMARY")
+    print(f"{'═'*60}")
+    print(f"  Coder version       : {coder_version_label(coder_version, episode)}")
+    print(f"  Final breaker tier  : {env.breaker.tier_name}")
+    print(f"  Avg pass rate       : {avg(ep_pass_rates):.2f}")
+    print(f"  Avg coder reward    : {avg(ep_coder_rewards):+.4f}")
+    print(f"  Avg breaker reward  : {avg(ep_breaker_rewards):+.4f}")
+    print(f"  Total fail count    : {sum(ep_fail_counts)}")
+    print(f"  Total error count   : {sum(ep_error_counts)}")
+    print(f"  Avg break rate      : {avg(ep_break_rates):.2f}")
+    print()
+    print("  Coach memory summary:")
+    summary = memory.summary()
+    print(f"    Lessons stored    : {summary.get('total_lessons', 0)}")
+    notes = summary.get("recent_coach_notes", [])
+    if notes:
+        print("    Recent coach notes:")
+        for note in notes:
+            print(f"      • {note}")
+    print()
+    print("  Log files updated:")
+    print_log_paths()
+    print(f"{'═'*60}")
+
 
+# ──────────────────────────────────────────────
+# Helpers
+# ──────────────────────────────────────────────
+
+def _banner() -> None:
+    print()
+    print("╔══════════════════════════════════════════════════════════╗")
+    print("║   FORGE-v4  |  Adversarial Code Generation Environment  ║")
+    print("╚══════════════════════════════════════════════════════════╝")
+
+
+def _print_help() -> None:
+    print("Usage: python app.py [OPTIONS]")
+    print()
+    print("Options:")
+    print("  --coder VERSION   Coder strategy to use:")
+    print("                      weak_coder_v1   (bubble sort — slow/weak)")
+    print("                      weak_coder_v2   (selection sort + abs() bug)")
+    print("                      improving_coder (adapts each episode)  [default]")
+    print("  --steps N         Override STEPS_PER_EPISODE for this run")
+    print("  --help / -h       Show this message")
+
+
+# ──────────────────────────────────────────────
+# Entry point
+# ──────────────────────────────────────────────
 
 def main() -> None:
-    """Entry point — parse minimal CLI args and run."""
     args = sys.argv[1:]
 
     if "--help" in args or "-h" in args:
-        print("Usage: python app.py [--steps N]")
-        print("  --steps N   Override STEPS_PER_EPISODE for this run (default: from config.py)")
+        _print_help()
         sys.exit(0)
 
-    # Optional: override step count via CLI
+    coder_version = DEFAULT_CODER_VERSION
+    if "--coder" in args:
+        idx = args.index("--coder")
+        try:
+            coder_version = args[idx + 1]
+            valid = ("weak_coder_v1", "weak_coder_v2", "improving_coder")
+            if coder_version not in valid:
+                print(f"Error: unknown coder version '{coder_version}'. Choose from: {valid}")
+                sys.exit(1)
+        except IndexError:
+            print("Error: --coder requires a version argument.")
+            sys.exit(1)
+
     if "--steps" in args:
         idx = args.index("--steps")
         try:
@@ -88,7 +209,7 @@ def main() -> None:
             print("Error: --steps requires an integer argument.")
             sys.exit(1)
 
-    run_demo_episode()
+    run_demo_episode(coder_version=coder_version)
 
 
 if __name__ == "__main__":

FORGE-v4/config.py

@@ -18,20 +18,43 @@ NUM_HIDDEN_TESTS = 5                 # Number of hidden test cases per task
 # ──────────────────────────────────────────────
 # Reward settings
 # ──────────────────────────────────────────────
-# Coder reward weights
-CODER_PASS_REWARD = 1.0              # Reward per passing hidden test
-CODER_FAIL_PENALTY = -0.5            # Penalty per failing hidden test
-CODER_ERROR_PENALTY = -1.0           # Penalty when code raises an error
+CODER_PASS_REWARD   =  1.0           # Reward per passing hidden test
+CODER_FAIL_PENALTY  = -0.5           # Penalty per failing hidden test
+CODER_ERROR_PENALTY = -1.0           # Penalty when code raises an error/timeout
 
-# Breaker reward weights
 BREAKER_BREAK_REWARD = 1.0           # Reward when breaker's test breaks coder
-BREAKER_FAIL_PENALTY = -0.3          # Penalty when breaker's test does NOT break coder
+BREAKER_FAIL_PENALTY = -0.3          # Penalty when coder survives a breaker test
 
 # ──────────────────────────────────────────────
-# Tier thresholds (coder skill levels)
+# Coder agent versions
+# ──────────────────────────────────────────────
+CODER_VERSIONS = ["weak_coder_v1", "weak_coder_v2", "improving_coder"]
+
+# improving_coder tier-up thresholds (episode numbers)
+IMPROVING_CODER_TIER1_UNTIL = 3      # Episodes 1–3 → uses weak strategy
+IMPROVING_CODER_TIER2_UNTIL = 6      # Episodes 4–6 → uses mid strategy
+
+# ──────────────────────────────────────────────
+# Breaker tier system
+# ──────────────────────────────────────────────
+BREAKER_TIER_NAMES = {
+    1: "Tier-1 (basic)",
+    2: "Tier-2 (edge cases)",
+    3: "Tier-3 (stress)",
+    4: "Tier-4 (boundary/extreme)",
+}
+
+# Minimum break_rate to unlock next tier
+BREAKER_TIER_UNLOCK_RATE  = 0.6      # 60% break rate needed to promote
+# Minimum episode before tier 3 unlocks (regardless of break rate)
+BREAKER_TIER3_MIN_EPISODE = 4
+BREAKER_TIER4_MIN_EPISODE = 7
+
+# ──────────────────────────────────────────────
+# Tier thresholds (coder skill levels — for display/labelling)
 # ──────────────────────────────────────────────
 TIER_THRESHOLDS = {
-    "novice":       (0.0,  0.4),     # pass-rate range [low, high)
+    "novice":       (0.0,  0.4),
     "intermediate": (0.4,  0.7),
     "advanced":     (0.7,  0.9),
     "expert":       (0.9,  1.01),
@@ -40,13 +63,18 @@ TIER_THRESHOLDS = {
 # ──────────────────────────────────────────────
 # Memory / logging
 # ──────────────────────────────────────────────
-MEMORY_FILE = "data/coach_memory.json"   # Persistent memory path
-LOG_DIR = "logs/"                        # Directory for episode logs
-MODELS_DIR = "models/"                   # Saved model checkpoints
-OUTPUTS_DIR = "outputs/"                 # Generated code outputs
+MEMORY_FILE   = "data/coach_memory.json"
+LOG_DIR       = "logs/"
+MODELS_DIR    = "models/"
+OUTPUTS_DIR   = "outputs/"
+
+# Log file paths (within LOG_DIR)
+LOG_REWARDS_FILE  = "logs/rewards.json"
+LOG_EPISODES_FILE = "logs/episodes.csv"
+LOG_SUMMARY_FILE  = "logs/summary.json"
 
 # ──────────────────────────────────────────────
 # Training placeholders
 # ──────────────────────────────────────────────
-MAX_EPISODES = 100                   # Default training episode count
-STEPS_PER_EPISODE = 10               # Steps per episode
+MAX_EPISODES      = 100
+STEPS_PER_EPISODE = 3                # Kept short for fast demo runs

FORGE-v4/env.py

@@ -1,12 +1,16 @@
 # env.py
 # Main OpenEnv-style reinforcement learning environment for FORGE-v4.
-# Manages the interaction between the Coder Agent, Breaker Agent, and Sandbox.
+# Manages Coder Agent, Breaker Agent, Sandbox, Rewards, Memory, and Logging.
 
+import uuid
 from typing import Any
-from tasks import generate_task, generate_breaker_task
+
+from tasks import generate_task
 from sandbox import run_code_against_tests
 from rewards import coder_reward, breaker_reward
 from memory import CoachMemory
+from agents import BreakerAgent, coder_version_label
+from logger import log_step
 from config import STEPS_PER_EPISODE
 
 
@@ -15,29 +19,68 @@ class FORGEEnv:
     Two-agent adversarial environment for code generation tasks.
 
     Agents:
-        - Coder:   writes Python code to solve array-sorting tasks.
-        - Breaker: generates adversarial test cases to break the Coder's solution.
+        - Coder:   submits Python code defining solution(arr).
+        - Breaker: submits adversarial test cases via a BreakerAgent.
 
     Episode flow:
-        1. reset()           → returns the initial task state
-        2. step(action)      × STEPS_PER_EPISODE steps
-        3. Rewards assigned to both agents at each step
+        1. reset()               → returns initial state
+        2. step(action) × N     → coder vs breaker, rewards, memory, logs
+        3. done=True             → call reset() for next episode
+
+    Action format passed to step():
+        {
+            "coder_code":    str,   # Python source defining solution(arr)
+            "coder_version": str,   # label, e.g. "weak_coder_v1"
+        }
+    The BreakerAgent is managed internally by the environment.
 
-    Action format:
+    State returned by get_state() / reset() / step():
         {
-            "coder_code":        str | None,   # Python source defining solution(arr)
-            "breaker_tests":     list | None,  # List of {"input": [...]} dicts
+            "task_id":              str,
+            "problem_description":  str,
+            "episode":              int,
+            "episode_step":         int,
+            "done":                 bool,
+            "coder_version":        str,
+            "current_tier":         int,
+            "recent_breaker_case":  list[int],
+            "pass_rate_history":    list[float],
+            "coach_memory_summary": dict,
+            "public_example":       dict,
         }
+
+    step() returns:
+        {
+            "state":          dict,
+            "coder_reward":   dict,   # from rewards.coder_reward()
+            "breaker_reward": dict,   # from rewards.breaker_reward()
+            "done":           bool,
+            "info":           dict,   # diagnostics
+        }
+
+    Explicit step() flow:
+        1. Run coder code against hidden tests in sandbox
+        2. Run breaker tests against coder code in sandbox
+        3. Assign coder_reward and breaker_reward
+        4. Update coach memory with structured lesson
+        5. Log step metrics to logs/rewards.json
+        6. Advance breaker tier based on break_rate
+        7. Return next_state, rewards, done, info
     """
 
     def __init__(self, memory: CoachMemory | None = None):
-        self.memory = memory or CoachMemory()
-        self.episode: int = 0
+        self.memory        = memory or CoachMemory()
+        self.breaker       = BreakerAgent()
+        self.episode: int  = 0
         self.step_count: int = 0
         self.current_task: dict[str, Any] = {}
-        self.done: bool = True
-        self._last_coder_code: str = ""
-        self._last_coder_pass_rate: float = 0.0
+        self.done: bool    = True
+
+        # Tracked across the episode
+        self._coder_version: str       = "unknown"
+        self._pass_rate_history: list[float] = []
+        self._recent_breaker_case: list[int] = []
+        self._last_coder_pass_rate: float    = 0.0
 
     # ──────────────────────────────────────────────
     # Core env methods
@@ -45,38 +88,42 @@ class FORGEEnv:
 
     def reset(self) -> dict[str, Any]:
         """
-        Start a new episode.
+        Start a new episode. Generates a fresh task and resets counters.
 
         Returns:
-            Initial state dict containing the task prompt and public example.
+            Initial state dict.
         """
         self.episode += 1
         self.step_count = 0
-        self.done = False
-        self._last_coder_code = ""
+        self.done       = False
+
+        self._coder_version        = "unknown"
+        self._pass_rate_history    = []
+        self._recent_breaker_case  = []
         self._last_coder_pass_rate = 0.0
 
         self.current_task = generate_task()
+        self.current_task["task_id"] = str(uuid.uuid4())[:8]
 
-        state = self.get_state()
-        return state
+        return self.get_state()
 
     def step(self, action: dict[str, Any]) -> dict[str, Any]:
         """
         Advance the environment by one step.
 
         Args:
-            action: dict with optional keys:
-                "coder_code"    – Python source defining solution(arr)
-                "breaker_tests" – list of {"input": [...]} dicts
+            action: {
+                "coder_code":    str   — Python source defining solution(arr)
+                "coder_version": str   — human label for the coder strategy used
+            }
 
         Returns:
             {
-                "state":          current env state,
-                "coder_reward":   coder reward info dict,
-                "breaker_reward": breaker reward info dict,
-                "done":           bool (True when episode ends),
-                "info":           extra diagnostics,
+                "state":          dict — next observable state,
+                "coder_reward":   dict — coder reward breakdown,
+                "breaker_reward": dict — breaker reward breakdown,
+                "done":           bool,
+                "info":           dict — diagnostics,
             }
         """
         if self.done:
@@ -84,33 +131,66 @@ class FORGEEnv:
 
         self.step_count += 1
         coder_code    = action.get("coder_code", "")
-        breaker_tests = action.get("breaker_tests", [])
+        coder_version = action.get("coder_version", "unknown")
+        self._coder_version = coder_version
 
-        # ── Evaluate Coder ────────────────────────────────────────────────
-        coder_info = self._evaluate_coder(coder_code)
+        # ── 1. Get breaker tests for this step ───────────────────────────
+        breaker_tests = self.breaker.get_tests(n_per_tier=2)
+        if breaker_tests:
+            self._recent_breaker_case = breaker_tests[-1]["input"]
 
-        # ── Evaluate Breaker ──────────────────────────────────────────────
+        # ── 2 & 3. Run sandbox + compute rewards ──────────────────────────
+        coder_info  = self._evaluate_coder(coder_code)
         breaker_info = self._evaluate_breaker(coder_code, breaker_tests, coder_info)
 
-        # ── Log to Coach Memory ───────────────────────────────────────────
+        self._pass_rate_history.append(coder_info["pass_rate"])
+        self._last_coder_pass_rate = coder_info["pass_rate"]
+
+        # ── 4. Update coach memory with rich lesson ───────────────────────
         self.memory.add_lesson(
             episode=self.episode,
             agent="env",
             observation=(
                 f"Step {self.step_count}: "
-                f"coder pass_rate={coder_info['pass_rate']:.2f}, "
-                f"breaker break_rate={breaker_info['break_rate']:.2f}"
+                f"coder={coder_version}, "
+                f"pass_rate={coder_info['pass_rate']:.2f}, "
+                f"breaker_tier={self.breaker.current_tier}, "
+                f"break_rate={breaker_info['break_rate']:.2f}"
             ),
             coder_reward=coder_info["total_reward"],
             breaker_reward=breaker_info["total_reward"],
             extra={
-                "step": self.step_count,
-                "coder_pass_rate": coder_info["pass_rate"],
-                "breaker_break_rate": breaker_info["break_rate"],
+                "step":                self.step_count,
+                "coder_version":       coder_version,
+                "breaker_tier":        self.breaker.current_tier,
+                "coder_pass_rate":     coder_info["pass_rate"],
+                "fail_count":          coder_info["fail_count"],
+                "error_count":         coder_info["error_count"],
+                "timeout_count":       coder_info["error_count"],   # errors include timeouts
+                "breaker_break_rate":  breaker_info["break_rate"],
+                "recent_breaker_case": self._recent_breaker_case,
             },
         )
 
-        # ── Check done ────────────────────────────────────────────────────
+        # ── 5. Log step metrics ───────────────────────────────────────────
+        log_step(
+            episode=self.episode,
+            step=self.step_count,
+            coder_version=coder_version,
+            breaker_tier=self.breaker.current_tier,
+            coder_reward=coder_info["total_reward"],
+            breaker_reward=breaker_info["total_reward"],
+            pass_rate=coder_info["pass_rate"],
+            fail_count=coder_info["fail_count"],
+            error_count=coder_info["error_count"],
+            timeout_count=coder_info["error_count"],
+            break_rate=breaker_info["break_rate"],
+        )
+
+        # ── 6. Advance breaker tier ────────────────────────────────────────
+        self.breaker.update_tier(breaker_info["break_rate"], self.episode)
+
+        # ── 7. Check done + return ────────────────────────────────────────
         if self.step_count >= STEPS_PER_EPISODE:
             self.done = True
 
@@ -120,26 +200,32 @@ class FORGEEnv:
             "breaker_reward": breaker_info,
             "done":           self.done,
             "info": {
-                "episode":    self.episode,
-                "step":       self.step_count,
+                "episode":         self.episode,
+                "step":            self.step_count,
+                "coder_version":   coder_version,
+                "breaker_tier":    self.breaker.current_tier,
+                "breaker_tier_name": self.breaker.tier_name,
             },
         }
 
     def get_state(self) -> dict[str, Any]:
-        """
-        Return the current observable state of the environment.
-        """
+        """Return the current observable state of the environment."""
         return {
-            "episode":        self.episode,
-            "step":           self.step_count,
-            "done":           self.done,
-            "task_prompt":    self.current_task.get("prompt", ""),
-            "public_example": self.current_task.get("public_example", {}),
-            "last_pass_rate": self._last_coder_pass_rate,
+            "task_id":              self.current_task.get("task_id", ""),
+            "problem_description":  self.current_task.get("prompt", ""),
+            "episode":              self.episode,
+            "episode_step":         self.step_count,
+            "done":                 self.done,
+            "coder_version":        self._coder_version,
+            "current_tier":         self.breaker.current_tier,
+            "recent_breaker_case":  self._recent_breaker_case,
+            "pass_rate_history":    list(self._pass_rate_history),
+            "coach_memory_summary": self.memory.summary(),
+            "public_example":       self.current_task.get("public_example", {}),
         }
 
     # ──────────────────────────────────────────────
-    # Private helpers
+    # Private evaluation helpers
     # ──────────────────────────────────────────────
 
     def _evaluate_coder(self, code: str) -> dict[str, Any]:
@@ -147,17 +233,11 @@ class FORGEEnv:
         hidden_tests = self.current_task.get("hidden_tests", [])
 
         if not code or not hidden_tests:
-            # No code submitted — max penalty
-            dummy_results = [{"status": "error"} for _ in hidden_tests or [{}]]
-            info = coder_reward(dummy_results)
-        else:
-            results = run_code_against_tests(code, hidden_tests)
-            info = coder_reward(results)
-
-        # Cache for Breaker quality multiplier
-        self._last_coder_code = code
-        self._last_coder_pass_rate = info["pass_rate"]
-        return info
+            dummy = [{"status": "error"} for _ in hidden_tests or [{}]]
+            return coder_reward(dummy)
+
+        results = run_code_against_tests(code, hidden_tests)
+        return coder_reward(results)
 
     def _evaluate_breaker(
         self,
@@ -165,9 +245,8 @@ class FORGEEnv:
         breaker_tests: list[dict[str, Any]],
         coder_info: dict[str, Any],
     ) -> dict[str, Any]:
-        """Run the coder's code against the breaker's adversarial tests."""
+        """Run the coder's code against breaker adversarial tests."""
         if not coder_code or not breaker_tests:
-            # No submission from one of the agents
             dummy = [{"status": "pass"} for _ in breaker_tests or [{}]]
             return breaker_reward(dummy, coder_base_pass_rate=coder_info["pass_rate"])
 

FORGE-v4/logger.py

@@ -0,0 +1,191 @@
+# logger.py
+# Metrics logging for FORGE-v4.
+# Writes structured logs to logs/rewards.json, logs/episodes.csv, logs/summary.json.
+
+import csv
+import json
+import os
+from datetime import datetime
+from typing import Any
+
+from config import LOG_REWARDS_FILE, LOG_EPISODES_FILE, LOG_SUMMARY_FILE, LOG_DIR
+
+
+# ──────────────────────────────────────────────
+# Internal helpers
+# ──────────────────────────────────────────────
+
+def _ensure_log_dir() -> None:
+    os.makedirs(LOG_DIR, exist_ok=True)
+
+
+def _load_json(path: str, default: Any) -> Any:
+    if os.path.exists(path):
+        try:
+            with open(path, "r", encoding="utf-8") as f:
+                return json.load(f)
+        except (json.JSONDecodeError, IOError):
+            pass
+    return default
+
+
+def _write_json(path: str, data: Any) -> None:
+    with open(path, "w", encoding="utf-8") as f:
+        json.dump(data, f, indent=2)
+
+
+# ──────────────────────────────────────────────
+# Step-level logging
+# ──────────────────────────────────────────────
+
+def log_step(
+    episode: int,
+    step: int,
+    coder_version: str,
+    breaker_tier: int,
+    coder_reward: float,
+    breaker_reward: float,
+    pass_rate: float,
+    fail_count: int,
+    error_count: int,
+    timeout_count: int,
+    break_rate: float,
+) -> None:
+    """
+    Append one step's metrics to logs/rewards.json.
+
+    Args:
+        episode:        Episode index.
+        step:           Step index within the episode.
+        coder_version:  Name of the coder strategy used.
+        breaker_tier:   Current breaker tier number.
+        coder_reward:   Total coder reward this step.
+        breaker_reward: Total breaker reward this step.
+        pass_rate:      Fraction of hidden tests passed.
+        fail_count:     Number of failing tests.
+        error_count:    Number of error/timeout tests.
+        timeout_count:  Number of sandbox timeouts specifically.
+        break_rate:     Fraction of breaker tests that broke the coder.
+    """
+    _ensure_log_dir()
+    records: list[dict[str, Any]] = _load_json(LOG_REWARDS_FILE, [])
+
+    record = {
+        "timestamp":      datetime.utcnow().isoformat(),
+        "episode":        episode,
+        "step":           step,
+        "coder_version":  coder_version,
+        "breaker_tier":   breaker_tier,
+        "coder_reward":   coder_reward,
+        "breaker_reward": breaker_reward,
+        "pass_rate":      pass_rate,
+        "fail_count":     fail_count,
+        "error_count":    error_count,
+        "timeout_count":  timeout_count,
+        "break_rate":     break_rate,
+    }
+    records.append(record)
+    _write_json(LOG_REWARDS_FILE, records)
+
+
+# ──────────────────────────────────────────────
+# Episode-level logging
+# ──────────────────────────────────────────────
+
+# CSV column order
+_CSV_FIELDS = [
+    "timestamp", "episode", "coder_version", "breaker_tier",
+    "avg_coder_reward", "avg_breaker_reward",
+    "avg_pass_rate", "total_fail_count", "total_error_count",
+    "total_timeout_count", "avg_break_rate", "steps",
+]
+
+
+def log_episode(
+    episode: int,
+    coder_version: str,
+    breaker_tier: int,
+    avg_coder_reward: float,
+    avg_breaker_reward: float,
+    avg_pass_rate: float,
+    total_fail_count: int,
+    total_error_count: int,
+    total_timeout_count: int,
+    avg_break_rate: float,
+    steps: int,
+) -> None:
+    """
+    Append one episode summary row to logs/episodes.csv.
+    """
+    _ensure_log_dir()
+    file_exists = os.path.exists(LOG_EPISODES_FILE)
+
+    row = {
+        "timestamp":          datetime.utcnow().isoformat(),
+        "episode":            episode,
+        "coder_version":      coder_version,
+        "breaker_tier":       breaker_tier,
+        "avg_coder_reward":   round(avg_coder_reward, 4),
+        "avg_breaker_reward": round(avg_breaker_reward, 4),
+        "avg_pass_rate":      round(avg_pass_rate, 4),
+        "total_fail_count":   total_fail_count,
+        "total_error_count":  total_error_count,
+        "total_timeout_count":total_timeout_count,
+        "avg_break_rate":     round(avg_break_rate, 4),
+        "steps":              steps,
+    }
+
+    with open(LOG_EPISODES_FILE, "a", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(f, fieldnames=_CSV_FIELDS)
+        if not file_exists:
+            writer.writeheader()
+        writer.writerow(row)
+
+
+# ──────────────────────────────────────────────
+# Summary logging
+# ──────────────────────────────────────────────
+
+def update_summary(
+    total_episodes: int,
+    coder_version: str,
+    final_breaker_tier: int,
+    all_coder_rewards: list[float],
+    all_breaker_rewards: list[float],
+    all_pass_rates: list[float],
+    all_break_rates: list[float],
+    coach_memory_summary: dict[str, Any],
+) -> None:
+    """
+    Overwrite logs/summary.json with the latest aggregate statistics.
+    """
+    _ensure_log_dir()
+
+    def avg(lst: list[float]) -> float:
+        return round(sum(lst) / len(lst), 4) if lst else 0.0
+
+    summary = {
+        "generated_at":         datetime.utcnow().isoformat(),
+        "total_episodes":       total_episodes,
+        "coder_version":        coder_version,
+        "final_breaker_tier":   final_breaker_tier,
+        "avg_coder_reward":     avg(all_coder_rewards),
+        "avg_breaker_reward":   avg(all_breaker_rewards),
+        "avg_pass_rate":        avg(all_pass_rates),
+        "avg_break_rate":       avg(all_break_rates),
+        "best_coder_reward":    round(max(all_coder_rewards), 4) if all_coder_rewards else 0.0,
+        "worst_coder_reward":   round(min(all_coder_rewards), 4) if all_coder_rewards else 0.0,
+        "coach_memory_summary": coach_memory_summary,
+    }
+    _write_json(LOG_SUMMARY_FILE, summary)
+
+
+# ──────────────────────────────────────────────
+# Convenience: print a compact log path report
+# ──────────────────────────────────────────────
+
+def print_log_paths() -> None:
+    """Print the paths of all updated log files."""
+    for path in [LOG_REWARDS_FILE, LOG_EPISODES_FILE, LOG_SUMMARY_FILE]:
+        exists = "✓" if os.path.exists(path) else "✗"
+        print(f"  {exists}  {path}")

FORGE-v4/logs/episodes.csv

@@ -0,0 +1,3 @@
+timestamp,episode,coder_version,breaker_tier,avg_coder_reward,avg_breaker_reward,avg_pass_rate,total_fail_count,total_error_count,total_timeout_count,avg_break_rate,steps
+2026-04-25T08:42:22.041578,1,improving_coder,1,5.0,-1.2,1.0,0,0,0,0.0,3
+2026-04-25T08:42:31.074377,1,weak_coder_v2,1,-1.0,-1.2,0.2,12,0,0,0.0,3

FORGE-v4/logs/rewards.json

@@ -0,0 +1,86 @@
+[
+  {
+    "timestamp": "2026-04-25T08:42:19.501582",
+    "episode": 1,
+    "step": 1,
+    "coder_version": "improving_coder",
+    "breaker_tier": 1,
+    "coder_reward": 5.0,
+    "breaker_reward": -1.2,
+    "pass_rate": 1.0,
+    "fail_count": 0,
+    "error_count": 0,
+    "timeout_count": 0,
+    "break_rate": 0.0
+  },
+  {
+    "timestamp": "2026-04-25T08:42:20.777574",
+    "episode": 1,
+    "step": 2,
+    "coder_version": "improving_coder",
+    "breaker_tier": 1,
+    "coder_reward": 5.0,
+    "breaker_reward": -1.2,
+    "pass_rate": 1.0,
+    "fail_count": 0,
+    "error_count": 0,
+    "timeout_count": 0,
+    "break_rate": 0.0
+  },
+  {
+    "timestamp": "2026-04-25T08:42:22.039872",
+    "episode": 1,
+    "step": 3,
+    "coder_version": "improving_coder",
+    "breaker_tier": 1,
+    "coder_reward": 5.0,
+    "breaker_reward": -1.2,
+    "pass_rate": 1.0,
+    "fail_count": 0,
+    "error_count": 0,
+    "timeout_count": 0,
+    "break_rate": 0.0
+  },
+  {
+    "timestamp": "2026-04-25T08:42:28.577096",
+    "episode": 1,
+    "step": 1,
+    "coder_version": "weak_coder_v2",
+    "breaker_tier": 1,
+    "coder_reward": -1.0,
+    "breaker_reward": -1.2,
+    "pass_rate": 0.2,
+    "fail_count": 4,
+    "error_count": 0,
+    "timeout_count": 0,
+    "break_rate": 0.0
+  },
+  {
+    "timestamp": "2026-04-25T08:42:29.829535",
+    "episode": 1,
+    "step": 2,
+    "coder_version": "weak_coder_v2",
+    "breaker_tier": 1,
+    "coder_reward": -1.0,
+    "breaker_reward": -1.2,
+    "pass_rate": 0.2,
+    "fail_count": 4,
+    "error_count": 0,
+    "timeout_count": 0,
+    "break_rate": 0.0
+  },
+  {
+    "timestamp": "2026-04-25T08:42:31.072423",
+    "episode": 1,
+    "step": 3,
+    "coder_version": "weak_coder_v2",
+    "breaker_tier": 1,
+    "coder_reward": -1.0,
+    "breaker_reward": -1.2,
+    "pass_rate": 0.2,
+    "fail_count": 4,
+    "error_count": 0,
+    "timeout_count": 0,
+    "break_rate": 0.0
+  }
+]

FORGE-v4/logs/summary.json

@@ -0,0 +1,23 @@
+{
+  "generated_at": "2026-04-25T08:42:31.075228",
+  "total_episodes": 1,
+  "coder_version": "weak_coder_v2",
+  "final_breaker_tier": 1,
+  "avg_coder_reward": -1.0,
+  "avg_breaker_reward": -1.2,
+  "avg_pass_rate": 0.2,
+  "avg_break_rate": 0.0,
+  "best_coder_reward": -1.0,
+  "worst_coder_reward": -1.0,
+  "coach_memory_summary": {
+    "total_lessons": 3,
+    "episodes_seen": 1,
+    "avg_coder_reward": -1.0,
+    "avg_breaker_reward": -1.2,
+    "recent_coach_notes": [
+      "Episode 1: Coder (weak_coder_v2) failed 4 test(s) at breaker 1 \u2192 review edge case handling",
+      "Episode 1: Coder (weak_coder_v2) failed 4 test(s) at breaker 1 \u2192 review edge case handling",
+      "Episode 1: Coder (weak_coder_v2) failed 4 test(s) at breaker 1 \u2192 review edge case handling"
+    ]
+  }
+}

FORGE-v4/memory.py

@@ -1,6 +1,6 @@
 # memory.py
 # Coach Memory system for FORGE-v4.
-# Stores lessons learned across episodes in a JSON file.
+# Stores structured lessons learned across episodes in a JSON file.
 
 import json
 import os
@@ -14,6 +14,8 @@ class CoachMemory:
     Persistent memory that accumulates lessons learned across training episodes.
 
     Lessons are stored as a list of dicts in a JSON file and loaded on startup.
+    Each lesson includes a human-readable "coach_note" derived from the metrics
+    so the history is understandable without post-processing.
     """
 
     def __init__(self, filepath: str = MEMORY_FILE):
@@ -41,16 +43,19 @@ class CoachMemory:
         Args:
             episode:        Episode index.
             agent:          "coder" | "breaker" | "env".
-            observation:    Human-readable description of what happened.
+            observation:    Raw observation string from the environment.
             coder_reward:   Total coder reward for this step.
             breaker_reward: Total breaker reward for this step.
-            extra:          Optional additional metadata.
+            extra:          Optional metadata (pass_rate, fail_count, etc.).
         """
+        coach_note = self._derive_coach_note(episode, extra or {})
+
         lesson = {
             "timestamp":      datetime.utcnow().isoformat(),
             "episode":        episode,
             "agent":          agent,
             "observation":    observation,
+            "coach_note":     coach_note,
             "coder_reward":   coder_reward,
             "breaker_reward": breaker_reward,
         }
@@ -60,16 +65,13 @@ class CoachMemory:
         self.lessons.append(lesson)
         self.save()
 
-    def get_lessons(self, agent: str | None = None, last_n: int | None = None) -> list[dict[str, Any]]:
+    def get_lessons(
+        self,
+        agent: str | None = None,
+        last_n: int | None = None,
+    ) -> list[dict[str, Any]]:
         """
         Retrieve stored lessons, optionally filtered by agent and/or limited to the last N.
-
-        Args:
-            agent:  Filter to a specific agent ("coder", "breaker", "env"), or None for all.
-            last_n: Return only the last N lessons if provided.
-
-        Returns:
-            List of lesson dicts.
         """
         result = self.lessons
         if agent is not None:
@@ -78,28 +80,29 @@ class CoachMemory:
             result = result[-last_n:]
         return result
 
+    def get_coach_notes(self, last_n: int = 5) -> list[str]:
+        """Return the most recent human-readable coach notes."""
+        return [l["coach_note"] for l in self.lessons[-last_n:] if l.get("coach_note")]
+
     def summary(self) -> dict[str, Any]:
-        """
-        Return a high-level summary of stored lessons.
-        """
+        """Return a high-level summary of stored lessons."""
         if not self.lessons:
             return {"total_lessons": 0, "episodes_seen": 0}
 
         episodes = {l["episode"] for l in self.lessons}
-        coder_rewards = [l["coder_reward"] for l in self.lessons]
+        coder_rewards   = [l["coder_reward"]   for l in self.lessons]
         breaker_rewards = [l["breaker_reward"] for l in self.lessons]
 
         return {
             "total_lessons":      len(self.lessons),
             "episodes_seen":      len(episodes),
-            "avg_coder_reward":   round(sum(coder_rewards) / len(coder_rewards), 4),
+            "avg_coder_reward":   round(sum(coder_rewards)   / len(coder_rewards),   4),
             "avg_breaker_reward": round(sum(breaker_rewards) / len(breaker_rewards), 4),
+            "recent_coach_notes": self.get_coach_notes(last_n=3),
         }
 
     def clear(self) -> None:
-        """
-        Wipe all stored lessons (use with caution).
-        """
+        """Wipe all stored lessons (use with caution)."""
         self.lessons = []
         self.save()
 
@@ -119,11 +122,89 @@ class CoachMemory:
                 with open(self.filepath, "r", encoding="utf-8") as f:
                     self.lessons = json.load(f)
             except (json.JSONDecodeError, IOError):
-                # Start fresh if file is corrupted
                 self.lessons = []
         else:
             self.lessons = []
 
+    # ──────────────────────────────────────────────
+    # Coach note derivation
+    # ──────────────────────────────────────────────
+
+    def _derive_coach_note(self, episode: int, extra: dict[str, Any]) -> str:
+        """
+        Generate a human-readable coaching note from step metadata.
+
+        Examples:
+            "Episode 4: Coder failed on duplicates → handle duplicate values safely"
+            "Episode 8: Coder timed out on large arrays → avoid O(n²) for large inputs"
+            "Episode 2: Strong performance (pass_rate=1.00) → keep current strategy"
+        """
+        pass_rate     = extra.get("coder_pass_rate",    None)
+        fail_count    = extra.get("fail_count",          0)
+        error_count   = extra.get("error_count",         0)
+        timeout_count = extra.get("timeout_count",       0)
+        breaker_tier  = extra.get("breaker_tier",        1)
+        coder_version = extra.get("coder_version",      "unknown")
+        recent_case   = extra.get("recent_breaker_case", [])
+
+        prefix = f"Episode {episode}"
+
+        # Timeout note
+        if timeout_count > 0:
+            return (
+                f"{prefix}: Coder timed out on {timeout_count} test(s)"
+                f" [tier={breaker_tier}] → avoid O(n²) or infinite loops for large inputs"
+            )
+
+        # Error note
+        if error_count > 0 and pass_rate is not None and pass_rate < 0.5:
+            return (
+                f"{prefix}: Coder raised errors on {error_count} test(s)"
+                f" → add input validation and handle edge cases"
+            )
+
+        # Negative/duplicate failure detection from recent breaker case
+        if fail_count > 0 and recent_case:
+            has_neg  = any(x < 0 for x in recent_case)
+            has_dups = len(recent_case) != len(set(recent_case))
+            is_large = len(recent_case) >= 10
+
+            if has_neg and has_dups:
+                return (
+                    f"{prefix}: Coder ({coder_version}) failed on negatives+duplicates"
+                    f" → ensure sort key uses true value, not abs()"
+                )
+            if has_neg:
+                return (
+                    f"{prefix}: Coder ({coder_version}) failed on negative values"
+                    f" → handle negative integers in comparison logic"
+                )
+            if has_dups:
+                return (
+                    f"{prefix}: Coder ({coder_version}) failed on duplicate values"
+                    f" → ensure stable sort handles equal elements correctly"
+                )
+            if is_large:
+                return (
+                    f"{prefix}: Coder ({coder_version}) failed on large array (n={len(recent_case)})"
+                    f" → consider O(n log n) algorithm"
+                )
+            return (
+                f"{prefix}: Coder ({coder_version}) failed {fail_count} test(s)"
+                f" at breaker {breaker_tier} → review edge case handling"
+            )
+
+        # Good performance
+        if pass_rate is not None and pass_rate >= 0.8:
+            return (
+                f"{prefix}: Strong performance (pass_rate={pass_rate:.2f})"
+                f" [{coder_version}] → breaker should escalate tier"
+            )
+
+        # Generic fallback
+        pr = f"{pass_rate:.2f}" if pass_rate is not None else "N/A"
+        return f"{prefix}: pass_rate={pr}, fail={fail_count}, errors={error_count}"
+
     # ──────────────────────────────────────────────
     # Internal helpers
     # ──────────────────────────────────────────────

FORGE-v4/trainer.py

@@ -1,47 +1,44 @@
 # trainer.py
-# Placeholder training loop hooks for FORGE-v4.
-# Ready for future TRL / Unsloth / Hugging Face integration.
+# Training loop for FORGE-v4.
+# Uses the real coder strategies and tiered BreakerAgent from agents.py.
+# Hook placeholders are ready for TRL / Unsloth / Hugging Face integration.
 
 from typing import Any, Callable
 from env import FORGEEnv
 from memory import CoachMemory
+from agents import get_coder_code, coder_version_label
+from logger import log_episode, update_summary
 from config import MAX_EPISODES, STEPS_PER_EPISODE
 
 
 # ──────────────────────────────────────────────
-# Placeholder agent policy functions
+# Built-in coder policies
 # ──────────────────────────────────────────────
 
-def default_coder_policy(state: dict[str, Any]) -> str:
+def make_coder_policy(version: str) -> Callable[[dict[str, Any]], dict[str, str]]:
     """
-    Placeholder Coder policy.
+    Factory: return a coder policy function for the given version name.
 
-    In production this will call a fine-tuned LLM (e.g. via TRL/Unsloth) to
-    generate Python code from the task prompt.
+    The returned callable takes a state dict and returns an action dict:
+        {"coder_code": str, "coder_version": str}
 
-    Currently returns a trivial reference solution so the environment runs.
+    Args:
+        version: "weak_coder_v1" | "weak_coder_v2" | "improving_coder"
     """
-    # TODO: Replace with LLM inference call
-    return "def solution(arr):\n    return sorted(arr)\n"
-
+    def policy(state: dict[str, Any]) -> dict[str, str]:
+        episode = state.get("episode", 1)
+        code    = get_coder_code(version, episode=episode)
+        return {"coder_code": code, "coder_version": version}
+    return policy
 
-def default_breaker_policy(state: dict[str, Any]) -> list[dict[str, Any]]:
-    """
-    Placeholder Breaker policy.
 
-    In production this will call a fine-tuned adversarial LLM to generate
-    adversarial test cases from the task prompt.
+# Convenience pre-built policies
+weak_coder_v1_policy    = make_coder_policy("weak_coder_v1")
+weak_coder_v2_policy    = make_coder_policy("weak_coder_v2")
+improving_coder_policy  = make_coder_policy("improving_coder")
 
-    Currently returns a fixed set of edge-case test inputs.
-    """
-    # TODO: Replace with adversarial LLM inference call
-    return [
-        {"input": [],                             "expected_output": []},
-        {"input": [1],                            "expected_output": [1]},
-        {"input": [3, 1, 2],                      "expected_output": [1, 2, 3]},
-        {"input": [-5, -1, -3],                   "expected_output": [-5, -3, -1]},
-        {"input": [0, 0, 0, 0],                   "expected_output": [0, 0, 0, 0]},
-    ]
+# Default used by app.py
+default_coder_policy    = improving_coder_policy
 
 
 # ──────────────────────────────────────────────
@@ -49,81 +46,133 @@ def default_breaker_policy(state: dict[str, Any]) -> list[dict[str, Any]]:
 # ──────────────────────────────────────────────
 
 def train(
-    coder_policy: Callable[[dict[str, Any]], str] = default_coder_policy,
-    breaker_policy: Callable[[dict[str, Any]], list[dict[str, Any]]] = default_breaker_policy,
+    coder_policy: Callable[[dict[str, Any]], dict[str, str]] = default_coder_policy,
     num_episodes: int = MAX_EPISODES,
     verbose: bool = True,
 ) -> dict[str, Any]:
     """
     Run the FORGE-v4 training loop.
 
+    The BreakerAgent is managed by the environment — it automatically tiers up
+    based on performance. Only the coder policy needs to be supplied here.
+
     Args:
-        coder_policy:   Callable(state) → Python source string.
-        breaker_policy: Callable(state) → list of test-case dicts.
-        num_episodes:   Number of training episodes to run.
-        verbose:        Print per-episode summaries when True.
+        coder_policy:  Callable(state) → {"coder_code": str, "coder_version": str}
+        num_episodes:  Number of episodes to run.
+        verbose:       Print per-episode summaries when True.
 
     Returns:
-        Training summary dict with per-episode reward histories.
+        Training summary dict.
     """
     memory = CoachMemory()
-    env = FORGEEnv(memory=memory)
+    env    = FORGEEnv(memory=memory)
 
     episode_history: list[dict[str, Any]] = []
 
+    # Aggregate accumulators for final summary
+    all_coder_rewards:   list[float] = []
+    all_breaker_rewards: list[float] = []
+    all_pass_rates:      list[float] = []
+    all_break_rates:     list[float] = []
+
     for ep in range(1, num_episodes + 1):
         state = env.reset()
-        episode_coder_rewards   = []
-        episode_breaker_rewards = []
 
-        for _ in range(STEPS_PER_EPISODE):
-            # ── Agent decisions ────────────────────────────────────────────
-            coder_code    = coder_policy(state)
-            breaker_tests = breaker_policy(state)
-
-            action = {
-                "coder_code":    coder_code,
-                "breaker_tests": breaker_tests,
-            }
+        ep_coder_rewards:   list[float] = []
+        ep_breaker_rewards: list[float] = []
+        ep_pass_rates:      list[float] = []
+        ep_fail_counts:     list[int]   = []
+        ep_error_counts:    list[int]   = []
+        ep_timeout_counts:  list[int]   = []
+        ep_break_rates:     list[float] = []
 
-            # ── Environment step ───────────────────────────────────────────
+        for _ in range(STEPS_PER_EPISODE):
+            action = coder_policy(state)
             result = env.step(action)
             state  = result["state"]
 
-            episode_coder_rewards.append(result["coder_reward"]["total_reward"])
-            episode_breaker_rewards.append(result["breaker_reward"]["total_reward"])
+            cr = result["coder_reward"]
+            br = result["breaker_reward"]
+
+            ep_coder_rewards.append(cr["total_reward"])
+            ep_breaker_rewards.append(br["total_reward"])
+            ep_pass_rates.append(cr["pass_rate"])
+            ep_fail_counts.append(cr["fail_count"])
+            ep_error_counts.append(cr["error_count"])
+            ep_timeout_counts.append(cr["error_count"])
+            ep_break_rates.append(br["break_rate"])
 
             if result["done"]:
                 break
 
         # ── Episode summary ────────────────────────────────────────────────
-        avg_cr = round(sum(episode_coder_rewards)   / len(episode_coder_rewards),   4)
-        avg_br = round(sum(episode_breaker_rewards) / len(episode_breaker_rewards), 4)
+        def avg(lst: list) -> float:
+            return round(sum(lst) / len(lst), 4) if lst else 0.0
 
         ep_summary = {
             "episode":              ep,
-            "avg_coder_reward":     avg_cr,
-            "avg_breaker_reward":   avg_br,
+            "coder_version":        action.get("coder_version", "unknown"),
+            "breaker_tier":         env.breaker.current_tier,
+            "avg_coder_reward":     avg(ep_coder_rewards),
+            "avg_breaker_reward":   avg(ep_breaker_rewards),
+            "avg_pass_rate":        avg(ep_pass_rates),
+            "avg_break_rate":       avg(ep_break_rates),
             "steps":                env.step_count,
         }
         episode_history.append(ep_summary)
 
+        # ── Log episode to CSV ─────────────────────────────────────────────
+        log_episode(
+            episode=ep,
+            coder_version=ep_summary["coder_version"],
+            breaker_tier=ep_summary["breaker_tier"],
+            avg_coder_reward=ep_summary["avg_coder_reward"],
+            avg_breaker_reward=ep_summary["avg_breaker_reward"],
+            avg_pass_rate=ep_summary["avg_pass_rate"],
+            total_fail_count=sum(ep_fail_counts),
+            total_error_count=sum(ep_error_counts),
+            total_timeout_count=sum(ep_timeout_counts),
+            avg_break_rate=ep_summary["avg_break_rate"],
+            steps=ep_summary["steps"],
+        )
+
+        # ── Accumulate for final summary ───────────────────────────────────
+        all_coder_rewards.extend(ep_coder_rewards)
+        all_breaker_rewards.extend(ep_breaker_rewards)
+        all_pass_rates.extend(ep_pass_rates)
+        all_break_rates.extend(ep_break_rates)
+
         if verbose:
+            label = coder_version_label(ep_summary["coder_version"], ep)
             print(
-                f"[Episode {ep:>4}/{num_episodes}]  "
-                f"Coder avg reward: {avg_cr:+.4f}  |  "
-                f"Breaker avg reward: {avg_br:+.4f}"
+                f"  [Ep {ep:>3}]  Coder: {label:<50}  "
+                f"pass={ep_summary['avg_pass_rate']:.2f}  "
+                f"reward={ep_summary['avg_coder_reward']:+.2f}  |  "
+                f"Breaker: {env.breaker.tier_name:<22}  "
+                f"break={ep_summary['avg_break_rate']:.2f}  "
+                f"reward={ep_summary['avg_breaker_reward']:+.2f}"
             )
 
-        # ── TRL / Unsloth hook placeholders ───────────────────────────────
+        # ── TRL / Unsloth hook ─────────────────────────────────────────────
         _on_episode_end(ep, ep_summary, memory)
 
-    training_summary = {
-        "total_episodes":      num_episodes,
-        "episode_history":     episode_history,
-        "memory_summary":      memory.summary(),
+    # ── Final summary JSON ────────────────────────────────────────────────
+    update_summary(
+        total_episodes=num_episodes,
+        coder_version=episode_history[-1]["coder_version"] if episode_history else "unknown",
+        final_breaker_tier=env.breaker.current_tier,
+        all_coder_rewards=all_coder_rewards,
+        all_breaker_rewards=all_breaker_rewards,
+        all_pass_rates=all_pass_rates,
+        all_break_rates=all_break_rates,
+        coach_memory_summary=memory.summary(),
+    )
+
+    return {
+        "total_episodes":  num_episodes,
+        "episode_history": episode_history,
+        "memory_summary":  memory.summary(),
     }
-    return training_summary
 
 
 # ──────────────────────────────────────────────
@@ -136,23 +185,20 @@ def _on_episode_end(
     memory: CoachMemory,
 ) -> None:
     """
-    Called at the end of every episode.
+    Called at end of every episode.
 
     TODO: Plug in TRL PPOTrainer / Unsloth model updates here.
     E.g.:
         trainer.step(queries, responses, rewards)
         model.save_pretrained(f"models/checkpoint-ep{episode}")
     """
-    pass  # placeholder
+    pass
 
 
-def _on_step_end(
-    step: int,
-    result: dict[str, Any],
-) -> None:
+def _on_step_end(step: int, result: dict[str, Any]) -> None:
     """
     Called after every environment step.
 
-    TODO: Plug in per-step reward logging (e.g. W&B, TensorBoard) here.
+    TODO: Plug in per-step reward logging (W&B, TensorBoard) here.
     """
-    pass  # placeholder
+    pass

attached_assets/Pasted-Upgrade-the-existing-FORGE-v4-project-from-starter-leve_1777106296176.txt

@@ -0,0 +1,163 @@
+Upgrade the existing FORGE-v4 project from starter-level skeleton into a stronger hackathon-ready backend prototype.
+
+Do NOT rebuild from scratch. Modify the current files intelligently.
+
+Current Issues To Fix:
+
+1. Placeholder Coder currently uses Python sorted() and is too perfect.
+2. Breaker attacks are static and too weak.
+3. Reward logs / metrics are not realistic enough.
+4. OpenEnv environment state/action flow needs stronger clarity.
+
+Your task is to upgrade the existing project with the following improvements:
+
+---
+
+## A. Replace Perfect Placeholder Coder
+
+Create multiple baseline coder strategies inside the project:
+
+1. weak_coder_v1
+
+* bubble sort style
+* slow for large arrays
+
+2. weak_coder_v2
+
+* handles normal arrays
+* fails on duplicates or negatives sometimes
+
+3. improving_coder
+
+* chooses stronger strategy based on episode count
+
+Use these instead of always using sorted().
+
+This is important so learning progress can be shown later.
+
+---
+
+## B. Upgrade Breaker into Tiered Adversarial System
+
+Implement breaker difficulty tiers.
+
+Tier 1:
+[]
+[1]
+[2,1]
+
+Tier 2:
+duplicates
+negative values
+already sorted
+reverse sorted
+
+Tier 3:
+large arrays
+many duplicates
+hard distributions
+
+Tier 4:
+boundary integer values
+stress tests
+
+Unlock next tier based on breaker success rate or episode progress.
+
+Breaker should dynamically choose test cases based on current tier.
+
+---
+
+## C. Add Real Metrics + Reward Logging
+
+Create logs folder outputs such as:
+
+logs/rewards.json
+logs/episodes.csv
+logs/summary.json
+
+Track:
+
+* episode number
+* coder reward
+* breaker reward
+* pass rate
+* current tier
+* number of failed tests
+* timeout count
+
+Also create helper functions to export metrics cleanly.
+
+---
+
+## D. Improve OpenEnv Style Clarity
+
+In env.py make state/action flow cleaner.
+
+Environment state should include:
+
+{
+task_id,
+problem_description,
+episode_step,
+coder_version,
+current_tier,
+recent_breaker_case,
+pass_rate_history,
+coach_memory_summary
+}
+
+step(action) should clearly:
+
+1. run coder
+2. run breaker
+3. sandbox evaluate
+4. assign rewards
+5. update memory
+6. log metrics
+7. return next_state
+
+---
+
+## E. Improve Coach Memory
+
+Store lessons like:
+
+Episode 4:
+Coder failed on duplicates
+Lesson: handle duplicate values safely
+
+Episode 8:
+Coder timed out on large arrays
+Lesson: avoid O(n²) for large arrays
+
+---
+
+## F. Keep Existing Structure
+
+Do not remove current modular structure.
+
+Files should still use:
+
+app.py
+env.py
+tasks.py
+rewards.py
+sandbox.py
+memory.py
+trainer.py
+config.py
+
+---
+
+## G. Final Result Needed
+
+After modifications, python app.py should run successfully and show:
+
+* coder version used
+* breaker tier used
+* test result summary
+* rewards
+* logs updated
+* coach lessons updated
+
+Keep code clean, modular, production-ready, and easy for later VS Code + Copilot + Google Colab upgrades.

replit.md

@@ -25,3 +25,43 @@ pnpm workspace monorepo using TypeScript. Each package manages its own dependenc
 - `pnpm --filter @workspace/api-server run dev` — run API server locally
 
 See the `pnpm-workspace` skill for workspace structure, TypeScript setup, and package details.
+
+## FORGE-v4  (Python — Adversarial RL Environment)
+
+Located at `FORGE-v4/`. A standalone Python project; run independently of the pnpm workspace.
+
+### Quick start
+```bash
+cd FORGE-v4
+python3 app.py                         # improving_coder vs tiered Breaker
+python3 app.py --coder weak_coder_v1   # bubble sort strategy
+python3 app.py --coder weak_coder_v2   # selection sort w/ abs() bug
+python3 app.py --steps 5              # override step count
+```
+
+### Key files
+| File | Purpose |
+|------|---------|
+| `app.py` | CLI entry point |
+| `env.py` | `FORGEEnv` — reset/step/get_state |
+| `agents.py` | Coder strategies + `BreakerAgent` (tiered) |
+| `tasks.py` | Task and hidden test generation |
+| `sandbox.py` | Subprocess code execution with timeout |
+| `rewards.py` | `coder_reward()` / `breaker_reward()` |
+| `memory.py` | `CoachMemory` — JSON-backed lessons |
+| `logger.py` | Writes `logs/rewards.json`, `logs/episodes.csv`, `logs/summary.json` |
+| `trainer.py` | Training loop + TRL/Unsloth hook placeholders |
+| `config.py` | All constants |
+
+### Coder strategies
+- `weak_coder_v1` — bubble sort (O(n²), slow on large arrays)
+- `weak_coder_v2` — selection sort with abs() bug (fails on negatives)
+- `improving_coder` — bubble sort → selection sort → `sorted()` by episode
+
+### Breaker tiers
+- Tier 1: empty / single element / tiny arrays
+- Tier 2: duplicates, negatives, sorted/reverse-sorted
+- Tier 3: large arrays, heavy duplicates, stress cases
+- Tier 4: boundary integers (±100), extreme stress
+
+Tier unlocks at 60% break rate; Tier 3 needs episode ≥ 4, Tier 4 needs episode ≥ 7.