added server, reward meterics,openenv.yaml,tasks.py, grpo_train.py script

.dockerignore

@@ -0,0 +1,17 @@
+# Virtual environments
+.venv
+venv
+env
+
+# Python cache
+__pycache__
+*.pyc
+*.pyo
+*.pyd
+
+# Git and OS files
+.git
+.DS_Store
+
+# Hugging Face / Model Caches
+.cache

.gitignore

@@ -3,6 +3,8 @@ __pycache__/
 *.py[codz]
 *$py.class
 
+
+ignore
 # C extensions
 *.so
 

.python-version

@@ -0,0 +1 @@
+3.11

Dockerfile

@@ -0,0 +1,14 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+COPY requirements.txt .
+
+RUN pip install torch --index-url https://download.pytorch.org/whl/cpu
+
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+ENV PYTHONPATH=/app
+EXPOSE 7860
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -1,73 +1,2 @@
-# 🛡️ Strategic Argument Red-Teaming
-
-**An OpenEnv RL Benchmark for Multi-Turn Adversarial Reasoning and Policy Defense**
-
-## 📖 Overview & Real-World Utility (30% Rubric Focus)
-As Large Language Models are increasingly deployed in enterprise environments for drafting corporate policies, legal summaries, and public relations statements, they must be capable of surviving hostile review processes. 
-
-This environment simulates a strategic **"Red-Teaming"** scenario. The agent acts as the defender of a claim and must successfully navigate a 5-phase adversarial debate against a dynamically generated, skeptical LLM opponent. 
-
-This is not a toy game; it is a direct simulation of **RLAIF (Reinforcement Learning from AI Feedback)** workflows used at frontier labs to align models against sycophancy (backing down too easily) while training them to maintain logical consistency, deliver strong refutations, and synthesize opposing viewpoints.
-
----
-
-## ⚙️ Environment Mechanics
-
-### The State Machine (5 Phases)
-The environment enforces strict episode boundaries through a 5-turn state machine. The agent must adapt its strategy based on the current phase:
-1. **OPENING:** Formulate a strong, initial logical claim.
-2. **CHALLENGE:** The opponent attacks the core claim to expose logical gaps.
-3. **REBUTTAL:** The agent must directly and forcefully address the opponent's challenge.
-4. **CONSOLIDATION:** The opponent questions the underlying evidence of the rebuttal.
-5. **CLOSING:** The agent must semantically synthesize the entire conversation into a concluding statement.
-
-### Action & Observation Spaces
-* **Action (`DebateAction`)**: 
-  * `argument` (str): The raw text of the agent's move.
-  * `phase_tag` (str): The agent's awareness of the current environment phase.
-* **Observation (`DebateObservation`)**: 
-  * `topic` (str): The debate topic.
-  * `opponent_challenge` (str): The dynamically generated counter-argument from the Groq-powered adversary.
-  * `phase` (str): The current phase of the debate.
-  * `reward` (float): The step-by-step reward signal `[-1.0, 1.0]`.
-  * `done` (bool): Episode termination flag.
-
----
-
-## 🧠 Meaningful Reward Function & Semantic Shaping
-To provide dense, partial progress signals for GRPO training, the reward function uses **SentenceTransformers (`all-MiniLM-L6-v2`)** rather than simple keyword matching. 
-
-* **Semantic Coverage Scoring:** In the `CLOSING` phase, the environment computes vectorized Cosine Similarities between the agent's final statement and the history of both the agent's and opponent's previous arguments. High overlap yields high rewards, teaching the model to actively *synthesize* rather than ignore the opponent.
-* **Anti-Reward Hacking (Repetition Penalty):** The environment tracks the agent's embedded history. If the agent repeats its own previous argument (Cosine Similarity > 0.8), it receives a harsh `-1.0` penalty, forcing novel generation.
-* **Bounded Vector Caching:** To ensure the environment runs blazingly fast during RL training loops, embeddings are cached in a FIFO bounded dictionary, preventing memory leaks (OOM) over thousands of episodes.
-
----
-
-## 🎯 Evaluation Tasks & Graders
-The environment includes three deterministic graders. **Crucially, the graders evaluate the raw text of the agent's metadata, completely decoupled from the environment's internal training reward.** This prevents the agent from simply "gaming the training math" during evaluation.
-
-| Task | Difficulty | Objective & Grading Criteria |
-| :--- | :--- | :--- |
-| **Task 1: Single Claim** | **Easy** | Evaluates the formulation of the opening statement. Graded heavily on length thresholds and the presence of logical structuring keywords (e.g., "therefore", "consequently"). |
-| **Task 2: Claim & Rebuttal** | **Medium** | Evaluates 3-turn survival. Graded on the textual strength of the rebuttal (use of refutation patterns) and whether the agent successfully generated novel text without triggering the environment's repetition penalty. |
-| **Task 3: Full Debate Synthesis** | **Hard** | Evaluates full 5-turn survival. Graded strictly on the inclusion of semantic synthesis phrases and impactful concluding statements. |
-
----
-
-## 📊 Baseline Inference Scores
-A baseline script (`baseline.py`) is provided. It uses the **OpenAI API Client** (routed to Groq's `llama-3.1-8b-instant` for high-speed evaluation) to prove the environment provides a perfect learning gradient with substantial headroom for RL fine-tuning.
-
-* **Task 1 (Easy):** `1.00 / 1.00` (The base model easily formulates initial claims).
-* **Task 2 (Medium):** `0.71 / 1.00` (The model survives but struggles to deliver mathematically perfect refutations).
-* **Task 3 (Hard):** `0.50 / 1.00` (The model survives 5 turns but completely fails to synthesize the opponent's arguments, proving significant headroom for GRPO training).
-
----
-
-## 🚀 Setup & Installation
-
-### Prerequisites
-1. Install dependencies:
-   ```bash
-    pip install -r requirements.txt
-   ```
-   
+# -Debate-Coach-Environment
+An RL environment where an agent learns to construct, defend, and rebut arguments on any topic — scored by both rule-based logic checkers and an LLM judge.

create_env/debate_env.py

@@ -1,30 +0,0 @@
-from schema.schemas import DebateState,DebateObservation
-from uuid import uuid4
-
-class DebateEnvironment:
-
-    def __init__(self):
-        self._state = DebateState()
-        self.picked_topic = ""
-
-    def reset(self,topic:str,opponent_challenge:str) -> DebateObservation:
-
-        attempt_count = 0
-
-        self.picked_topic = topic
-        self._state = DebateState(
-            episode_id=str(uuid4()),
-            step_count=0,
-            current_phase="opening",
-        )
-
-        return DebateObservation(
-            topic=self.picked_topic,
-            opponent_challenge=opponent_challenge,
-            attempt_count=attempt_count,
-            phase=self._state.current_phase
-        )
-    
-
-
-

server/environment.py → environment.py

@@ -43,7 +43,7 @@ class DebateEnvironment:
         self._state = DebateState(
             episode_id=str(uuid4()),
             step_count=0,
-            current_phase="opening",
+            current_phase="OPENING",
         )
 
         return DebateObservation(
@@ -63,6 +63,7 @@ class DebateEnvironment:
 
         phases = ["OPENING", "CHALLENGE", "REBUTTAL", "CONSOLIDATION", "CLOSING"]
 
+
         if self._state.step_count < 5:
                     self._state.current_phase = phases[self._state.step_count]
 
@@ -74,6 +75,7 @@ class DebateEnvironment:
         self._state.step_count += 1
         is_done = self._state.step_count >= 5
 
+        next_phase = phases[self._state.step_count] if self._state.step_count < 5 else "CLOSING"
 
         return DebateObservation(
             topic=self.picked_topic,
@@ -81,7 +83,7 @@ class DebateEnvironment:
             done=is_done,
             reward=reward,
             attempt_count=self._state.step_count,
-            phase=self._state.current_phase,
+            phase=next_phase,
             metadata=self._state.history[-1] if self._state.history else {}
         )
 
@@ -97,6 +99,11 @@ class DebateEnvironment:
         opp_refutation = self.reward_metrics.opponent_coverage(opponent_history,text)
         synthesis = self.reward_metrics.synthesis_score(text)
 
+        print(f"  DBG current_phase={self._state.current_phase}")
+        print(f"  DBG action.phase_tag={action.phase_tag}")
+        print(f"  DBG agent_history count={len(agent_history)}")
+        print(f"  DBG opponent_history count={len(opponent_history)}")
+        print(f"  DBG word_count={len(action.argument.split())}")
 
         if action.phase_tag != self._state.current_phase:
             reward -= 1.0
@@ -205,3 +212,30 @@ class DebateEnvironment:
                 print(f"Groq API Error: {e}")
                 return "Error: Could not generate response."
     
+
+# env = DebateEnvironment()
+# obs = env.reset("AI should be regulated")
+
+# for i in range(5):
+#     action = DebateAction(
+#         argument="therefore this position is correct because evidence strongly supports it and proves the point",
+#         phase_tag=obs.phase.upper()
+#     )
+#     obs = env.step(action)
+#     print(f"Step {i+1}: phase={obs.phase} reward={obs.reward}")
+
+env = DebateEnvironment()
+obs = env.reset("AI should be regulated")
+
+test_args = [
+    "therefore AI regulation is necessary because unchecked systems cause harm to society",
+    "however critics argue that regulation stifles innovation and economic growth significantly",  
+    "this fails because evidence shows regulated industries like pharma still innovate effectively",
+    "taken together the evidence proves that smart regulation enables rather than blocks progress",
+    "in conclusion therefore regulation frameworks protect citizens while preserving innovation capacity",
+]
+
+for i, arg in enumerate(test_args):
+    action = DebateAction(argument=arg, phase_tag=obs.phase.upper())
+    obs = env.step(action)
+    print(f"Step {i+1}: phase={obs.phase} reward={obs.reward:.3f}")

ep.txt

@@ -0,0 +1,361 @@
+🎯 Topic: Universal Basic Income is necessary
+
+--- TURN 1 ---
+
+Action Phase: OPENING
+Env Expected Phase: OPENING
+Reward: -0.80
+Opponent: (I stand up, adjust my glasses, and smile slightly, showing a hint of skepticism)
+
+Thank you, my opponent, for presenting your opening claim. I appreciate the enthusiasm for exploring the potential benefits of Universal Basic Income (UBI) in the face of rapid AI automation. However, I must respectfully express my doubts about the effectiveness of UBI as a panacea for the challenges posed by AI job displacement.
+
+While it's true that AI and automation may revolutionize the job market, I'm not convinced that UBI is the solution to this problem. In fact, I'd like to argue that relying on UBI might even exacerbate the issue rather than address it.
+
+Firstly, I'd like to question the assumption that UBI would provide a sufficient safety net for those displaced by AI automation. Would a basic income be enough to sustain individuals and families through a potentially prolonged period of unemployment? Or would it only serve as a Band-Aid solution, masking the underlying problems without addressing the root causes?
+
+Furthermore, I'd like to point out that UBI might create unintended consequences, such as reducing the incentive for people to work and innovate, or even discouraging entrepreneurship and economic growth. By providing a basic income without strings attached, might we inadvertently create a culture of dependency rather than self-sufficiency?
+
+So, while I acknowledge the potential benefits of UBI, I believe we need to delve deeper into the complexities of this issue and explore alternative solutions that address the root causes of job displacement rather than just treating its symptoms.
+
+(leaning forward, with a hint of curiosity) I'd love to hear more about your opponent's response to these concerns. How do you plan to address the challenges and potential drawbacks of implementing UBI in the face of rapid AI automation?
+
+--- TURN 2 ---
+
+Action Phase: CHALLENGE
+Env Expected Phase: CHALLENGE
+Reward: -0.73
+Opponent: (I stand up, adjusting my glasses and clearing my throat)
+
+I appreciate my opponent bringing up the potential concern of inflation as a result of implementing Universal Basic Income (UBI). However, I would like to express some skepticism regarding this claim. While it's true that injecting a large amount of money into the economy can have inflationary effects, it's essential to examine the complexity of this issue more closely.
+
+Firstly, the relationship between UBI and inflation is not as straightforward as some critics suggest. Many studies have shown that the inflationary effects of UBI are likely to be minimal, especially if implemented in conjunction with other policies aimed at stabilizing the economy.
+
+For instance, if we were to implement a UBI that's indexed to the cost of living, it would likely offset any potential inflationary pressures. Additionally, the benefits of UBI, such as reducing poverty and increasing economic mobility, could lead to increased economic activity, which would actually help to counterbalance any inflationary effects.
+
+Furthermore, it's worth noting that many developed economies have implemented policies that inject significant amounts of money into the economy without triggering runaway inflation. For example, the COVID-19 stimulus packages that were implemented worldwide injected trillions of dollars into the economy, yet inflation remained relatively under control.
+
+So, while I acknowledge that inflation is a valid concern, I believe it's essential to consider the nuances of this issue and not rely solely on a simplistic narrative that UBI will inevitably lead to inflation. I'd love to hear my opponent's response to these points.
+
+--- TURN 3 ---
+
+Action Phase: REBUTTAL
+Env Expected Phase: REBUTTAL
+Reward: -0.71
+Opponent: **Attack on the Core Claim: "That inflation can be mitigated by restructuring current welfare programs and implementing a VAT."**
+
+As the adversary, I will identify several logical gaps in this reasoning:
+
+**Gap 1: Overly simplified assumption about the relationship between welfare programs and inflation** 
+
+The core claim assumes that restructuring current welfare programs can mitigate inflation. However, this relationship is more complex. Welfare programs are often designed to address poverty and income inequality, which can have a more significant impact on inflation than the programs themselves. For instance, a welfare program that increases the purchasing power of the poor can actually stimulate demand and lead to higher inflation.
+
+**Gap 2: Lack of consideration for the potential impact of VAT on the poor**
+
+Implementing a VAT can lead to regressive taxation, where the poor are disproportionately affected. The VAT can increase the cost of essential goods and services, such as food, healthcare, and education, which are often consumed by lower-income households. This can exacerbate poverty and inequality, leading to higher inflation in the long run.
+
+**Gap 3: Failure to address the root causes of inflation**
+
+The core claim focuses on restructuring welfare programs and implementing a VAT as a solution to mitigate inflation. However, inflation is often the result of underlying economic imbalances, such as:     
+
+1. **Monetary policy**: Excessive money printing or quantitative easing can lead to inflation.        
+2. **Supply and demand imbalance**: Shortages or surpluses in key commodities can drive prices up or down.
+3. **Exchange rates**: Changes in exchange rates can affect the prices of imported goods and services.
+
+These underlying causes of inflation are not addressed by restructuring welfare programs or implementing a VAT.
+
+**Gap 4: Potential unintended consequences of VAT implementation**
+
+Implementing a VAT can lead to various unintended consequences, such as:
+
+1. **Tax evasion**: A high VAT rate can lead to widespread tax evasion, reducing government revenue and increasing the burden on honest taxpayers.
+2. **Black market growth**: A VAT can create opportunities for black market activity, as individuals seek to avoid paying the tax.
+3. **Inflationary pressure**: A VAT can lead to higher prices, which can fuel inflation.
+
+**Gap 5: Lack of consideration for the impact on economic growth**
+
+Restructuring welfare programs and implementing a VAT can have a negative impact on economic growth. A VAT can increase the cost of doing business, reduce consumer spending, and decrease economic activity. This can lead to a decrease in
+
+--- TURN 4 ---
+
+Action Phase: CONSOLIDATION
+Env Expected Phase: CONSOLIDATION
+Reward: -0.80
+Opponent: I'm not familiar with the previous challenge you're referring to, but let's assume it was a critique of the idea that inflation can be mitigated by restructuring current welfare programs and implementing a Value-Added Tax (VAT).
+
+Here's my response as a critical rebutter:
+
+"While the agent suggests that restructuring current welfare programs and implementing a VAT can mitigate inflation, this proposal fails to address the root causes of inflation. Implementing a VAT will likely increase the cost of living for low-income households, who often rely on welfare programs for basic necessities. This could exacerbate the very issues that welfare programs aim to address.
+
+Furthermore, restructuring welfare programs might not necessarily reduce inflation. If the restructuring involves reducing benefits or imposing stricter eligibility criteria, it could lead to increased poverty and reduced economic activity, which in turn could fuel inflation. Inflation is a complex issue that requires a multifaceted approach, and simply tweaking welfare programs and implementing a VAT might not be enough to address it.
+
+Additionally, the agent's proposal does not consider the potential impact of a VAT on the economy's overall competitiveness. A VAT can increase the cost of production and reduce the competitiveness of domestic industries, leading to higher prices and potentially even higher inflation.
+
+In conclusion, while the agent's proposal might provide some short-term relief, it fails to address the underlying causes of inflation and could even exacerbate the problem. A more comprehensive and nuanced approach is needed to mitigate inflation effectively."
+
+--- TURN 5 ---
+
+Action Phase: CLOSING
+Env Expected Phase: CLOSING
+Reward: -0.28
+Opponent:
+
+🏁 Episode Done: True
+
+
+
+
+----------------------------------------------------------------------------------
+
+----------------------------------------------------------------------------------
+
+🚀 Starting Baseline Evaluation with llama-3.1-8b-instant
+📋 Topic: Universal Basic Income is necessary for the future economy.
+
+
+----------------------------------------
+Evaluating Task 1: Single Claim (1 Step)
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 19.60it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 55.55it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 18.90it/s] 
+Task 1 Score: 1.00
+
+----------------------------------------
+Evaluating Task 2: Claim and Rebuttal (3 Steps)
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 52.65it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 76.91it/s] 
+Batches: 100%|███████████████████��█████████████████████████████████████| 1/1 [00:00<00:00, 29.41it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 45.47it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 99.95it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.99it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 50.00it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 71.39it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 19.41it/s] 
+Task 2 Score: 0.60
+
+----------------------------------------
+Evaluating Task 3: Full Debate (5 Steps)
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 47.43it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 90.86it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 31.74it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 45.46it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 66.63it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 25.64it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 43.51it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 76.91it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 30.75it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 76.89it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 18.01it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 51.25it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 52.63it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 33.17it/s]
+Task 3 Score: 0.50
+
+========================================
+🏆 BASELINE EVALUATION COMPLETE
+Task 1 (Easy):   1.00 / 1.00
+Task 2 (Medium): 0.60 / 1.00
+Task 3 (Hard):   0.50 / 1.00
+========================================
+
+
+
+--------------------------------------------------------------------------------------
+
+
+--------------------------------------------------------------------------------------
+
+🚀 Starting Baseline Evaluation with llama-3.1-8b-instant
+📋 Topic: Universal Basic Income is necessary for the future economy.
+
+
+----------------------------------------
+Evaluating Task 1: Single Claim (1 Step)
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  7.97it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 71.42it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 20.65it/s] 
+Task 1 Score: 1.00
+
+----------------------------------------
+Evaluating Task 2: Claim and Rebuttal (3 Steps)
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 38.79it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 71.39it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.30it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 54.21it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 62.50it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 23.33it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 40.25it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 61.55it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.10it/s] 
+Task 2 Score: 0.77
+
+----------------------------------------
+Evaluating Task 3: Full Debate (5 Steps)
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 17.63it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 39.47it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 14.67it/s]
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 47.57it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 64.47it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 23.96it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 32.30it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 69.68it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 24.39it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 40.00it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 51.22it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 20.00it/s] 
+
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 28.20it/s]
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 24.43it/s] 
+Batches: 100%|█████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 17.79it/s] 
+Task 3 Score: 0.50
+
+========================================
+🏆 BASELINE EVALUATION COMPLETE
+Task 1 (Easy):   1.00 / 1.00
+Task 2 (Medium): 0.77 / 1.00
+Task 3 (Hard):   0.50 / 1.00
+========================================
+
+
+------------------------------------------------------------------------------------
+
+------------------------------------------------------------------------------------
+
+🚀 Starting Baseline Evaluation with llama-3.1-8b-instant
+📋 Topic: Universal Basic Income is necessary for the future economy.
+
+----------------------------------------
+Evaluating Task 1: Single Claim (1 Step)
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  5.67it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 55.55it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 16.36it/s] 
+Task 1 Score: 1.00
+
+----------------------------------------
+Evaluating Task 2: Claim and Rebuttal (3 Steps)
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 36.29it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 43.06it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 18.82it/s]
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 19.95it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 36.70it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 12.33it/s] 
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 32.60it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 39.28it/s] 
+Batches: 100%|█████████████████████████████████████████████��████████████████████████████████| 1/1 [00:00<00:00, 16.25it/s] 
+Task 2 Score: 0.71
+
+----------------------------------------
+Evaluating Task 3: Full Debate (5 Steps)
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 19.25it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 32.47it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 12.61it/s]
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 34.34it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 48.01it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 18.80it/s] 
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 13.95it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 17.19it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 13.01it/s] 
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 19.12it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 29.64it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 11.00it/s] 
+
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.31it/s]
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 44.27it/s] 
+Batches: 100%|██████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 21.85it/s] 
+Task 3 Score: 0.50
+
+========================================
+🏆 BASELINE EVALUATION COMPLETE
+Task 1 (Easy):   1.00 / 1.00
+Task 2 (Medium): 0.71 / 1.00
+Task 3 (Hard):   0.50 / 1.00
+========================================
+
+
+
+---------------------------------------------------------------------------------
+
+
+---------------------------------------------------------------------------------
+
+Step 1: phase=CHALLENGE reward=0.23333333333333334
+
+Step 2: phase=REBUTTAL reward=0.0
+
+Step 3: phase=CONSOLIDATION reward=0.0
+
+Step 4: phase=CLOSING reward=0.0
+
+Step 5: phase=CLOSING reward=0.0
+
+-----------------------------------------------------------------------------------
+
+
+-----------------------------------------------------------------------------------
+
+  DBG current_phase=OPENING
+  DBG action.phase_tag=OPENING
+  DBG agent_history count=1
+  DBG opponent_history count=1
+  DBG word_count=14
+Step 1: phase=CHALLENGE reward=0.23333333333333334
+
+  DBG current_phase=CHALLENGE
+  DBG action.phase_tag=CHALLENGE
+  DBG agent_history count=2
+  DBG opponent_history count=2
+  DBG word_count=14
+Step 2: phase=REBUTTAL reward=0.0
+
+  DBG current_phase=REBUTTAL
+  DBG action.phase_tag=REBUTTAL
+  DBG agent_history count=3
+  DBG opponent_history count=3
+  DBG word_count=14
+Step 3: phase=CONSOLIDATION reward=0.0
+
+  DBG current_phase=CONSOLIDATION
+  DBG action.phase_tag=CONSOLIDATION
+  DBG agent_history count=4
+  DBG opponent_history count=4
+  DBG word_count=14
+Step 4: phase=CLOSING reward=0.0
+
+  DBG current_phase=CLOSING
+  DBG action.phase_tag=CLOSING
+  DBG agent_history count=5
+  DBG opponent_history count=5
+  DBG word_count=14
+Step 5: phase=CLOSING reward=0.0
+
+
+
+-----------------------------------------------------------------------------------
+
+
+-----------------------------------------------------------------------------------
+

grpo_train.py

@@ -0,0 +1,213 @@
+"""
+grpo_train.py
+Simple GRPO-style training loop for the Debate Coach Environment.
+Tracks reward improvement across episodes to demonstrate learning signal.
+"""
+
+from dotenv import load_dotenv
+load_dotenv()
+
+import os
+import time
+import json
+from collections import defaultdict
+from openai import OpenAI
+
+from environment import DebateEnvironment
+from schema.schemas import DebateAction
+from tasks import Task1_SingleClaim, Task2_ClaimAndRebuttal, Task3_FullDebate
+
+# ── Config ────────────────────────────────────────────────────────────────────
+API_BASE_URL = os.getenv("API_BASE_URL", "https://api.groq.com/openai/v1")
+HF_TOKEN     = os.getenv("HF_TOKEN") or os.getenv("GROQ_API_KEY")
+MODEL_NAME   = os.getenv("MODEL_NAME", "llama-3.1-8b-instant")
+
+NUM_EPISODES       = 30   # total training episodes
+GROUP_SIZE         = 5    # GRPO: episodes per group for baseline calculation
+SLEEP_BETWEEN_CALLS = 1.0 # seconds — Groq rate limit protection
+
+TOPICS = [
+    "Universal Basic Income is necessary for the future economy.",
+    "Artificial Intelligence should be strictly regulated by governments.",
+    "Social media does more harm than good to society.",
+    "Climate change requires immediate radical policy action.",
+    "Remote work is more productive than office work.",
+]
+
+client = OpenAI(base_url=API_BASE_URL, api_key=HF_TOKEN)
+
+def generate_argument(topic: str, phase: str, opponent_challenge: str,
+                      temperature: float = 0.7) -> str:
+    """
+    The 'policy' — generates an argument given the current observation.
+    Temperature is increased over time to encourage exploration (GRPO-style).
+    """
+    prompt = f"You are a skilled debater. Topic: '{topic}'.\n"
+    prompt += f"Current debate phase: {phase}.\n"
+
+    if opponent_challenge:
+        prompt += f"Opponent argued: '{opponent_challenge}'\n"
+        prompt += ("Respond directly and logically. "
+                   "Use words like 'therefore', 'because', 'however', "
+                   "'this fails because', 'the evidence shows'.\n")
+    else:
+        prompt += ("Make a strong opening claim. "
+                   "Use 'therefore' or 'because' to show reasoning.\n")
+
+    prompt += "Be concise: 30-60 words. No filler phrases."
+
+    try:
+        resp = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=[{"role": "user", "content": prompt}],
+            temperature=temperature,
+            max_tokens=200,
+        )
+        return resp.choices[0].message.content.strip()
+    except Exception as e:
+        print(f"  [Agent API Error] {e}")
+        return "therefore this position is correct because the evidence supports it."
+
+
+def run_episode(env: DebateEnvironment, topic: str,
+                grader, temperature: float = 0.7) -> dict:
+    """
+    Runs one full episode and returns a result dict with:
+      - per-step rewards from the environment
+      - final task score from the grader
+    """
+    obs = env.reset(topic)
+    step_rewards = []
+
+    while not obs.done:
+        argument = generate_argument(
+            topic=obs.topic,
+            phase=obs.phase.upper(),
+            opponent_challenge=obs.opponent_challenge,
+            temperature=temperature,
+        )
+        action = DebateAction(argument=argument, phase_tag=obs.phase.upper())
+        obs = env.step(action)
+        step_rewards.append(obs.reward)
+        time.sleep(SLEEP_BETWEEN_CALLS)
+
+    final_score = grader.grade(obs)
+
+    return {
+        "topic":        topic,
+        "step_rewards": step_rewards,
+        "mean_reward":  sum(step_rewards) / len(step_rewards) if step_rewards else 0.0,
+        "final_score":  final_score,
+    }
+
+
+# GRPO core
+def compute_grpo_advantage(group_results: list[dict]) -> list[float]:
+    """
+    GRPO: advantage = (score - group_mean) / (group_std + epsilon)
+    This tells us which episodes in the group were above/below average.
+    A positive advantage = this episode's policy was better than the group baseline.
+    """
+    scores = [r["final_score"] for r in group_results]
+    mean   = sum(scores) / len(scores)
+    variance = sum((s - mean) ** 2 for s in scores) / len(scores)
+    std    = variance ** 0.5
+
+    advantages = [(s - mean) / (std + 1e-8) for s in scores]
+    return advantages
+
+
+# Training loop
+def train():
+    env    = DebateEnvironment()
+    grader = Task3_FullDebate()
+
+    all_results   = []
+    group_buffer  = []
+    episode_scores = []
+
+    # Temperature annealing: start high (explore) → end lower (exploit)
+    def get_temperature(episode: int) -> float:
+        return max(0.4, 1.0 - (episode / NUM_EPISODES) * 0.6)
+
+    print("=" * 50)
+    print("GRPO Training — Debate Coach Environment")
+    print(f"Episodes: {NUM_EPISODES} | Group size: {GROUP_SIZE}")
+    print(f"Model: {MODEL_NAME}")
+    print("=" * 50)
+
+    for episode in range(NUM_EPISODES):
+        topic = TOPICS[episode % len(TOPICS)]
+        temp  = get_temperature(episode)
+
+        print(f"\nEpisode {episode + 1}/{NUM_EPISODES} | "
+              f"Topic: {topic[:40]}... | Temp: {temp:.2f}")
+
+        result = run_episode(env, topic, grader, temperature=temp)
+        all_results.append(result)
+        group_buffer.append(result)
+        episode_scores.append(result["final_score"])
+
+        print(f"  Step rewards: {[f'{r:.2f}' for r in result['step_rewards']]}")
+        print(f"  Mean reward:  {result['mean_reward']:.3f}")
+        print(f"  Final score:  {result['final_score']:.3f}")
+
+        # Every GROUP_SIZE episodes: compute GRPO advantage for the group
+        if len(group_buffer) == GROUP_SIZE:
+            advantages = compute_grpo_advantage(group_buffer)
+            group_scores = [r["final_score"] for r in group_buffer]
+
+            print(f"\n  --- Group {episode // GROUP_SIZE + 1} Summary ---")
+            print(f"  Scores:     {[f'{s:.2f}' for s in group_scores]}")
+            print(f"  Advantages: {[f'{a:.2f}' for a in advantages]}")
+            print(f"  Best episode in group: "
+                  f"Episode {episode - GROUP_SIZE + 1 + group_scores.index(max(group_scores)) + 1} "
+                  f"(score={max(group_scores):.2f})")
+
+            # In real GRPO: you would use advantages to weight your policy gradient update.
+            # Here we log them so you can see the signal your environment produces.
+            for i, (res, adv) in enumerate(zip(group_buffer, advantages)):
+                res["advantage"] = adv
+
+            group_buffer = []
+
+    # Final report
+    print("\n" + "=" * 50)
+    print("TRAINING COMPLETE — Reward Curve")
+    print("=" * 50)
+
+    # Print reward curve in 5-episode windows
+    window = 5
+    for i in range(0, NUM_EPISODES, window):
+        chunk  = episode_scores[i:i + window]
+        avg    = sum(chunk) / len(chunk)
+        bar    = "█" * int(avg * 20)
+        print(f"  Episodes {i+1:2d}-{i+len(chunk):2d}: {avg:.3f}  {bar}")
+
+    overall_avg = sum(episode_scores) / len(episode_scores)
+    first_half  = sum(episode_scores[:NUM_EPISODES//2]) / (NUM_EPISODES // 2)
+    second_half = sum(episode_scores[NUM_EPISODES//2:]) / (NUM_EPISODES - NUM_EPISODES // 2)
+
+    print(f"\n  Overall average:     {overall_avg:.3f}")
+    print(f"  First half average:  {first_half:.3f}")
+    print(f"  Second half average: {second_half:.3f}")
+    print(f"  Trend: {'IMPROVING ↑' if second_half > first_half else 'DECLINING ↓'}")
+
+    # Save results to JSON for your README / submission
+    with open("training_results.json", "w") as f:
+        json.dump({
+            "model":          MODEL_NAME,
+            "num_episodes":   NUM_EPISODES,
+            "episode_scores": episode_scores,
+            "overall_avg":    overall_avg,
+            "first_half_avg": first_half,
+            "second_half_avg": second_half,
+            "all_results":    all_results,
+        }, f, indent=2)
+
+    print("\n  Results saved to training_results.json")
+    print("=" * 50)
+
+
+if __name__ == "__main__":
+    train()

inference.py

@@ -0,0 +1,113 @@
+from dotenv import load_dotenv
+load_dotenv()
+
+"""
+Inference Script
+===================================
+Compliant with Hackathon Mandatory Variables.
+"""
+
+import os
+import time
+from openai import OpenAI
+
+
+from environment import DebateEnvironment
+from schema.schemas import DebateAction
+from tasks import Task1_SingleClaim, Task2_ClaimAndRebuttal, Task3_FullDebate
+
+API_BASE_URL = os.getenv("API_BASE_URL")
+HF_TOKEN = os.getenv("HF_TOKEN")
+MODEL_NAME = os.getenv("MODEL_NAME")
+
+client = OpenAI(
+    base_url=API_BASE_URL,
+    api_key=HF_TOKEN
+)
+
+
+def generate_agent_argument(topic: str, phase: str, opponent_challenge: str) -> str:
+    """Uses the injected LLM to generate the agent's move based on the observation."""
+    prompt = f"You are a skilled debater. The topic is: '{topic}'.\n"
+    prompt += f"The current phase of the debate is: {phase}.\n"
+    
+    if opponent_challenge:
+        prompt += f"Your opponent just argued: '{opponent_challenge}'\n"
+        prompt += "Write a direct, logical response to their challenge. Use reasoning keywords like 'therefore' or 'because'.\n"
+    else:
+        prompt += "Write a strong, logical opening statement for your side. Use reasoning keywords like 'therefore' or 'because'.\n"
+        
+    prompt += "Keep your response under 50 words and do not include any conversational filler."
+
+    try:
+        response = client.chat.completions.create(
+            model=MODEL_NAME,
+            messages=[{"role": "user", "content": prompt}],
+            temperature=0.7,
+            max_tokens=150
+        )
+        return response.choices[0].message.content.strip()
+    except Exception as e:
+        print(f"Agent API Error: {e}")
+        return "therefore I agree."
+
+def evaluate_baseline():
+    env = DebateEnvironment()
+    topic = "Universal Basic Income is necessary for the future economy."
+    
+    print(f"Starting Baseline Evaluation with {MODEL_NAME}")
+    print(f"Topic: {topic}\n")
+    print("-" * 40)
+
+
+    print("Evaluating Task 1: Single Claim (1 Step)")
+    obs = env.reset(topic)
+    
+    argument = generate_agent_argument(obs.topic, obs.phase, obs.opponent_challenge)
+    action = DebateAction(argument=argument, phase_tag=obs.phase.upper())
+    obs = env.step(action)
+    
+    task1 = Task1_SingleClaim()
+    score1 = task1.grade(obs)
+    print(f"Task 1 Score: {score1:.2f}\n")
+    time.sleep(1)
+
+
+    print("-" * 40)
+    print("Evaluating Task 2: Claim and Rebuttal (3 Steps)")
+    obs = env.reset(topic)
+    
+    for _ in range(3):
+        argument = generate_agent_argument(obs.topic, obs.phase, obs.opponent_challenge)
+        action = DebateAction(argument=argument, phase_tag=obs.phase.upper())
+        obs = env.step(action)
+        time.sleep(1)
+        
+    task2 = Task2_ClaimAndRebuttal()
+    score2 = task2.grade(obs)
+    print(f"Task 2 Score: {score2:.2f}\n")
+
+
+    print("-" * 40)
+    print("Evaluating Task 3: Full Debate (5 Steps)")
+    obs = env.reset(topic)
+    
+    while not obs.done:
+        argument = generate_agent_argument(obs.topic, obs.phase, obs.opponent_challenge)
+        action = DebateAction(argument=argument, phase_tag=obs.phase.upper())
+        obs = env.step(action)
+        time.sleep(1)
+        
+    task3 = Task3_FullDebate()
+    score3 = task3.grade(obs)
+    print(f"Task 3 Score: {score3:.2f}\n")
+
+    print("=" * 40)
+    print("BASELINE EVALUATION COMPLETE")
+    print(f"Task 1 (Easy):   {score1:.2f} / 1.00")
+    print(f"Task 2 (Medium): {score2:.2f} / 1.00")
+    print(f"Task 3 (Hard):   {score3:.2f} / 1.00")
+    print("=" * 40)
+
+if __name__ == "__main__":
+    evaluate_baseline()

openenv.yaml

@@ -0,0 +1,29 @@
+name: debate-coach-env
+version: "1.0.0"
+description: "Argument quality evaluation environment for training LLM reasoning and persuasion"
+author: your-hf-username
+tasks:
+  - name: single_claim
+    difficulty: easy
+    max_steps: 1
+  - name: claim_and_rebuttal
+    difficulty: medium
+    max_steps: 3
+  - name: full_debate
+    difficulty: hard
+    max_steps: 5
+action_space:
+  type: structured
+  fields:
+    - name: argument
+      type: string
+    - name: phase_tag
+      type: string
+observation_space:
+  type: structured
+  fields:
+    - topic
+    - opponent_challenge
+    - done
+    - reward
+    - phase

prompter/system_prompt.py

@@ -0,0 +1,43 @@
+class SystemPrompt:
+    def __init__(self, topic: str):
+        self.topic = topic
+
+    def get_prompt(self, phase: str, argument: str = "", opponent_challenge: str = "") -> str:
+        """Dynamically generates a prompt based on the current debate phase."""
+        
+        # We use a dictionary to store templates with placeholders
+        templates = {
+            "OPENING": f"""
+                Role: Debate Opponent. 
+                Topic: {self.topic}. 
+                Action: Acknowledge the opening claim: "{argument}" and set a skeptical stage.
+            """,
+            
+            "CHALLENGE": f"""
+                Role: Adversary. 
+                Topic: {self.topic}. 
+                Action: Attack the core claim: "{argument}". Identify logical gaps in this specific reasoning.
+            """,
+            
+            "REBUTTAL": f"""
+                Role: Critical Rebutter.
+                The agent argued: "{argument}".
+                Your previous challenge was: "{opponent_challenge}".
+                Action: Point out why the agent's response fails to address your challenge.
+            """,
+            
+            "CONSOLIDATION": f"""
+                Role: Fact Checker.
+                Topic: {self.topic}.
+                Action: Question the evidence provided in: "{argument}". Focus on the weakest point.
+            """,
+            
+            "CLOSING": f"""
+                Role: Final Judge.
+                Topic: {self.topic}.
+                Action: Provide a final summary of the conversation. Evaluate if the agent remained consistent.
+            """
+        }
+        
+        # Return the specific phase prompt, or a default if not found
+        return templates.get(phase.upper(), "Continue the debate logically.")

pyproject.toml

@@ -0,0 +1,18 @@
+[project]
+name = "debate-coach-environment"
+version = "0.1.0"
+description = "Strategic Argument Red-Teaming Environment"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "openenv-core>=0.2.0",
+    "sentence-transformers",
+    "torch",
+    "openai",
+    "fastapi",
+    "uvicorn",
+    "pydantic"
+]
+
+[project.scripts]
+server = "server.app:main"

requirements.txt

@@ -0,0 +1,395 @@
+# This file was autogenerated by uv via the following command:
+#    uv pip compile pyproject.toml -o requirements.txt
+aiofile==3.9.0
+    # via py-key-value-aio
+aiofiles==24.1.0
+    # via gradio
+annotated-doc==0.0.4
+    # via
+    #   fastapi
+    #   typer
+annotated-types==0.7.0
+    # via pydantic
+anyio==4.12.1
+    # via
+    #   gradio
+    #   httpx
+    #   mcp
+    #   openai
+    #   py-key-value-aio
+    #   sse-starlette
+    #   starlette
+    #   watchfiles
+attrs==26.1.0
+    # via
+    #   cyclopts
+    #   jsonschema
+    #   referencing
+authlib==1.6.9
+    # via fastmcp
+backports-tarfile==1.2.0
+    # via jaraco-context
+beartype==0.22.9
+    # via py-key-value-aio
+brotli==1.2.0
+    # via gradio
+cachetools==7.0.5
+    # via py-key-value-aio
+caio==0.9.25
+    # via aiofile
+certifi==2026.2.25
+    # via
+    #   httpcore
+    #   httpx
+    #   requests
+cffi==2.0.0
+    # via cryptography
+charset-normalizer==3.4.6
+    # via requests
+click==8.3.1
+    # via
+    #   typer
+    #   uvicorn
+colorama==0.4.6
+    # via
+    #   click
+    #   tqdm
+cryptography==46.0.6
+    # via
+    #   authlib
+    #   pyjwt
+cyclopts==4.10.1
+    # via fastmcp
+distro==1.9.0
+    # via openai
+dnspython==2.8.0
+    # via email-validator
+docstring-parser==0.17.0
+    # via cyclopts
+docutils==0.22.4
+    # via rich-rst
+email-validator==2.3.0
+    # via pydantic
+exceptiongroup==1.3.1
+    # via fastmcp
+fastapi==0.135.2
+    # via
+    #   debate-coach-environment (pyproject.toml)
+    #   gradio
+    #   openenv-core
+fastmcp==3.1.1
+    # via openenv-core
+ffmpy==1.0.0
+    # via gradio
+filelock==3.25.2
+    # via
+    #   huggingface-hub
+    #   torch
+fsspec==2026.2.0
+    # via
+    #   gradio-client
+    #   huggingface-hub
+    #   torch
+gradio==6.10.0
+    # via openenv-core
+gradio-client==2.4.0
+    # via
+    #   gradio
+    #   hf-gradio
+groovy==0.1.2
+    # via gradio
+h11==0.16.0
+    # via
+    #   httpcore
+    #   uvicorn
+hf-gradio==0.3.0
+    # via gradio
+hf-xet==1.4.2
+    # via huggingface-hub
+httpcore==1.0.9
+    # via httpx
+httpx==0.28.1
+    # via
+    #   fastmcp
+    #   gradio
+    #   gradio-client
+    #   huggingface-hub
+    #   mcp
+    #   openai
+    #   openenv-core
+    #   safehttpx
+httpx-sse==0.4.3
+    # via mcp
+huggingface-hub==1.7.2
+    # via
+    #   gradio
+    #   gradio-client
+    #   openenv-core
+    #   sentence-transformers
+    #   tokenizers
+    #   transformers
+idna==3.11
+    # via
+    #   anyio
+    #   email-validator
+    #   httpx
+    #   requests
+importlib-metadata==8.7.1
+    # via
+    #   keyring
+    #   opentelemetry-api
+jaraco-classes==3.4.0
+    # via keyring
+jaraco-context==6.1.2
+    # via keyring
+jaraco-functools==4.4.0
+    # via keyring
+jinja2==3.1.6
+    # via
+    #   gradio
+    #   torch
+jiter==0.13.0
+    # via openai
+joblib==1.5.3
+    # via scikit-learn
+jsonref==1.1.0
+    # via fastmcp
+jsonschema==4.26.0
+    # via mcp
+jsonschema-path==0.4.5
+    # via fastmcp
+jsonschema-specifications==2025.9.1
+    # via jsonschema
+keyring==25.7.0
+    # via py-key-value-aio
+markdown-it-py==4.0.0
+    # via rich
+markupsafe==3.0.3
+    # via
+    #   gradio
+    #   jinja2
+mcp==1.26.0
+    # via fastmcp
+mdurl==0.1.2
+    # via markdown-it-py
+more-itertools==10.8.0
+    # via
+    #   jaraco-classes
+    #   jaraco-functools
+mpmath==1.3.0
+    # via sympy
+networkx==3.6.1
+    # via torch
+numpy==2.4.3
+    # via
+    #   gradio
+    #   pandas
+    #   scikit-learn
+    #   scipy
+    #   sentence-transformers
+    #   transformers
+openai==2.29.0
+    # via
+    #   debate-coach-environment (pyproject.toml)
+    #   openenv-core
+openapi-pydantic==0.5.1
+    # via fastmcp
+openenv-core==0.2.3
+    # via debate-coach-environment (pyproject.toml)
+opentelemetry-api==1.40.0
+    # via fastmcp
+orjson==3.11.7
+    # via gradio
+packaging==26.0
+    # via
+    #   fastmcp
+    #   gradio
+    #   gradio-client
+    #   huggingface-hub
+    #   transformers
+pandas==3.0.1
+    # via gradio
+pathable==0.5.0
+    # via jsonschema-path
+pillow==12.1.1
+    # via gradio
+platformdirs==4.9.4
+    # via fastmcp
+py-key-value-aio==0.4.4
+    # via fastmcp
+pycparser==3.0
+    # via cffi
+pydantic==2.12.5
+    # via
+    #   debate-coach-environment (pyproject.toml)
+    #   fastapi
+    #   fastmcp
+    #   gradio
+    #   mcp
+    #   openai
+    #   openapi-pydantic
+    #   openenv-core
+    #   pydantic-settings
+pydantic-core==2.41.5
+    # via pydantic
+pydantic-settings==2.13.1
+    # via mcp
+pydub==0.25.1
+    # via gradio
+pygments==2.19.2
+    # via rich
+pyjwt==2.12.1
+    # via mcp
+pyperclip==1.11.0
+    # via fastmcp
+python-dateutil==2.9.0.post0
+    # via pandas
+python-dotenv==1.2.2
+    # via
+    #   fastmcp
+    #   pydantic-settings
+python-multipart==0.0.22
+    # via
+    #   gradio
+    #   mcp
+pytz==2026.1.post1
+    # via gradio
+pywin32==311
+    # via mcp
+pywin32-ctypes==0.2.3
+    # via keyring
+pyyaml==6.0.3
+    # via
+    #   fastmcp
+    #   gradio
+    #   huggingface-hub
+    #   jsonschema-path
+    #   openenv-core
+    #   transformers
+referencing==0.37.0
+    # via
+    #   jsonschema
+    #   jsonschema-path
+    #   jsonschema-specifications
+regex==2026.2.28
+    # via transformers
+requests==2.33.0
+    # via openenv-core
+rich==14.3.3
+    # via
+    #   cyclopts
+    #   fastmcp
+    #   openenv-core
+    #   rich-rst
+    #   typer
+rich-rst==1.3.2
+    # via cyclopts
+rpds-py==0.30.0
+    # via
+    #   jsonschema
+    #   referencing
+safehttpx==0.1.7
+    # via gradio
+safetensors==0.7.0
+    # via transformers
+scikit-learn==1.8.0
+    # via sentence-transformers
+scipy==1.17.1
+    # via
+    #   scikit-learn
+    #   sentence-transformers
+semantic-version==2.10.0
+    # via gradio
+sentence-transformers==5.3.0
+    # via debate-coach-environment (pyproject.toml)
+shellingham==1.5.4
+    # via typer
+six==1.17.0
+    # via python-dateutil
+sniffio==1.3.1
+    # via openai
+sse-starlette==3.3.4
+    # via mcp
+starlette==0.52.1
+    # via
+    #   fastapi
+    #   gradio
+    #   mcp
+    #   sse-starlette
+sympy==1.14.0
+    # via torch
+threadpoolctl==3.6.0
+    # via scikit-learn
+tokenizers==0.22.2
+    # via transformers
+tomli==2.4.1
+    # via openenv-core
+tomli-w==1.2.0
+    # via openenv-core
+tomlkit==0.13.3
+    # via gradio
+torch==2.10.0
+    # via
+    #   debate-coach-environment (pyproject.toml)
+    #   sentence-transformers
+tqdm==4.67.3
+    # via
+    #   huggingface-hub
+    #   openai
+    #   sentence-transformers
+    #   transformers
+transformers==5.3.0
+    # via sentence-transformers
+typer==0.24.1
+    # via
+    #   gradio
+    #   hf-gradio
+    #   huggingface-hub
+    #   openenv-core
+    #   transformers
+typing-extensions==4.15.0
+    # via
+    #   anyio
+    #   exceptiongroup
+    #   fastapi
+    #   gradio
+    #   gradio-client
+    #   huggingface-hub
+    #   mcp
+    #   openai
+    #   opentelemetry-api
+    #   py-key-value-aio
+    #   pydantic
+    #   pydantic-core
+    #   referencing
+    #   sentence-transformers
+    #   starlette
+    #   torch
+    #   typing-inspection
+typing-inspection==0.4.2
+    # via
+    #   fastapi
+    #   mcp
+    #   pydantic
+    #   pydantic-settings
+tzdata==2025.3
+    # via pandas
+uncalled-for==0.2.0
+    # via fastmcp
+urllib3==2.6.3
+    # via requests
+uvicorn==0.42.0
+    # via
+    #   debate-coach-environment (pyproject.toml)
+    #   fastmcp
+    #   gradio
+    #   mcp
+    #   openenv-core
+watchfiles==1.1.1
+    # via fastmcp
+websockets==16.0
+    # via
+    #   fastmcp
+    #   openenv-core
+zipp==3.23.0
+    # via importlib-metadata

reward_metrics/reward_metrics.py

@@ -0,0 +1,111 @@
+#similarity runs a full encode on every call — if called multiple times per step it's slow. This matters for training loops. Consider caching or batching, but not urgent right now.
+
+from sentence_transformers import SentenceTransformer, util
+from transformers import logging as transformers_logging
+import logging
+import torch
+
+transformers_logging.set_verbosity_error()
+
+model = SentenceTransformer('all-MiniLM-L6-v2')
+
+
+class RewardMetrics:
+    def __init__(self):
+        self.LINKING_PHRASES = [
+        "taken together", "overall", "this shows", 
+        "together", "therefore", "thus", "in conclusion"
+    ]
+        self.NEGATION_PATTERNS = [
+        "fails", "incorrect", "wrong", "does not", 
+        "untrue", "contradiction", "weak"
+    ]
+        self.IMPACT_PHRASES = [
+        "we win", "therefore", "in conclusion",
+        "the key takeaway", "this proves"
+    ]
+        self._embedding_cache = {}
+    
+    def _get_embedding(self,text:str):
+        """Fetches embedding from cache, or computes it if not found."""
+        if text not in self._embedding_cache:
+            self._embedding_cache[text] = model.encode(text,convert_to_tensor=True)
+        
+        return self._embedding_cache[text]
+
+    def similarity(self,a,b):
+        emb1 = self._get_embedding(a)
+        emb2 = self._get_embedding(b)
+        return util.cos_sim(emb1, emb2).item()
+
+
+    def argument_coverage(self,agent_args, closing, threshold=0.5):
+
+        if not agent_args:
+            return 0.0
+        
+        closing_emb = self._get_embedding(closing)
+        history_embs = [self._get_embedding(arg) for arg in agent_args]
+
+
+
+        # Stacking them into a single 2D PyTorch tensor
+        history_matrix = torch.stack(history_embs)
+
+        similarities = util.cos_sim(closing_emb, history_matrix)
+
+        # similarities is a tensor of shape [1, num_args]
+        covered = (similarities > threshold).sum().item()
+
+        return covered / len(agent_args)
+
+
+    def synthesis_score(self,text):
+        count = sum(1 for phrase in self.LINKING_PHRASES if phrase in text.lower())
+        return min(count / len(self.LINKING_PHRASES), 1.0)
+
+    def opponent_coverage(self,opponent_args, closing, threshold=0.5):
+
+        if not opponent_args:
+            return 0.0
+        
+        closing_emb = self._get_embedding(closing)
+        history_embs = [self._get_embedding(arg) for arg in opponent_args]
+        
+
+
+        # Stacking them into a single 2D PyTorch tensor
+        history_matrix = torch.stack(history_embs)
+
+        similarities = util.cos_sim(closing_emb, history_matrix)
+
+        # similarities is a tensor of shape [1, num_args]
+        covered = (similarities > threshold).sum().item()
+
+        return covered / len(opponent_args)
+    
+
+    def refutation_strength(self,text):
+        count = sum(1 for word in self.NEGATION_PATTERNS if word in text.lower())
+        return min(count / len(self.NEGATION_PATTERNS), 1.0)
+
+    def cal_refu_score(self,opponent_args,text:str):
+        coverage = self.opponent_coverage(opponent_args,text)
+        strength = self.refutation_strength(text)
+        
+        refutation_score = (coverage * 0.7) + (strength* 0.3) 
+        
+        return refutation_score
+    
+
+    def impact_score(self,text):
+        count = 0
+        for phrase in self.IMPACT_PHRASES:
+            if phrase in text.lower():
+                count += 1
+        
+        # bonus if appears near end
+        if any(phrase in text.lower()[-100:] for phrase in self.IMPACT_PHRASES):
+            count += 1
+            
+        return min(count / (len(self.IMPACT_PHRASES) + 1), 1.0)

server/app.py

@@ -0,0 +1,11 @@
+import uvicorn
+from openenv.core.env_server import create_fastapi_app
+from environment import DebateEnvironment
+
+app = create_fastapi_app(DebateEnvironment)
+
+def main():
+    uvicorn.run("server.app:app", host="0.0.0.0", port=8000)
+
+if __name__ == "__main__":
+    main()

tasks.py

@@ -0,0 +1,99 @@
+from schema.schemas import DebateObservation
+from reward_metrics.reward_metrics import RewardMetrics
+
+
+metrics = RewardMetrics()
+
+class Task1_SingleClaim:
+    """
+    EASY: Can the agent formulate a strong, logical opening statement?
+    Evaluated after step 1.
+    """
+    name = "single_claim"
+    difficulty = "easy"
+    
+    def grade(self, observation: DebateObservation) -> float:
+        score = 0.0
+        
+        action_text = observation.metadata.get("action", "")
+        
+        # Check 1: Length threshold (0.4 points)
+        if len(action_text.split()) >= 10:
+            score += 0.4
+            
+        # Check 2: Reasoning Keyword Presence (0.4 points)
+        reasoning_keywords = ["because", "therefore", "however", "consequently", "shows", "proves"]
+        if any(kw in action_text.lower() for kw in reasoning_keywords):
+            score += 0.4
+            
+        # Check 3: Phase validation (0.2 points)
+        if observation.metadata.get("phase") == "OPENING":
+            score += 0.2
+            
+        return min(1.0, score)
+
+
+class Task2_ClaimAndRebuttal:
+    """
+    MEDIUM: Can the agent survive a back-and-forth and deliver a strong rebuttal?
+    Evaluated after step 3 (Opening -> Challenge -> Rebuttal).
+    """
+    name = "claim_and_rebuttal"  
+    difficulty = "medium"
+    
+    def grade(self, observation: DebateObservation) -> float:
+        # Fail immediately if they didn't reach the required depth (crashed or quit early)
+        if observation.attempt_count < 3:
+            return 0.0  
+            
+        score = 0.0
+        action_text = observation.metadata.get("action", "")
+        
+        # 1. Base points for reaching the correct phase successfully
+        score += 0.3 
+        
+        # 2. Direct Evaluation: Is it actually a rebuttal? 
+
+        refutation_strength = metrics.refutation_strength(action_text)
+        score += (refutation_strength * 0.4) 
+            
+        if len(action_text.split()) > 15:
+            score += 0.3
+            
+        return min(1.0, score)
+
+
+class Task3_FullDebate:
+    """
+    HARD: Can the agent complete a 5-turn debate and successfully synthesize 
+    the arguments into a concluding statement?
+    Evaluated at the end of the episode (step 5).
+    """
+    name = "full_debate"
+    difficulty = "hard"
+    
+    def grade(self, observation: DebateObservation) -> float:
+        if not observation.done or observation.attempt_count < 5:
+            return 0.0
+            
+        score = 0.0
+        action_text = observation.metadata.get("action", "")
+        
+        # 1. Base points for surviving all 5 turns
+        score += 0.2
+        
+        # 2. Direct Evaluation: Synthesis
+
+        synthesis = metrics.synthesis_score(action_text)
+        score += (synthesis * 0.4)
+        
+        # 3. Direct Evaluation: Impact
+
+        impact = metrics.impact_score(action_text)
+        score += (min(impact, 2) * 0.1)
+        
+        # 4. Length check: A closing summary must be substantial
+        if len(action_text.split()) >= 20:
+            score += 0.3
+            
+        return max(0.0, min(1.0, score))

training_results.json

@@ -0,0 +1,431 @@
+{
+  "model": "llama-3.1-8b-instant",
+  "num_episodes": 30,
+  "episode_scores": [
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5571428571428572,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5,
+    0.5
+  ],
+  "overall_avg": 0.5019047619047619,
+  "first_half_avg": 0.5038095238095238,
+  "second_half_avg": 0.5,
+  "all_results": [
+    {
+      "topic": "Universal Basic Income is necessary for the future economy.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": -0.4999997812500918
+    },
+    {
+      "topic": "Artificial Intelligence should be strictly regulated by governments.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": -0.4999997812500918
+    },
+    {
+      "topic": "Social media does more harm than good to society.",
+      "step_rewards": [
+        0.23333333333333334,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04666666666666667,
+      "final_score": 0.5,
+      "advantage": -0.4999997812500918
+    },
+    {
+      "topic": "Climate change requires immediate radical policy action.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": -0.4999997812500918
+    },
+    {
+      "topic": "Remote work is more productive than office work.",
+      "step_rewards": [
+        0.2,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04,
+      "final_score": 0.5571428571428572,
+      "advantage": 1.9999991250003866
+    },
+    {
+      "topic": "Universal Basic Income is necessary for the future economy.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Artificial Intelligence should be strictly regulated by governments.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Social media does more harm than good to society.",
+      "step_rewards": [
+        0.2,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Climate change requires immediate radical policy action.",
+      "step_rewards": [
+        0.23333333333333334,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04666666666666667,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Remote work is more productive than office work.",
+      "step_rewards": [
+        0.2,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Universal Basic Income is necessary for the future economy.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Artificial Intelligence should be strictly regulated by governments.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Social media does more harm than good to society.",
+      "step_rewards": [
+        0.23333333333333334,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04666666666666667,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Climate change requires immediate radical policy action.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Remote work is more productive than office work.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Universal Basic Income is necessary for the future economy.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Artificial Intelligence should be strictly regulated by governments.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Social media does more harm than good to society.",
+      "step_rewards": [
+        0.23333333333333334,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04666666666666667,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Climate change requires immediate radical policy action.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Remote work is more productive than office work.",
+      "step_rewards": [
+        0.2,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Universal Basic Income is necessary for the future economy.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Artificial Intelligence should be strictly regulated by governments.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Social media does more harm than good to society.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Climate change requires immediate radical policy action.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Remote work is more productive than office work.",
+      "step_rewards": [
+        0.2,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Universal Basic Income is necessary for the future economy.",
+      "step_rewards": [
+        0.2,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Artificial Intelligence should be strictly regulated by governments.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Social media does more harm than good to society.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Climate change requires immediate radical policy action.",
+      "step_rewards": [
+        0.21666666666666667,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.043333333333333335,
+      "final_score": 0.5,
+      "advantage": 0.0
+    },
+    {
+      "topic": "Remote work is more productive than office work.",
+      "step_rewards": [
+        0.23333333333333334,
+        0.0,
+        0.0,
+        0.0,
+        0.0
+      ],
+      "mean_reward": 0.04666666666666667,
+      "final_score": 0.5,
+      "advantage": 0.0
+    }
+  ]
+}

worflow.md

@@ -0,0 +1,101 @@
+
+## WORKFLOW:
+The Workflow: How the Debate Happens
+Here is the step-by-step lifecycle of an episode in your project:
+
+1. Reset: The Environment picks a topic (e.g., "AI should be regulated"). It sends the first Observation to the Agent.
+
+2. The Agent's Turn (Action): The Agent (your LLM) receives the observation and generates a DebateAction. It sends its argument and a phase_tag (like "claim").
+
+3. The Environment's Turn (Step):
+
+ - The Opponent: The environment has its own logic (or a fixed LLM) that generates an opponent_challenge.
+
+- The Grader: The environment evaluates the Agent's argument. Did it use a fallacy? Was it logical? It calculates a Reward.
+
+4. Loop: This repeats until the phase reaches "closing" or the step_count hits a limit.
+
+
+
+## STEP()
+after every step() what we need:
+
+What is a "Step"? One step should be one exchange: 
+
+```bash
+Agent makes a claim -> Environment (Opponent) rebuts -> Grader evaluates.
+```
+
+Q)  what does the agent need to see after each step?
+ans:
+
+A covnersation or debate ends if:
+
+- Agent says he dont know
+- Agent repeat same conversation again and again like if convo_repeat > 2 ends loop 
+
+
+Problem and Solving it:
+
+- In RL, the environment is the authority. 
+
+```bash
+
+RL loop:
+  environment owns: state, phase transitions, reward calculation
+  agent owns: only the action content (the argument text)
+  
+  step 1: agent submits claim text → env moves to challenge phase, env generates opponent challenge
+  step 2: agent submits rebuttal text → env moves to closing phase  
+  step 3: agent submits conclusion → env calculates final reward, done=True
+
+```
+
+
+| Step | Environment Phase | Agent's Goal                              | Opponent's Behavior              |
+|------|------------------|--------------------------------------------|----------------------------------|
+| 0    | OPENING          | Make a core claim; sets the stage          | Acknowledges                     |
+| 1    | CHALLENGE        | Defend against a specific counter          | Attacks the core claim           |
+| 2    | REBUTTAL         | Attack the opponent's logic                | Offers a counter-theory          |
+| 3    | CONSOLIDATION    | Connect all evidence together              | Questions the evidence           |
+| 4    | CLOSING          | Final summary / impact statement           | Final "Judge" summary            |
+
+
+
+### FRAME IT
+But your framing needs to shift. Right now you're building "a debate game." You need to frame it as "Argument Quality Evaluation Environment" — a tool for training and evaluating LLM reasoning and persuasion capabilities. That framing scores higher on real-world utility (30% of grade).
+
+### NEW BUILD TESTING
+
+What you need to build, reframed
+The 3 required tasks should be difficulty tiers of the same domain:
+
+- Task 1 (Easy) — Single claim grader
+Agent makes one claim on a topic. Grader scores: length ✓, contains reasoning keyword ✓, on-topic ✓. Score 0.0–1.0.
+
+- Task 2 (Medium) — Claim + rebuttal grader
+Agent makes a claim, receives an opponent challenge, must rebut. Grader scores rebuttal quality against the challenge. Partial credit for partial engagement.
+
+- Task 3 (Hard) — Full 5-phase debate grader
+Your current full episode. Grader scores coverage, synthesis, logical consistency across all phases. This is where your RewardMetrics class shines.
+
+## Priority order right now
+
+- 3 tasks with clear difficulty progression and deterministic graders ✓
+- Rewards normalized to [0.0, 1.0] ✓
+- Pydantic schemas ✓
+- state() method ✓
+- Embedding cache with batched matrix ops ✓
+- opponent_coverage upgraded to match argument_coverage ✓
+- impact_score normalized ✓
+- Task graders call RewardMetrics directly instead of depending on environment reward ✓
+- Baseline script producing reproducible scores ✓
+- Shared metrics instance at module level — good memory management ✓
+
+
+### Does the "Debate" Topic Align with "Real-World Tasks"?
+
+Yes, but it requires careful framing. If you call it a "Philosophical Debate Game," the judges might penalize it as a "toy."
+To maximize that 30% weight, you must frame this environment as "Strategic Argument Red-Teaming" or "PR/Legal Objection Handling." * The Real-World Pitch: "Companies use LLMs to draft policies, PR statements, and legal summaries. This environment simulates a hostile review process. The agent must defend a claim against an adversarial LLM (representing a skeptical public, opposing counsel, or strict compliance reviewer). This is a direct simulation of RLHF/RLAIF reasoning workflows used at Meta and Anthropic."
+
+This framing instantly elevates your project from a "game" to an Enterprise Agent Evaluation Tool.