cleanup comments and readme

Dockerfile

@@ -5,5 +5,6 @@ RUN pip install --no-cache-dir -r requirements.txt
 COPY . .
 ENV PYTHONPATH=/
 ENV PYTHONIOENCODING=utf-8
+ENV ENABLE_WEB_INTERFACE=true
 EXPOSE 7860
 CMD ["python", "-m", "uvicorn", "code_review_env.server.app:app", "--host", "0.0.0.0", "--port", "7860"]

README.md

@@ -9,125 +9,118 @@ tags:
   - openenv
 ---
 
-# CodeReviewEnv: Triage CVEs Like a Pro
+# CodeReviewEnv
 
-*Built for the Meta/PyTorch OpenEnv Hackathon*
+An RL environment for vulnerability triage, built on real CVE data from the NVD.
 
-Hey! 👋 Welcome to **CodeReviewEnv**. 
+The idea: most RL envs are toy problems (gridworld, cartpole, etc). We wanted something closer to what devs actually deal with — triaging security patches across a codebase with limited time and attention.
 
-Most RL environments are toy setups like GridWorld or simple mazes. We wanted to tackle a real problem that we actually face as developers: **Vulnerability Triage**. 
+The agent gets a stream of files from a real CVE patch and has to decide: **flag** this file for human review, or **skip** it. There's a fixed review budget so you can't just flag everything.
 
-We built `CodeReviewEnv` using real-world data scraped directly from patches in the National Vulnerability Database (NVD). The agent connects to a repository, scans the files, and uses heuristics (like codebase churn, complexity, and recency) to decide whether to `flag` a file for manual security review or `skip` it and move on.
-
-* 🚀 **Hugging Face Space (Live Environment)**: [https://huggingface.co/spaces/lucid987654/code-review-env](https://huggingface.co/spaces/lucid987654/code-review-env)
-* 📁 **GitHub Repository**: [https://github.com/subwaycookiecrunch/Meta-project](https://github.com/subwaycookiecrunch/Meta-project)
+* **HF Space**: https://huggingface.co/spaces/lucid987654/code-review-env
+* **GitHub**: https://github.com/subwaycookiecrunch/Meta-project
 
 ---
 
-### The Problem
+### Data
 
-We have 1715 files across 65 authentic CVEs pulled directly from actual GitHub vulnerabilities. 
-We hit the agent with **Asymmetric Rewards**. In the real world, missing a critical bug (False Negative) is infinitely worse than accidentally flagging a safe file for review (False Positive). 
+1715 files across 65 CVEs scraped from actual GitHub vulnerability patches. Each file has four features extracted from the commit history: churn, complexity, TODO count, and recency.
 
-Our reward table forces the agent to balance its paranoia:
+### Rewards
 
-| Outcome | Reward | Rationale |
-|---------|--------|-----------|
-| True Positive (found a real bug) | **+1.0** | Highest reward — catching vulnerabilities is the goal |
-| True Negative (correctly skipped safe file) | **+0.8** | Good judgment that saves review budget |
-| False Positive (flagged a safe file) | **-0.4** | Wastes review budget, penalized |
-| False Negative (missed a real bug) | **-0.2** | Worst failure — penalty provides learning signal |
-| Over-budget flag attempt | **-0.5** | Hard constraint — budget is non-negotiable |
+Asymmetric on purpose — missing a real bug is worse than wasting a review slot on a clean file.
 
-Oh, and there's a strict **Review Budget**. You can't just flag everything, or you run out of budget and get heavily penalized!
+| Outcome | Reward | Why |
+|---------|--------|-----|
+| True Positive | +1.0 | found a real bug |
+| True Negative | +0.8 | correctly skipped clean file |
+| False Positive | -0.4 | wasted budget on safe file |
+| False Negative | -0.2 | missed a bug |
+| Over-budget flag | -0.5 | budget is a hard limit |
 
----
+### Tasks
+
+Three difficulty levels based on repo size:
+
+- **easy**: ≤15 files, generous budget
+- **medium**: 16-29 files
+- **hard**: 30+ files, tight budget — agent really has to pick its spots
 
-### 🧩 Environment Specifications
+Grading is F1 score (precision × recall), always in [0, 1].
 
-#### Action Space
-The action space is a straightforward, string-based categorical action:
-* `decision`: `"flag"` or `"skip"`
+---
 
-#### Observation Space
-At each step, the environment provides a rich state vector. The key metrics include:
-* `file_path` & tracking metrics (`file_index`, `files_remaining`, `total_files`)
-* **Difficulty & Context**: `difficulty_level`, `cve_id`, `repo_name`
-* **Static Analysis Features**: 
-  * `churn_score` — lines changed in the file (higher = more volatile)
-  * `complexity_score` — cyclomatic complexity proxy (higher = harder to review)
-  * `todo_score` — count of TODOs/FIXMEs (higher = more tech debt)
-  * `recency_score` — how recently the file was modified (higher = more recent)
-* **Limits**: `review_budget` and `files_flagged`
-* **Terminal Metrics**: `precision`, `recall`, `f1_score`, `true_positives`, `false_positives`, `false_negatives`, `true_negatives`
+### Observation fields
 
-#### 🎯 The Three Tasks (Difficulty Tiers)
-We've partitioned the environment into three distinct difficulty tasks, scaling gracefully by the size of the repository logic the agent needs to parse over its fixed review budget:
-1. **Easy (`difficulty="easy"`)**: Small repositories and pull requests (≤ 15 files). The budget is relatively forgiving.
-2. **Medium (`difficulty="medium"`)**: Average-sized PRs (16-29 files) requiring more scrutiny.
-3. **Hard (`difficulty="hard"`)**: Large-scale patches (30+ files). The agent is strapped for budget and must be extremely selective about utilizing its flags.
+Each step gives you:
+- `file_path`, `file_index`, `total_files`, `files_remaining`
+- `churn_score`, `complexity_score`, `todo_score`, `recency_score`
+- `review_budget`, `files_flagged`
+- `difficulty_level`, `cve_id`, `repo_name`
+- terminal: `precision`, `recall`, `f1_score`, `true_positives`, `false_positives`, etc.
 
-Each task includes a **programmatic grader** that returns a score between 0.0 and 1.0 (the F1-score), combining precision and recall into a single metric. The grader is deterministic and reproducible.
+Action is just `{"decision": "flag"}` or `{"decision": "skip"}`.
 
 ---
 
-## Setup & Running
+## Running it
 
-**1. Install deps:**
+**Install:**
 ```bash
 pip install openenv-core openai
 ```
 
-**2. Spin up the FastAPI Server via Docker:**
+**Docker:**
 ```bash
 docker build -t codereviewenv .
 docker run -p 7860:7860 codereviewenv
 ```
-*(If you are viewing this on Hugging Face Spaces, the server is automatically running!)*
 
-**3. Run the inference script:**
+**Inference:**
 ```bash
-export HF_TOKEN="your_huggingface_token"
+export HF_TOKEN="your_token"
 python inference.py
 ```
 
 ---
 
-## The Agents (We built two!)
+## Agents
+
+### LLM baseline (`inference.py`)
 
-### 1. The Zero-Shot LLM Baseline (`inference.py`)
-This is the standard OpenEnv submission script required by the Hackathon. We wrote a wrapper that passes the environment state into an OpenAI-compatible LLM to see if a huge model can reason through the file stats to allocate its budget. It evaluates the environment sequentially over the **Easy**, **Medium**, and **Hard** tasks.
+Sends the file stats to Qwen2.5-Coder-32B via the HF inference API and asks it to flag or skip. Runs all three difficulty levels.
 
-**Baseline Scores:**
+Rough zero-shot numbers:
 
-| Difficulty | Model | F1-Score | Precision | Recall |
-|-----------|-------|----------|-----------|--------|
-| Easy | Qwen2.5-Coder-32B | ~0.15 | ~0.12 | ~0.25 |
-| Medium | Qwen2.5-Coder-32B | ~0.10 | ~0.08 | ~0.18 |
-| Hard | Qwen2.5-Coder-32B | ~0.08 | ~0.06 | ~0.15 |
+| Difficulty | F1 | Precision | Recall |
+|---|---|---|---|
+| Easy | ~0.15 | ~0.12 | ~0.25 |
+| Medium | ~0.10 | ~0.08 | ~0.18 |
+| Hard | ~0.08 | ~0.06 | ~0.15 |
 
-These are zero-shot scores — the LLM has no training on this specific task, demonstrating there is real room for improvement via RL training.
+Lots of room to improve — the LLM has no training signal, it's just guessing from feature names.
 
 ```bash
-export HF_TOKEN="your_huggingface_token"
+export HF_TOKEN="your_token"
 python inference.py
 ```
 
-### 2. The Native PyTorch Agent (`train_pytorch_agent.py`)
-**Flex Warning.** LLMs are cool, but they struggle to implicitly understand strict mathematical bounds (like rationing a flag budget perfectly over exactly 29 files with precise asymmetric scoring). So, we went bare-metal.
+### PyTorch agent (`train_pytorch_agent.py`)
+
+REINFORCE with a 3-layer MLP. Takes the 6 observation features as input, outputs flag/skip probabilities. Trains directly against the env reward signal.
 
-We built a custom Deep Reinforcement Learning Agent using native PyTorch Policy Gradients (REINFORCE) to interface perfectly with the OpenEnv API. It iteratively converges to find the perfect risk/reward strategy.
 ```bash
 pip install torch
 python train_pytorch_agent.py
 ```
 
-## Hackathon Repo Tour
-* `Dockerfile` & `openenv.yaml`: The OpenEnv backend deployment wrappers
-* `inference.py`: The mandatory LLM endpoint validation script
-* `train_pytorch_agent.py`: Our custom PyTorch REINFORCE brain
-* `/models.py`: Pydantic Models for Actions & Observations
-* `/server/environment.py`: Where the magic reward mathematics happen
-* `/data/`: The actual scraped CVE GitHub dataset 
+## File layout
+
+- `Dockerfile` + `openenv.yaml` — deployment config
+- `inference.py` — LLM baseline (hackathon submission script)
+- `train_pytorch_agent.py` — pytorch RL agent
+- `models.py` — pydantic action/observation/state types
+- `server/environment.py` — core env logic + reward math
+- `data/` — the CVE dataset
 
-*MIT License. Thanks for checking it out!*
+MIT License

client.py

@@ -51,7 +51,9 @@ class CodeReviewEnv(EnvClient[CodeReviewAction, CodeReviewObservation, CodeRevie
             total_files=payload.get("total_files", 0),
             total_bugs=payload.get("total_bugs", 0),
             current_file_index=payload.get("current_file_index", 0),
+            difficulty_level=payload.get("difficulty_level", "medium"),
             files_flagged=payload.get("files_flagged", 0),
             correct_flags=payload.get("correct_flags", 0),
             review_budget=payload.get("review_budget", 0),
+            cumulative_reward=payload.get("cumulative_reward", 0.0),
         )

code_review_env.egg-info/PKG-INFO

@@ -10,81 +10,141 @@ Requires-Dist: openenv-core>=0.2.0
 Requires-Dist: fastapi>=0.100.0
 Requires-Dist: uvicorn>=0.20.0
 Requires-Dist: pydantic>=2.0.0
+Provides-Extra: train
+Requires-Dist: torch>=2.0.0; extra == "train"
+Provides-Extra: inference
+Requires-Dist: openai>=1.0.0; extra == "inference"
 
-# CodeReviewEnv
+---
+title: CodeReviewEnv
+emoji: 🛡️
+colorFrom: yellow
+colorTo: red
+sdk: docker
+pinned: false
+tags:
+  - openenv
+---
 
-RL environment for training agents to triage code files for vulnerabilities. Built on top of OpenEnv (Meta/PyTorch).
+# CodeReviewEnv: Triage CVEs Like a Pro
 
-Uses real labeled data from NVD - 1715 file samples across 65 CVEs. The agent reviews files from a repo and has to figure out which ones are actually buggy based on features like churn, complexity, TODO count, and recency.
+*Built for the Meta/PyTorch OpenEnv Hackathon*
 
-## How it works
+Hey! 👋 Welcome to **CodeReviewEnv**. 
 
-Each episode loads files from a real CVE-affected codebase. The agent goes through files one by one and decides `flag` or `skip`. At the end it gets scored on precision/recall/f1.
+Most RL environments are toy setups like GridWorld or simple mazes. We wanted to tackle a real problem that we actually face as developers: **Vulnerability Triage**. 
 
-Rewards are intentionally asymmetric - missing a real bug hurts way more than flagging a false positive, because thats how it works in practice.
+We built `CodeReviewEnv` using real-world data scraped directly from patches in the National Vulnerability Database (NVD). The agent connects to a repository, scans the files, and uses heuristics (like codebase churn, complexity, and recency) to decide whether to `flag` a file for manual security review or `skip` it and move on.
 
-| outcome | reward |
-|---------|--------|
-| correctly flag a bug | 1.0 |
-| correctly skip safe file | 0.8 |
-| flag a safe file | 0.4 |
-| miss a real bug | 0.0 |
+* 🚀 **Hugging Face Space (Live Environment)**: [https://huggingface.co/spaces/lucid987654/code-review-env](https://huggingface.co/spaces/lucid987654/code-review-env)
+* 📁 **GitHub Repository**: [https://github.com/subwaycookiecrunch/Meta-project](https://github.com/subwaycookiecrunch/Meta-project)
 
-The agent also has a limited flag budget per episode so it can't just flag everything.
+---
 
-## Setup
+### The Problem
 
+We have 1715 files across 65 authentic CVEs pulled directly from actual GitHub vulnerabilities. 
+We hit the agent with **Asymmetric Rewards**. In the real world, missing a critical bug (False Negative) is infinitely worse than accidentally flagging a safe file for review (False Positive). 
+
+Our reward table forces the agent to balance its paranoia:
+
+| Outcome | Reward | Rationale |
+|---------|--------|-----------|
+| True Positive (found a real bug) | **+1.0** | Highest reward — catching vulnerabilities is the goal |
+| True Negative (correctly skipped safe file) | **+0.8** | Good judgment that saves review budget |
+| False Positive (flagged a safe file) | **-0.4** | Wastes review budget, penalized |
+| False Negative (missed a real bug) | **-0.2** | Worst failure — penalty provides learning signal |
+| Over-budget flag attempt | **-0.5** | Hard constraint — budget is non-negotiable |
+
+Oh, and there's a strict **Review Budget**. You can't just flag everything, or you run out of budget and get heavily penalized!
+
+---
+
+### 🧩 Environment Specifications
+
+#### Action Space
+The action space is a straightforward, string-based categorical action:
+* `decision`: `"flag"` or `"skip"`
+
+#### Observation Space
+At each step, the environment provides a rich state vector. The key metrics include:
+* `file_path` & tracking metrics (`file_index`, `files_remaining`, `total_files`)
+* **Difficulty & Context**: `difficulty_level`, `cve_id`, `repo_name`
+* **Static Analysis Features**: 
+  * `churn_score` — lines changed in the file (higher = more volatile)
+  * `complexity_score` — cyclomatic complexity proxy (higher = harder to review)
+  * `todo_score` — count of TODOs/FIXMEs (higher = more tech debt)
+  * `recency_score` — how recently the file was modified (higher = more recent)
+* **Limits**: `review_budget` and `files_flagged`
+* **Terminal Metrics**: `precision`, `recall`, `f1_score`, `true_positives`, `false_positives`, `false_negatives`, `true_negatives`
+
+#### 🎯 The Three Tasks (Difficulty Tiers)
+We've partitioned the environment into three distinct difficulty tasks, scaling gracefully by the size of the repository logic the agent needs to parse over its fixed review budget:
+1. **Easy (`difficulty="easy"`)**: Small repositories and pull requests (≤ 15 files). The budget is relatively forgiving.
+2. **Medium (`difficulty="medium"`)**: Average-sized PRs (16-29 files) requiring more scrutiny.
+3. **Hard (`difficulty="hard"`)**: Large-scale patches (30+ files). The agent is strapped for budget and must be extremely selective about utilizing its flags.
+
+Each task includes a **programmatic grader** that returns a score between 0.0 and 1.0 (the F1-score), combining precision and recall into a single metric. The grader is deterministic and reproducible.
+
+---
+
+## Setup & Running
+
+**1. Install deps:**
 ```bash
 pip install openenv-core openai
 ```
 
-To run the server locally:
+**2. Spin up the FastAPI Server via Docker:**
 ```bash
 docker build -t codereviewenv .
-docker run -p 8000:8000 codereviewenv
+docker run -p 7860:7860 codereviewenv
 ```
+*(If you are viewing this on Hugging Face Spaces, the server is automatically running!)*
 
-## Client usage
-
-To run the evaluation script:
+**3. Run the inference script:**
 ```bash
-export HF_TOKEN="your_hugging_face_token"
+export OPENAI_API_KEY="your_api_key"  # or export HF_TOKEN="your_hf_token"
 python inference.py
 ```
 
-## Observation fields
+---
 
-- `file_path` - current file being reviewed
-- `churn_score` / `complexity_score` / `todo_score` / `recency_score` - features (0-100)
-- `review_budget` - how many flags left
-- `files_remaining` - files left in episode
-- `cve_id` / `cvss_score` - which vulnerability this episode is about
+## The Agents (We built two!)
 
-## Data
+### 1. The Zero-Shot LLM Baseline (`inference.py`)
+This is the standard OpenEnv submission script required by the Hackathon. We wrote a wrapper that passes the environment state into an OpenAI-compatible LLM to see if a huge model can reason through the file stats to allocate its budget. It evaluates the environment sequentially over the **Easy**, **Medium**, and **Hard** tasks.
 
-The training data comes from DeathClock's CVE pipeline:
-1. Pull CVEs from NVD API with GitHub refs
-2. Fetch file metadata and commit history from those repos
-3. Compute feature scores for each file
-4. Label files based on which ones were actually patched in the CVE fix
+**Baseline Scores:**
 
-65 CVEs, 1715 files total. 12 confirmed buggy files across 7 episodes - rest are negative examples that the agent needs to learn to skip.
+| Difficulty | Model | F1-Score | Precision | Recall |
+|-----------|-------|----------|-----------|--------|
+| Easy | Qwen2.5-Coder-32B | ~0.15 | ~0.12 | ~0.25 |
+| Medium | Qwen2.5-Coder-32B | ~0.10 | ~0.08 | ~0.18 |
+| Hard | Qwen2.5-Coder-32B | ~0.08 | ~0.06 | ~0.15 |
 
-## Files
+These are zero-shot scores — the LLM has no training on this specific task, demonstrating there is real room for improvement via RL training.
 
+```bash
+export OPENAI_API_KEY="your_api_key"
+python inference.py
 ```
-code_review_env/
-├── Dockerfile         - project root dockerfile (for openenv)
-├── inference.py       - generic evaluator script
-├── models.py          - action/observation/state types
-├── client.py          - websocket client
-├── demo.py            - runs baseline agents locally without docker
-├── data/              - cve training data (json)
-└── server/
-    ├── environment.py - core env logic
-    └── app.py         - fastapi server
+
+### 2. The Native PyTorch Agent (`train_pytorch_agent.py`)
+**Flex Warning.** LLMs are cool, but they struggle to implicitly understand strict mathematical bounds (like rationing a flag budget perfectly over exactly 29 files with precise asymmetric scoring). So, we went bare-metal.
+
+We built a custom Deep Reinforcement Learning Agent using native PyTorch Policy Gradients (REINFORCE) to interface perfectly with the OpenEnv API. It iteratively converges to find the perfect risk/reward strategy.
+```bash
+pip install torch
+python train_pytorch_agent.py
 ```
 
-## License
+## Hackathon Repo Tour
+* `Dockerfile` & `openenv.yaml`: The OpenEnv backend deployment wrappers
+* `inference.py`: The mandatory LLM endpoint validation script
+* `train_pytorch_agent.py`: Our custom PyTorch REINFORCE brain
+* `/models.py`: Pydantic Models for Actions & Observations
+* `/server/environment.py`: Where the magic reward mathematics happen
+* `/data/`: The actual scraped CVE GitHub dataset 
 
-MIT
+*MIT License. Thanks for checking it out!*

code_review_env.egg-info/requires.txt

@@ -2,3 +2,9 @@ openenv-core>=0.2.0
 fastapi>=0.100.0
 uvicorn>=0.20.0
 pydantic>=2.0.0
+
+[inference]
+openai>=1.0.0
+
+[train]
+torch>=2.0.0

demo.py

@@ -12,7 +12,7 @@ def run_episode(env, agent_fn, name, episode=None):
     if episode:
         orig = random.choice
         random.choice = lambda x: episode
-        obs = env.reset()
+        obs = env.reset(difficulty="medium")
         random.choice = orig
     else:
         obs = env.reset()

inference.py

@@ -11,7 +11,6 @@ API_BASE_URL = os.getenv("API_BASE_URL", "https://api-inference.huggingface.co/v
 MODEL_NAME = os.getenv("MODEL_NAME", "Qwen/Qwen2.5-Coder-32B-Instruct")
 LOCAL_IMAGE_NAME = os.getenv("LOCAL_IMAGE_NAME", "codereviewenv")
 
-# HF_TOKEN is primary per sample inference.py, API_KEY as fallback
 API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")
 
 if API_KEY is None:

models.py

@@ -40,3 +40,4 @@ class CodeReviewState(State):
     files_flagged: int = 0
     correct_flags: int = 0
     review_budget: int = 0
+    cumulative_reward: float = 0.0

openenv.yaml

@@ -1,3 +1,8 @@
 name: code_review_env
 version: "1.0.0"
-description: RL env for code vulnerability triage using CVE data from NVD
+description: RL env for triaging CVE vulnerabilities in code repos
+entry_point: code_review_env.server.app:app
+tags:
+  - openenv
+  - security
+  - code-review

pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "code-review-env"
 version = "1.0.0"
-description = "RL environment for code vulnerability triage using real CVE data"
+description = "RL env for CVE triage on real vulnerability data"
 readme = "README.md"
 requires-python = ">=3.10"
 license = {text = "MIT"}
@@ -16,8 +16,15 @@ dependencies = [
     "fastapi>=0.100.0",
     "uvicorn>=0.20.0",
     "pydantic>=2.0.0",
+]
+
+[project.optional-dependencies]
+train = [
     "torch>=2.0.0",
 ]
+inference = [
+    "openai>=1.0.0",
+]
 
 [tool.setuptools.packages.find]
 include = ["code_review_env*"]

server/app.py

@@ -6,7 +6,7 @@ app = create_fastapi_app(CodeReviewEnvironment, CodeReviewAction, CodeReviewObse
 
 @app.get("/")
 def root():
-    return {"status": "success", "message": "CodeReviewEnv is active and ready for OpenEnv clients!"}
+    return {"status": "ok", "env": "code_review"}
 
 
 def main():

server/environment.py

@@ -73,7 +73,7 @@ class CodeReviewEnvironment(Environment):
         if difficulty not in ["easy", "medium", "hard"]:
             difficulty = random.choice(["easy", "medium", "hard"])
 
-        # Filter by difficulty
+
         if difficulty == "easy":
             size_filter = lambda e: len(e["files"]) <= 15
         elif difficulty == "medium":
@@ -81,22 +81,19 @@ class CodeReviewEnvironment(Environment):
         else:
             size_filter = lambda e: len(e["files"]) >= 30
 
-        # Strongly prefer episodes that actually contain bugs —
-        # without bugs F1 is always 0.0 regardless of agent behavior,
-        # which would trip the "graders always return same score" disqualification.
+        # need episodes w/ bugs or f1 is stuck at 0
         buggy_candidates = [e for e in BUGGY_EPISODES if size_filter(e)]
         if buggy_candidates:
             ep = random.choice(buggy_candidates)
         else:
-            # Fallback: pick any episode matching size, then inject synthetic bugs
+            # fallback: any matching ep, add synthetic bugs if clean
             all_candidates = [e for e in EPISODES if size_filter(e)]
             if not all_candidates:
                 all_candidates = BUGGY_EPISODES if BUGGY_EPISODES else EPISODES
             ep = random.choice(all_candidates)
 
-            # If the chosen episode has no bugs, inject some
             if ep["total_bugs"] == 0:
-                ep = dict(ep)  # shallow copy to avoid mutating global
+                ep = dict(ep)
                 files = [dict(f) for f in ep["files"]]
                 n_inject = max(1, len(files) // 8)
                 targets = random.sample(range(len(files)), min(n_inject, len(files)))

train_pytorch_agent.py

@@ -45,7 +45,7 @@ def main():
     optimizer = optim.Adam(policy_net.parameters(), lr=learning_rate)
     
     num_episodes = 50
-    print(f"Initializing PyTorch REINFORCE Agent natively on CodeReviewEnv for {num_episodes} episodes...")
+    print(f"training for {num_episodes} episodes")
     
     env = CodeReviewEnvironment()
     
@@ -104,7 +104,7 @@ def main():
                   f"Reward: {sum(rewards):.1f} | "
                   f"F1: {obs.f1_score:.2f} (P: {obs.precision:.2f}, R: {obs.recall:.2f})")
             
-    print("\nTraining complete! PyTorch internal weights updated.")
+    print("\ndone.")
     
 if __name__ == "__main__":
     main()