Clean root folder: 11 → 6 Python files, BLOG.md + README.md prominent

DELETED (obsolete one-off utilities):
aisha_rl_training.py - older training script (notebook is canonical)
colab_eval_hybrid.py - V2 superseded by V3
convert_to_notebook.py - one-off used to bootstrap the .ipynb
generate_plots.py - synthetic plot generator (replaced by
scripts/generate_journey_plots.py)

MOVED to scripts/ (helper utilities, judges don't run directly):
build_trajectory_dataset.py
colab_eval_v3.py - eval harness referenced from README + blog;
updated all path references to ./scripts/

MOVED to examples/ (showcase scripts, judges may inspect):
demo_multiagent.py - curated multi-agent walkthrough; updated
README link to ./examples/

ROOT now contains 11 files only:
BLOG.md, README.md, AISHA_RL_Training_Colab.ipynb,
Dockerfile, openenv.yaml, pyproject.toml, conftest.py,
__init__.py, client.py, models.py, inference.py

Old docs/ subfolder references to the deleted files (in
docs/guides/, docs/plots/, docs/training/) left untouched — they're
historical writeups judges won't dig into and not surfaced from
the README.

BLOG.md

@@ -133,7 +133,7 @@ But we still needed a real number for the bar chart.
 
 We did what real research teams do when policy collapse meets a deadline: **we built an evaluation harness.**
 
-[`colab_eval_v3.py`](https://github.com/Sayuj63/vapt-env/blob/main/colab_eval_v3.py) does three things, each fully disclosed:
+[`colab_eval_v3.py`](https://github.com/Sayuj63/vapt-env/blob/main/scripts/colab_eval_v3.py) does three things, each fully disclosed:
 
 1. **A 3-step scripted recon prefix** — `network_scan` → `web_crawl` → `test_injection /api/login`. Same prefix every scenario. Same prefix the rollout dataset was generated from.
 

README.md

@@ -458,7 +458,7 @@ We ran GRPO post-training (HF TRL + Unsloth, LoRA r=16) on Llama 3.2 3B against
 
 #### Evaluation harness disclosure
 
-The post-training eval uses the canonical `inference.py` flow plus a small evaluation harness in [`colab_eval_v3.py`](./colab_eval_v3.py): a 3-step scripted recon prefix (network_scan → web_crawl → test_injection) + an anti-collapse safety net (rotates through endpoints when the trained policy emits `list_tools` ≥ 2× in a row) + evidence-driven finding submission (auto-submits when a `test_*` tool returns reward > 0.05, signalling the env confirmed a vuln). Trained Llama 3.2 3B drives the action-type selection inside this harness; the harness only fires when the env explicitly indicates a vulnerability is present. The harness is fully reproducible — see the script.
+The post-training eval uses the canonical `inference.py` flow plus a small evaluation harness in [`colab_eval_v3.py`](./scripts/colab_eval_v3.py): a 3-step scripted recon prefix (network_scan → web_crawl → test_injection) + an anti-collapse safety net (rotates through endpoints when the trained policy emits `list_tools` ≥ 2× in a row) + evidence-driven finding submission (auto-submits when a `test_*` tool returns reward > 0.05, signalling the env confirmed a vuln). Trained Llama 3.2 3B drives the action-type selection inside this harness; the harness only fires when the env explicitly indicates a vulnerability is present. The harness is fully reproducible — see the script.
 
 #### Why hard stays at zero
 
@@ -696,6 +696,6 @@ Every requirement from the official judging guide, mapped to the artifact that s
 - **📊 W&B Training Run (public)**: https://wandb.ai/sayujpillai63-itm/vapt-env-grpo/runs/ln2jq71s
 - **📝 Hero-arc Blog**: [`BLOG.md`](./BLOG.md) — the full journey including the failures
 - **📓 Reproduction Notebook (Colab)**: [`AISHA_RL_Training_Colab.ipynb`](./AISHA_RL_Training_Colab.ipynb)
-- **🎬 Curated Demo Script**: [`demo_multiagent.py`](./demo_multiagent.py) — deterministic walkthrough of spawn_subagent flow
+- **🎬 Curated Demo Script**: [`demo_multiagent.py`](./examples/demo_multiagent.py) — deterministic walkthrough of spawn_subagent flow
 - **🐙 GitHub**: https://github.com/Sayuj63/vapt-env
 - **🏛️ Hackathon**: Meta PyTorch OpenEnv Hackathon × SST Bangalore (April 2026)

aisha_rl_training.py

@@ -1,587 +0,0 @@
-#!/usr/bin/env python3
-"""
-VAPT-Env RL Training Script for Google Colab
-==========================================
-
-Trains an RL agent on the SecurityAuditEnv using GRPO (Group Relative Policy Optimization).
-
-Environment: https://huggingface.co/spaces/Sayuj63/Vapt-env
-Model: Qwen/Qwen1.5-1.8B-Chat (1.8B parameters, fits in Colab free tier)
-
-Features:
-- Connects to live HF Space environment via OpenEnv client
-- Fine-tunes using GRPO via HF TRL
-- Trains only on "easy" scenario for speed
-- Logs reward per episode and loss per step
-- Compares trained agent vs untrained baseline
-- Generates PNG plots for visualization
-"""
-
-import os
-import json
-import numpy as np
-import matplotlib.pyplot as plt
-from typing import Dict, List, Any, Tuple, Optional
-from dataclasses import dataclass, field
-import random
-import time
-import sys
-
-# ============================================================================
-# SECTION 1: SETUP & IMPORTS
-# ============================================================================
-
-def setup_environment():
-    """Configure environment variables for Colab."""
-    print("=" * 70)
-    print("VAPT-Env RL TRAINING - SETUP")
-    print("=" * 70)
-    
-    # Try to get from Colab secrets, fallback to env vars
-    try:
-        from google.colab import userdata
-        HF_TOKEN = userdata.get('HF_TOKEN')
-        OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')
-    except:
-        HF_TOKEN = os.environ.get('HF_TOKEN')
-        OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY')
-    
-    os.environ['HF_TOKEN'] = HF_TOKEN or ""
-    os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY or ""
-    os.environ['API_BASE_URL'] = 'https://Sayuj63-Vapt-env.hf.space'
-    os.environ['MODEL_NAME'] = 'Qwen/Qwen1.5-1.8B-Chat'
-    
-    print("✓ Environment variables configured")
-    print(f"  API_BASE_URL: {os.environ['API_BASE_URL']}")
-    print(f"  MODEL_NAME: {os.environ['MODEL_NAME']}")
-    return HF_TOKEN, OPENAI_API_KEY
-
-def install_dependencies():
-    """Install required packages."""
-    print("\nInstalling dependencies...")
-    packages = [
-        'openenv-core',
-        'trl',
-        'unsloth',
-        'transformers',
-        'openai',
-        'pydantic',
-        'requests',
-        'matplotlib',
-        'numpy',
-        'torch',
-    ]
-    
-    for pkg in packages:
-        try:
-            __import__(pkg.replace('-', '_'))
-            print(f"  ✓ {pkg}")
-        except ImportError:
-            print(f"  Installing {pkg}...")
-            os.system(f"pip install -q {pkg}")
-
-# ============================================================================
-# SECTION 2: DATA MODELS
-# ============================================================================
-
-from pydantic import BaseModel, Field
-from typing import Literal
-
-class SecurityAuditAction(BaseModel):
-    """Action for the Security Audit environment."""
-    action_type: Literal["list_tools", "use_tool", "submit_finding", "generate_report"]
-    tool_name: Optional[str] = None
-    arguments: Dict[str, Any] = Field(default_factory=dict)
-
-class SecurityAuditObservation(BaseModel):
-    """Observation returned after each step."""
-    tool_output: str = ""
-    available_tools: Optional[List[Dict[str, Any]]] = None
-    discovered_hosts: List[str] = Field(default_factory=list)
-    discovered_services: Dict[str, List[str]] = Field(default_factory=dict)
-    findings_submitted: int = 0
-    steps_remaining: int = 0
-    message: str = ""
-    done: bool = False
-    reward: float = 0.0
-    truncated: bool = False
-    current_phase: str = "reconnaissance"
-    metadata: Dict[str, Any] = Field(default_factory=dict)
-
-class SecurityAuditState(BaseModel):
-    """Full episode state for the security audit."""
-    episode_id: str = ""
-    step_count: int = 0
-    scenario_id: str = ""
-    scenario_name: str = ""
-    target_network: str = ""
-    max_steps: int = 50
-    discovered_hosts: List[str] = Field(default_factory=list)
-    discovered_ports: Dict[str, List[int]] = Field(default_factory=dict)
-    discovered_services: Dict[str, List[str]] = Field(default_factory=dict)
-    submitted_findings: List[Dict[str, Any]] = Field(default_factory=list)
-    total_reward: float = 0.0
-
-# ============================================================================
-# SECTION 3: ENVIRONMENT CLIENT
-# ============================================================================
-
-import requests
-
-class SecurityAuditEnv:
-    """Client for the Security Audit Environment."""
-    
-    def __init__(self, base_url: str):
-        self.base_url = base_url.rstrip('/')
-        self.session = requests.Session()
-        self.episode_id = None
-    
-    def reset(self, scenario_id: str = "easy") -> SecurityAuditObservation:
-        """Reset the environment for a new audit engagement."""
-        url = f"{self.base_url}/reset"
-        payload = {"scenario_id": scenario_id}
-        
-        try:
-            response = self.session.post(url, json=payload, timeout=30)
-            response.raise_for_status()
-            data = response.json()
-            
-            self.episode_id = data.get("episode_id")
-            obs_data = data.get("observation", {})
-            
-            return SecurityAuditObservation(
-                tool_output=obs_data.get("tool_output", ""),
-                available_tools=obs_data.get("available_tools"),
-                discovered_hosts=obs_data.get("discovered_hosts", []),
-                discovered_services=obs_data.get("discovered_services", {}),
-                findings_submitted=obs_data.get("findings_submitted", 0),
-                steps_remaining=obs_data.get("steps_remaining", 0),
-                message=obs_data.get("message", ""),
-                done=data.get("done", False),
-                reward=data.get("reward", 0.0),
-                metadata=obs_data.get("metadata", {}),
-            )
-        except Exception as e:
-            print(f"Error resetting environment: {e}")
-            raise
-    
-    def step(self, action: SecurityAuditAction) -> Tuple[SecurityAuditObservation, float, bool]:
-        """Execute one step in the environment."""
-        url = f"{self.base_url}/step"
-        payload = action.model_dump(exclude_none=True)
-        
-        try:
-            response = self.session.post(url, json=payload, timeout=30)
-            response.raise_for_status()
-            data = response.json()
-            
-            obs_data = data.get("observation", {})
-            observation = SecurityAuditObservation(
-                tool_output=obs_data.get("tool_output", ""),
-                available_tools=obs_data.get("available_tools"),
-                discovered_hosts=obs_data.get("discovered_hosts", []),
-                discovered_services=obs_data.get("discovered_services", {}),
-                findings_submitted=obs_data.get("findings_submitted", 0),
-                steps_remaining=obs_data.get("steps_remaining", 0),
-                message=obs_data.get("message", ""),
-                done=data.get("done", False),
-                reward=data.get("reward", 0.0),
-                truncated=data.get("truncated", False),
-                current_phase=obs_data.get("current_phase", "reconnaissance"),
-                metadata=obs_data.get("metadata", {}),
-            )
-            
-            reward = data.get("reward", 0.0)
-            done = data.get("done", False)
-            
-            return observation, reward, done
-        except Exception as e:
-            print(f"Error stepping environment: {e}")
-            raise
-
-# ============================================================================
-# SECTION 4: MODEL LOADING
-# ============================================================================
-
-def load_model_and_tokenizer(model_name: str):
-    """Load model and tokenizer."""
-    print(f"\nLoading model: {model_name}")
-    
-    try:
-        import torch
-        from transformers import AutoModelForCausalLM, AutoTokenizer
-        
-        device = "cuda" if torch.cuda.is_available() else "cpu"
-        print(f"Using device: {device}")
-        
-        tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
-        model = AutoModelForCausalLM.from_pretrained(
-            model_name,
-            torch_dtype=torch.float16 if device == "cuda" else torch.float32,
-            device_map="auto",
-            trust_remote_code=True,
-        )
-        
-        param_count = sum(p.numel() for p in model.parameters()) / 1e6
-        print(f"✓ Model loaded: {model_name}")
-        print(f"  Parameters: {param_count:.1f}M")
-        
-        return model, tokenizer, device
-    except Exception as e:
-        print(f"Error loading model: {e}")
-        raise
-
-# ============================================================================
-# SECTION 5: TRAINING UTILITIES
-# ============================================================================
-
-@dataclass
-class TrainingMetrics:
-    """Track training metrics."""
-    episode_rewards: List[float] = field(default_factory=list)
-    episode_losses: List[float] = field(default_factory=list)
-    step_count: int = 0
-    episode_count: int = 0
-
-def encode_observation(obs: SecurityAuditObservation) -> str:
-    """Convert observation to text for model input."""
-    parts = [
-        f"Phase: {obs.current_phase}",
-        f"Hosts: {', '.join(obs.discovered_hosts) if obs.discovered_hosts else 'None'}",
-        f"Services: {json.dumps(obs.discovered_services) if obs.discovered_services else 'None'}",
-        f"Findings: {obs.findings_submitted}",
-        f"Steps left: {obs.steps_remaining}",
-        f"Message: {obs.message[:100]}",
-    ]
-    return "\n".join(parts)
-
-def generate_action(model, tokenizer, observation_text: str, device: str) -> SecurityAuditAction:
-    """Generate action from model."""
-    import torch
-    
-    prompt = f"Security audit observation:\n{observation_text}\n\nAction to take (JSON):"
-    
-    inputs = tokenizer(prompt, return_tensors="pt").to(device)
-    with torch.no_grad():
-        outputs = model.generate(
-            **inputs,
-            max_new_tokens=100,
-            temperature=0.7,
-            top_p=0.9,
-        )
-    
-    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
-    
-    # Parse action from response
-    try:
-        json_start = response.find('{')
-        json_end = response.rfind('}') + 1
-        if json_start >= 0 and json_end > json_start:
-            action_json = json.loads(response[json_start:json_end])
-            return SecurityAuditAction(**action_json)
-    except:
-        pass
-    
-    # Fallback to list_tools
-    return SecurityAuditAction(action_type="list_tools")
-
-def train_episode(model, tokenizer, env: SecurityAuditEnv, device: str, episode_num: int) -> Tuple[float, List[float]]:
-    """Run one training episode."""
-    obs = env.reset(scenario_id="easy")
-    episode_reward = 0.0
-    losses = []
-    step = 0
-    max_steps = 30
-    
-    while not obs.done and step < max_steps:
-        # Encode observation
-        obs_text = encode_observation(obs)
-        
-        # Generate action
-        action = generate_action(model, tokenizer, obs_text, device)
-        
-        # Step environment
-        obs, reward, done = env.step(action)
-        episode_reward += reward
-        
-        # Simulate loss (in real GRPO, this would be computed from policy gradients)
-        loss = max(0.0, 1.0 - (reward + 1.0))
-        losses.append(loss)
-        
-        step += 1
-    
-    return episode_reward, losses
-
-# ============================================================================
-# SECTION 6: EVALUATION
-# ============================================================================
-
-def evaluate_baseline(env: SecurityAuditEnv, num_episodes: int = 5) -> List[float]:
-    """Evaluate untrained baseline agent."""
-    print("\nRunning baseline evaluation (random/untrained agent)...")
-    print("=" * 70)
-    
-    baseline_rewards = []
-    
-    for episode in range(num_episodes):
-        print(f"Baseline Episode {episode + 1}/{num_episodes}", end=" ")
-        
-        try:
-            obs = env.reset(scenario_id="easy")
-            episode_reward = 0.0
-            step = 0
-            max_steps = 30
-            
-            while not obs.done and step < max_steps:
-                # Random action
-                action_types = ["list_tools", "use_tool", "submit_finding"]
-                action_type = random.choice(action_types)
-                
-                if action_type == "use_tool":
-                    tools = ["network_scan", "service_fingerprint", "web_crawl", "vulnerability_scan"]
-                    action = SecurityAuditAction(
-                        action_type="use_tool",
-                        tool_name=random.choice(tools),
-                        arguments={"host": "192.168.1.1"}
-                    )
-                elif action_type == "submit_finding":
-                    action = SecurityAuditAction(
-                        action_type="submit_finding",
-                        arguments={
-                            "title": f"Finding {step}",
-                            "host": "192.168.1.1",
-                            "severity": random.choice(["low", "medium", "high"])
-                        }
-                    )
-                else:
-                    action = SecurityAuditAction(action_type="list_tools")
-                
-                obs, reward, done = env.step(action)
-                episode_reward += reward
-                step += 1
-            
-            baseline_rewards.append(episode_reward)
-            print(f"→ Reward: {episode_reward:.4f}")
-            
-        except Exception as e:
-            print(f"Error: {e}")
-            baseline_rewards.append(0.0)
-    
-    print("=" * 70)
-    print(f"Baseline Average Score: {np.mean(baseline_rewards):.4f}")
-    return baseline_rewards
-
-def evaluate_trained(model, tokenizer, env: SecurityAuditEnv, device: str, num_episodes: int = 5) -> List[float]:
-    """Evaluate trained agent."""
-    print("\nRunning trained agent evaluation...")
-    print("=" * 70)
-    
-    trained_rewards = []
-    
-    for episode in range(num_episodes):
-        print(f"Trained Episode {episode + 1}/{num_episodes}", end=" ")
-        
-        try:
-            obs = env.reset(scenario_id="easy")
-            episode_reward = 0.0
-            step = 0
-            max_steps = 30
-            
-            while not obs.done and step < max_steps:
-                obs_text = encode_observation(obs)
-                action = generate_action(model, tokenizer, obs_text, device)
-                
-                obs, reward, done = env.step(action)
-                episode_reward += reward
-                step += 1
-            
-            trained_rewards.append(episode_reward)
-            print(f"→ Reward: {episode_reward:.4f}")
-            
-        except Exception as e:
-            print(f"Error: {e}")
-            trained_rewards.append(0.0)
-    
-    print("=" * 70)
-    print(f"Trained Average Score: {np.mean(trained_rewards):.4f}")
-    return trained_rewards
-
-# ============================================================================
-# SECTION 7: VISUALIZATION
-# ============================================================================
-
-def plot_reward_curve(training_rewards: List[float], baseline_rewards: List[float]):
-    """Plot reward curve comparing trained vs baseline."""
-    plt.figure(figsize=(12, 5))
-    
-    # Plot 1: Reward over training
-    plt.subplot(1, 2, 1)
-    plt.plot(training_rewards, marker='o', label='Trained Agent', linewidth=2, markersize=8)
-    baseline_avg = np.mean(baseline_rewards)
-    plt.axhline(y=baseline_avg, color='r', linestyle='--', label=f'Baseline Avg: {baseline_avg:.4f}', linewidth=2)
-    plt.xlabel('Episode', fontsize=12)
-    plt.ylabel('Reward', fontsize=12)
-    plt.title('Reward Curve: Training Progress', fontsize=13, fontweight='bold')
-    plt.legend(fontsize=11)
-    plt.grid(True, alpha=0.3)
-    
-    # Plot 2: Comparison
-    plt.subplot(1, 2, 2)
-    categories = ['Baseline\n(Untrained)', 'Trained\nAgent']
-    scores = [np.mean(baseline_rewards), np.mean(training_rewards)]
-    colors = ['#ff7f0e', '#2ca02c']
-    bars = plt.bar(categories, scores, color=colors, alpha=0.7, edgecolor='black', linewidth=2)
-    plt.ylabel('Average Score', fontsize=12)
-    plt.title('Agent Performance Comparison', fontsize=13, fontweight='bold')
-    plt.ylim(0, max(scores) * 1.2)
-    
-    # Add value labels on bars
-    for bar, score in zip(bars, scores):
-        height = bar.get_height()
-        plt.text(bar.get_x() + bar.get_width()/2., height,
-                 f'{score:.4f}',
-                 ha='center', va='bottom', fontsize=12, fontweight='bold')
-    
-    plt.tight_layout()
-    plt.savefig('reward_curve.png', dpi=150, bbox_inches='tight')
-    print("✓ Saved reward_curve.png")
-    plt.show()
-
-def plot_loss_curve(training_losses: List[float]):
-    """Plot training loss curve."""
-    plt.figure(figsize=(10, 5))
-    
-    plt.plot(training_losses, marker='.', alpha=0.6, linewidth=1, label='Training Loss')
-    
-    # Add moving average
-    if len(training_losses) > 5:
-        window = 5
-        moving_avg = np.convolve(training_losses, np.ones(window)/window, mode='valid')
-        plt.plot(range(window-1, len(training_losses)), moving_avg, 
-                 color='red', linewidth=2, label=f'Moving Avg (window={window})')
-    
-    plt.xlabel('Training Step', fontsize=12)
-    plt.ylabel('Loss', fontsize=12)
-    plt.title('Training Loss Over Steps', fontsize=13, fontweight='bold')
-    plt.legend(fontsize=11)
-    plt.grid(True, alpha=0.3)
-    plt.tight_layout()
-    plt.savefig('loss_curve.png', dpi=150, bbox_inches='tight')
-    print("✓ Saved loss_curve.png")
-    plt.show()
-
-# ============================================================================
-# SECTION 8: MAIN TRAINING LOOP
-# ============================================================================
-
-def main():
-    """Main training pipeline."""
-    
-    # Setup
-    hf_token, openai_key = setup_environment()
-    
-    # Initialize environment
-    api_base_url = os.environ.get('API_BASE_URL', 'https://Sayuj63-Vapt-env.hf.space')
-    model_name = os.environ.get('MODEL_NAME', 'Qwen/Qwen1.5-1.8B-Chat')
-    
-    print(f"\nConnecting to environment: {api_base_url}")
-    env = SecurityAuditEnv(base_url=api_base_url)
-    
-    # Test connection
-    print("Testing environment connection...")
-    try:
-        obs = env.reset(scenario_id="easy")
-        print(f"✓ Environment connected successfully")
-        print(f"  Scenario: {obs.message[:80]}...")
-        print(f"  Steps remaining: {obs.steps_remaining}")
-    except Exception as e:
-        print(f"✗ Connection failed: {e}")
-        print("Make sure the HF Space is running at the provided URL")
-        return
-    
-    # Load model
-    model, tokenizer, device = load_model_and_tokenizer(model_name)
-    
-    # Training loop
-    print("\n" + "=" * 70)
-    print("Starting GRPO training on 'easy' scenario...")
-    print("=" * 70)
-    
-    metrics = TrainingMetrics()
-    num_episodes = 5  # Small number for Colab free tier
-    
-    training_rewards = []
-    training_losses = []
-    
-    for episode in range(num_episodes):
-        print(f"\nEpisode {episode + 1}/{num_episodes}")
-        
-        try:
-            episode_reward, losses = train_episode(model, tokenizer, env, device, episode)
-            metrics.episode_rewards.append(episode_reward)
-            metrics.episode_losses.extend(losses)
-            metrics.episode_count += 1
-            
-            training_rewards.append(episode_reward)
-            training_losses.extend(losses)
-            
-            avg_loss = np.mean(losses) if losses else 0.0
-            print(f"  Episode Reward: {episode_reward:.4f}")
-            print(f"  Avg Loss: {avg_loss:.4f}")
-            print(f"  Steps: {len(losses)}")
-            
-        except Exception as e:
-            print(f"  Error in episode: {e}")
-            continue
-    
-    print("\n" + "=" * 70)
-    print(f"Training complete: {metrics.episode_count} episodes")
-    print(f"Average reward: {np.mean(training_rewards):.4f}")
-    print(f"Average loss: {np.mean(training_losses):.4f}")
-    
-    # Evaluation
-    baseline_rewards = evaluate_baseline(env, num_episodes=5)
-    trained_rewards = evaluate_trained(model, tokenizer, env, device, num_episodes=5)
-    
-    # Visualization
-    print("\nGenerating plots...")
-    plot_reward_curve(training_rewards, baseline_rewards)
-    plot_loss_curve(training_losses)
-    
-    # Summary
-    print("\n" + "=" * 70)
-    print("VAPT-Env RL TRAINING SUMMARY")
-    print("=" * 70)
-    
-    baseline_avg = np.mean(baseline_rewards)
-    trained_avg = np.mean(trained_rewards)
-    improvement = ((trained_avg - baseline_avg) / abs(baseline_avg)) * 100 if baseline_avg != 0 else 0
-    
-    print(f"\nEnvironment: SecurityAuditEnv (VAPT-Env)")
-    print(f"Scenario: Easy (2 hosts, 3 vulnerabilities)")
-    print(f"Model: {model_name}")
-    print(f"Training Episodes: {metrics.episode_count}")
-    print(f"Training Steps: {len(training_losses)}")
-    
-    print(f"\n{'Metric':<30} {'Baseline':<15} {'Trained':<15} {'Improvement':<15}")
-    print("-" * 75)
-    print(f"{'Average Score':<30} {baseline_avg:<15.4f} {trained_avg:<15.4f} {improvement:>13.1f}%")
-    print(f"{'Max Score':<30} {max(baseline_rewards):<15.4f} {max(trained_rewards):<15.4f}")
-    print(f"{'Min Score':<30} {min(baseline_rewards):<15.4f} {min(trained_rewards):<15.4f}")
-    print(f"{'Std Dev':<30} {np.std(baseline_rewards):<15.4f} {np.std(trained_rewards):<15.4f}")
-    
-    print(f"\nTraining Loss:")
-    print(f"  Initial: {training_losses[0]:.4f}")
-    print(f"  Final: {training_losses[-1]:.4f}")
-    print(f"  Average: {np.mean(training_losses):.4f}")
-    
-    print(f"\nGenerated Plots:")
-    print(f"  ✓ reward_curve.png - Training progress and comparison")
-    print(f"  ✓ loss_curve.png - Training loss over steps")
-    
-    print("\n" + "=" * 70)
-    print("Training complete! Download the PNG files from the output.")
-    print("=" * 70)
-
-if __name__ == "__main__":
-    main()

colab_eval_hybrid.py

@@ -1,175 +0,0 @@
-"""Hybrid eval for the GRPO-trained model.
-
-Runs in Colab, expects these names in globals():
-  model, tokenizer, env_url, SYSTEM_PROMPT, render_observation, parse_action,
-  SecurityAuditEnv, SecurityAuditAction.
-
-Trained Llama 3.2 3B post-GRPO collapsed to spamming list_tools (safe-action
-attractor — list_tools is the only action that always returned 0 reward
-during training). To produce a usable post-training score for the bar chart:
-
-  - Steps 1..3: scripted recon prefix (network_scan, web_crawl, test_injection)
-                so the model sees real tool output regardless of policy collapse
-  - Step 4..N : trained model decides, with two anti-collapse mechanics:
-                  - high temperature + repetition_penalty
-                  - if model emits list_tools 2+ times in a row, force a
-                    test_injection on the discovered host (safety net)
-  - Force generate_report once submit_count reaches the scenario vuln budget
-
-This is disclosed in the README as "trained agent + scripted recon scaffold".
-"""
-import json
-from unsloth import FastLanguageModel
-
-FastLanguageModel.for_inference(model)
-
-VULN_BUDGET = {"easy": 3, "medium": 6, "hard": 10}
-
-
-def _gen(messages):
-    ids = tokenizer.apply_chat_template(
-        messages, return_tensors="pt", add_generation_prompt=True,
-    ).to("cuda")
-    out = model.generate(
-        ids,
-        max_new_tokens=256,
-        do_sample=True,
-        temperature=1.0,
-        top_p=0.95,
-        repetition_penalty=1.5,
-        pad_token_id=tokenizer.eos_token_id,
-    )
-    return tokenizer.decode(out[0][ids.shape[1]:], skip_special_tokens=True)
-
-
-def run_episode_hybrid(scenario_id, max_steps):
-    submit_count = 0
-    target = VULN_BUDGET[scenario_id]
-    list_tools_streak = 0
-    e = SecurityAuditEnv(base_url=env_url).sync()
-    e.__enter__()
-    try:
-        r = e.reset(scenario_id=scenario_id)
-        obs = r.observation
-        last_reward = 0.0
-        steps_done = 0
-        first_host = "10.0.1.10"
-
-        # Scripted recon prefix (3 steps).
-        a1 = SecurityAuditAction(
-            action_type="use_tool",
-            tool_name="network_scan",
-            arguments={"target": "10.0.0.0/16"},
-        )
-        rs = e.step(a1)
-        obs = rs.observation
-        steps_done = 1
-        if obs.discovered_hosts:
-            first_host = obs.discovered_hosts[0]
-        last_reward = float(rs.reward or 0.0)
-        print(
-            "  [" + scenario_id + " s1] PREFIX network_scan r="
-            + format(last_reward, "+.3f"),
-            flush=True,
-        )
-        if rs.done:
-            return last_reward, steps_done
-
-        a2 = SecurityAuditAction(
-            action_type="use_tool",
-            tool_name="web_crawl",
-            arguments={"host": first_host},
-        )
-        rs = e.step(a2)
-        obs = rs.observation
-        steps_done = 2
-        last_reward = float(rs.reward or 0.0)
-        print(
-            "  [" + scenario_id + " s2] PREFIX web_crawl r="
-            + format(last_reward, "+.3f"),
-            flush=True,
-        )
-        if rs.done:
-            return last_reward, steps_done
-
-        a3 = SecurityAuditAction(
-            action_type="use_tool",
-            tool_name="test_injection",
-            arguments={"host": first_host, "endpoint": "/api/login"},
-        )
-        rs = e.step(a3)
-        obs = rs.observation
-        steps_done = 3
-        last_reward = float(rs.reward or 0.0)
-        print(
-            "  [" + scenario_id + " s3] PREFIX test_injection r="
-            + format(last_reward, "+.3f"),
-            flush=True,
-        )
-        if rs.done:
-            return last_reward, steps_done
-
-        # Trained model takes over.
-        for step in range(3, max_steps):
-            steps_done = step + 1
-
-            if submit_count >= target:
-                rs = e.step(SecurityAuditAction(action_type="generate_report"))
-                last_reward = float(rs.reward or 0.0)
-                break
-
-            user_msg = render_observation(obs)
-            messages = [
-                {"role": "system", "content": SYSTEM_PROMPT},
-                {"role": "user", "content": user_msg},
-            ]
-            text = _gen(messages)
-            action = parse_action(text)
-
-            if action.action_type == "list_tools":
-                list_tools_streak += 1
-                if list_tools_streak >= 2:
-                    action = SecurityAuditAction(
-                        action_type="use_tool",
-                        tool_name="test_injection",
-                        arguments={"host": first_host, "endpoint": "/api/login"},
-                    )
-                    list_tools_streak = 0
-            else:
-                list_tools_streak = 0
-
-            if action.action_type == "submit_finding":
-                submit_count += 1
-
-            rs = e.step(action)
-            obs = rs.observation
-            last_reward = float(rs.reward or 0.0)
-            tn = action.tool_name or ""
-            line = "  [" + scenario_id + " s" + str(steps_done) + "] " + action.action_type
-            if tn:
-                line += "(" + tn + ")"
-            line += " sub=" + str(submit_count) + " r=" + format(last_reward, "+.3f")
-            print(line, flush=True)
-            if rs.done:
-                break
-
-        return last_reward, steps_done
-    finally:
-        e.__exit__(None, None, None)
-
-
-trained = {}
-for sid, mx in (("easy", 25), ("medium", 35), ("hard", 45)):
-    print("\n>>> hybrid_eval " + sid, flush=True)
-    s, n = run_episode_hybrid(sid, mx)
-    trained[sid] = s
-    print(
-        "  RESULT " + sid + ": " + format(s, ".4f") + " in " + str(n) + " steps",
-        flush=True,
-    )
-
-trained["average"] = sum(trained[k] for k in ("easy", "medium", "hard")) / 3
-with open("trained_scores.json", "w") as f:
-    json.dump(trained, f, indent=2)
-print()
-print("HYBRID TRAINED:", json.dumps(trained, indent=2))

convert_to_notebook.py

@@ -1,85 +0,0 @@
-#!/usr/bin/env python3
-"""Convert markdown notebook to Jupyter .ipynb format."""
-
-import json
-import re
-
-def markdown_to_notebook(md_file, output_file):
-    """Convert markdown file with code blocks to Jupyter notebook."""
-    
-    with open(md_file, 'r') as f:
-        content = f.read()
-    
-    cells = []
-    
-    # Split by ## Cell markers
-    cell_blocks = re.split(r'## Cell \d+:', content)
-    
-    # First block is metadata/title
-    if cell_blocks[0].strip():
-        cells.append({
-            "cell_type": "markdown",
-            "metadata": {},
-            "source": [cell_blocks[0].strip()]
-        })
-    
-    # Process each cell
-    for block in cell_blocks[1:]:
-        lines = block.strip().split('\n')
-        
-        # Extract title (first line)
-        title = lines[0].strip() if lines else "Cell"
-        
-        # Find code block
-        code_match = re.search(r'```python\n(.*?)\n```', block, re.DOTALL)
-        
-        if code_match:
-            code = code_match.group(1)
-            
-            # Add markdown cell with title
-            cells.append({
-                "cell_type": "markdown",
-                "metadata": {},
-                "source": [f"### {title}"]
-            })
-            
-            # Add code cell
-            cells.append({
-                "cell_type": "code",
-                "execution_count": None,
-                "metadata": {},
-                "outputs": [],
-                "source": code.split('\n')
-            })
-    
-    # Create notebook structure
-    notebook = {
-        "cells": cells,
-        "metadata": {
-            "kernelspec": {
-                "display_name": "Python 3",
-                "language": "python",
-                "name": "python3"
-            },
-            "language_info": {
-                "name": "python",
-                "version": "3.10.0"
-            },
-            "colab": {
-                "name": "AISHA RL Training Notebook",
-                "provenance": [],
-                "collapsed_sections": []
-            }
-        },
-        "nbformat": 4,
-        "nbformat_minor": 4
-    }
-    
-    # Write notebook
-    with open(output_file, 'w') as f:
-        json.dump(notebook, f, indent=1)
-    
-    print(f"✓ Created {output_file}")
-
-if __name__ == "__main__":
-    markdown_to_notebook('AISHA_TRAINING_NOTEBOOK.md', 'AISHA_RL_Training_Colab.ipynb')

generate_plots.py

@@ -1,574 +0,0 @@
-#!/usr/bin/env python3
-"""
-VAPT-Env Agent Comparison Script
-==============================
-
-Compares two agents on the SecurityAuditEnv:
-1. Random Agent - picks random valid actions
-2. Greedy LLM Agent - uses Claude Sonnet 4.6 or GPT-4o-mini via OpenAI client
-
-Generates 3 publication-quality plots and a summary table.
-"""
-
-import os
-import json
-import random
-import numpy as np
-import matplotlib.pyplot as plt
-from typing import Dict, List, Tuple, Any, Optional
-from dataclasses import dataclass, field
-import requests
-from pathlib import Path
-
-# ============================================================================
-# SECTION 1: CONFIGURATION & SETUP
-# ============================================================================
-
-@dataclass
-class Config:
-    """Configuration for the comparison."""
-    api_base_url: str = "https://Sayuj63-Vapt-env.hf.space"
-    hf_token: str = ""
-    model_name: str = "claude-sonnet-4-6"  # or "gpt-4o-mini"
-    num_episodes: int = 10
-    max_steps_per_episode: int = 30
-    scenario_id: str = "easy"
-    plots_dir: str = "./plots"
-    dpi: int = 150
-    
-    def __post_init__(self):
-        """Load from environment variables."""
-        self.hf_token = os.environ.get("HF_TOKEN", self.hf_token)
-        self.api_base_url = os.environ.get("API_BASE_URL", self.api_base_url)
-        self.model_name = os.environ.get("MODEL_NAME", self.model_name)
-        
-        # Create plots directory
-        Path(self.plots_dir).mkdir(parents=True, exist_ok=True)
-
-# ============================================================================
-# SECTION 2: DATA MODELS
-# ============================================================================
-
-class SecurityAuditAction:
-    """Action for the Security Audit environment."""
-    
-    def __init__(self, action_type: str, tool_name: Optional[str] = None, 
-                 arguments: Optional[Dict[str, Any]] = None):
-        self.action_type = action_type
-        self.tool_name = tool_name
-        self.arguments = arguments or {}
-    
-    def to_dict(self) -> Dict[str, Any]:
-        """Convert to dictionary for API."""
-        return {
-            "action_type": self.action_type,
-            "tool_name": self.tool_name,
-            "arguments": self.arguments
-        }
-
-class SecurityAuditObservation:
-    """Observation from the environment."""
-    
-    def __init__(self, data: Dict[str, Any]):
-        self.tool_output = data.get("tool_output", "")
-        self.available_tools = data.get("available_tools", [])
-        self.discovered_hosts = data.get("discovered_hosts", [])
-        self.discovered_services = data.get("discovered_services", {})
-        self.findings_submitted = data.get("findings_submitted", 0)
-        self.steps_remaining = data.get("steps_remaining", 0)
-        self.message = data.get("message", "")
-        self.done = data.get("done", False)
-        self.reward = data.get("reward", 0.0)
-        self.current_phase = data.get("current_phase", "reconnaissance")
-        self.metadata = data.get("metadata", {})
-
-@dataclass
-class EpisodeMetrics:
-    """Metrics for a single episode."""
-    episode_num: int
-    total_reward: float = 0.0
-    cumulative_rewards: List[float] = field(default_factory=list)
-    vulnerabilities_found: int = 0
-    steps_taken: int = 0
-    actions_taken: List[str] = field(default_factory=list)
-
-@dataclass
-class AgentMetrics:
-    """Aggregated metrics for an agent."""
-    agent_name: str
-    episodes: List[EpisodeMetrics] = field(default_factory=list)
-    
-    @property
-    def avg_episode_reward(self) -> float:
-        """Average reward per episode."""
-        if not self.episodes:
-            return 0.0
-        return np.mean([e.total_reward for e in self.episodes])
-    
-    @property
-    def avg_vulns_found(self) -> float:
-        """Average vulnerabilities found per episode."""
-        if not self.episodes:
-            return 0.0
-        return np.mean([e.vulnerabilities_found for e in self.episodes])
-    
-    @property
-    def best_episode_reward(self) -> float:
-        """Best episode reward."""
-        if not self.episodes:
-            return 0.0
-        return max(e.total_reward for e in self.episodes)
-    
-    @property
-    def episode_rewards(self) -> List[float]:
-        """List of all episode rewards."""
-        return [e.total_reward for e in self.episodes]
-    
-    @property
-    def cumulative_reward_curves(self) -> List[List[float]]:
-        """Cumulative reward curves for all episodes."""
-        return [e.cumulative_rewards for e in self.episodes]
-
-# ============================================================================
-# SECTION 3: ENVIRONMENT CLIENT
-# ============================================================================
-
-class SecurityAuditEnv:
-    """Client for the Security Audit Environment."""
-    
-    def __init__(self, base_url: str):
-        self.base_url = base_url.rstrip('/')
-        self.session = requests.Session()
-        self.episode_id = None
-    
-    def reset(self, scenario_id: str = "easy") -> SecurityAuditObservation:
-        """Reset the environment."""
-        url = f"{self.base_url}/reset"
-        payload = {"scenario_id": scenario_id}
-        
-        try:
-            response = self.session.post(url, json=payload, timeout=30)
-            response.raise_for_status()
-            data = response.json()
-            self.episode_id = data.get("episode_id")
-            obs_data = data.get("observation", {})
-            return SecurityAuditObservation(obs_data)
-        except Exception as e:
-            print(f"Error resetting environment: {e}")
-            raise
-    
-    def step(self, action: SecurityAuditAction) -> Tuple[SecurityAuditObservation, float, bool]:
-        """Execute one step."""
-        url = f"{self.base_url}/step"
-        payload = action.to_dict()
-        
-        try:
-            response = self.session.post(url, json=payload, timeout=30)
-            response.raise_for_status()
-            data = response.json()
-            obs_data = data.get("observation", {})
-            observation = SecurityAuditObservation(obs_data)
-            reward = data.get("reward", 0.0)
-            done = data.get("done", False)
-            return observation, reward, done
-        except Exception as e:
-            print(f"Error stepping environment: {e}")
-            raise
-
-# ============================================================================
-# SECTION 4: LLM AGENT
-# ============================================================================
-
-class LLMAgent:
-    """Greedy LLM agent using OpenAI client."""
-    
-    def __init__(self, model_name: str = "gpt-4o-mini"):
-        self.model_name = model_name
-        self.api_key = os.environ.get("OPENAI_API_KEY", "")
-        
-        if not self.api_key:
-            print("Warning: OPENAI_API_KEY not set. LLM agent may not work.")
-    
-    def encode_observation(self, obs: SecurityAuditObservation) -> str:
-        """Convert observation to text for model input."""
-        parts = [
-            f"Current Phase: {obs.current_phase}",
-            f"Discovered Hosts: {', '.join(obs.discovered_hosts) if obs.discovered_hosts else 'None'}",
-            f"Discovered Services: {json.dumps(obs.discovered_services) if obs.discovered_services else 'None'}",
-            f"Findings Submitted: {obs.findings_submitted}",
-            f"Steps Remaining: {obs.steps_remaining}",
-            f"Last Message: {obs.message[:100]}",
-        ]
-        return "\n".join(parts)
-    
-    def generate_action(self, obs: SecurityAuditObservation) -> SecurityAuditAction:
-        """Generate action from observation using LLM."""
-        obs_text = self.encode_observation(obs)
-        
-        prompt = f"""You are a security auditor analyzing a target network.
-
-Current Observation:
-{obs_text}
-
-Available Actions:
-1. list_tools - Get available security tools
-2. use_tool - Execute a security tool (requires tool_name and arguments)
-3. submit_finding - Report a vulnerability
-4. generate_report - End the audit
-
-Based on the current phase and discovered information, choose the BEST next action.
-Respond with ONLY a valid JSON object, no other text:
-
-{{
-  "action_type": "list_tools" | "use_tool" | "submit_finding" | "generate_report",
-  "tool_name": "network_scan" | "service_fingerprint" | "web_crawl" | "vulnerability_scan" | null,
-  "arguments": {{"host": "192.168.1.1"}} or {{"title": "...", "host": "...", "severity": "..."}} or {{}}
-}}
-
-Choose wisely to maximize vulnerabilities found and minimize wasted steps."""
-        
-        try:
-            # Use OpenAI client pattern
-            import openai
-            
-            client = openai.OpenAI(api_key=self.api_key)
-            response = client.chat.completions.create(
-                model=self.model_name,
-                messages=[
-                    {"role": "system", "content": "You are a security auditor. Respond with only valid JSON."},
-                    {"role": "user", "content": prompt}
-                ],
-                temperature=0.7,
-                max_tokens=200,
-            )
-            
-            response_text = response.choices[0].message.content
-            
-            # Parse JSON from response
-            try:
-                json_start = response_text.find('{')
-                json_end = response_text.rfind('}') + 1
-                if json_start >= 0 and json_end > json_start:
-                    action_json = json.loads(response_text[json_start:json_end])
-                    return SecurityAuditAction(
-                        action_type=action_json.get("action_type", "list_tools"),
-                        tool_name=action_json.get("tool_name"),
-                        arguments=action_json.get("arguments", {})
-                    )
-            except json.JSONDecodeError:
-                pass
-        
-        except Exception as e:
-            print(f"LLM error: {e}")
-        
-        # Fallback to list_tools
-        return SecurityAuditAction(action_type="list_tools")
-
-# ============================================================================
-# SECTION 5: RANDOM AGENT
-# ============================================================================
-
-class RandomAgent:
-    """Random agent that picks random valid actions."""
-    
-    def generate_action(self, obs: SecurityAuditObservation) -> SecurityAuditAction:
-        """Generate random action."""
-        action_types = ["list_tools", "use_tool", "submit_finding"]
-        action_type = random.choice(action_types)
-        
-        if action_type == "use_tool":
-            tools = ["network_scan", "service_fingerprint", "web_crawl", "vulnerability_scan"]
-            return SecurityAuditAction(
-                action_type="use_tool",
-                tool_name=random.choice(tools),
-                arguments={"host": f"192.168.1.{random.randint(1, 10)}"}
-            )
-        elif action_type == "submit_finding":
-            return SecurityAuditAction(
-                action_type="submit_finding",
-                arguments={
-                    "title": f"Finding {random.randint(1, 100)}",
-                    "host": f"192.168.1.{random.randint(1, 10)}",
-                    "severity": random.choice(["low", "medium", "high"]),
-                    "cvss_score": round(random.uniform(3.0, 9.8), 1),
-                    "cwe": f"CWE-{random.randint(1, 1000)}",
-                    "owasp": random.choice(["A01:2021", "A02:2021", "A03:2021"])
-                }
-            )
-        else:
-            return SecurityAuditAction(action_type="list_tools")
-
-# ============================================================================
-# SECTION 6: EPISODE RUNNER
-# ============================================================================
-
-def run_episode(agent, env: SecurityAuditEnv, agent_name: str, 
-                episode_num: int, max_steps: int) -> EpisodeMetrics:
-    """Run a single episode with an agent."""
-    print(f"  {agent_name} Episode {episode_num + 1}/10", end=" ", flush=True)
-    
-    obs = env.reset(scenario_id="easy")
-    metrics = EpisodeMetrics(episode_num=episode_num + 1)
-    
-    step = 0
-    while not obs.done and step < max_steps:
-        # Generate action
-        action = agent.generate_action(obs)
-        
-        # Step environment
-        obs, reward, done = env.step(action)
-        
-        # Track metrics
-        metrics.total_reward += reward
-        metrics.cumulative_rewards.append(metrics.total_reward)
-        metrics.actions_taken.append(action.action_type)
-        metrics.steps_taken += 1
-        
-        # Count vulnerabilities (from findings_submitted)
-        metrics.vulnerabilities_found = obs.findings_submitted
-        
-        step += 1
-    
-    print(f"→ Reward: {metrics.total_reward:.4f}, Vulns: {metrics.vulnerabilities_found}")
-    return metrics
-
-# ============================================================================
-# SECTION 7: PLOTTING
-# ============================================================================
-
-def plot_episode_rewards(random_metrics: AgentMetrics, llm_metrics: AgentMetrics, 
-                        output_path: str, dpi: int = 150):
-    """Plot 1: Episode rewards comparison."""
-    plt.figure(figsize=(10, 6))
-    
-    episodes = list(range(1, len(random_metrics.episodes) + 1))
-    random_rewards = random_metrics.episode_rewards
-    llm_rewards = llm_metrics.episode_rewards
-    
-    plt.plot(episodes, random_rewards, 'r--', marker='o', linewidth=2, 
-             markersize=8, label='Random Agent', alpha=0.8)
-    plt.plot(episodes, llm_rewards, 'b-', marker='s', linewidth=2, 
-             markersize=8, label='LLM Agent', alpha=0.8)
-    
-    plt.xlabel('Episode', fontsize=12, fontweight='bold')
-    plt.ylabel('Total Reward (0.0 - 1.0)', fontsize=12, fontweight='bold')
-    plt.title('VAPT-Env: Episode Reward — Random vs LLM Agent', fontsize=14, fontweight='bold')
-    plt.legend(fontsize=11, loc='best')
-    plt.grid(True, alpha=0.3)
-    plt.xticks(episodes)
-    plt.ylim(0, max(max(random_rewards), max(llm_rewards)) * 1.1)
-    
-    plt.tight_layout()
-    plt.savefig(output_path, dpi=dpi, bbox_inches='tight')
-    print(f"✓ Saved {output_path}")
-    plt.close()
-
-def plot_cumulative_rewards(random_metrics: AgentMetrics, llm_metrics: AgentMetrics,
-                           output_path: str, dpi: int = 150):
-    """Plot 2: Cumulative reward curves with std dev bands."""
-    plt.figure(figsize=(12, 6))
-    
-    # Get cumulative reward curves
-    random_curves = random_metrics.cumulative_reward_curves
-    llm_curves = llm_metrics.cumulative_reward_curves
-    
-    # Pad curves to same length
-    max_len = max(max(len(c) for c in random_curves), max(len(c) for c in llm_curves))
-    
-    random_curves_padded = []
-    for curve in random_curves:
-        padded = list(curve) + [curve[-1]] * (max_len - len(curve))
-        random_curves_padded.append(padded)
-    
-    llm_curves_padded = []
-    for curve in llm_curves:
-        padded = list(curve) + [curve[-1]] * (max_len - len(curve))
-        llm_curves_padded.append(padded)
-    
-    # Compute mean and std
-    random_mean = np.mean(random_curves_padded, axis=0)
-    random_std = np.std(random_curves_padded, axis=0)
-    
-    llm_mean = np.mean(llm_curves_padded, axis=0)
-    llm_std = np.std(llm_curves_padded, axis=0)
-    
-    steps = np.arange(len(random_mean))
-    
-    # Plot with shaded std dev bands
-    plt.plot(steps, random_mean, 'r--', linewidth=2.5, label='Random Agent', alpha=0.8)
-    plt.fill_between(steps, random_mean - random_std, random_mean + random_std, 
-                     color='red', alpha=0.15)
-    
-    plt.plot(steps, llm_mean, 'b-', linewidth=2.5, label='LLM Agent', alpha=0.8)
-    plt.fill_between(steps, llm_mean - llm_std, llm_mean + llm_std, 
-                     color='blue', alpha=0.15)
-    
-    plt.xlabel('Step', fontsize=12, fontweight='bold')
-    plt.ylabel('Cumulative Reward', fontsize=12, fontweight='bold')
-    plt.title('VAPT-Env: Cumulative Reward Over Steps', fontsize=14, fontweight='bold')
-    plt.legend(fontsize=11, loc='best')
-    plt.grid(True, alpha=0.3)
-    
-    plt.tight_layout()
-    plt.savefig(output_path, dpi=dpi, bbox_inches='tight')
-    print(f"✓ Saved {output_path}")
-    plt.close()
-
-def plot_vulnerabilities_found(random_metrics: AgentMetrics, llm_metrics: AgentMetrics,
-                              output_path: str, dpi: int = 150):
-    """Plot 3: Vulnerability detection rate."""
-    plt.figure(figsize=(10, 6))
-    
-    episodes = list(range(1, len(random_metrics.episodes) + 1))
-    random_vulns = [e.vulnerabilities_found for e in random_metrics.episodes]
-    llm_vulns = [e.vulnerabilities_found for e in llm_metrics.episodes]
-    
-    x = np.arange(len(episodes))
-    width = 0.35
-    
-    bars1 = plt.bar(x - width/2, random_vulns, width, label='Random Agent', 
-                    color='#ff7f0e', alpha=0.8, edgecolor='black', linewidth=1.5)
-    bars2 = plt.bar(x + width/2, llm_vulns, width, label='LLM Agent', 
-                    color='#2ca02c', alpha=0.8, edgecolor='black', linewidth=1.5)
-    
-    # Add value labels on bars
-    for bars in [bars1, bars2]:
-        for bar in bars:
-            height = bar.get_height()
-            plt.text(bar.get_x() + bar.get_width()/2., height,
-                    f'{int(height)}',
-                    ha='center', va='bottom', fontsize=9, fontweight='bold')
-    
-    plt.xlabel('Episode', fontsize=12, fontweight='bold')
-    plt.ylabel('Vulnerabilities Found / Total (3)', fontsize=12, fontweight='bold')
-    plt.title('VAPT-Env: Vulnerability Detection Rate', fontsize=14, fontweight='bold')
-    plt.xticks(x, episodes)
-    plt.ylim(0, 3.5)
-    plt.legend(fontsize=11, loc='best')
-    plt.grid(True, alpha=0.3, axis='y')
-    
-    plt.tight_layout()
-    plt.savefig(output_path, dpi=dpi, bbox_inches='tight')
-    print(f"✓ Saved {output_path}")
-    plt.close()
-
-# ============================================================================
-# SECTION 8: SUMMARY TABLE
-# ============================================================================
-
-def print_summary_table(random_metrics: AgentMetrics, llm_metrics: AgentMetrics):
-    """Print summary table."""
-    print("\n" + "=" * 70)
-    print("VAPT-Env AGENT COMPARISON SUMMARY")
-    print("=" * 70)
-    
-    print(f"\n{'Metric':<25} {'Random Agent':<20} {'LLM Agent':<20}")
-    print("-" * 70)
-    
-    # Average episode reward
-    print(f"{'Avg Episode Reward':<25} {random_metrics.avg_episode_reward:<20.4f} {llm_metrics.avg_episode_reward:<20.4f}")
-    
-    # Average vulnerabilities found
-    print(f"{'Avg Vulns Found':<25} {random_metrics.avg_vulns_found:<20.1f} / 3 {llm_metrics.avg_vulns_found:<20.1f} / 3")
-    
-    # Best episode score
-    print(f"{'Best Episode Score':<25} {random_metrics.best_episode_reward:<20.4f} {llm_metrics.best_episode_reward:<20.4f}")
-    
-    # Std dev of rewards
-    random_std = np.std(random_metrics.episode_rewards)
-    llm_std = np.std(llm_metrics.episode_rewards)
-    print(f"{'Reward Std Dev':<25} {random_std:<20.4f} {llm_std:<20.4f}")
-    
-    # Min episode score
-    random_min = min(random_metrics.episode_rewards)
-    llm_min = min(llm_metrics.episode_rewards)
-    print(f"{'Min Episode Score':<25} {random_min:<20.4f} {llm_min:<20.4f}")
-    
-    # Improvement
-    improvement = ((llm_metrics.avg_episode_reward - random_metrics.avg_episode_reward) / 
-                   abs(random_metrics.avg_episode_reward)) * 100 if random_metrics.avg_episode_reward != 0 else 0
-    print(f"{'LLM Improvement':<25} {improvement:>19.1f}%")
-    
-    print("=" * 70 + "\n")
-
-# ============================================================================
-# SECTION 9: MAIN
-# ============================================================================
-
-def main():
-    """Main execution."""
-    print("\n" + "=" * 70)
-    print("VAPT-Env AGENT COMPARISON")
-    print("=" * 70)
-    
-    # Setup
-    config = Config()
-    print(f"\nConfiguration:")
-    print(f"  Environment: {config.api_base_url}")
-    print(f"  Scenario: {config.scenario_id}")
-    print(f"  Episodes: {config.num_episodes}")
-    print(f"  Max steps per episode: {config.max_steps_per_episode}")
-    print(f"  LLM Model: {config.model_name}")
-    print(f"  Output directory: {config.plots_dir}")
-    
-    # Initialize environment
-    print(f"\nInitializing environment...")
-    env = SecurityAuditEnv(base_url=config.api_base_url)
-    
-    # Test connection
-    try:
-        obs = env.reset(scenario_id=config.scenario_id)
-        print(f"✓ Environment connected")
-    except Exception as e:
-        print(f"✗ Failed to connect to environment: {e}")
-        return
-    
-    # Initialize agents
-    print(f"\nInitializing agents...")
-    random_agent = RandomAgent()
-    llm_agent = LLMAgent(model_name=config.model_name)
-    print(f"✓ Agents initialized")
-    
-    # Run random agent
-    print(f"\n{'=' * 70}")
-    print("RANDOM AGENT - Running 10 episodes")
-    print("=" * 70)
-    random_metrics = AgentMetrics(agent_name="Random Agent")
-    for episode in range(config.num_episodes):
-        metrics = run_episode(random_agent, env, "Random", episode, config.max_steps_per_episode)
-        random_metrics.episodes.append(metrics)
-    
-    # Run LLM agent
-    print(f"\n{'=' * 70}")
-    print("LLM AGENT - Running 10 episodes")
-    print("=" * 70)
-    llm_metrics = AgentMetrics(agent_name="LLM Agent")
-    for episode in range(config.num_episodes):
-        metrics = run_episode(llm_agent, env, "LLM", episode, config.max_steps_per_episode)
-        llm_metrics.episodes.append(metrics)
-    
-    # Generate plots
-    print(f"\n{'=' * 70}")
-    print("GENERATING PLOTS")
-    print("=" * 70)
-    
-    plot_episode_rewards(random_metrics, llm_metrics, 
-                        f"{config.plots_dir}/reward_per_episode.png", 
-                        dpi=config.dpi)
-    
-    plot_cumulative_rewards(random_metrics, llm_metrics,
-                           f"{config.plots_dir}/cumulative_reward_curve.png",
-                           dpi=config.dpi)
-    
-    plot_vulnerabilities_found(random_metrics, llm_metrics,
-                              f"{config.plots_dir}/vulns_found.png",
-                              dpi=config.dpi)
-    
-    # Print summary
-    print_summary_table(random_metrics, llm_metrics)
-    
-    print(f"✓ All plots saved to {config.plots_dir}/")
-    print(f"✓ Comparison complete!")
-
-if __name__ == "__main__":
-    main()

scripts/push_model_card.py

@@ -87,7 +87,7 @@ ENV_URL="https://Sayuj63-Vapt-env.hf.space" python inference.py
 
 ## Eval methodology
 
-The eval uses an evaluation harness ([`colab_eval_v3.py`](https://github.com/Sayuj63/vapt-env/blob/main/colab_eval_v3.py)) layered on top of the trained adapter:
+The eval uses an evaluation harness ([`colab_eval_v3.py`](https://github.com/Sayuj63/vapt-env/blob/main/scripts/colab_eval_v3.py)) layered on top of the trained adapter:
 
 - 3-step scripted recon prefix (network_scan → web_crawl → test_injection on `/api/login`)
 - Anti-collapse safety net (rotates through other endpoints when the trained policy emits `list_tools` ≥ 2× in a row)