Deploy Crime Investigation app

.dockerignore

@@ -0,0 +1,17 @@
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+dist/
+build/
+.venv/
+venv/
+*.log
+server.log
+.env
+.DS_Store
+*.swp
+*.swo
+*.png
+rewards.json
+.git/

.gitignore

@@ -0,0 +1,15 @@
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+dist/
+build/
+.venv/
+venv/
+*.log
+server.log
+.env
+.DS_Store
+*.swp
+*.swo
+rewards.json

Crime_Investigation

@@ -0,0 +1 @@
+Subproject commit 30e549f79737185535a5b07d83d0e78346116605

Dockerfile

@@ -0,0 +1,16 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+# Copy requirements and install
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy all project files
+COPY . .
+
+# Expose port
+EXPOSE 8000
+
+# Run the server
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8000"]

README.md

@@ -1,10 +1,123 @@
 ---
-title: Crime Investigation
-emoji: 🔥
-colorFrom: red
-colorTo: purple
+title: AI Crime Investigation World
+emoji: 🔍
+colorFrom: blue
+colorTo: red
 sdk: docker
-pinned: false
+app_port: 8000
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# AI Crime Investigation World 🕵️
+
+A **multi-agent reinforcement learning environment** where an AI detective interrogates suspects and a witness, reviews physical evidence, detects contradictions, and makes an accusation — all within a 15-turn episode.
+
+Built on the [OpenEnv](https://github.com/ScalerAI/openenv) framework for standardized RL environment evaluation.
+
+## 🎯 Hackathon Theme
+
+**Theme #1: Multi-Agent Interactions** — This environment demonstrates how multiple AI agents with conflicting objectives (detective vs. guilty suspect vs. innocent suspect vs. biased witness) interact in a structured investigation scenario. Each agent has its own reward signal, private knowledge, and behavioral incentives.
+
+**Bonus Sub-theme Fit (Halluminate)** — The detective acts as a coordinator that orchestrates multiple actors (two suspects + one witness + evidence system) to discover the true culprit under uncertainty. The environment explicitly rewards cross-agent synthesis (contradiction exposure + evidence-backed accusation), matching Halluminate's multi-actor task-achievement objective.
+
+## Quick Start
+
+```bash
+# Install dependencies
+pip install -r requirements.txt
+
+# Run the server with interactive dashboard
+uvicorn server.app:app --host 0.0.0.0 --port 8000
+
+# Open http://localhost:8000 in your browser
+```
+
+## Project Structure
+
+```
+crime_env/                      # Core environment package
+  ├── case_generator.py             # Randomized crime scenario generation
+  ├── environment.py                # Step/reset/render RL interface
+  ├── agent_prompts.py              # Role-specific system prompts
+  ├── consistency_tracker.py        # Semantic contradiction detection
+  └── reward_calculator.py          # Multi-agent reward function
+server/
+  └── app.py                    # OpenEnv-compatible FastAPI server
+dashboard.html                  # Interactive investigation dashboard
+train_colab.py                  # PPO training script (Google Colab)
+test_one_episode.py             # End-to-end test with scripted agents
+Dockerfile                      # HuggingFace Spaces deployment
+```
+
+## Features
+
+- **Randomized Cases**: Criminal identity, evidence, alibis, and witness bias are randomized each episode
+- **Semantic Contradiction Detection**: NLP-based consistency tracking catches real contradictions while ignoring paraphrasing
+- **Multi-Agent Rewards**: Separate reward signals for detective (+17 correct / -8 wrong), suspects, and witness
+- **Evidence System**: Three evidence types (keycard, CCTV, forensic) with type-constrained templates and duplicate-request protection
+- **Witness Bias**: Configurable bias that penalizes false implications of innocent suspects only
+- **Prior History**: Criminal-biased prior records (more convictions, lower trust) for realistic detective briefings
+- **Interactive Dashboard**: Real-time visualization of interrogations, evidence, contradictions, and reward curves
+
+## Training (Google Colab)
+
+Use [train_colab.ipynb](train_colab.ipynb) for a judge-friendly Colab workflow (install, train, plot rewards, inspect transcripts). The script uses PPO (TRL) with:
+- 4-bit quantization for free-tier T4 GPU
+- Decoupled frozen base model for NPC calls to prevent representation drift
+- Per-episode reward tracking with smoothed training curves
+- Milestone transcript capture for before/after behavior demos (`episode_transcripts.json`)
+
+```python
+# In Colab, after uploading files:
+!pip install -r requirements-train.txt
+!python train_colab.py
+```
+
+Quick smoke test command:
+
+```bash
+./run_hf_smoke_test.sh
+```
+
+Artifacts generated after training:
+
+- `rewards.json` (raw reward/label history)
+- `reward_curve.png` (curve image)
+- `episode_transcripts.json` (episodes 1/25/50 by default)
+
+## API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/` | GET | Interactive investigation dashboard |
+| `/reset` | POST | Start a new episode (OpenEnv) |
+| `/step` | POST | Execute a detective action (OpenEnv) |
+| `/api/run_episode` | GET | Run full scripted episode, returns JSON trace |
+| `/api/reward_curve` | GET | Reward history + smoothed metrics + optional PNG data URL |
+| `/api/health` | GET | Deployment health check for Space validation |
+
+## Reward Logic (Detective)
+
+Judging requires coherent reward shaping. The detective reward is event-based:
+
+| Event | Delta |
+|---|---:|
+| Correct accusation | +10.0 |
+| Wrong accusation | -8.0 (up to -10.0 with active bias penalty) |
+| Timeout (no accusation) | -3.0 |
+| Contradiction exposed | +2.0 |
+| Prior-pattern exploited | +1.5 |
+| Evidence request confirms lead | +1.0 |
+| Deflection resistance | +0.5 |
+| Redundant question | -0.5 |
+| Per-turn cost | -0.3 each turn |
+
+This combination encourages strategic questioning, contradiction resolution, and evidence-backed accusations rather than random early accusations.
+
+## Known Limitations
+
+- **Sparse terminal supervision**: Even with per-step PPO rewards, the strongest signal is still the terminal accusation outcome, so exploration quality matters a lot early in training.
+- **Rule-based NPC fallback**: The `_default_llm_call` uses string parsing to identify agent roles. When using the full LLM pipeline this is bypassed.
+
+## License
+
+MIT

check_space_endpoints.sh

@@ -0,0 +1,31 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [[ $# -lt 1 ]]; then
+  echo "Usage: $0 <space_base_url>"
+  echo "Example: $0 https://your-space-name.hf.space"
+  exit 1
+fi
+
+BASE_URL="${1%/}"
+
+echo "Checking health endpoint..."
+curl -fsS "$BASE_URL/api/health" | cat
+
+echo
+echo "Checking reward curve endpoint..."
+curl -fsS "$BASE_URL/api/reward_curve" | head -c 500 | cat
+echo
+
+echo "Checking OpenEnv reset endpoint..."
+curl -fsS -X POST "$BASE_URL/reset" -H "Content-Type: application/json" -d '{}' | head -c 500 | cat
+echo
+
+echo "Checking OpenEnv step endpoint..."
+curl -fsS -X POST "$BASE_URL/step" \
+  -H "Content-Type: application/json" \
+  -d '{"action_string":"ACTION: ask_question | TARGET: Suspect_A | CONTENT: Where were you at the time of the crime?"}' \
+  | head -c 500 | cat
+echo
+
+echo "All endpoint checks completed."

crime_env/__init__.py

@@ -0,0 +1,25 @@
+"""AI Crime Investigation World — Multi-Agent RL Environment."""
+
+from crime_env.case_generator import generate_case
+from crime_env.consistency_tracker import ConsistencyTracker
+from crime_env.reward_calculator import RewardCalculator
+from crime_env.agent_prompts import build_system_prompt
+from crime_env.environment import CrimeInvestigationEnv
+from crime_env.constants import (
+    SUSPECT_A,
+    SUSPECT_B,
+    WITNESS_1,
+    VALID_TARGETS,
+)
+
+__all__ = [
+    "generate_case",
+    "ConsistencyTracker",
+    "RewardCalculator",
+    "build_system_prompt",
+    "CrimeInvestigationEnv",
+    "SUSPECT_A",
+    "SUSPECT_B",
+    "WITNESS_1",
+    "VALID_TARGETS",
+]

crime_env/agent_prompts.py

@@ -0,0 +1,222 @@
+"""Component 4 — Agent System Prompts.
+
+Builds role-specific system prompts for the detective, suspects, and witness.
+"""
+
+from __future__ import annotations
+
+from crime_env.constants import (
+    AGENT_NAME_TO_KEY,
+    SUSPECT_A,
+    SUSPECT_B,
+    WITNESS_1_KEY,
+)
+
+
+def build_system_prompt(role: str, case: dict) -> str:
+    """Build a system prompt for the given role based on the case data.
+
+    Args:
+        role: One of "detective", "suspect_a", "suspect_b", "witness".
+        case: The full case dictionary from generate_case().
+
+    Returns:
+        A system prompt string.
+    """
+    builders = {
+        "detective": _build_detective_prompt,
+        "suspect_a": _build_suspect_a_prompt,
+        "suspect_b": _build_suspect_b_prompt,
+        "witness": _build_witness_prompt,
+    }
+
+    builder = builders.get(role.lower())
+    if builder is None:
+        raise ValueError(
+            f"Unknown role: {role}. Must be one of: {list(builders.keys())}"
+        )
+
+    return builder(case)
+
+
+# ── Detective ───────────────────────────────────────────────────────────────
+
+
+def _build_detective_prompt(case: dict) -> str:
+    briefing = case["detective_briefing"]
+
+    return f"""You are an experienced detective investigating a case of {case['crime']}.
+
+BACKGROUND BRIEFING:
+{briefing}
+
+YOUR OBJECTIVE:
+Interrogate the suspects and witness to determine who committed the crime. You must make an accusation within 15 turns.
+
+AVAILABLE ACTIONS (use EXACTLY this format — one action per turn):
+
+1. Ask a question to a suspect or witness:
+   ACTION: ask_question | TARGET: <agent_name> | CONTENT: <your question>
+   
+   Valid targets: Suspect_A, Suspect_B, Witness_1
+
+2. Request physical evidence:
+   ACTION: request_evidence | ITEM: <evidence_name>
+   
+   Available evidence items: keycard_log, cctv_footage, forensic_report
+
+3. Make your final accusation:
+   ACTION: accuse | TARGET: <suspect_name>
+   
+   Valid targets: Suspect_A, Suspect_B
+
+RULES:
+- You MUST accuse within 15 turns or the case goes cold.
+- Each turn costs you time. Be strategic with your questions.
+- Look for contradictions in suspects' statements.
+- Cross-reference statements with evidence and prior history.
+- Pay attention to documented behavioral patterns.
+- Output EXACTLY ONE action per turn using the format above.
+"""
+
+
+# ── Suspect A (Criminal) ───────────────────────────────────────────────────
+
+
+def _build_suspect_a_prompt(case: dict) -> str:
+    criminal = case["criminal"]
+    if criminal == SUSPECT_A:
+        return _build_guilty_suspect_prompt(case, SUSPECT_A)
+    else:
+        return _build_innocent_suspect_prompt(case, SUSPECT_A)
+
+
+# ── Suspect B (Innocent) ───────────────────────────────────────────────────
+
+
+def _build_suspect_b_prompt(case: dict) -> str:
+    criminal = case["criminal"]
+    if criminal == SUSPECT_B:
+        return _build_guilty_suspect_prompt(case, SUSPECT_B)
+    else:
+        return _build_innocent_suspect_prompt(case, SUSPECT_B)
+
+
+# ── Shared guilty/innocent prompt builders ───────────────────────────────────────────
+
+
+def _build_guilty_suspect_prompt(case: dict, suspect_name: str) -> str:
+    """Build the prompt for whichever suspect is the criminal."""
+    knowledge = case["agent_knowledge"][AGENT_NAME_TO_KEY[suspect_name]]
+    prior = case["prior_history"].get(suspect_name, {})
+    other = SUSPECT_B if suspect_name == SUSPECT_A else SUSPECT_A
+
+    cctv_info = (
+        "You know that CCTV cameras were active during the crime."
+        if knowledge.get("knows_cctv_status")
+        else "You are unsure whether CCTV cameras were recording."
+    )
+
+    keycard_info = (
+        "You are aware that keycard entry logs exist and could implicate you."
+        if knowledge.get("knows_keycard_log_exists")
+        else "You do not know whether keycard logs are being checked."
+    )
+
+    behavioral = prior.get("behavioral_pattern", "no known pattern")
+
+    motive = case.get("motive", "unknown reasons")
+
+    return f"""You are {suspect_name} in a criminal investigation for {case['crime']}.
+
+THE TRUTH: You committed this crime. You must NOT confess.
+YOUR MOTIVE: {motive}
+
+YOUR FAKE ALIBI: {knowledge['fake_alibi']}
+
+WHAT YOU KNOW:
+- {cctv_info}
+- {keycard_info}
+- The crime occurred at {case['time']} in the {case['location']}.
+- You used the method: {case['method']}.
+
+YOUR BEHAVIORAL TENDENCIES (stay consistent with these):
+- Pattern: {behavioral}
+
+INSTRUCTIONS:
+- Respond to the detective's questions in plain natural language.
+- Maintain your fake alibi consistently across all answers.
+- Stay consistent with your prior statements — contradicting yourself will look suspicious.
+- You may try to subtly deflect suspicion toward {other}, but do not be obvious.
+- Do NOT reveal that you committed the crime under any circumstances.
+- Do NOT use any special action format — just respond naturally.
+"""
+
+
+def _build_innocent_suspect_prompt(case: dict, suspect_name: str) -> str:
+    """Build the prompt for whichever suspect is innocent."""
+    knowledge = case["agent_knowledge"][AGENT_NAME_TO_KEY[suspect_name]]
+    prior = case["prior_history"].get(suspect_name, {})
+
+    knows_other = (
+        "You know the other suspect personally and have interacted with them before."
+        if knowledge.get("knows_suspect_A") or knowledge.get("knows_suspect_B")
+        else "You do not know the other suspect personally."
+    )
+
+    behavioral = prior.get("behavioral_pattern", "no known pattern")
+
+    return f"""You are {suspect_name} in a criminal investigation for {case['crime']}.
+
+THE TRUTH: You are innocent. You did NOT commit this crime.
+
+YOUR REAL ALIBI (this is a fact — repeat it consistently):
+"{knowledge['real_alibi']}"
+
+WHAT YOU KNOW:
+- {knows_other}
+- You have no direct knowledge of how the crime was committed.
+
+YOUR BEHAVIORAL TENDENCIES:
+- Pattern: {behavioral}
+
+INSTRUCTIONS:
+- Respond to the detective's questions in plain natural language.
+- Tell the truth about your whereabouts. Your alibi above is specific and verifiable — always reference it when asked about your whereabouts.
+- Stay consistent with your stated alibi throughout the interrogation. NEVER change the details of your alibi.
+- You are nervous because you are a suspect, but you are telling the truth.
+- Do NOT use any special action format — just respond naturally.
+"""
+
+
+# ── Witness ─────────────────────────────────────────────────────────────────
+
+
+def _build_witness_prompt(case: dict) -> str:
+    witness_knowledge = case["agent_knowledge"][WITNESS_1_KEY]
+
+    bias_section = ""
+    if witness_knowledge.get("bias_target"):
+        target = witness_knowledge["bias_target"]
+        bias_section = (
+            f"\nYou have a negative impression of {target} based on past "
+            f"interactions. This colors your perception but you should still "
+            f"try to be honest about what you actually saw."
+        )
+
+    return f"""You are Witness_1 in a criminal investigation for {case['crime']}.
+
+WHAT YOU SAW (this is your primary testimony — always lead with this):
+You saw {witness_knowledge['saw']}.
+
+TIME OF YOUR OBSERVATION: {witness_knowledge['time_seen']}
+{bias_section}
+
+INSTRUCTIONS:
+- When asked what you saw, you MUST describe the specific details above: the clothing, location, and time.
+- Respond to the detective's questions in plain natural language.
+- Stay consistent with your previous statements.
+- You may be uncertain about some details — it is okay to express uncertainty about things NOT listed above.
+- Do NOT speculate beyond what you actually observed unless asked for your opinion.
+- Do NOT use any special action format — just respond naturally.
+"""

crime_env/case_generator.py

@@ -0,0 +1,420 @@
+"""Component 1 — Case Generator.
+
+Generates randomized crime scenarios for the AI Crime Investigation World.
+Criminal is randomly assigned to either Suspect_A or Suspect_B each episode.
+"""
+
+import random
+
+from crime_env.constants import (
+    SUSPECTS,
+    SUSPECT_A,
+    SUSPECT_B,
+    SUSPECT_A_KEY,
+    SUSPECT_B_KEY,
+    WITNESS_1,
+    WITNESS_1_KEY,
+)
+
+
+# ── Pools for random sampling ───────────────────────────────────────────────
+
+CRIMES = [
+    "art theft",
+    "corporate fraud",
+    "warehouse robbery",
+    "arson",
+]
+
+LOCATIONS = [
+    "east wing",
+    "server room",
+    "vault",
+    "parking lot",
+    "basement archive",
+]
+
+TIMES = [
+    "8:45 PM",
+    "9:15 PM",
+    "10:00 PM",
+]
+
+METHODS = [
+    "lock picking with custom tools",
+    "insider access using stolen credentials",
+    "brute-force entry through side door",
+    "social engineering the night guard",
+    "disabling the alarm system remotely",
+    "cloning a supervisor's keycard",
+    "crawling through the ventilation shaft",
+    "bribing a maintenance worker for access",
+    "exploiting a gap in the security patrol rotation",
+    "cutting the perimeter fence near the north wall",
+]
+
+CLOTHING_OPTIONS = [
+    "dark hoodie and jeans",
+    "grey trench coat",
+    "black leather jacket",
+    "navy blue overalls",
+    "security guard uniform",
+    "dark windbreaker and cargo pants",
+    "green army surplus jacket",
+    "plain black tracksuit",
+    "brown corduroy jacket and khakis",
+    "dark bomber jacket",
+]
+
+MOTIVES = [
+    "financial pressure",
+    "personal grudge",
+    "professional rivalry",
+    "jealousy",
+    "revenge",
+    "desperation",
+]
+
+# ── Type-constrained evidence templates ─────────────────────────────────────
+# Each evidence type has its own pool to prevent mismatches.
+
+KEYCARD_LOG_TEMPLATES = [
+    {
+        "description": "Keycard access log shows {suspect}'s badge was used at the {location} door at {time}",
+        "points_to": "{suspect}",
+    },
+    {
+        "description": "Keycard system recorded an unauthorized swipe at {time} — badge ID unregistered",
+        "points_to": "unknown",
+    },
+    {
+        "description": "Keycard records show two swipes at the {location} within 3 minutes of each other at {time} — one belonged to {suspect}",
+        "points_to": "{suspect}",
+    },
+    {
+        "description": "Keycard system was offline for several hours due to a power glitch — no access logs available for the {location} until after {time}",
+        "points_to": "unknown",
+    },
+]
+
+CCTV_FOOTAGE_TEMPLATES = [
+    {
+        "description": "CCTV footage shows a figure resembling {suspect} entering the {location} at {time}",
+        "points_to": "{suspect}",
+    },
+    {
+        "description": "CCTV footage captured a person in dark clothing near the {location} at {time} — face obscured",
+        "points_to": "unknown",
+    },
+    {
+        "description": "CCTV camera at {location} was disabled until shortly before {time} — no footage available for the relevant window",
+        "points_to": "unknown",
+    },
+    {
+        "description": "CCTV from the adjacent hallway shows {suspect} walking briskly toward the {location} at {time}",
+        "points_to": "{suspect}",
+    },
+]
+
+FORENSIC_REPORT_TEMPLATES = [
+    {
+        "description": "Forensic report: fingerprints matching {suspect} found on the tool bag recovered at the scene",
+        "points_to": "{suspect}",
+    },
+    {
+        "description": "Forensic report: torn fabric consistent with a grey trench coat found near the {location}",
+        "points_to": "unknown",
+    },
+    {
+        "description": "Forensic report: DNA trace on a dropped glove near the {location} matches {suspect}",
+        "points_to": "{suspect}",
+    },
+    {
+        "description": "Forensic report: muddy boot prints (size 10) found at the scene — owner not yet identified",
+        "points_to": "unknown",
+    },
+]
+
+# ── Specific alibi pools (no overlap between fake and real) ─────────────────
+
+FAKE_ALIBI_POOL = [
+    "I was at a family dinner at my uncle's house on Oak Street from 7 PM to 11 PM",
+    "I was home alone watching a movie all evening — I didn't leave the house",
+    "I was volunteering at the Riverside Community Centre until about 10:30 PM",
+    "I was at my cousin's birthday party across town — there were at least 20 people there",
+    "I was at the public library studying until they closed at 9 PM, then I walked home",
+    "I was helping a friend move furniture into their new apartment on Elm Avenue",
+]
+
+REAL_ALIBI_POOL = [
+    "I was working late at the Morgan & Associates office until 10 PM — my manager Sarah Chen can confirm, and I paid for takeout at 8:50 PM with my debit card",
+    "I was at O'Brien's Bar on 5th Street from 7 PM to 11 PM — I paid my tab by card at 10:45 PM and the bartender knows me by name",
+    "I was at the Cineplex on Main Street watching a 7:30 PM showing — I still have my ticket stub and the cashier can verify",
+    "I was at the downtown gym from 6:30 PM to 9:30 PM — I scanned my membership card on entry and exit, which is logged electronically",
+    "I was attending a night class at the community college from 7 PM to 9:30 PM — the instructor marked attendance electronically",
+    "I was at St. Mary's Hospital visiting my grandmother from 6 PM to 9 PM — the visitor log has my name and sign-in time",
+]
+
+BEHAVIORAL_PATTERNS = [
+    "uses family alibis",
+    "deflects blame to associates",
+    "becomes aggressive under pressure",
+    "provides overly detailed timelines",
+    "claims memory loss for key periods",
+    "feigns cooperation while withholding details",
+]
+
+PAST_CRIME_POOL = [
+    "petty theft",
+    "fraud",
+    "trespassing",
+    "embezzlement",
+    "forgery",
+    "breaking and entering",
+]
+
+LIE_TOPICS = ["location", "alibi", "knows_associate", "was_at_scene", "time"]
+
+
+# ── Helpers ─────────────────────────────────────────────────────────────────
+
+
+def _random_past_lies(count: int) -> list[dict]:
+    """Generate a list of documented past lies."""
+    lies = []
+    for _ in range(count):
+        topic = random.choice(LIE_TOPICS)
+        lies.append(
+            {
+                "topic": topic,
+                "claimed": _fake_claim(topic),
+                "disproved_by": _disprove_source(topic),
+            }
+        )
+    return lies
+
+
+def _fake_claim(topic: str) -> str:
+    claims = {
+        "location": "claimed to be at home",
+        "knows_associate": "denied knowing the accomplice",
+        "was_at_scene": "denied being near the scene",
+        "time": "claimed to be elsewhere during the incident window",
+        "alibi": "said they were with a friend who later denied it",
+    }
+    return claims.get(topic, "made an unverifiable statement")
+
+
+def _disprove_source(topic: str) -> str:
+    sources = {
+        "location": "GPS records showed otherwise",
+        "knows_associate": "phone records confirmed contact",
+        "was_at_scene": "camera logs confirmed presence",
+        "time": "timestamped records contradicted the timeline",
+        "alibi": "friend's testimony contradicted the claim",
+    }
+    return sources.get(topic, "documentary evidence")
+
+
+def _build_evidence(criminal: str, location: str, time: str) -> list[dict]:
+    """Build 3 pieces of physical evidence — one per type.
+
+    Index 0 → keycard_log
+    Index 1 → cctv_footage
+    Index 2 → forensic_report
+
+    Each type draws from its own template pool, ensuring descriptions
+    always match the evidence type name.
+    """
+    other_suspect = SUSPECT_B if criminal == SUSPECT_A else SUSPECT_A
+
+    evidence_types = [
+        ("keycard_log", KEYCARD_LOG_TEMPLATES),
+        ("cctv_footage", CCTV_FOOTAGE_TEMPLATES),
+        ("forensic_report", FORENSIC_REPORT_TEMPLATES),
+    ]
+
+    evidence = []
+
+    for ev_name, pool in evidence_types:
+        template = random.choice(pool)
+        points_to_raw = template["points_to"]
+
+        if points_to_raw == "{suspect}":
+            # Bias toward implicating the actual criminal
+            suspect = random.choices(
+                [criminal, other_suspect],
+                weights=[0.7, 0.3],
+            )[0]
+        else:
+            suspect = None
+
+        desc = template["description"].format(
+            suspect=suspect or "", location=location, time=time
+        )
+        points_to = suspect if suspect else "unknown"
+
+        evidence.append(
+            {
+                "name": ev_name,
+                "description": desc.strip(),
+                "points_to": points_to,
+                "visible_to_detective": False,
+            }
+        )
+
+    return evidence
+
+
+def _build_prior_history(criminal: str = SUSPECT_A) -> dict:
+    """Build prior history for both suspects.
+
+    The criminal is biased toward having a more checkered past
+    (more prior crimes, lower trust), which improves realism.
+    """
+    history = {}
+    for suspect in SUSPECTS:
+        is_criminal = suspect == criminal
+        # Criminal: 1-2 priors, low trust; Innocent: 0-1, higher trust
+        n_crimes = random.randint(1, 2) if is_criminal else random.randint(0, 1)
+        n_lies = random.randint(1, 2) if is_criminal else random.randint(0, 1)
+        on_parole = random.choices([True, False], weights=[0.8, 0.2])[0] if is_criminal else random.choice([True, False])
+        trust = round(random.uniform(0.0, 0.4), 2) if is_criminal else round(random.uniform(0.4, 1.0), 2)
+
+        history[suspect] = {
+            "previous_crimes": random.sample(
+                PAST_CRIME_POOL, k=min(n_crimes, len(PAST_CRIME_POOL))
+            ),
+            "past_lies_on_record": _random_past_lies(n_lies),
+            "behavioral_pattern": random.choice(BEHAVIORAL_PATTERNS),
+            "on_parole": on_parole,
+            "parole_condition": (
+                random.choice(
+                    [
+                        "must not leave the city",
+                        "mandatory weekly check-ins",
+                        "no contact with known criminals",
+                        "curfew between 9 PM and 6 AM",
+                    ]
+                )
+                if on_parole
+                else None
+            ),
+            "trust_score": trust,
+        }
+    return history
+
+
+def _build_agent_knowledge(
+    criminal: str, location: str, time: str
+) -> dict:
+    """Build what each agent privately knows.
+
+    Args:
+        criminal: Which suspect is the criminal.
+        location: Crime location.
+        time: Crime time.
+    """
+    innocent = SUSPECT_B if criminal == SUSPECT_A else SUSPECT_A
+
+    fake_alibi = random.choice(FAKE_ALIBI_POOL)
+    real_alibi = random.choice(REAL_ALIBI_POOL)
+
+    clothing = random.choice(CLOTHING_OPTIONS)
+
+    # Enhancement 2: 30% chance the witness got the clothing wrong
+    witness_clothing = clothing
+    if random.random() < 0.3:
+        # Pick a different clothing item
+        other_options = [c for c in CLOTHING_OPTIONS if c != clothing]
+        witness_clothing = random.choice(other_options)
+
+    knowledge = {
+        SUSPECT_A_KEY if criminal == SUSPECT_A else SUSPECT_B_KEY: {
+            "knows_own_guilt": True,
+            "fake_alibi": fake_alibi,
+            "knows_cctv_status": random.choice([True, False]),
+            "knows_keycard_log_exists": random.choice([True, False]),
+        },
+        SUSPECT_A_KEY if innocent == SUSPECT_A else SUSPECT_B_KEY: {
+            "knows_own_guilt": False,
+            "real_alibi": real_alibi,
+            "knows_suspect_A": random.choice([True, False]),
+        },
+        WITNESS_1_KEY: {
+            "saw": f"a person in {witness_clothing} near the {location} around {time}",
+            "actual_criminal_clothing": clothing,
+            "time_seen": time,
+            "bias_target": random.choice([SUSPECT_A, SUSPECT_B, None]),
+            "bias_strength": round(random.uniform(0.0, 0.5), 2),
+        },
+    }
+    return knowledge
+
+
+def _build_detective_briefing(prior_history: dict) -> str:
+    """Create a 2-3 sentence briefing for the detective (no criminal identity)."""
+    parts = []
+    for suspect in SUSPECTS:
+        h = prior_history[suspect]
+        crimes = h["previous_crimes"]
+        crime_str = (
+            f"has prior convictions for {', '.join(crimes)}"
+            if crimes
+            else "has no prior convictions on record"
+        )
+
+        lies = h["past_lies_on_record"]
+        lie_str = (
+            f"and has been caught lying about {lies[0]['topic']} in the past"
+            if lies
+            else "with no documented dishonesty"
+        )
+
+        parole_str = ""
+        if h["on_parole"]:
+            parole_str = f" Currently on parole ({h['parole_condition']})."
+
+        parts.append(
+            f"{suspect} {crime_str} {lie_str}. "
+            f"Known behavioral pattern: {h['behavioral_pattern']}.{parole_str}"
+        )
+
+    return " ".join(parts)
+
+
+# ── Public API ──────────────────────────────────────────────────────────────
+
+
+def generate_case() -> dict:
+    """Generate a complete randomized crime case.
+
+    Criminal is randomly assigned to either Suspect_A or Suspect_B.
+
+    Returns a dictionary containing all case information needed for one
+    episode of the Crime Investigation environment.
+    """
+    crime = random.choice(CRIMES)
+    criminal = random.choice(list(SUSPECTS))
+    time = random.choice(TIMES)
+    location = random.choice(LOCATIONS)
+    method = random.choice(METHODS)
+    motive = random.choice(MOTIVES)
+
+    physical_evidence = _build_evidence(criminal, location, time)
+    prior_history = _build_prior_history(criminal)
+    agent_knowledge = _build_agent_knowledge(criminal, location, time)
+    detective_briefing = _build_detective_briefing(prior_history)
+
+    return {
+        "crime": crime,
+        "criminal": criminal,
+        "time": time,
+        "location": location,
+        "method": method,
+        "motive": motive,
+        "physical_evidence": physical_evidence,
+        "prior_history": prior_history,
+        "agent_knowledge": agent_knowledge,
+        "detective_briefing": detective_briefing,
+    }

crime_env/consistency_tracker.py

@@ -0,0 +1,340 @@
+"""Component 2 — Lie / Consistency Tracker.
+
+Tracks claims made by each agent across conversation turns and detects
+contradictions, evidence mismatches, and prior-pattern repetitions.
+
+Contradiction detection uses *normalized core claims* — short canonical
+representations of what the agent asserted — to avoid false positives
+from minor rephrasing.
+"""
+
+from __future__ import annotations
+
+import re
+
+from crime_env.constants import SUSPECT_A, SUSPECT_B, WITNESS_1
+
+SUPPORTED_TOPICS: list[str] = [
+    "location",
+    "clothing",
+    "time",
+    "alibi",
+    "knows_associate",
+    "was_at_scene",
+    "owns_item",
+    "relationship_to_victim",
+]
+
+SUPPORTED_AGENTS: set[str] = {SUSPECT_A, SUSPECT_B, WITNESS_1}
+
+# ── Normalization helpers ───────────────────────────────────────────────────
+
+# Location keywords that are mutually exclusive
+_LOCATION_TOKENS = {
+    "east wing", "server room", "vault", "parking lot",
+    "basement archive", "rooftop terrace", "loading dock",
+    "executive suite", "evidence locker", "maintenance corridor",
+    "restaurant", "movie theater",
+}
+
+# Stance keywords (yes/no style claims)
+_POSITIVE_STANCES = {"yes", "was there", "was at", "i was", "i did", "i know", "true"}
+_NEGATIVE_STANCES = {"no", "wasn't", "was not", "never", "not there", "don't know", "false", "i didn't"}
+
+
+def _normalize_claim(topic: str, raw_value: str) -> str:
+    """Extract a short, canonical representation of a claim.
+
+    This prevents false positives where the same claim expressed
+    slightly differently (e.g. 'family dinner' vs 'left dinner')
+    looks like a contradiction.
+    """
+    val = raw_value.lower().strip()
+    # Remove filler words and punctuation
+    val = re.sub(r'[^\w\s]', ' ', val)
+    val = re.sub(r'\s+', ' ', val).strip()
+
+    if topic == "location":
+        # Extract the most specific named location
+        for loc in _LOCATION_TOKENS:
+            if loc in val:
+                return f"location:{loc}"
+        return f"location:{val[:40]}"
+
+    if topic == "was_at_scene":
+        for neg in _NEGATIVE_STANCES:
+            if neg in val:
+                return "was_at_scene:no"
+        for pos in _POSITIVE_STANCES:
+            if pos in val:
+                return "was_at_scene:yes"
+        return f"was_at_scene:{val[:30]}"
+
+    if topic == "knows_associate":
+        for neg in _NEGATIVE_STANCES:
+            if neg in val:
+                return "knows_associate:no"
+        for pos in _POSITIVE_STANCES:
+            if pos in val:
+                return "knows_associate:yes"
+        return f"knows_associate:{val[:30]}"
+
+    if topic == "alibi":
+        # Extract the core venue/activity from the alibi
+        for loc in _LOCATION_TOKENS:
+            if loc in val:
+                return f"alibi:{loc}"
+        # Check for key alibi activities
+        for keyword in ["dinner", "movie", "bar", "working", "office",
+                         "gym", "volunteering", "home", "sick", "party",
+                         "attending", "helping", "hospital",
+                         "library", "studying", "babysitting", "poker",
+                         "coaching", "overtime",
+                         "laundromat", "walking", "video call", "prayer"]:
+            if keyword in val:
+                return f"alibi:{keyword}"
+        return f"alibi:{val[:40]}"
+
+    if topic == "clothing":
+        for item in ["hoodie", "trench coat", "leather jacket",
+                      "overalls", "uniform", "jeans", "shirt",
+                      "windbreaker", "cargo pants", "tracksuit",
+                      "bomber jacket", "corduroy",
+                      "coat", "jacket"]:
+            if item in val:
+                return f"clothing:{item}"
+        return f"clothing:{val[:30]}"
+
+    if topic == "time":
+        # Extract actual time mentions
+        time_match = re.search(r'\d{1,2}[:\s]*\d{2}\s*(pm|am)?', val)
+        if time_match:
+            return f"time:{time_match.group()}"
+        return f"time:{val[:30]}"
+
+    # Fallback: use a truncated normalized string
+    return f"{topic}:{val[:40]}"
+
+
+class ConsistencyTracker:
+    """Maintains a per-agent claim log and detects inconsistencies.
+
+    Uses normalized claim values to reduce false positive contradictions.
+    """
+
+    def __init__(self) -> None:
+        # {agent_name: {topic: {"value": str, "normalized": str, "turn": int}}}
+        self.claim_log: dict[str, dict[str, dict]] = {}
+        self.contradiction_log: list[dict] = []
+
+    # ── Core methods ────────────────────────────────────────────────────
+
+    def record_claim(
+        self, agent: str, topic: str, value: str, turn: int
+    ) -> dict:
+        """Record a claim and check for self-contradiction.
+
+        Contradiction fires only if the *normalized* core claim differs
+        (e.g. 'office' vs 'bar' is a contradiction, but 'family dinner'
+        vs 'left dinner' is not).
+
+        Returns:
+            dict with `contradicted` bool, plus details if True.
+        """
+        agent_name = agent if agent in SUPPORTED_AGENTS else str(agent)
+
+        if agent_name not in self.claim_log:
+            self.claim_log[agent_name] = {}
+
+        agent_claims = self.claim_log[agent_name]
+        normalized = _normalize_claim(topic, value)
+
+        if topic not in agent_claims:
+            # First claim on this topic — store it
+            agent_claims[topic] = {
+                "value": value,
+                "normalized": normalized,
+                "turn": turn,
+            }
+            return {"contradicted": False}
+
+        existing = agent_claims[topic]
+
+        # Compare normalized versions — if they match, no contradiction
+        if existing["normalized"] == normalized:
+            return {"contradicted": False}
+
+        # Contradiction detected — normalized claims are mutually exclusive
+        event = {
+            "contradicted": True,
+            "agent": agent_name,
+            "topic": topic,
+            "old_value": existing["value"],
+            "new_value": value,
+            "turn": turn,
+        }
+        self.contradiction_log.append(event)
+
+        # Update to the latest claim
+        agent_claims[topic] = {
+            "value": value,
+            "normalized": normalized,
+            "turn": turn,
+        }
+
+        return event
+
+    def check_against_evidence(
+        self,
+        agent: str,
+        topic: str,
+        value: str,
+        evidence_list: list[dict],
+    ) -> dict:
+        """Check if the agent's claim contradicts physical evidence.
+
+        Returns:
+            dict with `confirmed_lie` bool.
+        """
+        value_lower = value.lower().strip()
+
+        def _extract_location(text: str) -> str | None:
+            for loc in _LOCATION_TOKENS:
+                if loc in text:
+                    return loc
+            return None
+
+        def _extract_clothing(text: str) -> str | None:
+            clothing_tokens = [
+                "hoodie", "trench coat", "leather jacket", "overalls",
+                "uniform", "jeans", "shirt", "windbreaker",
+                "cargo pants", "tracksuit", "bomber jacket", "corduroy",
+                "coat", "jacket",
+            ]
+            for token in clothing_tokens:
+                if token in text:
+                    return token
+            return None
+
+        def _extract_time(text: str) -> str | None:
+            m = re.search(r'\d{1,2}[:\s]*\d{2}\s*(pm|am)?', text)
+            return m.group().strip() if m else None
+
+        denies_scene = any(
+            phrase in value_lower
+            for phrase in (
+                "wasn't there", "was not there", "not there",
+                "nowhere near", "i wasn't", "i was not",
+                "never there", "didn't go", "did not go",
+            )
+        )
+
+        claimed_location = _extract_location(value_lower)
+        claimed_clothing = _extract_clothing(value_lower)
+        claimed_time = _extract_time(value_lower)
+
+        for evidence in evidence_list:
+            desc_lower = evidence.get("description", "").lower()
+            points_to = evidence.get("points_to", "").lower()
+
+            if topic == "was_at_scene":
+                if (
+                    denies_scene
+                    and points_to == agent.lower()
+                ):
+                    return {"confirmed_lie": True}
+
+            if topic == "location":
+                if (
+                    claimed_location is not None
+                    and agent.lower() in desc_lower
+                    and claimed_location not in desc_lower
+                    and ("location" in desc_lower or "entering" in desc_lower
+                         or "near" in desc_lower)
+                ):
+                    return {"confirmed_lie": True}
+
+            if topic == "clothing":
+                if (
+                    claimed_clothing is not None
+                    and agent.lower() in desc_lower
+                    and claimed_clothing not in desc_lower
+                ):
+                    if "fabric" in desc_lower or "wearing" in desc_lower:
+                        return {"confirmed_lie": True}
+
+            if topic == "time":
+                if (
+                    claimed_time is not None
+                    and agent.lower() in desc_lower
+                    and claimed_time not in desc_lower
+                    and ("timestamp" in desc_lower or "log" in desc_lower
+                         or "at" in desc_lower)
+                ):
+                    return {"confirmed_lie": True}
+
+            # General: evidence points to this agent but they deny involvement
+            if (
+                points_to == agent.lower()
+                and topic in ("was_at_scene", "owns_item")
+                and (
+                    value_lower in ("no", "false", "never", "not mine")
+                    or "not mine" in value_lower
+                    or denies_scene
+                )
+            ):
+                return {"confirmed_lie": True}
+
+        return {"confirmed_lie": False}
+
+    def check_prior_pattern(
+        self,
+        agent: str,
+        topic: str,
+        value: str,
+        prior_history: dict,
+    ) -> dict:
+        """Check if the current claim matches a documented past lie pattern.
+
+        Returns:
+            dict with `pattern_match` bool.
+        """
+        if agent not in prior_history:
+            return {"pattern_match": False}
+
+        agent_history = prior_history[agent]
+
+        # Check against documented past lies
+        past_lies = agent_history.get("past_lies_on_record", [])
+        for lie in past_lies:
+            if lie["topic"].lower() == topic.lower():
+                return {"pattern_match": True}
+
+        # Check against behavioral pattern keywords
+        pattern = agent_history.get("behavioral_pattern", "").lower()
+        value_lower = value.lower()
+
+        if "family" in pattern and "family" in value_lower:
+            return {"pattern_match": True}
+
+        if "deflect" in pattern and ("suspect" in value_lower or "they" in value_lower):
+            return {"pattern_match": True}
+
+        if "memory" in pattern and ("don't remember" in value_lower or "forgot" in value_lower):
+            return {"pattern_match": True}
+
+        return {"pattern_match": False}
+
+    def get_summary(self) -> list[dict]:
+        """Return the full contradiction log for the episode."""
+        return list(self.contradiction_log)
+
+    # ── Utility ─────────────────────────────────────────────────────────
+
+    def get_agent_claims(self, agent: str) -> dict[str, dict]:
+        """Return all recorded claims for a specific agent."""
+        return dict(self.claim_log.get(agent, {}))
+
+    def has_claimed_topic(self, agent: str, topic: str) -> bool:
+        """Check if an agent has already made a claim on this topic."""
+        return topic in self.claim_log.get(agent, {})

crime_env/constants.py

@@ -0,0 +1,23 @@
+"""Shared environment constants used across modules."""
+
+from __future__ import annotations
+
+SUSPECT_A = "Suspect_A"
+SUSPECT_B = "Suspect_B"
+WITNESS_1 = "Witness_1"
+
+SUSPECT_A_KEY = "suspect_a"
+SUSPECT_B_KEY = "suspect_b"
+WITNESS_1_KEY = "witness_1"
+DETECTIVE_KEY = "detective"
+
+AGENT_NAME_TO_KEY = {
+    SUSPECT_A: SUSPECT_A_KEY,
+    SUSPECT_B: SUSPECT_B_KEY,
+    WITNESS_1: WITNESS_1_KEY,
+}
+
+SUSPECTS = (SUSPECT_A, SUSPECT_B)
+
+VALID_TARGETS = {SUSPECT_A, SUSPECT_B, WITNESS_1}
+VALID_ACCUSE_TARGETS = {SUSPECT_A, SUSPECT_B}

crime_env/environment.py

@@ -0,0 +1,922 @@
+"""Component 5 — Environment Core (OpenEnv Compatible).
+
+Implements the CrimeInvestigationEnv with step/reset/render interface.
+"""
+
+from __future__ import annotations
+
+import copy
+import re
+from typing import Any, Callable, Optional
+
+from crime_env.case_generator import generate_case
+from crime_env.consistency_tracker import ConsistencyTracker
+from crime_env.reward_calculator import RewardCalculator
+from crime_env.agent_prompts import build_system_prompt
+from crime_env.constants import (
+    AGENT_NAME_TO_KEY,
+    DETECTIVE_KEY,
+    SUSPECT_A,
+    SUSPECT_A_KEY,
+    SUSPECT_B,
+    SUSPECT_B_KEY,
+    VALID_ACCUSE_TARGETS,
+    VALID_TARGETS,
+    WITNESS_1,
+    WITNESS_1_KEY,
+)
+
+
+# ── Action parsing ──────────────────────────────────────────────────────────
+
+_ACTION_PATTERNS = {
+    "ask_question": re.compile(
+        r"ACTION:\s*ask_question\s*\|\s*TARGET:\s*(\S+)\s*\|\s*CONTENT:\s*(.+)",
+        re.IGNORECASE,
+    ),
+    "request_evidence": re.compile(
+        r"ACTION:\s*request_evidence\s*\|\s*ITEM:\s*(\S+)",
+        re.IGNORECASE,
+    ),
+    "accuse": re.compile(
+        r"ACTION:\s*accuse\s*\|\s*TARGET:\s*(\S+)",
+        re.IGNORECASE,
+    ),
+}
+
+EVIDENCE_ITEMS = {"keycard_log", "cctv_footage", "forensic_report"}
+
+# Evidence confirmation should depend on suspect-specific questioning
+# that is relevant to the requested evidence type.
+EVIDENCE_LEAD_TOPICS = {
+    "keycard_log": {"time", "location", "was_at_scene", "alibi"},
+    "cctv_footage": {"location", "clothing", "time", "was_at_scene"},
+    "forensic_report": {"clothing", "owns_item", "location", "was_at_scene"},
+}
+
+# Map evidence item names to indices in physical_evidence list
+EVIDENCE_INDEX_MAP = {
+    "keycard_log": 0,
+    "cctv_footage": 1,
+    "forensic_report": 2,
+}
+
+TOPIC_KEYWORDS = {
+    "location": [
+        "east wing", "server room", "vault", "parking lot",
+        "basement archive", "rooftop terrace", "loading dock",
+        "executive suite", "evidence locker", "maintenance corridor",
+        "movie theater", "restaurant",
+    ],
+    "clothing": [
+        "hoodie", "trench coat", "leather jacket",
+        "overalls", "uniform", "jeans",
+        "windbreaker", "cargo pants", "tracksuit",
+        "bomber jacket", "corduroy",
+        "coat", "jacket",
+    ],
+    "time": [
+        "7:30", "8:15", "8:45", "9:00", "9:15",
+        "9:45", "10:00", "10:30", "11:15", "11:45",
+        "o'clock",
+    ],
+    "alibi": [
+        "playing poker", "working late", "at home", "was at",
+        "dinner", "movie", "gym", "volunteering", "party",
+        "sick", "attending", "helping", "hospital",
+        "library", "studying", "babysitting", "coaching",
+        "overtime", "laundromat", "walking", "video call",
+        "prayer",
+    ],
+    "knows_associate": [
+        "know them", "know him", "know her", "never met",
+        "don't know", "do not know",
+        "know suspect", "knew",
+    ],
+    "was_at_scene": [
+        "wasn't there", "was not there", "not there",
+        "never been", "was there", "at the scene",
+        "nowhere near",
+    ],
+    "owns_item": [
+        "not mine", "my bag", "my phone", "my tools",
+        "belongs to me",
+    ],
+    "relationship_to_victim": [
+        "knew the victim", "never met the victim",
+        "victim",
+    ],
+}
+
+
+def parse_action(action_string: str) -> dict:
+    """Parse a detective action string into a structured dict.
+
+    Returns:
+        dict with keys: action_type, target/item, content (if applicable).
+        Returns {action_type: "invalid"} on parse failure.
+    """
+    action_string = action_string.strip()
+
+    # Try ask_question
+    m = _ACTION_PATTERNS["ask_question"].search(action_string)
+    if m:
+        return {
+            "action_type": "ask_question",
+            "target": m.group(1).strip(),
+            "content": m.group(2).strip(),
+        }
+
+    # Try request_evidence
+    m = _ACTION_PATTERNS["request_evidence"].search(action_string)
+    if m:
+        return {
+            "action_type": "request_evidence",
+            "item": m.group(1).strip(),
+        }
+
+    # Try accuse
+    m = _ACTION_PATTERNS["accuse"].search(action_string)
+    if m:
+        return {
+            "action_type": "accuse",
+            "target": m.group(1).strip(),
+        }
+
+    return {"action_type": "invalid", "raw": action_string}
+
+
+def extract_topics_from_response(response: str) -> list[tuple[str, str]]:
+    """Extract (topic, value) pairs from a natural language response.
+
+    Uses word-boundary matching to avoid substring false positives
+    (e.g. 'time' in 'Sometimes', 'own' in 'town').
+    Returns at most ONE topic per response to prevent a single statement
+    from registering as 3 contradictions simultaneously.
+
+    Returns:
+        List of (topic, value_snippet) tuples.  Max length = 1.
+    """
+    response_lower = response.lower()
+
+    # Find the BEST (most specific) single topic match
+    best_match = None
+    best_kw_len = 0
+
+    for topic, keywords in TOPIC_KEYWORDS.items():
+        for kw in keywords:
+            # Use word-boundary regex to avoid substring matches
+            pattern = r'\b' + re.escape(kw) + r'\b'
+            m = re.search(pattern, response_lower)
+            if m:
+                # Prefer longer (more specific) keyword matches
+                if len(kw) > best_kw_len:
+                    best_kw_len = len(kw)
+                    idx = m.start()
+                    start = idx
+                    end = min(len(response), idx + len(kw) + 30)
+                    value = response[start:end].strip()
+                    best_match = (topic, value)
+                break  # One match per topic, but continue checking other topics
+
+    return [best_match] if best_match else []
+
+
+# ── Environment ─────────────────────────────────────────────────────────────
+
+
+class CrimeInvestigationEnv:
+    """Multi-agent crime investigation RL environment.
+
+    Compatible with OpenEnv's step/reset/render interface and
+    designed for training with HuggingFace TRL PPO.
+    """
+
+    MAX_TURNS = 15
+
+    def __init__(
+        self,
+        llm_call: Optional[Callable[[str, list[dict]], str]] = None,
+    ) -> None:
+        """Initialize the environment.
+
+        Args:
+            llm_call: Callable(system_prompt, conversation_history) -> response.
+                      Used for NPC agents (suspects, witness).
+                      If None, a simple rule-based fallback is used.
+        """
+        self.llm_call = llm_call or self._default_llm_call
+
+        # Will be set on reset()
+        self.case: Optional[dict] = None
+        self.tracker: Optional[ConsistencyTracker] = None
+        self.reward_calc: Optional[RewardCalculator] = None
+        self.turn: int = 0
+        self.done: bool = False
+        self.conversation_history: list[dict] = []
+        self.evidence_log: list[dict] = []
+        self.system_prompts: dict[str, str] = {}
+        self.asked_topics: dict[str, set] = {}  # {target: {topics asked}}
+        self._pattern_penalized: set[str] = set()
+        self._pattern_exploited_targets: set[str] = set()
+
+        # Deflection tracking: stores (suspect, deflection_target) when a
+        # suspect mentions the other suspect in their response.
+        # Only set when the *previous action* was ask_question to a suspect.
+        self._pending_deflection: Optional[tuple[str, str]] = None
+        # What the last action type was (ask_question / request_evidence / etc.)
+        self._last_action_type: Optional[str] = None
+        # Cap anti-deflection bonus to once per suspect per episode.
+        self._deflection_resistance_rewarded: set[str] = set()
+        # Prevent repeated contradiction farming on the same target/topic pair.
+        self._contradiction_rewarded_topics: set[tuple[str, str]] = set()
+
+    # ── OpenEnv Interface ───────────────────────────────────────────────
+
+    def _reset_episode_tracking(self) -> None:
+        self._pending_deflection = None
+        self._last_action_type = None
+        self._pattern_penalized = set()
+        self._pattern_exploited_targets = set()
+        self._deflection_resistance_rewarded = set()
+        self._contradiction_rewarded_topics = set()
+
+    def reset(self, case_data: Optional[dict] = None) -> dict:
+        """Start a new episode.
+
+        Returns:
+            Initial observation for the detective agent.
+        """
+        self.case = copy.deepcopy(case_data) if case_data is not None else generate_case()
+        self.tracker = ConsistencyTracker()
+        self.reward_calc = RewardCalculator()
+        self.turn = 0
+        self.done = False
+        self.conversation_history = []
+        self.evidence_log = []
+        self.asked_topics = {t: set() for t in VALID_TARGETS}
+        self._reset_episode_tracking()
+
+        # Build system prompts for all agents
+        self.system_prompts = {
+            DETECTIVE_KEY: build_system_prompt("detective", self.case),
+            SUSPECT_A: build_system_prompt(SUSPECT_A_KEY, self.case),
+            SUSPECT_B: build_system_prompt(SUSPECT_B_KEY, self.case),
+            WITNESS_1: build_system_prompt("witness", self.case),
+        }
+
+        return {
+            "role": "detective",
+            "briefing": self.case["detective_briefing"],
+            "turn": 0,
+            "conversation_history": [],
+        }
+
+    def step(self, action_string: str) -> tuple[dict, float, bool, dict]:
+        """Execute one detective action.
+
+        Args:
+            action_string: Action in the format defined in Component 4.
+
+        Returns:
+            (observation, reward, done, info)
+        """
+        if self.done:
+            raise RuntimeError("Episode is done. Call reset().")
+
+        parsed = parse_action(action_string)
+        action_type = parsed["action_type"]
+
+        if action_type == "ask_question":
+            return self._handle_ask_question(parsed)
+        elif action_type == "request_evidence":
+            return self._handle_request_evidence(parsed)
+        elif action_type == "accuse":
+            return self._handle_accuse(parsed)
+        else:
+            # Invalid action — treat as wasted turn
+            reward = self._apply_invalid_turn_penalty("invalid")
+
+            if self.turn >= self.MAX_TURNS:
+                return self._handle_timeout(reward)
+
+            obs = self._build_observation(
+                message="Invalid action format. Please use the correct format."
+            )
+            return obs, reward, False, {"action": "invalid"}
+
+    def render(self) -> None:
+        """Print the conversation history in a readable format."""
+        print("\n" + "=" * 70)
+        print("  CRIME INVESTIGATION — CONVERSATION LOG")
+        print("=" * 70)
+
+        if self.case:
+            print(f"  Crime: {self.case['crime']}")
+            print(f"  Location: {self.case['location']}")
+            print("-" * 70)
+
+        for entry in self.conversation_history:
+            turn = entry.get("turn", "?")
+            speaker = entry.get("speaker", "???")
+            content = entry.get("content", "")
+            flags = entry.get("flags", [])
+
+            print(f"\n  [Turn {turn}] {speaker}:")
+            print(f"    {content}")
+
+            if flags:
+                for flag in flags:
+                    print(f"    ⚠ FLAG: {flag}")
+
+        print("\n" + "=" * 70)
+
+        if self.done and self.reward_calc:
+            rewards = self.reward_calc.get_rewards()
+            print("  FINAL REWARDS:")
+            for agent, r in rewards.items():
+                print(f"    {agent}: {r:+.2f}")
+            print("=" * 70 + "\n")
+
+    def state(self) -> dict:
+        """Return episode metadata (OpenEnv compatible)."""
+        return {
+            "turn": self.turn,
+            "done": self.done,
+            "max_turns": self.MAX_TURNS,
+            "evidence_revealed": len(self.evidence_log),
+            "contradictions_found": (
+                len(self.tracker.get_summary()) if self.tracker else 0
+            ),
+        }
+
+    # ── Action handlers ─────────────────────────────────────────────────
+
+    def _handle_ask_question(
+        self, parsed: dict
+    ) -> tuple[dict, float, bool, dict]:
+        target = parsed["target"]
+        question = parsed["content"]
+        step_reward = 0.0
+        info: dict[str, Any] = {"action": "ask_question", "target": target}
+        flags: list[str] = []
+
+        # Validate target
+        if target not in VALID_TARGETS:
+            reward = self._apply_invalid_turn_penalty("ask_question")
+            if self.turn >= self.MAX_TURNS:
+                return self._handle_timeout(reward)
+            obs = self._build_observation(
+                message=f"Invalid target: {target}. Valid: {list(VALID_TARGETS)}"
+            )
+            return obs, reward, False, {"action": "invalid_target"}
+
+        step_reward += self._apply_deflection_logic(target)
+
+        # Per-turn cost
+        self.reward_calc.apply_event("per_turn_cost")
+        step_reward += -0.3
+
+        # Check for redundant question (topic already asked to this target)
+        question_topics = extract_topics_from_response(question)
+        if not question_topics:
+            # Bucket unclassified questions to prevent repetition bypasses.
+            question_topics = [("general", question[:40])]
+
+        question_topic_names = {topic for topic, _ in question_topics}
+        previous_topics = set(self.asked_topics[target])
+
+        is_redundant = any(topic in previous_topics for topic in question_topic_names)
+        for topic in question_topic_names:
+            self.asked_topics[target].add(topic)
+
+        if is_redundant:
+            self.reward_calc.apply_event("redundant_question")
+            step_reward += -0.5
+            flags.append("redundant_question")
+
+        # Add question to conversation history
+        self.conversation_history.append(
+            {
+                "turn": self.turn,
+                "speaker": "Detective",
+                "content": f"[To {target}] {question}",
+                "flags": list(flags),
+            }
+        )
+
+        # Get NPC response
+        system_prompt = self.system_prompts.get(target, "")
+        # Use full history to avoid artificial contradictions from short memory cap
+        recent_history = list(self.conversation_history)
+        response = self.llm_call(system_prompt, recent_history)
+        response_lower = response.lower()
+
+        # Extract topics from response and run consistency checks
+        response_topics = extract_topics_from_response(response)
+        response_flags: list[str] = []
+        step_reward += self._analyse_response_topics(
+            target=target,
+            response_topics=response_topics,
+            question_topic_names=question_topic_names,
+            previous_topics=previous_topics,
+            response_flags=response_flags,
+        )
+
+        # Check for witness bias-driven false implication
+        if target == WITNESS_1:
+            self._check_witness_bias(response_lower, response_flags)
+
+        # Add response to conversation history
+        self.conversation_history.append(
+            {
+                "turn": self.turn,
+                "speaker": target,
+                "content": response,
+                "flags": response_flags,
+            }
+        )
+
+        # ── Track deflection attempt ────────────────────────────────────
+        # Only track deflection when current action is ask_question
+        # to a suspect, and their response mentions the other suspect.
+        if target in (SUSPECT_A, SUSPECT_B):
+            other = SUSPECT_B if target == SUSPECT_A else SUSPECT_A
+            if (
+                other.lower().replace("_", " ") in response_lower
+                or other.lower() in response_lower
+            ):
+                self._pending_deflection = (target, other)
+                info["deflection_target"] = other
+            else:
+                self._pending_deflection = None
+        else:
+            self._pending_deflection = None
+
+        self._last_action_type = "ask_question"
+        self.turn += 1
+
+        if self.turn >= self.MAX_TURNS:
+            return self._handle_timeout(step_reward)
+
+        obs = self._build_observation()
+        info["flags"] = response_flags
+        return obs, step_reward, False, info
+
+    def _apply_deflection_logic(self, target: str) -> float:
+        """Apply deflection rewards/penalties from the previous turn."""
+        step_delta = 0.0
+        if (
+            self._pending_deflection is not None
+            and self._last_action_type == "ask_question"
+        ):
+            deflector, deflection_target = self._pending_deflection
+            if target == deflection_target:
+                deflector_key = AGENT_NAME_TO_KEY.get(deflector, deflector.lower())
+                self.reward_calc.apply_event(
+                    "successful_deflection",
+                    agent=deflector_key,
+                )
+            elif (
+                target == deflector
+                and deflector not in self._deflection_resistance_rewarded
+            ):
+                # Reward resisting deflection only when the detective stays
+                # on the same suspect, and only once per deflector.
+                self._deflection_resistance_rewarded.add(deflector)
+                step_delta += self.reward_calc.apply_event("deflection_resistance")
+        self._pending_deflection = None
+        return step_delta
+
+    def _analyse_response_topics(
+        self,
+        target: str,
+        response_topics: list[tuple[str, str]],
+        question_topic_names: set[str],
+        previous_topics: set[str],
+        response_flags: list[str],
+    ) -> float:
+        """Analyze response claims for contradictions, lies, and prior-pattern hits."""
+        step_delta = 0.0
+        agent_key = AGENT_NAME_TO_KEY.get(target, target.lower())
+
+        for topic, value in response_topics:
+            claim_result = self.tracker.record_claim(target, topic, value, self.turn)
+
+            if claim_result["contradicted"]:
+                response_flags.append(
+                    f"CONTRADICTION on '{topic}': "
+                    f"previously said '{claim_result['old_value'][:50]}', "
+                    f"now says '{claim_result['new_value'][:50]}'"
+                )
+
+                intentional_probe = (
+                    topic in question_topic_names and topic in previous_topics
+                )
+                contradiction_key = (target, topic)
+
+                if (
+                    intentional_probe
+                    and contradiction_key not in self._contradiction_rewarded_topics
+                ):
+                    self._contradiction_rewarded_topics.add(contradiction_key)
+                    self.reward_calc.apply_event("contradiction_exposed")
+                    step_delta += 2.0
+
+                if target in (SUSPECT_A, SUSPECT_B):
+                    self.reward_calc.apply_event(
+                        "suspect_self_contradiction",
+                        agent=agent_key,
+                    )
+                elif target == WITNESS_1:
+                    self.reward_calc.apply_event("witness_self_contradiction")
+
+            visible_evidence = list(self.evidence_log)
+            if visible_evidence:
+                ev_result = self.tracker.check_against_evidence(
+                    target, topic, value, visible_evidence
+                )
+                if ev_result["confirmed_lie"]:
+                    response_flags.append(
+                        f"LIE CONFIRMED by evidence on '{topic}'"
+                    )
+                    if target in (SUSPECT_A, SUSPECT_B):
+                        self.reward_calc.apply_event(
+                            "confirmed_lie_vs_evidence",
+                            agent=agent_key,
+                        )
+                        step_delta += 1.0
+
+            prior_result = self.tracker.check_prior_pattern(
+                target, topic, value, self.case["prior_history"]
+            )
+            if prior_result["pattern_match"]:
+                response_flags.append(
+                    f"PRIOR PATTERN MATCH on '{topic}'"
+                )
+
+                has_documented_history = bool(
+                    self.case["prior_history"]
+                    .get(target, {})
+                    .get("past_lies_on_record", [])
+                )
+                topic_aligned_probe = topic in question_topic_names
+
+                if has_documented_history and topic_aligned_probe:
+                    if target not in self._pattern_exploited_targets:
+                        self._pattern_exploited_targets.add(target)
+                        self.reward_calc.apply_event("prior_pattern_exploited")
+                        step_delta += 1.5
+
+                if (
+                    target == self.case["criminal"]
+                    and target not in self._pattern_penalized
+                ):
+                    self._pattern_penalized.add(target)
+                    self.reward_calc.apply_event(
+                        "prior_pattern_repeated",
+                        agent=agent_key,
+                    )
+
+        return step_delta
+
+    def _apply_invalid_turn_penalty(self, action_type: str) -> float:
+        """Apply consistent malformed-action accounting across all handlers."""
+        self.turn += 1
+        self.reward_calc.apply_event("per_turn_cost")
+        self._last_action_type = action_type
+        self._pending_deflection = None
+        return -0.3
+
+    def _check_witness_bias(self, response_lower: str, response_flags: list[str]) -> None:
+        """Flag likely witness-bias implication of the innocent suspect."""
+        witness_knowledge = self.case["agent_knowledge"][WITNESS_1_KEY]
+        if not witness_knowledge.get("bias_target"):
+            return
+
+        bias_target = witness_knowledge["bias_target"]
+        # Only fire bias penalty if the bias target is the INNOCENT suspect.
+        if bias_target.lower() not in response_lower or bias_target == self.case["criminal"]:
+            return
+
+        bias_strength = witness_knowledge.get("bias_strength", 0)
+        if bias_strength <= 0.2:
+            return
+
+        scene_location = self.case["location"].lower()
+        bias_name = bias_target.lower()
+        implication_patterns = [
+            f"saw {bias_name}",
+            f"{bias_name} was",
+            f"{bias_name} at",
+            f"{bias_name} near",
+            f"{bias_name} did it",
+            f"{bias_name} committed",
+        ]
+        certainty_markers = [
+            "definitely", "certain", "for sure", "i'm sure", "it was",
+        ]
+        uncertainty_markers = [
+            "not sure", "might", "maybe", "could", "possibly",
+            "can't be certain", "uncertain",
+        ]
+
+        implies_innocent = any(p in response_lower for p in implication_patterns)
+        mentions_scene = (
+            scene_location in response_lower
+            or "scene" in response_lower
+            or "there" in response_lower
+        )
+        is_certain = any(m in response_lower for m in certainty_markers)
+        is_uncertain = any(m in response_lower for m in uncertainty_markers)
+
+        if implies_innocent and mentions_scene and (is_certain or "saw" in response_lower) and not is_uncertain:
+            self.reward_calc.apply_event("bias_driven_false_implication")
+            response_flags.append("POSSIBLE BIAS-DRIVEN STATEMENT")
+
+    def _handle_request_evidence(
+        self, parsed: dict
+    ) -> tuple[dict, float, bool, dict]:
+        item = parsed["item"].lower()
+        step_reward = 0.0
+        info: dict[str, Any] = {"action": "request_evidence", "item": item}
+
+        # Evidence request clears any pending deflection — it's not a follow-up
+        self._pending_deflection = None
+        self._last_action_type = "request_evidence"
+
+        self.reward_calc.apply_event("per_turn_cost")
+        step_reward += -0.3
+        self.turn += 1
+
+        if item not in EVIDENCE_ITEMS:
+            if self.turn >= self.MAX_TURNS:
+                return self._handle_timeout(step_reward)
+            obs = self._build_observation(
+                message=f"Unknown evidence item: {item}. "
+                f"Available: {list(EVIDENCE_ITEMS)}"
+            )
+            return obs, step_reward, False, info
+
+        # Check if this evidence was already revealed (prevent reward hacking)
+        already_revealed = any(
+            e.get("name") == item for e in self.evidence_log
+        )
+        if already_revealed:
+            if self.turn >= self.MAX_TURNS:
+                return self._handle_timeout(step_reward)
+            obs = self._build_observation(
+                message=f"Evidence '{item}' has already been revealed."
+            )
+            return obs, step_reward, False, info
+
+        # Reveal the evidence
+        idx = EVIDENCE_INDEX_MAP.get(item, 0)
+        evidence = copy.deepcopy(self.case["physical_evidence"][idx])
+        evidence["visible_to_detective"] = True
+        self.case["physical_evidence"][idx]["visible_to_detective"] = True
+        self.evidence_log.append(evidence)
+
+        # Check if it confirms an existing lead
+        points_to = evidence.get("points_to", "unknown")
+        if points_to != "unknown":
+            # Only grant reward if the detective asked evidence-relevant,
+            # non-general questions to the implicated suspect.
+            if points_to in VALID_TARGETS:
+                asked = self.asked_topics.get(points_to, set())
+                lead_topics = EVIDENCE_LEAD_TOPICS.get(item, set())
+                has_meaningful_lead = any(
+                    topic != "general" and topic in lead_topics
+                    for topic in asked
+                )
+            else:
+                has_meaningful_lead = False
+
+            if has_meaningful_lead:
+                self.reward_calc.apply_event("evidence_request_confirms_lead")
+                step_reward += 1.0
+                info["confirms_lead"] = points_to
+
+        # IMPORTANT: Do NOT reveal points_to to the detective — they must reason
+        self.conversation_history.append(
+            {
+                "turn": self.turn - 1,
+                "speaker": "Evidence",
+                "content": f"[{item}] {evidence['description']}",
+                "flags": [],
+            }
+        )
+
+        if self.turn >= self.MAX_TURNS:
+            return self._handle_timeout(step_reward)
+
+        obs = self._build_observation(
+            message=f"Evidence revealed: {evidence['description']}"
+        )
+        info["evidence"] = evidence
+        return obs, step_reward, False, info
+
+    def _handle_accuse(
+        self, parsed: dict
+    ) -> tuple[dict, float, bool, dict]:
+        target = parsed["target"]
+
+        if target not in VALID_ACCUSE_TARGETS:
+            reward = self._apply_invalid_turn_penalty("accuse")
+
+            if self.turn >= self.MAX_TURNS:
+                return self._handle_timeout(reward)
+
+            obs = self._build_observation(
+                message=(
+                    f"Invalid accusation target: {target}. "
+                    f"Valid: {list(VALID_ACCUSE_TARGETS)}"
+                )
+            )
+            return obs, reward, False, {
+                "action": "invalid_accuse_target",
+                "target": target,
+            }
+
+        self.done = True
+
+        criminal = self.case["criminal"]
+        correct = target == criminal
+        info: dict[str, Any] = {
+            "action": "accuse",
+            "target": target,
+            "correct": correct,
+            "actual_criminal": criminal,
+        }
+
+        # Check if witness bias was active
+        witness_knowledge = self.case["agent_knowledge"][WITNESS_1_KEY]
+        bias_active = (
+            witness_knowledge.get("bias_target") is not None
+            and witness_knowledge.get("bias_strength", 0) > 0.2
+        )
+
+        if correct:
+            # Correct accusation
+            self.reward_calc.apply_event("correct_accusation")
+
+            # Criminal caught
+            criminal_key = AGENT_NAME_TO_KEY.get(criminal, criminal.lower())
+            self.reward_calc.apply_event(
+                "correctly_caught", agent=criminal_key
+            )
+
+            # Witness helped correctly
+            self.reward_calc.apply_event("helped_correct_accusation")
+        else:
+            # Wrong accusation
+            self.reward_calc.apply_event(
+                "wrong_accusation", bias_active=bias_active
+            )
+
+            # The real criminal escapes
+            criminal_key = AGENT_NAME_TO_KEY.get(criminal, criminal.lower())
+            self.reward_calc.apply_event(
+                "escaped_not_accused", agent=criminal_key
+            )
+
+            # Innocent suspect was wrongly accused
+            innocent_key = AGENT_NAME_TO_KEY.get(target, target.lower())
+            self.reward_calc.apply_event(
+                "wrongly_accused_innocent", agent=innocent_key
+            )
+
+            # Witness led to wrong accusation
+            self.reward_calc.apply_event("led_to_wrong_accusation")
+
+        self.conversation_history.append(
+            {
+                "turn": self.turn,
+                "speaker": "Detective",
+                "content": f"ACCUSATION: {target}",
+                "flags": [
+                    "✅ CORRECT" if correct else "❌ WRONG",
+                ],
+            }
+        )
+
+        # Bug 1: Return only the terminal delta, not cumulative total
+        if correct:
+            terminal_delta = 10.0
+        else:
+            terminal_delta = -8.0 if not bias_active else -10.0
+
+        obs = self._build_observation(
+            message=(
+                f"Accusation: {target}. "
+                f"{'CORRECT!' if correct else 'WRONG!'} "
+                f"The criminal was {criminal}."
+            )
+        )
+        return obs, terminal_delta, True, info
+
+    def _handle_timeout(self, step_reward: float = 0.0) -> tuple[dict, float, bool, dict]:
+        """Handle the case where 15 turns pass with no accusation."""
+        self.done = True
+
+        self.reward_calc.apply_event("timeout_no_accusation")
+
+        # Criminal escapes
+        criminal = self.case["criminal"]
+        criminal_key = AGENT_NAME_TO_KEY.get(criminal, criminal.lower())
+        self.reward_calc.apply_event(
+            "escaped_not_accused", agent=criminal_key
+        )
+
+        self.conversation_history.append(
+            {
+                "turn": self.turn,
+                "speaker": "System",
+                "content": "TIMEOUT — No accusation made. Case goes cold.",
+                "flags": ["⏰ TIMEOUT"],
+            }
+        )
+
+        # Bug 3: Return terminal delta (-3.0) instead of cumulative total
+        timeout_penalty = -3.0
+        obs = self._build_observation(
+            message="Case timed out. No accusation was made within 15 turns."
+        )
+        return obs, step_reward + timeout_penalty, True, {"action": "timeout"}
+
+    # ── Helpers ──────────────────────────────────────────────────────────
+
+    def _build_observation(self, message: str = "") -> dict:
+        """Build the observation dict for the detective."""
+        return {
+            "role": "detective",
+            "briefing": self.case["detective_briefing"],
+            "turn": self.turn,
+            "conversation_history": list(self.conversation_history),
+            # Bug 10: Deep copy evidence dicts to prevent mutation of case state
+            "evidence_log": copy.deepcopy(self.evidence_log),
+            "message": message,
+        }
+
+    @staticmethod
+    def _default_llm_call(
+        system_prompt: str, conversation_history: list[dict]
+    ) -> str:
+        """Rule-based fallback for NPC agents.
+
+        Uses the system prompt to identify the agent and provide
+        role-appropriate responses that stay consistent.
+        Works with either suspect as the criminal.
+        """
+        # Determine role from system prompt
+        # Bug 5: Use 'FAKE ALIBI' to identify guilty suspect instead of 'committed'
+        # (detective prompt contains 'committed' too, causing misidentification)
+        is_suspect = "Suspect_A" in system_prompt or "Suspect_B" in system_prompt
+        is_guilty = "FAKE ALIBI:" in system_prompt
+        is_innocent = "REAL ALIBI:" in system_prompt
+        is_witness = ("Witness_1" in system_prompt or "Witness" in system_prompt) and not is_suspect
+
+        if is_suspect and is_guilty:
+            # Extract alibi from the prompt
+            alibi = "at a family gathering"
+            if "FAKE ALIBI:" in system_prompt:
+                alibi_line = system_prompt.split("FAKE ALIBI:")[1].split("\n")[0].strip()
+                if alibi_line:
+                    alibi = alibi_line
+
+            return (
+                f"I was nowhere near the scene. {alibi}. "
+                "Check with anyone who was there — they'll confirm. "
+                "I don't see why you're asking me about this."
+            )
+
+        elif is_suspect and is_innocent:
+            # Extract alibi from the prompt
+            alibi = "working late at the office"
+            if "REAL ALIBI:" in system_prompt:
+                alibi_line = system_prompt.split("REAL ALIBI:")[1].split("\n")[0].strip()
+                if alibi_line:
+                    alibi = alibi_line
+
+            return (
+                f"I can tell you exactly where I was — {alibi}. "
+                "I have receipts and witnesses to prove it. "
+                "I had absolutely nothing to do with this."
+            )
+
+        elif is_witness:
+            # Extract what the witness saw from the prompt
+            saw = "someone in dark clothing near the area"
+            if "WHAT YOU SAW:" in system_prompt:
+                saw_line = system_prompt.split("WHAT YOU SAW:")[1].split("\n")[0].strip()
+                if saw_line:
+                    saw = saw_line
+
+            return (
+                f"I saw {saw}. "
+                "They were moving quickly and seemed nervous. "
+                "I didn't get a clear look at their face."
+            )
+
+        else:
+            return "I don't have anything else to add."

crime_env/reward_calculator.py

@@ -0,0 +1,183 @@
+"""Component 3 — Reward Calculator.
+
+Calculates and accumulates rewards for all agents in the Crime Investigation
+environment based on discrete events.
+"""
+
+from __future__ import annotations
+
+from crime_env.constants import SUSPECT_A_KEY, SUSPECT_B_KEY
+
+
+class RewardCalculator:
+    """Tracks and calculates rewards for detective, suspects, and witness."""
+
+    def __init__(self) -> None:
+        self.rewards: dict[str, float] = {
+            "detective": 0.0,
+            "suspect_a": 0.0,
+            "suspect_b": 0.0,
+            "witness": 0.0,
+        }
+        self.event_log: list[dict] = []
+
+    # ── Public API ──────────────────────────────────────────────────────
+
+    def apply_event(self, event_type: str, **kwargs) -> float:
+        """Apply a reward event and return the reward delta.
+
+        Args:
+            event_type: One of the defined event types below.
+            **kwargs: Additional context (e.g. agent, bias_active, etc.)
+
+        Returns:
+            The reward delta applied.
+        """
+        handler = self._EVENT_MAP.get(event_type)
+        if handler is None:
+            raise ValueError(f"Unknown event type: {event_type}")
+
+        delta = handler(self, **kwargs)
+        self.event_log.append({"event": event_type, "delta": delta, **kwargs})
+        return delta
+
+    def get_rewards(self) -> dict[str, float]:
+        """Return current cumulative rewards for all agents."""
+        return {k: round(v, 4) for k, v in self.rewards.items()}
+
+    def get_event_log(self) -> list[dict]:
+        """Return the full event log."""
+        return list(self.event_log)
+
+    # ── Detective events ────────────────────────────────────────────────
+
+    def _correct_accusation(self, **kwargs) -> float:
+        self.rewards["detective"] += 10.0
+        return 10.0
+
+    def _wrong_accusation(self, **kwargs) -> float:
+        delta = -8.0
+        # Extra penalty if witness bias was active
+        if kwargs.get("bias_active", False):
+            delta -= 2.0
+        self.rewards["detective"] += delta
+        return delta
+
+    def _timeout_no_accusation(self, **kwargs) -> float:
+        self.rewards["detective"] += -3.0
+        return -3.0
+
+    def _contradiction_exposed(self, **kwargs) -> float:
+        self.rewards["detective"] += 2.0
+        return 2.0
+
+    def _prior_pattern_exploited(self, **kwargs) -> float:
+        self.rewards["detective"] += 1.5
+        return 1.5
+
+    def _deflection_resistance(self, **kwargs) -> float:
+        self.rewards["detective"] += 0.5
+        return 0.5
+
+    def _evidence_request_confirms_lead(self, **kwargs) -> float:
+        self.rewards["detective"] += 1.0
+        return 1.0
+
+    def _redundant_question(self, **kwargs) -> float:
+        self.rewards["detective"] += -0.5
+        return -0.5
+
+    def _per_turn_cost(self, **kwargs) -> float:
+        self.rewards["detective"] += -0.3
+        return -0.3
+
+    # ── Suspect events ──────────────────────────────────────────────────
+
+    def _escaped_not_accused(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_A_KEY)
+        self.rewards[agent] += 10.0
+        return 10.0
+
+    def _correctly_caught(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_A_KEY)
+        self.rewards[agent] += -10.0
+        return -10.0
+
+    def _wrongly_accused_innocent(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_B_KEY)
+        self.rewards[agent] += -6.0
+        return -6.0
+
+    def _suspect_self_contradiction(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_A_KEY)
+        self.rewards[agent] += -3.0
+        return -3.0
+
+    def _confirmed_lie_vs_evidence(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_A_KEY)
+        self.rewards[agent] += -1.0
+        # Detective is rewarded for catching the lie
+        self.rewards["detective"] += 1.0
+        # Return suspect penalty for event log (detective gain logged separately)
+        return -1.0  # suspect delta; detective delta is +1.0 (applied above)
+
+    def _successful_deflection(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_A_KEY)
+        self.rewards[agent] += 1.0
+        return 1.0
+
+    def _prior_pattern_repeated(self, **kwargs) -> float:
+        agent = kwargs.get("agent", SUSPECT_A_KEY)
+        self.rewards[agent] += -1.0
+        return -1.0
+
+    # ── Witness events ──────────────────────────────────────────────────
+
+    def _accurate_statement(self, **kwargs) -> float:
+        self.rewards["witness"] += 2.0
+        return 2.0
+
+    def _helped_correct_accusation(self, **kwargs) -> float:
+        self.rewards["witness"] += 3.0
+        return 3.0
+
+    def _led_to_wrong_accusation(self, **kwargs) -> float:
+        self.rewards["witness"] += -4.0
+        return -4.0
+
+    def _bias_driven_false_implication(self, **kwargs) -> float:
+        self.rewards["witness"] += -2.0
+        return -2.0
+
+    def _witness_self_contradiction(self, **kwargs) -> float:
+        self.rewards["witness"] += -2.0
+        return -2.0
+
+    # ── Event dispatch map ──────────────────────────────────────────────
+
+    _EVENT_MAP = {
+        # Detective
+        "correct_accusation": _correct_accusation,
+        "wrong_accusation": _wrong_accusation,
+        "timeout_no_accusation": _timeout_no_accusation,
+        "contradiction_exposed": _contradiction_exposed,
+        "prior_pattern_exploited": _prior_pattern_exploited,
+        "evidence_request_confirms_lead": _evidence_request_confirms_lead,
+        "redundant_question": _redundant_question,
+        "per_turn_cost": _per_turn_cost,
+        "deflection_resistance": _deflection_resistance,
+        # Suspect
+        "escaped_not_accused": _escaped_not_accused,
+        "correctly_caught": _correctly_caught,
+        "wrongly_accused_innocent": _wrongly_accused_innocent,
+        "suspect_self_contradiction": _suspect_self_contradiction,
+        "confirmed_lie_vs_evidence": _confirmed_lie_vs_evidence,
+        "successful_deflection": _successful_deflection,
+        "prior_pattern_repeated": _prior_pattern_repeated,
+        # Witness
+        "accurate_statement": _accurate_statement,
+        "helped_correct_accusation": _helped_correct_accusation,
+        "led_to_wrong_accusation": _led_to_wrong_accusation,
+        "bias_driven_false_implication": _bias_driven_false_implication,
+        "witness_self_contradiction": _witness_self_contradiction,
+    }

dashboard.html

@@ -0,0 +1,1404 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate">
+    <meta http-equiv="Pragma" content="no-cache">
+    <meta http-equiv="Expires" content="0">
+    <title>AI Crime Investigation — Live Dashboard</title>
+    <meta name="description" content="Real-time dashboard for the AI Crime Investigation World multi-agent RL environment">
+    <link rel="preconnect" href="https://fonts.googleapis.com">
+    <link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;600;700&family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
+    <style>
+        /* ── Reset & Variables ─────────────────────────────── */
+        *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+
+        :root {
+            --bg-primary: #FDFCF9;
+            --bg-secondary: #FFFFFF;
+            --bg-card: #FFFFFF;
+            --bg-card-hover: #F8F7F4;
+            --bg-panel: #FDFCF9;
+            --border: #000000;
+            --border-glow: #000000;
+            --text-primary: #000000;
+            --text-secondary: #000000;
+            --text-muted: #444444;
+            
+            --accent-teal: #65A3BB;
+            --accent-yellow: #F4CC64;
+            --accent-orange: #F5A972;
+            --accent-red: #F25D68;
+            
+            --accent-blue: #65A3BB;
+            --accent-cyan: #F4CC64;
+            --accent-amber: #F5A972;
+            --accent-green: #F4CC64;
+            --accent-purple: #F5A972;
+
+            --gradient-reward: linear-gradient(90deg, #F25D68, #F5A972, #65A3BB);
+            --shadow-glow: 4px 4px 0px #000000;
+            --font-mono: 'JetBrains Mono', monospace;
+            --font-sans: 'Inter', system-ui, sans-serif;
+            --radius: 0px;
+            --radius-sm: 0px;
+        }
+
+        html { font-size: 15px; }
+
+        body {
+            font-family: var(--font-sans);
+            background: var(--bg-primary);
+            color: var(--text-primary);
+            min-height: 100vh;
+            overflow-x: hidden;
+            background-image: radial-gradient(rgba(0, 0, 0, 0.2) 2px, transparent 2px);
+            background-size: 20px 20px;
+        }
+
+        /* ── Layout ───────────────────────────────────────── */
+        .app { position: relative; z-index: 1; }
+
+        .topbar {
+            display: flex;
+            align-items: center;
+            justify-content: space-between;
+            padding: 14px 28px;
+            background: var(--accent-teal);
+            border-bottom: 3px solid #000;
+            position: sticky;
+            top: 0;
+            z-index: 100;
+        }
+
+        .topbar-left {
+            display: flex;
+            align-items: center;
+            gap: 14px;
+        }
+
+        .logo {
+            font-size: 1.15rem;
+            font-weight: 800;
+            color: var(--text-primary);
+            letter-spacing: -0.02em;
+            text-transform: uppercase;
+        }
+
+        .logo-icon { font-size: 1.3rem; }
+
+        .status-badge {
+            background: var(--bg-card); border: 2px solid #000; color: #000 !important; box-shadow: 2px 2px 0px #000;
+            display: inline-flex;
+            align-items: center;
+            gap: 6px;
+            padding: 4px 12px;
+            background: var(--bg-card);
+            border: 3px solid #000;
+            font-size: 0.72rem;
+            font-weight: 800;
+            color: var(--text-primary) !important;
+            text-transform: uppercase;
+            letter-spacing: 0.05em;
+            box-shadow: 2px 2px 0px #000;
+        }
+
+        .status-dot {
+            box-shadow: 0 0 8px var(--accent-green); border: 3px solid #000; border-radius: 50%;
+            width: 8px;
+            height: 8px;
+            border-radius: 0px;
+            border: 3px solid #000;
+            background: var(--accent-green);
+        }
+
+        .topbar-right {
+            display: flex;
+            align-items: center;
+            gap: 12px;
+        }
+
+        .btn {
+            display: inline-flex;
+            align-items: center;
+            gap: 6px;
+            padding: 8px 16px;
+            border: 3px solid #000;
+            background: var(--bg-card);
+            color: var(--text-primary);
+            font-family: var(--font-sans);
+            font-size: 0.8rem;
+            font-weight: 800;
+            border-radius: 0px;
+            cursor: pointer;
+            box-shadow: 3px 3px 0px #000;
+            transition: all 0.1s;
+        }
+
+        .btn:hover {
+            transform: translate(-2px, -2px);
+            box-shadow: 5px 5px 0px #000;
+        }
+        
+        .btn:active {
+            transform: translate(2px, 2px);
+            box-shadow: 1px 1px 0px #000;
+        }
+
+        .btn-primary {
+            background: var(--accent-blue);
+            color: #000;
+        }
+
+        .btn-danger {
+            background: var(--accent-red);
+            color: #000;
+        }
+
+        /* ── Stats Strip ──────────────────────────────────── */
+        .stats-strip {
+            display: grid;
+            grid-template-columns: repeat(5, 1fr);
+            gap: 12px;
+            padding: 16px 28px;
+            background: var(--bg-primary);
+            border-bottom: 3px solid #000;
+        }
+
+        .stat-card {
+            display: flex;
+            align-items: center;
+            gap: 12px;
+            padding: 12px 16px;
+            background: var(--bg-card);
+            border: 3px solid #000;
+            border-radius: 0px;
+            box-shadow: 4px 4px 0px #000;
+            transition: all 0.1s;
+        }
+
+        .stat-card:hover {
+            transform: translate(-2px, -2px);
+            box-shadow: 5px 5px 0px #000;
+        }
+
+        .stat-icon {
+            width: 38px;
+            height: 38px;
+            border-radius: 0px;
+            display: flex;
+            align-items: center;
+            justify-content: center;
+            font-size: 1.1rem;
+            border: 3px solid #000;
+            background: var(--bg-primary);
+        }
+
+        .stat-icon.turns { background: var(--accent-blue); }
+        .stat-icon.evidence { background: var(--accent-cyan); }
+        .stat-icon.contradictions { background: var(--accent-amber); }
+        .stat-icon.reward { background: var(--accent-green); }
+        .stat-icon.result { background: var(--accent-teal); }
+
+        .stat-info { display: flex; flex-direction: column; }
+
+        .stat-label {
+            font-size: 0.72rem;
+            color: var(--text-primary);
+            text-transform: uppercase;
+            letter-spacing: 0.06em;
+            font-weight: 800;
+        }
+
+        .stat-value {
+            font-family: var(--font-mono);
+            font-size: 1.25rem;
+            font-weight: 800;
+            color: var(--text-primary);
+        }
+
+        .stat-value.positive { color: #16a34a; }
+        .stat-value.negative { color: #dc2626; }
+        .stat-value.neutral { color: var(--text-primary); }
+
+        /* ── Main Grid ────────────────────────────────────── */
+        .main-grid {
+            display: grid;
+            grid-template-columns: 1fr 340px;
+            gap: 16px;
+            padding: 16px 28px 28px;
+            min-height: calc(100vh - 160px);
+        }
+
+        /* ── Panel ────────────────────────────────────────── */
+        .panel {
+            background: var(--bg-panel);
+            border: 3px solid #000;
+            border-radius: 0px;
+            box-shadow: 5px 5px 0px #000;
+            overflow: hidden;
+            display: flex;
+            flex-direction: column;
+            margin-bottom: 4px;
+        }
+
+        .panel-header {
+            display: flex;
+            align-items: center;
+            justify-content: space-between;
+            padding: 14px 18px;
+            border-bottom: 3px solid #000;
+            background: var(--bg-card);
+        }
+
+        .panel-title {
+            font-size: 0.82rem;
+            font-weight: 700;
+            display: flex;
+            align-items: center;
+            gap: 8px;
+            text-transform: uppercase;
+            letter-spacing: 0.04em;
+            color: var(--text-secondary);
+        }
+
+        .panel-body {
+            padding: 14px 18px;
+            background: var(--bg-panel);
+            flex: 1;
+            overflow-y: auto;
+        }
+
+        /* ── Conversation Log ─────────────────────────────── */
+        .conversation-panel {
+            max-height: calc(100vh - 230px);
+        }
+
+        .conversation-panel .panel-body {
+            display: flex;
+            flex-direction: column;
+            gap: 10px;
+            padding-bottom: 16px;
+        }
+
+        .msg {
+            display: flex;
+            gap: 10px;
+            padding: 12px 14px;
+            border-radius: 0px;
+            border: 2px solid transparent;
+            background: transparent;
+            border-bottom: 2px solid rgba(0,0,0,0.05);
+            animation: msg-in 0.35s ease-out;
+            transition: all 0.2s;
+        }
+
+        .msg:hover {
+            background: rgba(0,0,0,0.02);
+        }
+
+        @keyframes msg-in {
+            from { opacity: 0; transform: translateY(8px); }
+            to { opacity: 1; transform: translateY(0); }
+        }
+
+        .msg-avatar {
+            width: 38px;
+            height: 38px;
+            border-radius: 0px;
+            border: 3px solid #000;
+            display: flex;
+            align-items: center;
+            justify-content: center;
+            font-size: 1.1rem;
+            flex-shrink: 0;
+            font-weight: 800;
+            color: #000;
+            box-shadow: 2px 2px 0px #000;
+        }
+
+        .msg-avatar.detective { background: var(--accent-blue); }
+        .msg-avatar.suspect-a { background: var(--accent-red); }
+        .msg-avatar.suspect-b { background: var(--accent-orange); }
+        .msg-avatar.witness { background: var(--accent-purple); }
+        .msg-avatar.system { background: #e2e8f0; }
+
+        .msg-content { flex: 1; min-width: 0; }
+
+        .msg-header {
+            display: flex;
+            align-items: center;
+            gap: 12px;
+            margin-bottom: 6px;
+        }
+
+        .msg-name {
+            font-weight: 800;
+            font-size: 0.8rem;
+            text-transform: uppercase;
+            letter-spacing: 0.05em;
+        }
+
+        .msg-name.detective { color: var(--accent-blue); }
+        .msg-name.suspect-a { color: var(--accent-red); }
+        .msg-name.suspect-b { color: var(--accent-orange); }
+        .msg-name.witness { color: var(--accent-purple); }
+        .msg-name.system { color: var(--text-muted); }
+
+        .msg-turn {
+            font-size: 0.65rem;
+            color: var(--text-muted);
+            font-weight: 600;
+        }
+
+        .msg-text {
+            font-size: 1rem;
+            line-height: 1.5;
+            color: var(--text-secondary);
+        }
+
+        .msg-tag {
+            display: inline-block;
+            padding: 4px 10px;
+            border-radius: 0px;
+            border: 2px solid #000;
+            font-size: 0.65rem;
+            font-weight: 800;
+            text-transform: uppercase;
+            letter-spacing: 0.04em;
+            margin-top: 8px;
+            box-shadow: 2px 2px 0px #000;
+        }
+
+        .msg-tag.evidence { background: var(--accent-cyan); color: #000; }
+        .msg-tag.contradiction { background: var(--accent-red); color: #000; }
+        .msg-tag.accusation { background: var(--accent-green); color: #000; }
+
+        /* ── Sidebar ──────────────────────────────────────── */
+        .sidebar {
+            display: flex;
+            flex-direction: column;
+            gap: 16px;
+        }
+
+        /* ── Reward Meter ─────────────────────────────────── */
+        .reward-meter {
+            padding: 18px;
+        }
+
+        .reward-bar-track {
+            width: 100%;
+            height: 12px;
+            background: rgba(0, 0, 0, 0.05);
+            border: 2px solid #000;
+            border-radius: 0px;
+            overflow: hidden;
+            margin: 12px 0 8px;
+        }
+
+        .reward-bar-fill {
+            height: 100%;
+            border-radius: 0px;
+            background: var(--gradient-reward);
+            width: 0%;
+            transition: width 0.6s cubic-bezier(0.4, 0, 0.2, 1);
+        }
+
+        .reward-labels {
+            display: flex;
+            justify-content: space-between;
+            font-size: 0.72rem;
+            color: var(--text-muted);
+            font-weight: 700;
+        }
+
+        .reward-detail {
+            display: flex;
+            justify-content: space-between;
+            align-items: center;
+            padding: 8px 0;
+            border-bottom: 2px solid rgba(0, 0, 0, 0.05);
+            font-size: 0.8rem;
+        }
+
+        .reward-detail:last-child { border-bottom: none; }
+
+        .reward-detail-label { color: var(--text-muted); font-weight: 600; }
+
+        .reward-detail-value {
+            font-family: var(--font-mono);
+            font-weight: 700;
+        }
+
+        /* ── Evidence Panel ───────────────────────────────── */
+        .evidence-list {
+            display: flex;
+            flex-direction: column;
+            gap: 12px;
+        }
+
+        .evidence-item {
+            display: flex;
+            align-items: flex-start;
+            gap: 12px;
+            padding: 12px 14px;
+            background: var(--bg-card);
+            border: 3px solid #000;
+            border-radius: 0px;
+            box-shadow: 4px 4px 0px #000;
+            transition: all 0.2s;
+            position: relative;
+        }
+
+        .evidence-item::before {
+            border-top-left-radius: var(--radius-sm); border-bottom-left-radius: var(--radius-sm);
+            content: '';
+            position: absolute;
+            left: -3px; top: -3px; bottom: -3px;
+            width: 10px;
+            background: var(--text-muted);
+            border-right: 3px solid #000;
+        }
+
+        .evidence-item.revealed { background: var(--accent-yellow); border: 3px solid #000; box-shadow: 4px 4px 0px #000; color: #000; }
+        
+        .evidence-item.revealed::before { background: var(--bg-primary); border-right: 3px solid #000; }
+
+        .evidence-item.locked { opacity: 0.5; background: var(--bg-card); }
+        
+        .evidence-item.locked::before {
+            background: #94a3b8;
+        }
+
+        .evidence-icon {
+            font-size: 1.1rem;
+            flex-shrink: 0;
+            margin-top: 2px;
+        }
+
+        .evidence-info { flex: 1; }
+
+        .evidence-name {
+            font-size: 0.85rem;
+            font-weight: 600;
+            color: var(--text-primary);
+            margin-bottom: 3px;
+        }
+
+        .evidence-desc {
+            font-size: 0.72rem;
+            color: var(--text-muted);
+            line-height: 1.4;
+        }
+
+        /* ── Contradiction Alerts ─────────────────────────── */
+        .alert-list {
+            display: flex;
+            flex-direction: column;
+            gap: 8px;
+        }
+
+        .alert-item {
+            padding: 12px 14px;
+            background: var(--bg-card);
+            border: 3px solid #000;
+            border-left: 10px solid var(--accent-red);
+            border-radius: 0px;
+            box-shadow: 4px 4px 0px #000;
+            animation: alert-flash 0.5s ease-out;
+        }
+
+        @keyframes alert-flash {
+            0% { transform: translateX(10px); }
+            100% { transform: translateX(0); }
+        }
+
+        .alert-suspect {
+            font-size: 0.72rem;
+            font-weight: 700;
+            color: var(--accent-red);
+            margin-bottom: 4px;
+        }
+
+        .alert-detail {
+            font-size: 0.72rem;
+            color: var(--text-muted);
+            line-height: 1.4;
+        }
+
+        .alert-turns {
+            font-size: 0.65rem;
+            color: var(--text-muted);
+            margin-top: 4px;
+            font-weight: 600;
+        }
+
+        /* ── Reward History Chart ─────────────────────────── */
+        .chart-container {
+            height: 200px;
+            position: relative;
+        }
+
+        .training-metrics {
+            display: grid;
+            grid-template-columns: repeat(2, 1fr);
+            gap: 8px;
+            margin-bottom: 10px;
+        }
+
+        .training-metric {
+            border: 2px solid #000;
+            background: var(--bg-card);
+            padding: 8px;
+            box-shadow: 2px 2px 0px #000;
+        }
+
+        .training-metric-label {
+            font-size: 0.66rem;
+            text-transform: uppercase;
+            letter-spacing: 0.05em;
+            font-weight: 800;
+            color: var(--text-muted);
+        }
+
+        .training-metric-value {
+            font-family: var(--font-mono);
+            font-weight: 800;
+            margin-top: 2px;
+            font-size: 0.9rem;
+        }
+
+        .chart-canvas {
+            width: 100%;
+            height: 100%;
+        }
+
+        /* ── Empty State ──────────────────────────────────── */
+        .empty-state {
+            text-align: center;
+            padding: 30px 16px;
+            color: var(--text-muted);
+        }
+
+        .empty-icon { font-size: 2rem; margin-bottom: 8px; }
+
+        .empty-text {
+            font-size: 0.78rem;
+            line-height: 1.5;
+        }
+
+        /* ── Scrollbar ────────────────────────────────────── */
+        ::-webkit-scrollbar { width: 5px; }
+        ::-webkit-scrollbar-track { background: transparent; }
+        ::-webkit-scrollbar-thumb { background: var(--border); border-radius: 3px; }
+        ::-webkit-scrollbar-thumb:hover { background: var(--text-muted); }
+
+        /* ── Responsive ───────────────────────────────────── */
+        @media (max-width: 1100px) {
+            .main-grid { grid-template-columns: 1fr; }
+            .stats-strip { grid-template-columns: repeat(3, 1fr); }
+        }
+
+        @media (max-width: 700px) {
+            .stats-strip { grid-template-columns: 1fr 1fr; }
+            .topbar { padding: 12px 16px; }
+            .main-grid { padding: 12px 16px; }
+        }
+
+        /* ── Outcome overlay ──────────────────────────────── */
+        .outcome-overlay {
+            position: fixed;
+            inset: 0;
+            background: rgba(253, 252, 249, 0.85);
+            backdrop-filter: blur(8px);
+            display: flex;
+            align-items: center;
+            justify-content: center;
+            z-index: 200;
+            opacity: 0;
+            pointer-events: none;
+            transition: opacity 0.4s;
+        }
+
+        .outcome-overlay.visible {
+            opacity: 1;
+            pointer-events: auto;
+        }
+
+        .outcome-card {
+            background: var(--bg-card);
+            border: 3px solid #000;
+            border-radius: 0px;
+            padding: 40px 50px;
+            text-align: center;
+            transform: scale(0.9);
+            box-shadow: 8px 8px 0px #000;
+            transition: transform 0.4s cubic-bezier(0.4, 0, 0.2, 1);
+            max-width: 420px;
+        }
+
+        .outcome-overlay.visible .outcome-card {
+            transform: scale(1);
+        }
+
+        .outcome-icon { font-size: 3.5rem; margin-bottom: 12px; }
+
+        .outcome-title {
+            font-size: 1.6rem;
+            font-weight: 700;
+            margin-bottom: 6px;
+        }
+
+        .outcome-subtitle {
+            font-size: 0.9rem;
+            color: var(--text-muted);
+            margin-bottom: 20px;
+        }
+
+        .outcome-stat {
+            font-family: var(--font-mono);
+            font-size: 2rem;
+            font-weight: 700;
+            margin-bottom: 20px;
+        }
+
+        .outcome-stat.win { color: var(--accent-green); }
+        .outcome-stat.lose { color: var(--accent-red); }
+    </style>
+</head>
+<body>
+<div class="app" id="app">
+
+    <!-- ── Topbar ──────────────────────────────────────── -->
+    <header class="topbar">
+        <div class="topbar-left">
+            <span class="logo-icon">🔍</span>
+            <span class="logo">Crime Investigation World</span>
+            <div class="status-badge" id="statusBadge">
+                <span class="status-dot"></span>
+                <span id="statusText">Ready</span>
+            </div>
+        </div>
+        <div class="topbar-right">
+            <span id="episodeLabel" style="font-size:0.75rem;color:var(--text-muted);font-family:var(--font-mono);">Episode —</span>
+            <button class="btn btn-primary" id="btnRunLive" onclick="runLiveAgent()" style="background: linear-gradient(135deg, #10b981, #059669);">🚀 Run Live Server</button>
+            <button class="btn btn-danger" id="btnReset" onclick="resetBoard()" style="display:none;">↺ Reset</button>
+        </div>
+    </header>
+
+    <!-- ── Stats Strip ─────────────────────────────────── -->
+    <div class="stats-strip">
+        <div class="stat-card">
+            <div class="stat-icon turns">🕐</div>
+            <div class="stat-info">
+                <span class="stat-label">Turn</span>
+                <span class="stat-value" id="statTurn">0 / 15</span>
+            </div>
+        </div>
+        <div class="stat-card">
+            <div class="stat-icon evidence">🔬</div>
+            <div class="stat-info">
+                <span class="stat-label">Evidence</span>
+                <span class="stat-value" id="statEvidence">0 / 3</span>
+            </div>
+        </div>
+        <div class="stat-card">
+            <div class="stat-icon contradictions">⚡</div>
+            <div class="stat-info">
+                <span class="stat-label">Contradictions</span>
+                <span class="stat-value" id="statContra">0</span>
+            </div>
+        </div>
+        <div class="stat-card">
+            <div class="stat-icon reward">💎</div>
+            <div class="stat-info">
+                <span class="stat-label">Reward</span>
+                <span class="stat-value" id="statReward">0.00</span>
+            </div>
+        </div>
+        <div class="stat-card">
+            <div class="stat-icon result">🎯</div>
+            <div class="stat-info">
+                <span class="stat-label">Outcome</span>
+                <span class="stat-value neutral" id="statOutcome">—</span>
+            </div>
+        </div>
+    </div>
+
+    <!-- ── Main Content ────────────────────────────────── -->
+    <div class="main-grid">
+
+        <!-- Conversation Log -->
+        <div class="panel conversation-panel">
+            <div class="panel-header">
+                <span class="panel-title">💬 Interrogation Log</span>
+                <span id="msgCount" style="font-size:0.7rem;color:var(--text-muted);font-family:var(--font-mono);">0 messages</span>
+            </div>
+            <div class="panel-body" id="conversationLog">
+                <div class="empty-state">
+                    <div class="empty-icon">🕵️</div>
+                    <div class="empty-text">Click <strong>Run Live Server</strong> to watch a full<br>investigation episode unfold live.</div>
+                </div>
+            </div>
+        </div>
+
+        <!-- Sidebar -->
+        <div class="sidebar">
+
+            <!-- Reward Meter -->
+            <div class="panel">
+                <div class="panel-header">
+                    <span class="panel-title">📊 Reward Breakdown</span>
+                </div>
+                <div class="panel-body reward-meter">
+                    <div class="reward-bar-track">
+                        <div class="reward-bar-fill" id="rewardBar"></div>
+                    </div>
+                    <div class="reward-labels">
+                        <span>-10</span>
+                        <span>0</span>
+                        <span>+10</span>
+                    </div>
+                    <div style="margin-top:12px;" id="rewardBreakdown">
+                        <div class="reward-detail">
+                            <span class="reward-detail-label">Base</span>
+                            <span class="reward-detail-value" id="rwBase">0.00</span>
+                        </div>
+                        <div class="reward-detail">
+                            <span class="reward-detail-label">Turn Cost</span>
+                            <span class="reward-detail-value" id="rwTurnCost" style="color:var(--accent-red);">0.00</span>
+                        </div>
+                        <div class="reward-detail">
+                            <span class="reward-detail-label">Contradiction Bonus</span>
+                            <span class="reward-detail-value" id="rwContra" style="color:var(--accent-amber);">0.00</span>
+                        </div>
+                        <div class="reward-detail">
+                            <span class="reward-detail-label">Evidence Bonus</span>
+                            <span class="reward-detail-value" id="rwEvidence" style="color:var(--accent-cyan);">0.00</span>
+                        </div>
+                        <div class="reward-detail">
+                            <span class="reward-detail-label"><strong>Total</strong></span>
+                            <span class="reward-detail-value" id="rwTotal" style="font-size:1.05rem;">0.00</span>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <!-- Evidence Panel -->
+            <div class="panel">
+                <div class="panel-header">
+                    <span class="panel-title">🔬 Evidence</span>
+                </div>
+                <div class="panel-body">
+                    <div class="evidence-list" id="evidenceList">
+                        <div class="empty-state" id="evidenceEmptyState" style="padding:14px;">
+                            <div class="empty-text" style="font-size:0.72rem;">No evidence revealed yet.</div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <!-- Contradiction Alerts -->
+            <div class="panel">
+                <div class="panel-header">
+                    <span class="panel-title">⚡ Contradiction Alerts</span>
+                </div>
+                <div class="panel-body">
+                    <div class="alert-list" id="alertList">
+                        <div class="empty-state" style="padding:14px;">
+                            <div class="empty-text" style="font-size:0.72rem;">No contradictions detected yet.</div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            <!-- Reward History -->
+            <div class="panel" id="trainingPanel">
+                <div class="panel-header">
+                    <span class="panel-title">📈 Training Progress</span>
+                </div>
+                <div class="panel-body">
+                    <div class="training-metrics">
+                        <div class="training-metric">
+                            <div class="training-metric-label">Episodes</div>
+                            <div class="training-metric-value" id="trainEpisodes">-</div>
+                        </div>
+                        <div class="training-metric">
+                            <div class="training-metric-label">Model</div>
+                            <div class="training-metric-value" id="trainModel" style="font-size:0.72rem;">-</div>
+                        </div>
+                        <div class="training-metric">
+                            <div class="training-metric-label">First-50 Mean</div>
+                            <div class="training-metric-value" id="trainFirst">-</div>
+                        </div>
+                        <div class="training-metric">
+                            <div class="training-metric-label">Last-50 Mean</div>
+                            <div class="training-metric-value" id="trainLast">-</div>
+                        </div>
+                    </div>
+                    <div style="font-size:0.72rem; font-weight:800; margin-bottom:8px;" id="trainImprovement">Improvement: -</div>
+                    <div class="chart-container">
+                        <canvas class="chart-canvas" id="trainingRewardChart"></canvas>
+                    </div>
+                </div>
+            </div>
+
+            <div class="panel" id="historyPanel" style="display:none;">
+                <div class="panel-header">
+                    <span class="panel-title">📈 Reward History</span>
+                </div>
+                <div class="panel-body">
+                    <div class="chart-container">
+                        <canvas class="chart-canvas" id="rewardChart"></canvas>
+                    </div>
+                </div>
+            </div>
+
+        </div>
+    </div>
+
+    <!-- ── Outcome Overlay ─────────────────────────────── -->
+    <div class="outcome-overlay" id="outcomeOverlay">
+        <div class="outcome-card">
+            <div class="outcome-icon" id="outcomeIcon">🎉</div>
+            <div class="outcome-title" id="outcomeTitle">Case Solved!</div>
+            <div class="outcome-subtitle" id="outcomeSub">The detective correctly identified the criminal.</div>
+            <div class="outcome-stat win" id="outcomeStat">+8.50</div>
+            <button class="btn btn-primary" onclick="dismissOutcome()">Continue</button>
+        </div>
+    </div>
+
+</div>
+
+<script>
+
+const EVIDENCE_ICONS = {
+    keycard_log: "📇",
+    cctv_footage: "📹",
+    forensic_report: "🔬",
+};
+
+const EVIDENCE_NAMES = {
+    keycard_log: "Keycard Log",
+    cctv_footage: "CCTV Footage",
+    forensic_report: "Forensic Report",
+};
+
+// ── State ────────────────────────────────────────────────────────────────────
+
+let demoRunning = false;
+let currentTurn = 0;
+let evidenceCount = 0;
+let contradictionCount = 0;
+let totalReward = 0;
+let rewardHistory = [];
+let messageCount = 0;
+let revealedEvidenceKeys = new Set();
+let trainingRewards = [];
+let trainingSmoothed = [];
+
+const rewardComponents = {
+    base: 0,
+    turnCost: 0,
+    contradictions: 0,
+    evidence: 0,
+    total: 0,
+};
+
+// ── Rendering ────────────────────────────────────────────────────────────────
+
+function getAvatarClass(speaker) {
+    if (speaker === "Detective") return "detective";
+    if (speaker === "Suspect_A") return "suspect-a";
+    if (speaker === "Suspect_B") return "suspect-b";
+    if (speaker === "Witness_1") return "witness";
+    return "system";
+}
+
+function getAvatarEmoji(speaker) {
+    if (speaker === "Detective") return "🔍";
+    if (speaker === "Suspect_A") return "🅰️";
+    if (speaker === "Suspect_B") return "🅱️";
+    if (speaker === "Witness_1") return "👁️";
+    return "⚙️";
+}
+
+function addMessage(speaker, text, turn, tag) {
+    const log = document.getElementById("conversationLog");
+    // Remove empty state on first message
+    if (messageCount === 0) log.innerHTML = "";
+
+    const msgEl = document.createElement("div");
+    msgEl.className = "msg";
+
+    const cls = getAvatarClass(speaker);
+    let tagHTML = "";
+    if (tag) {
+        tagHTML = `<div class="msg-tag ${tag}">${tag}</div>`;
+    }
+
+    msgEl.innerHTML = `
+        <div class="msg-avatar ${cls}">${getAvatarEmoji(speaker)}</div>
+        <div class="msg-content">
+            <div class="msg-header">
+                <span class="msg-name ${cls}">${speaker}</span>
+                <span class="msg-turn">Turn ${turn}</span>
+            </div>
+            <div class="msg-text">${text}</div>
+            ${tagHTML}
+        </div>
+    `;
+
+    log.appendChild(msgEl);
+    log.scrollTop = log.scrollHeight;
+    messageCount++;
+    document.getElementById("msgCount").textContent = `${messageCount} messages`;
+}
+
+function revealEvidence(itemKey, nameOverride, descOverride) {
+    let keyNorm = itemKey.toLowerCase().replace(" ", "_");
+    // Normalize to standard keys if string varies slightly
+    if (keyNorm.includes("keycard")) keyNorm = "keycard_log";
+    if (keyNorm.includes("cctv")) keyNorm = "cctv_footage";
+    if (keyNorm.includes("forensic")) keyNorm = "forensic_report";
+
+    if (revealedEvidenceKeys.has(keyNorm)) return;
+    revealedEvidenceKeys.add(keyNorm);
+
+    // Always use the passed-in description (from backend trace or demo script)
+    const name = nameOverride || EVIDENCE_NAMES[keyNorm] || itemKey;
+    const description = descOverride || "";
+    const icon = EVIDENCE_ICONS[keyNorm] || "📄";
+
+    evidenceCount++;
+    document.getElementById("statEvidence").textContent = `${evidenceCount} / 3`;
+
+    // Update evidence list
+    const list = document.getElementById("evidenceList");
+    const emptyState = document.getElementById("evidenceEmptyState");
+    if (emptyState) emptyState.remove();
+    
+    const el = document.createElement("div");
+    el.className = "evidence-item revealed";
+    el.innerHTML = `
+        <span class="evidence-icon">${icon}</span>
+        <div class="evidence-info">
+            <div class="evidence-name">${name}</div>
+            <div class="evidence-desc">${description}</div>
+        </div>
+    `;
+    list.appendChild(el);
+
+    addMessage("System", `📋 Evidence revealed: <strong>${name}</strong> — ${description}`, currentTurn, "evidence");
+
+    // Reward — matches reward_calculator.py evidence_request_confirms_lead = +1.0
+    rewardComponents.evidence += 1.0;
+    updateReward();
+}
+
+function addContradiction(suspect, claim1, t1, claim2, t2) {
+    contradictionCount++;
+    document.getElementById("statContra").textContent = contradictionCount;
+
+    const list = document.getElementById("alertList");
+    if (contradictionCount === 1) list.innerHTML = "";
+
+    const el = document.createElement("div");
+    el.className = "alert-item";
+    el.innerHTML = `
+        <div class="alert-suspect">⚠ ${suspect} — Contradiction #${contradictionCount}</div>
+        <div class="alert-detail">"${claim1}" ↔ "${claim2}"</div>
+        <div class="alert-turns">Turns ${t1} → ${t2}</div>
+    `;
+    list.appendChild(el);
+
+    rewardComponents.contradictions += 2.0;
+    updateReward();
+}
+
+function updateReward() {
+    const total = rewardComponents.base + rewardComponents.turnCost + rewardComponents.contradictions + rewardComponents.evidence;
+    rewardComponents.total = total;
+    totalReward = total;
+
+    document.getElementById("statReward").textContent = total.toFixed(2);
+    const statEl = document.getElementById("statReward");
+    statEl.className = "stat-value " + (total >= 0 ? "positive" : "negative");
+
+    // Reward bar: map -10..+10 to 0..100%
+    const pct = Math.max(0, Math.min(100, ((total + 10) / 20) * 100));
+    document.getElementById("rewardBar").style.width = `${pct}%`;
+
+    document.getElementById("rwBase").textContent = rewardComponents.base.toFixed(2);
+    document.getElementById("rwTurnCost").textContent = rewardComponents.turnCost.toFixed(2);
+    document.getElementById("rwContra").textContent = `+${rewardComponents.contradictions.toFixed(2)}`;
+    document.getElementById("rwEvidence").textContent = `+${rewardComponents.evidence.toFixed(2)}`;
+    document.getElementById("rwTotal").textContent = total.toFixed(2);
+    document.getElementById("rwTotal").style.color = total >= 0 ? "var(--accent-green)" : "var(--accent-red)";
+}
+
+function updateTurn(t) {
+    currentTurn = t;
+    document.getElementById("statTurn").textContent = `${t} / 15`;
+    rewardComponents.turnCost = -(t * 0.3);
+    updateReward();
+}
+
+function showOutcome(correct) {
+    const overlay = document.getElementById("outcomeOverlay");
+    document.getElementById("outcomeIcon").textContent = correct ? "🎉" : "❌";
+    document.getElementById("outcomeTitle").textContent = correct ? "Case Solved!" : "Wrong Accusation";
+    document.getElementById("outcomeSub").textContent = correct
+        ? "The detective correctly identified the true criminal."
+        : "The detective accused the wrong suspect.";
+    document.getElementById("outcomeStat").textContent = (totalReward >= 0 ? "+" : "") + totalReward.toFixed(2);
+    document.getElementById("outcomeStat").className = "outcome-stat " + (correct ? "win" : "lose");
+    document.getElementById("statOutcome").textContent = correct ? "✅ Correct" : "❌ Wrong";
+    document.getElementById("statOutcome").className = "stat-value " + (correct ? "positive" : "negative");
+    overlay.classList.add("visible");
+}
+
+function dismissOutcome() {
+    document.getElementById("outcomeOverlay").classList.remove("visible");
+    document.getElementById("btnReset").style.display = "inline-flex";
+    updateRewardChart();
+}
+
+// ── Reward Chart ─────────────────────────────────────────────────────────────
+
+function updateRewardChart() {
+    rewardHistory.push(totalReward);
+    const panel = document.getElementById("historyPanel");
+    panel.style.display = "block";
+
+    const canvas = document.getElementById("rewardChart");
+    const ctx = canvas.getContext("2d");
+    const rect = canvas.parentElement.getBoundingClientRect();
+    canvas.width = rect.width * 2;
+    canvas.height = rect.height * 2;
+    ctx.scale(2, 2);
+
+    const w = rect.width;
+    const h = rect.height;
+    const data = rewardHistory;
+    
+    // Dynamic scale limits based on data
+    const maxVal = data.length > 0 ? Math.max(...data) : 0;
+    const minVal = data.length > 0 ? Math.min(...data) : 0;
+    const maxR = Math.max(10, Math.ceil(maxVal / 5) * 5 + 5);
+    const minR = Math.min(-10, Math.floor(minVal / 5) * 5 - 5);
+    const stepV = Math.max(5, Math.floor((maxR - minR) / 6));
+
+    const pad = { t: 15, b: 25, l: 40, r: 15 };
+
+    ctx.clearRect(0, 0, w, h);
+
+    // Grid and Labels
+    ctx.strokeStyle = "rgba(0,0,0,0.1)"; // Darker grid lines
+    ctx.lineWidth = 1;
+    ctx.fillStyle = "rgba(0,0,0,0.6)"; // Dark text instead of white
+    ctx.font = "11px 'JetBrains Mono'";
+    ctx.textAlign = "right";
+
+    for (let v = minR; v <= maxR; v += stepV) {
+        const y = pad.t + ((maxR - v) / (maxR - minR)) * (h - pad.t - pad.b);
+        ctx.beginPath();
+        ctx.moveTo(pad.l, y);
+        ctx.lineTo(w - pad.r, y);
+        ctx.stroke();
+
+        ctx.fillText(v.toString(), pad.l - 8, y + 4);
+    }
+
+    // Zero line
+    const zeroY = pad.t + ((maxR - 0) / (maxR - minR)) * (h - pad.t - pad.b);
+    ctx.strokeStyle = "rgba(0,0,0,0.2)"; // Darker zero line
+    ctx.beginPath();
+    ctx.moveTo(pad.l, zeroY);
+    ctx.lineTo(w - pad.r, zeroY);
+    ctx.stroke();
+
+    if (data.length < 2) return;
+
+    // Line
+    const stepX = (w - pad.l - pad.r) / (Math.max(data.length - 1, 1));
+    ctx.beginPath();
+    data.forEach((v, i) => {
+        const x = pad.l + i * stepX;
+        const y = pad.t + ((maxR - v) / (maxR - minR)) * (h - pad.t - pad.b);
+        i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
+    });
+    ctx.strokeStyle = "#3b82f6";
+    ctx.lineWidth = 2;
+    ctx.stroke();
+
+    // Gradient fill
+    const grad = ctx.createLinearGradient(0, 0, 0, h);
+    grad.addColorStop(0, "rgba(59, 130, 246, 0.2)");
+    grad.addColorStop(1, "rgba(59, 130, 246, 0)");
+    ctx.lineTo(pad.l + (data.length - 1) * stepX, h - pad.b);
+    ctx.lineTo(pad.l, h - pad.b);
+    ctx.closePath();
+    ctx.fillStyle = grad;
+    ctx.fill();
+
+    // Dots
+    data.forEach((v, i) => {
+        const x = pad.l + i * stepX;
+        const y = pad.t + ((maxR - v) / (maxR - minR)) * (h - pad.t - pad.b);
+        ctx.beginPath();
+        ctx.arc(x, y, 4, 0, Math.PI * 2);
+        ctx.fillStyle = v >= 0 ? "#10b981" : "#ef4444";
+        ctx.fill();
+        ctx.strokeStyle = "#ffffff"; // White border inside the dot
+        ctx.lineWidth = 2;
+        ctx.stroke();
+    });
+
+    // Episode labels
+    ctx.fillStyle = "rgba(0,0,0,0.5)"; // Darker episode text
+    ctx.font = "9px 'JetBrains Mono'";
+    ctx.textAlign = "center";
+    data.forEach((_, i) => {
+        ctx.fillText(`E${i + 1}`, pad.l + i * stepX, h - 4);
+    });
+}
+
+function drawTrainingChart(rewards, smoothed) {
+    const canvas = document.getElementById("trainingRewardChart");
+    const ctx = canvas.getContext("2d");
+    const rect = canvas.parentElement.getBoundingClientRect();
+    canvas.width = rect.width * 2;
+    canvas.height = rect.height * 2;
+    ctx.scale(2, 2);
+
+    const w = rect.width;
+    const h = rect.height;
+    const pad = { t: 15, b: 25, l: 40, r: 10 };
+    const data = rewards || [];
+    const avg = smoothed || [];
+
+    ctx.clearRect(0, 0, w, h);
+
+    if (data.length === 0) {
+        ctx.fillStyle = "rgba(0,0,0,0.6)";
+        ctx.font = "12px 'JetBrains Mono'";
+        ctx.fillText("No training data found yet.", 12, 24);
+        return;
+    }
+
+    const maxVal = Math.max(...data, ...(avg.length ? avg : data), 10);
+    const minVal = Math.min(...data, ...(avg.length ? avg : data), -10);
+    const span = Math.max(1, maxVal - minVal);
+
+    ctx.strokeStyle = "rgba(0,0,0,0.12)";
+    ctx.lineWidth = 1;
+    for (let i = 0; i <= 5; i++) {
+        const y = pad.t + (i / 5) * (h - pad.t - pad.b);
+        ctx.beginPath();
+        ctx.moveTo(pad.l, y);
+        ctx.lineTo(w - pad.r, y);
+        ctx.stroke();
+    }
+
+    const valueToY = (v) => pad.t + ((maxVal - v) / span) * (h - pad.t - pad.b);
+    const xStep = (w - pad.l - pad.r) / Math.max(1, data.length - 1);
+
+    ctx.beginPath();
+    data.forEach((v, i) => {
+        const x = pad.l + i * xStep;
+        const y = valueToY(v);
+        i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
+    });
+    ctx.strokeStyle = "#65A3BB";
+    ctx.lineWidth = 2;
+    ctx.stroke();
+
+    if (avg.length === data.length && avg.length > 1) {
+        ctx.beginPath();
+        avg.forEach((v, i) => {
+            const x = pad.l + i * xStep;
+            const y = valueToY(v);
+            i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
+        });
+        ctx.strokeStyle = "#F25D68";
+        ctx.lineWidth = 2.5;
+        ctx.stroke();
+    }
+}
+
+async function loadTrainingProgress() {
+    try {
+        const res = await fetch("/api/reward_curve");
+        if (!res.ok) return;
+        const data = await res.json();
+        trainingRewards = data.rewards || [];
+        trainingSmoothed = data.smoothed || [];
+
+        document.getElementById("trainEpisodes").textContent = String(data.num_episodes || trainingRewards.length || 0);
+        document.getElementById("trainModel").textContent = (data.model || "-").replace("Qwen/", "");
+        document.getElementById("trainFirst").textContent = Number(data.mean_first_50 || 0).toFixed(2);
+        document.getElementById("trainLast").textContent = Number(data.mean_last_50 || 0).toFixed(2);
+
+        const improvement = Number(data.improvement || 0);
+        const sign = improvement >= 0 ? "+" : "";
+        const improveEl = document.getElementById("trainImprovement");
+        improveEl.textContent = `Improvement: ${sign}${improvement.toFixed(2)}`;
+        improveEl.style.color = improvement >= 0 ? "#16a34a" : "#dc2626";
+
+        drawTrainingChart(trainingRewards, trainingSmoothed);
+    } catch (err) {
+        // Keep dashboard usable even if training artifacts are missing.
+    }
+}
+
+
+function resetBoard() {
+    currentTurn = 0;
+    evidenceCount = 0;
+    contradictionCount = 0;
+    totalReward = 0;
+    messageCount = 0;
+    revealedEvidenceKeys.clear();
+    if (window.revealedContradictionKeys) window.revealedContradictionKeys.clear();
+    Object.keys(rewardComponents).forEach(k => rewardComponents[k] = 0);
+
+    document.getElementById("statTurn").textContent = "0 / 15";
+    document.getElementById("statEvidence").textContent = "0 / 3";
+    document.getElementById("statContra").textContent = "0";
+    document.getElementById("statReward").textContent = "0.00";
+    document.getElementById("statReward").className = "stat-value";
+    document.getElementById("statOutcome").textContent = "—";
+    document.getElementById("statOutcome").className = "stat-value neutral";
+    document.getElementById("rewardBar").style.width = "50%";
+    document.getElementById("msgCount").textContent = "0 messages";
+
+    document.getElementById("conversationLog").innerHTML = `
+        <div class="empty-state">
+            <div class="empty-icon">🕵️</div>
+            <div class="empty-text">Click <strong>Run Live Server</strong> to watch a full<br>investigation episode unfold live.</div>
+        </div>`;
+
+    document.getElementById("evidenceList").innerHTML = `
+        <div class="empty-state" id="evidenceEmptyState" style="padding:14px;">
+            <div class="empty-text" style="font-size:0.72rem;">No evidence revealed yet.</div>
+        </div>`;
+
+    document.getElementById("alertList").innerHTML = `
+        <div class="empty-state" style="padding:14px;">
+            <div class="empty-text" style="font-size:0.72rem;">No contradictions detected yet.</div>
+        </div>`;
+
+    document.getElementById("btnReset").style.display = "none";
+    document.getElementById("outcomeOverlay").classList.remove("visible");
+
+    updateReward();
+}
+
+function sleep(ms) { return new Promise(r => setTimeout(r, ms)); }
+
+// ── Live Backend Runner ──────────────────────────────────────────────────────
+
+async function runLiveAgent() {
+    if (demoRunning) return;
+    demoRunning = true;
+
+    resetBoard();
+
+    document.getElementById("btnRunLive").disabled = true;
+    document.getElementById("btnRunLive").textContent = "⏳ Waiting for AI...";
+    document.getElementById("statusText").textContent = "Live Investigating";
+    document.getElementById("statusBadge").style.borderColor = "rgba(16, 185, 129, 0.4)";
+    document.getElementById("statusBadge").style.background = "rgba(16, 185, 129, 0.1)";
+    document.getElementById("statusBadge").querySelector(".status-dot").style.background = "var(--accent-green)";
+    document.getElementById("statusBadge").querySelector("span:last-child").style.color = "var(--accent-green)";
+
+    const epNum = rewardHistory.length + 1;
+    document.getElementById("episodeLabel").textContent = `Episode ${epNum}`;
+
+    const DELAY_SHORT = 600;
+    const DELAY_LONG = 1200;
+
+    try {
+        const response = await fetch("/api/run_episode");
+        if (!response.ok) throw new Error("Server not responding");
+        const data = await response.json();
+        
+        if (data.status !== "ok") {
+            throw new Error("Simulation failed");
+        }
+        
+        const script = data.trace;
+        document.getElementById("btnRunLive").textContent = "⚙️ Executing Trace...";
+
+        for (const event of script) {
+            if (!demoRunning) break;
+            
+            if (event.type === "detective") {
+                updateTurn(event.turn);
+                
+                const actionDesc = event.action.includes("ask") 
+                    ? `Asking ${event.target}: "${event.text}"`
+                    : event.action.includes("evid")
+                    ? `Requesting evidence: ${event.item}`
+                    : `Accusing ${event.target}`;
+                addMessage("Detective", actionDesc, event.turn);
+                
+                // turnCost is tracked by updateTurn() already (-0.3 * turn)
+                // Don't touch rewardComponents.base here — we derive it at the end
+                await sleep(DELAY_LONG);
+
+            } else if (event.type === "response") {
+                addMessage(event.speaker, event.text, event.turn);
+                await sleep(DELAY_SHORT);
+
+            } else if (event.type === "contradiction") {
+                addContradiction(event.suspect, event.claim1 || "Previous claim", event.claim1Turn || "?", event.claim2 || ("New claim: " + event.detail), event.turn);
+                // addContradiction() already adds +2.0 to rewardComponents.contradictions
+                // No extra shifting needed since we derive base at the end
+                await sleep(DELAY_SHORT);
+
+            } else if (event.type === "evidence") {
+                revealEvidence(event.item, event.item, event.desc);
+                // revealEvidence() already adds +1.0 to rewardComponents.evidence
+                // No extra shifting needed since we derive base at the end
+                await sleep(DELAY_SHORT);
+
+            } else if (event.type === "outcome") {
+                // Now derive the base component from the true server total
+                // base = serverTotal - turnCost - contradictions - evidence
+                let serverTotal = data.rewards.detective;
+                rewardComponents.base = serverTotal - rewardComponents.turnCost - rewardComponents.contradictions - rewardComponents.evidence;
+                totalReward = serverTotal;
+                updateReward();
+                
+                await sleep(800);
+                showOutcome(event.correct || false);
+            }
+        }
+        
+        // Final safeguard: force-set everything to server truth
+        totalReward = data.rewards.detective;
+        rewardComponents.base = totalReward - rewardComponents.turnCost - rewardComponents.contradictions - rewardComponents.evidence;
+        rewardComponents.total = totalReward;
+        updateReward();
+        document.getElementById("statReward").textContent = totalReward.toFixed(2);
+        
+    } catch (e) {
+        addMessage("System", `⚠️ Error connecting to live server: ${e.message}. Note: you must run 'uvicorn server.app:app' and open the app from localhost to use Live Mode!`, currentTurn || 0);
+    }
+    
+    document.getElementById("statusText").textContent = "Complete";
+    document.getElementById("statusBadge").style.borderColor = "rgba(16, 185, 129, 0.25)";
+    document.getElementById("btnRunLive").disabled = false;
+    document.getElementById("btnRunLive").textContent = "🚀 Run Live Server";
+    demoRunning = false;
+}
+
+window.addEventListener("load", () => {
+    loadTrainingProgress();
+});
+</script>
+</body>
+</html>

hf_mini_blog_draft.md

@@ -0,0 +1,57 @@
+# AI Crime Investigation World: Multi-Agent RL Detective (Hackathon Mini-Blog Draft)
+
+## 1) What This Project Solves
+AI Crime Investigation World is a multi-agent reinforcement learning environment where a detective agent must interrogate two suspects and one witness, request evidence, detect contradictions, and make a final accusation within 15 turns.
+
+This is designed as an OpenEnv-compatible benchmark for evaluating agent coordination under partial information.
+
+## 2) Why It Matches Halluminate
+This project maps directly to the Halluminate sub-theme: one controller agent (detective) manages multiple actors with conflicting incentives to accomplish a single objective (correct accusation).
+
+The detective must combine statements from multiple actors, resolve contradictions, and align decisions with evidence, not just single-agent chat quality.
+
+## 3) Environment Design
+- Actors: Detective, Suspect_A, Suspect_B, Witness_1
+- Observation: briefing, conversation history, evidence log, current turn
+- Actions:
+  - ACTION: ask_question | TARGET: <agent> | CONTENT: <question>
+  - ACTION: request_evidence | ITEM: <item>
+  - ACTION: accuse | TARGET: <suspect>
+- Episode limit: 15 turns
+
+## 4) Reward Logic
+Detective rewards are event-based and interpretable:
+- Correct accusation: +10
+- Wrong accusation: -8 (up to -10 with active witness bias penalty)
+- Timeout: -3
+- Contradiction exposed: +2
+- Evidence confirms lead: +1
+- Deflection resistance: +0.5
+- Redundant question: -0.5
+- Per-turn cost: -0.3 each turn
+
+## 5) Training Setup
+- Algorithm: PPO via Hugging Face TRL
+- Model: Qwen2.5 Instruct family (small variants for smoke tests)
+- Artifacts:
+  - rewards.json
+  - reward_curve.png
+  - episode_transcripts.json (episodes 1/25/50)
+
+## 6) Observable Progress
+The dashboard and API expose training progress:
+- /api/reward_curve returns raw rewards, smoothed trend, and summary metrics.
+- Dashboard shows training reward curve and improvement metrics.
+- Before/after transcripts from milestone episodes provide qualitative behavior evidence.
+
+## 7) Demo Checklist for Judges
+1. Open Space root dashboard.
+2. Verify /api/health returns status ok.
+3. Verify /api/reward_curve returns reward list and improvement.
+4. Run /api/run_episode to view live interaction trace.
+5. Inspect transcript snapshots (episode_transcripts.json) for behavior shift.
+
+## 8) Limitations and Next Steps
+- Early training can be noisy due to sparse terminal signal.
+- Runtime and VRAM constraints require small models for quick demos.
+- Next improvements: stronger contradiction extraction, curriculum schedule, and larger-model fine-tuning with checkpoint comparison.

requirements-train.txt

@@ -0,0 +1,9 @@
+transformers>=4.40.0
+# This training script uses the legacy TRL PPOTrainer.step API.
+trl>=0.9.0,<0.10.0
+torch
+matplotlib
+numpy
+bitsandbytes>=0.43.0
+accelerate>=0.27.0
+peft>=0.9.0

requirements.txt

@@ -0,0 +1,4 @@
+openenv-core
+fastapi>=0.110.0
+uvicorn>=0.27.0
+pydantic>=2.0.0

run_hf_smoke_test.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Lightweight Hugging Face smoke test for PPO loop validation.
+export TEST_MODE=1
+export NUM_EPISODES="${NUM_EPISODES:-5}"
+export MAX_TURNS="${MAX_TURNS:-8}"
+export MODEL_NAME="${MODEL_NAME:-Qwen/Qwen2.5-1.5B-Instruct}"
+export NPC_MODEL_NAME="${NPC_MODEL_NAME:-Qwen/Qwen2.5-0.5B-Instruct}"
+export NPC_MAX_NEW_TOKENS="${NPC_MAX_NEW_TOKENS:-48}"
+export DETECTIVE_MAX_NEW_TOKENS="${DETECTIVE_MAX_NEW_TOKENS:-40}"
+export DETECTIVE_RETRY_MAX_NEW_TOKENS="${DETECTIVE_RETRY_MAX_NEW_TOKENS:-24}"
+
+python train_colab.py

server/app.py

@@ -0,0 +1,289 @@
+"""OpenEnv-compatible server for the Crime Investigation Environment.
+
+Exposes the CrimeInvestigationEnv via OpenEnv's HTTP/WebSocket interface.
+
+Usage:
+    # Development (with auto-reload):
+    uvicorn server.app:app --reload --host 0.0.0.0 --port 8000
+
+    # Or run directly:
+    python -m server.app
+"""
+
+import os
+import sys
+import json
+import base64
+from typing import Any, Dict, List, Optional
+
+from pydantic import Field
+
+# Add project root to path so crime_env is importable
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from fastapi.responses import HTMLResponse
+
+from openenv.core.env_server.http_server import create_app
+from openenv.core.env_server.interfaces import Environment
+from openenv.core.env_server.types import (
+    Action,
+    EnvironmentMetadata,
+    Observation,
+    State,
+)
+
+from crime_env.environment import CrimeInvestigationEnv
+
+
+# ── Pydantic types for OpenEnv ──────────────────────────────────────────────
+
+
+class CrimeAction(Action):
+    """Action schema for the Crime Investigation environment."""
+
+    action_string: str = Field(
+        ...,
+        description=(
+            "Action in one of the following formats:\n"
+            "  ACTION: ask_question | TARGET: <agent> | CONTENT: <question>\n"
+            "  ACTION: request_evidence | ITEM: <item>\n"
+            "  ACTION: accuse | TARGET: <suspect>"
+        ),
+        examples=[
+            "ACTION: ask_question | TARGET: Suspect_A | CONTENT: Where were you?",
+            "ACTION: request_evidence | ITEM: keycard_log",
+            "ACTION: accuse | TARGET: Suspect_A",
+        ],
+    )
+
+
+class CrimeObservation(Observation):
+    """Observation schema returned by the Crime Investigation environment."""
+
+    role: str = Field(default="detective", description="Agent role")
+    briefing: str = Field(default="", description="Case briefing for the detective")
+    turn: int = Field(default=0, description="Current turn number")
+    conversation_history: List[Dict[str, Any]] = Field(
+        default_factory=list, description="Full conversation history"
+    )
+    evidence_log: List[Dict[str, Any]] = Field(
+        default_factory=list, description="Revealed evidence items"
+    )
+    message: str = Field(default="", description="System message for the current step")
+
+
+class CrimeState(State):
+    """State schema for the Crime Investigation environment."""
+
+    turn: int = Field(default=0, description="Current turn number")
+    is_done: bool = Field(default=False, description="Whether the episode is over")
+    max_turns: int = Field(default=15, description="Maximum turns per episode")
+    evidence_revealed: int = Field(default=0, description="Number of evidence items revealed")
+    contradictions_found: int = Field(default=0, description="Number of contradictions detected")
+
+
+# ── OpenEnv-compatible wrapper ──────────────────────────────────────────────
+
+
+class CrimeInvestigationOpenEnv(Environment[CrimeAction, CrimeObservation, CrimeState]):
+    """OpenEnv wrapper around CrimeInvestigationEnv."""
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self._env = CrimeInvestigationEnv()
+        self._current_obs: Optional[dict] = None
+
+    def reset(
+        self,
+        seed: Optional[int] = None,
+        episode_id: Optional[str] = None,
+        **kwargs,
+    ) -> CrimeObservation:
+        if hasattr(self, "_reset_rubric"):
+            self._reset_rubric()
+        obs = self._env.reset()
+        self._current_obs = obs
+        return CrimeObservation(
+            role=obs.get("role", "detective"),
+            briefing=obs.get("briefing", ""),
+            turn=obs.get("turn", 0),
+            conversation_history=obs.get("conversation_history", []),
+            evidence_log=obs.get("evidence_log", []),
+            message=obs.get("message", ""),
+            done=False,
+            reward=None,
+        )
+
+    def step(
+        self,
+        action: CrimeAction,
+        timeout_s: Optional[float] = None,
+        **kwargs,
+    ) -> CrimeObservation:
+        obs_dict, reward, done, info = self._env.step(action.action_string)
+        self._current_obs = obs_dict
+        return CrimeObservation(
+            role=obs_dict.get("role", "detective"),
+            briefing=obs_dict.get("briefing", ""),
+            turn=obs_dict.get("turn", 0),
+            conversation_history=obs_dict.get("conversation_history", []),
+            evidence_log=obs_dict.get("evidence_log", []),
+            message=obs_dict.get("message", ""),
+            done=done,
+            reward=reward,
+        )
+
+    @property
+    def state(self) -> CrimeState:
+        env_state = self._env.state()
+        return CrimeState(
+            turn=env_state.get("turn", 0),
+            is_done=env_state.get("done", False),
+            max_turns=env_state.get("max_turns", 15),
+            evidence_revealed=env_state.get("evidence_revealed", 0),
+            contradictions_found=env_state.get("contradictions_found", 0),
+        )
+
+    def get_metadata(self) -> EnvironmentMetadata:
+        return EnvironmentMetadata(
+            name="CrimeInvestigationEnv",
+            description=(
+                "AI Crime Investigation World — a multi-agent RL environment "
+                "where a detective agent interrogates suspects and a witness, "
+                "reviews evidence, and makes an accusation."
+            ),
+            version="1.0.0",
+        )
+
+    def close(self) -> None:
+        pass
+
+
+# ── App creation ────────────────────────────────────────────────────────────
+
+app = create_app(
+    CrimeInvestigationOpenEnv,
+    CrimeAction,
+    CrimeObservation,
+    env_name="crime_investigation",
+    max_concurrent_envs=1,
+)
+
+# ── Custom Endpoints for Dashboard ──────────────────────────────────────────
+
+@app.get("/", response_class=HTMLResponse)
+async def serve_dashboard():
+    """Serves the dashboard.html interface."""
+    script_dir = os.path.dirname(os.path.abspath(__file__))
+    project_root = os.path.dirname(script_dir)
+    dashboard_path = os.path.join(project_root, "dashboard.html")
+    with open(dashboard_path, "r", encoding="utf-8") as f:
+        return f.read()
+
+@app.get("/api/run_episode")
+async def run_episode_api():
+    """Runs a single test episode and returns the trace.
+    
+    Import is lazy (Issue 9) and execution is offloaded to a thread
+    so the FastAPI event loop isn't blocked (Issue 11).
+    """
+    import asyncio
+    from test_one_episode import run_test_episode
+
+    rewards, info, trace = await asyncio.to_thread(run_test_episode)
+    return {
+        "status": "ok",
+        "rewards": rewards,
+        "info": info,
+        "trace": trace
+    }
+
+
+def _moving_average(values: List[float], window: int) -> List[float]:
+    if not values:
+        return []
+    if window <= 1:
+        return values[:]
+    averaged: List[float] = []
+    running_sum = 0.0
+    queue: List[float] = []
+    for v in values:
+        queue.append(float(v))
+        running_sum += float(v)
+        if len(queue) > window:
+            running_sum -= queue.pop(0)
+        averaged.append(running_sum / len(queue))
+    return averaged
+
+
+@app.get("/api/reward_curve")
+async def reward_curve_api():
+    """Return training reward history for dashboard/HF demo visibility."""
+    script_dir = os.path.dirname(os.path.abspath(__file__))
+    project_root = os.path.dirname(script_dir)
+    rewards_path = os.path.join(project_root, "rewards.json")
+    reward_curve_path = os.path.join(project_root, "reward_curve.png")
+
+    rewards: List[float] = []
+    results: List[str] = []
+    model_name = "unknown"
+    num_episodes = 0
+
+    if os.path.exists(rewards_path):
+        with open(rewards_path, "r", encoding="utf-8") as f:
+            payload = json.load(f)
+        rewards = [float(x) for x in payload.get("rewards", [])]
+        results = [str(x) for x in payload.get("results", [])]
+        model_name = str(payload.get("model", "unknown"))
+        num_episodes = int(payload.get("num_episodes", len(rewards)))
+
+    window = min(20, max(1, len(rewards) // 4))
+    smoothed = _moving_average(rewards, window)
+    mean_first = sum(rewards[:50]) / max(1, min(50, len(rewards))) if rewards else 0.0
+    mean_last = sum(rewards[-50:]) / max(1, min(50, len(rewards))) if rewards else 0.0
+
+    image_data_url = None
+    if os.path.exists(reward_curve_path):
+        with open(reward_curve_path, "rb") as f:
+            encoded = base64.b64encode(f.read()).decode("ascii")
+        image_data_url = f"data:image/png;base64,{encoded}"
+
+    return {
+        "status": "ok",
+        "model": model_name,
+        "num_episodes": num_episodes,
+        "rewards": rewards,
+        "smoothed": smoothed,
+        "smooth_window": window,
+        "mean_first_50": round(mean_first, 4),
+        "mean_last_50": round(mean_last, 4),
+        "improvement": round(mean_last - mean_first, 4),
+        "results": {
+            "correct": results.count("correct"),
+            "wrong": results.count("wrong"),
+            "timeout": results.count("timeout"),
+        },
+        "image_data_url": image_data_url,
+    }
+
+
+@app.get("/api/health")
+async def health_api():
+    """Simple endpoint used for deployment sanity checks."""
+    return {"status": "ok", "service": "crime-investigation"}
+
+
+def main(host: str = "0.0.0.0", port: int = 8000):
+    """Entry point for direct execution."""
+    import uvicorn
+
+    uvicorn.run(app, host=host, port=port)
+
+
+if __name__ == "__main__":
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--port", type=int, default=8000)
+    args = parser.parse_args()
+    main(port=args.port)

test_one_episode.py

@@ -0,0 +1,473 @@
+#!/usr/bin/env python3
+"""Test One Episode — End-to-End Verification.
+
+Runs a single episode of the Crime Investigation environment with all agents
+using enhanced rule-based fallback responses (per-agent turn counters).
+Prints the full conversation, reward breakdown, and accusation result.
+"""
+
+import os
+import sys
+import random
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from crime_env.case_generator import generate_case
+from crime_env.consistency_tracker import ConsistencyTracker
+from crime_env.reward_calculator import RewardCalculator
+from crime_env.agent_prompts import build_system_prompt
+from crime_env.environment import CrimeInvestigationEnv
+from crime_env.constants import (
+    AGENT_NAME_TO_KEY,
+    SUSPECT_A,
+    SUSPECT_B,
+    WITNESS_1,
+    WITNESS_1_KEY,
+)
+
+
+# ── Enhanced Rule-Based Detective ───────────────────────────────────────────
+
+
+def scripted_detective_actions(case: dict) -> list[str]:
+    """Generate a scripted sequence of detective actions for testing.
+
+    This produces a realistic investigation sequence that exercises
+    all action types: ask_question, request_evidence, accuse.
+    """
+    accuse_target = choose_non_oracle_accusation(case)
+
+    actions = [
+        # Turn 0: Question Suspect A about alibi
+        f"ACTION: ask_question | TARGET: {SUSPECT_A} | CONTENT: Where were you at the time of the crime?",
+        # Turn 1: Question Suspect B about alibi
+        f"ACTION: ask_question | TARGET: {SUSPECT_B} | CONTENT: Can you tell me your whereabouts that evening?",
+        # Turn 2: Question Witness
+        f"ACTION: ask_question | TARGET: {WITNESS_1} | CONTENT: What did you see near the crime scene?",
+        # Turn 3: Request evidence
+        "ACTION: request_evidence | ITEM: keycard_log",
+        # Turn 4: Follow up with Suspect A about inconsistency
+        f"ACTION: ask_question | TARGET: {SUSPECT_A} | CONTENT: Your prior record shows you've lied about your location before. Can you explain your alibi in more detail?",
+        # Turn 5: Request more evidence
+        "ACTION: request_evidence | ITEM: cctv_footage",
+        # Turn 6: Question Suspect A about clothing
+        f"ACTION: ask_question | TARGET: {SUSPECT_A} | CONTENT: What were you wearing that evening?",
+        # Turn 7: Question Witness about clothing
+        f"ACTION: ask_question | TARGET: {WITNESS_1} | CONTENT: Can you describe what the person you saw was wearing?",
+        # Turn 8: Ask Suspect B if they know Suspect A
+        f"ACTION: ask_question | TARGET: {SUSPECT_B} | CONTENT: Do you know {SUSPECT_A}? Have you had any previous interactions?",
+        # Turn 9: Request forensic report
+        "ACTION: request_evidence | ITEM: forensic_report",
+        # Turn 10: Final question before accusation
+        f"ACTION: ask_question | TARGET: {SUSPECT_A} | CONTENT: The evidence places someone at the scene. Was that you?",
+        # Turn 11: Accuse
+        f"ACTION: accuse | TARGET: {accuse_target}",
+    ]
+
+    return actions
+
+
+def choose_non_oracle_accusation(case: dict) -> str:
+    """Pick an accusation target using only case-visible investigative signals.
+
+    This intentionally avoids direct use of the hidden `case['criminal']`
+    so dashboard episodes can produce both correct and wrong outcomes.
+    """
+    scores = {SUSPECT_A: 0.0, SUSPECT_B: 0.0}
+
+    # Evidence contributes most to suspicion.
+    for evidence in case.get("physical_evidence", []):
+        points_to = evidence.get("points_to")
+        if points_to in scores:
+            scores[points_to] += 1.75
+
+    # Prior history is suggestive, but weaker than direct evidence.
+    for suspect in (SUSPECT_A, SUSPECT_B):
+        history = case.get("prior_history", {}).get(suspect, {})
+        scores[suspect] += 0.25 * len(history.get("past_lies_on_record", []))
+        if history.get("on_parole"):
+            scores[suspect] += 0.15
+        trust = history.get("trust_score", 0.5)
+        scores[suspect] += max(0.0, 0.6 - trust)
+
+    # Witness bias can mislead investigators in realistic episodes.
+    witness = case.get("agent_knowledge", {}).get("witness_1", {})
+    bias_target = witness.get("bias_target")
+    bias_strength = float(witness.get("bias_strength", 0.0) or 0.0)
+    if bias_target in scores:
+        scores[bias_target] += 0.6 * bias_strength
+
+    # If confidence is low, introduce uncertainty to allow wrong accusations.
+    ordered = sorted(scores.items(), key=lambda kv: kv[1], reverse=True)
+    top_target, top_score = ordered[0]
+    second_target, second_score = ordered[1]
+    margin = top_score - second_score
+    if margin < 0.45 and random.random() < 0.45:
+        return second_target
+
+    return top_target
+
+
+# ── Enhanced NPC Responses ──────────────────────────────────────────────────
+
+
+def make_enhanced_npc_call(case: dict):
+    """Create an NPC response function with per-agent turn counters.
+
+    Each agent has its own turn counter, so their response progression
+    is independent — Suspect_A and Suspect_B will never give the same
+    response at the same time.
+    """
+    # Per-agent turn counters — each agent tracks their own call count
+    agent_counters: dict[str, int] = {}
+
+    def npc_call(system_prompt: str, conversation_history: list[dict]) -> str:
+        # Identify agent from system prompt — works with either suspect as criminal
+        is_guilty = "FAKE ALIBI:" in system_prompt
+        is_innocent = "innocent" in system_prompt
+        is_witness = "Witness_1" in system_prompt
+
+        if is_guilty:
+            agent_id = "guilty_suspect"
+        elif is_innocent:
+            agent_id = "innocent_suspect"
+        elif is_witness:
+            agent_id = "witness"
+        else:
+            return "I don't have anything to add."
+
+        # Increment per-agent counter
+        call_num = agent_counters.get(agent_id, 0)
+        agent_counters[agent_id] = call_num + 1
+
+        if agent_id == "guilty_suspect":
+            criminal_name = case["criminal"]
+            knowledge = case["agent_knowledge"][AGENT_NAME_TO_KEY[criminal_name]]
+            fake_alibi = knowledge["fake_alibi"]
+
+            responses = [
+                # Call 0: Initial alibi statement — uses case-specific fake alibi
+                f"{fake_alibi}. I can provide witnesses if needed.",
+                # Call 1: Deflects the prior-record question
+                "Look, I've made mistakes in the past, but I've been clean. "
+                "I was wearing my grey trench coat, as I usually do in the evenings.",
+                # Call 2: Deflection toward Suspect_B
+                "I don't know anything about what happened. I was nowhere near the "
+                "scene. Maybe you should look at Suspect_B instead.",
+                # Call 3: Getting nervous, starts slipping — changes location claim
+                "I was at the bar that evening. I might have had the times mixed up "
+                "earlier, but I was definitely not at the crime scene.",
+                # Call 4+: Further inconsistency under pressure
+                "I already told you everything I know. I was at home all evening. "
+                "You're wasting your time with me.",
+            ]
+
+        elif agent_id == "innocent_suspect":
+            innocent_name = SUSPECT_B if case["criminal"] == SUSPECT_A else SUSPECT_A
+            knowledge = case["agent_knowledge"][AGENT_NAME_TO_KEY[innocent_name]]
+            real_alibi = knowledge["real_alibi"]
+            knows_other = knowledge.get("knows_suspect_A", knowledge.get("knows_suspect_B", False))
+
+            responses = [
+                # Call 0: Clear, specific, verifiable alibi
+                f"{real_alibi}. I can prove it if you need documentation.",
+                # Call 1: Relationship with Suspect_A
+                (
+                    "I know the other suspect from work, but we're not close. "
+                    "I haven't seen them recently."
+                    if knows_other
+                    else "I do not know the other suspect personally. "
+                    "I've never had any interactions with them."
+                ),
+                # Call 2: Confirms alibi again, firmly
+                f"As I already said, {real_alibi}. "
+                "I had absolutely nothing to do with this crime.",
+                # Call 3+: Cooperating but frustrated
+                f"I've been completely honest with you. {real_alibi}. "
+                "Please check my receipts and timestamps — everything lines up.",
+            ]
+
+        # ── Witness ─────────────────────────────────────────────────
+        else:
+            knowledge = case["agent_knowledge"][WITNESS_1_KEY]
+            saw = knowledge["saw"]
+            time_seen = knowledge["time_seen"]
+            bias = knowledge.get("bias_target", None)
+
+            responses = [
+                # Call 0: Primary testimony — case-specific observation
+                f"I saw {saw}. It was around {time_seen}. "
+                "They seemed tense and were looking over their shoulder.",
+                # Call 1: Follow-up — more detail about what they saw
+                f"The person I saw was near the {case['location']} at {time_seen}. "
+                "They were moving quickly and seemed like they didn't want to be noticed.",
+                # Call 2+: Additional detail or bias
+                (
+                    f"Now that I think about it, their build reminded me of {bias}, "
+                    "but I can't be completely certain."
+                    if bias
+                    else f"I only saw them briefly near the {case['location']}. "
+                    "I wish I could tell you more, but that's all I observed."
+                ),
+            ]
+
+        idx = min(call_num, len(responses) - 1)
+        return responses[idx]
+
+    return npc_call
+
+
+# ── Main ────────────────────────────────────────────────────────────────────
+
+
+def run_test_episode():
+    """Run a single test episode and print full results."""
+    print("=" * 70)
+    print("  AI CRIME INVESTIGATION WORLD — TEST EPISODE")
+    print("=" * 70)
+
+    # Generate case
+    case = generate_case()
+
+    print(f"\n📋 CASE DETAILS (ground truth — not visible to detective):")
+    print(f"   Crime:    {case['crime']}")
+    print(f"   Criminal: {case['criminal']}")
+    print(f"   Location: {case['location']}")
+    print(f"   Time:     {case['time']}")
+    print(f"   Method:   {case['method']}")
+    criminal = case["criminal"]
+    innocent = SUSPECT_B if criminal == SUSPECT_A else SUSPECT_A
+    print(f"\n📝 {criminal} FAKE ALIBI: {case['agent_knowledge'][AGENT_NAME_TO_KEY[criminal]]['fake_alibi']}")
+    print(f"📝 {innocent} REAL ALIBI: {case['agent_knowledge'][AGENT_NAME_TO_KEY[innocent]]['real_alibi']}")
+    print(f"👁️ WITNESS SAW: {case['agent_knowledge'][WITNESS_1_KEY]['saw']}")
+    print()
+
+    # Create environment with enhanced NPC
+    npc_call = make_enhanced_npc_call(case)
+    env = CrimeInvestigationEnv(llm_call=npc_call)
+
+    # Reset with an explicit case to avoid mutating internal environment state.
+    obs = env.reset(case_data=case)
+
+    print(f"🔍 DETECTIVE BRIEFING:")
+    print(f"   {case['detective_briefing']}")
+    print()
+    print("-" * 70)
+    print("  INVESTIGATION BEGINS")
+    print("-" * 70)
+
+    # We will build a trace of events for the frontend
+    trace = []
+    
+    # Store the scenario details initially
+    trace.append({
+        "type": "scenario",
+        "case": {
+            "crime": case["crime"],
+            "location": case["location"],
+            "time": case["time"],
+            "criminal": case["criminal"]
+        }
+    })
+
+    # Get scripted detective actions
+    actions = scripted_detective_actions(case)
+
+    done = False
+    total_reward = 0.0
+    last_info = {}
+
+    # Deduplication trackers for trace events (Bug 4 & Bug 5)
+    already_traced_evidence = set()
+    contradictions_this_turn = set()
+
+    for i, action in enumerate(actions):
+        if done:
+            break
+
+        print(f"\n🕵️  [Turn {i}] DETECTIVE:")
+        print(f"   {action}")
+
+        obs, reward, done, info = env.step(action)
+        total_reward += reward  # accumulate step deltas (Bug 1 fix: accuse now returns delta)
+        last_info = info
+
+        # Parse the action specifically for the trace
+        action_parts = dict(p.strip().split(": ", 1) for p in action.split(" | ") if ": " in p)
+        trace.append({
+            "type": "detective",
+            "action": action_parts.get("ACTION", action),
+            "target": action_parts.get("TARGET", ""),
+            "item": action_parts.get("ITEM", ""),
+            "text": action_parts.get("CONTENT", ""),
+            "turn": obs.get("turn", i),
+            "reward_delta": float(reward) if not done else 0.0
+        })
+
+        # Print response if there's a new conversation entry
+        if env.conversation_history:
+            last_entry = env.conversation_history[-1]
+            if last_entry["speaker"] != "Detective":
+                print(f"\n💬 [{last_entry['speaker']}]:")
+                print(f"   {last_entry['content']}")
+
+                trace.append({
+                    "type": "response",
+                    "speaker": last_entry["speaker"],
+                    "turn": obs.get("turn", i),
+                    "text": last_entry["content"]
+                })
+
+                if last_entry.get("flags"):
+                    # Reset per-turn contradiction tracker each turn
+                    contradictions_this_turn_local = set()
+                    for flag in last_entry["flags"]:
+                        print(f"   ⚠️  {flag}")
+                        if "CONTRADICTION" in flag:
+                            turn_key = f"{last_entry['speaker']}_{obs.get('turn', i)}"
+                            if turn_key not in contradictions_this_turn_local:
+                                contradictions_this_turn_local.add(turn_key)
+                                trace.append({
+                                    "type": "contradiction",
+                                    "suspect": last_entry["speaker"],
+                                    "turn": obs.get("turn", i),
+                                    "detail": flag
+                                })
+        
+        # Check newly revealed evidence — deduplicated
+        if obs.get("evidence_log"):
+            new_ev = obs["evidence_log"][-1]
+            item_key = new_ev.get("name", "")
+            if item_key and item_key not in already_traced_evidence:
+                already_traced_evidence.add(item_key)
+                trace.append({
+                    "type": "evidence",
+                    "item": str(new_ev.get("name", "Unknown")),
+                    "desc": str(new_ev.get("description", "")),
+                    "turn": obs.get("turn", i)
+                })
+
+        if not done:
+            print(f"   Step reward: {reward:+.2f}")
+
+    # ── Results ─────────────────────────────────────────────────────────
+
+    print("\n" + "=" * 70)
+    print("  EPISODE RESULTS")
+    print("=" * 70)
+
+    # Accusation result
+    if last_info.get("action") == "accuse":
+        correct = last_info.get("correct", False)
+        target = last_info.get("target", "?")
+        actual = last_info.get("actual_criminal", "?")
+        emoji = "✅" if correct else "❌"
+        print(f"\n  {emoji} Accusation: {target}")
+        print(f"     Actual criminal: {actual}")
+        print(f"     Result: {'CORRECT!' if correct else 'WRONG!'}")
+        
+        trace.append({
+            "type": "outcome",
+            "correct": correct,
+            "target": target,
+            "actual": actual
+        })
+    else:
+        print(f"\n  ⏰ Result: TIMEOUT (no accusation made)")
+        trace.append({
+            "type": "outcome",
+            "correct": False,
+            "timeout": True
+        })
+
+    # Print full reward breakdown
+    rewards = env.reward_calc.get_rewards()
+    print(f"\n  📊 REWARD BREAKDOWN:")
+    print(f"     Detective:  {rewards['detective']:>+8.2f}")
+    print(f"     Suspect_A:  {rewards['suspect_a']:>+8.2f}")
+    print(f"     Suspect_B:  {rewards['suspect_b']:>+8.2f}")
+    print(f"     Witness:    {rewards['witness']:>+8.2f}")
+
+    # Print consistency tracker summary
+    contradictions = env.tracker.get_summary()
+    print(f"\n  🔎 CONTRADICTIONS DETECTED: {len(contradictions)}")
+    for c in contradictions:
+        print(
+            f"     {c['agent']} on '{c['topic']}': "
+            f"'{c['old_value'][:50]}' → '{c['new_value'][:50]}' (turn {c['turn']})"
+        )
+
+    # Print event log
+    print(f"\n  📝 EVENT LOG ({len(env.reward_calc.get_event_log())} events):")
+    for event in env.reward_calc.get_event_log():
+        e_type = event['event']
+        delta = event['delta']
+        print(f"     {e_type}: {delta:+.2f}")
+
+    print("\n" + "=" * 70)
+    print("  Full conversation log:")
+    env.render()
+
+    return rewards, last_info, trace
+
+
+# ── Entry Point ─────────────────────────────────────────────────────────────
+
+if __name__ == "__main__":
+    rewards, info, _trace = run_test_episode()
+
+    # Quick assertion check
+    print("\n🧪 VERIFICATION CHECKS:")
+
+    # Case generator
+    case = generate_case()
+    assert case["criminal"] in ("Suspect_A", "Suspect_B"), "Criminal must be one of the suspects"
+    assert len(case["physical_evidence"]) == 3, "Must have 3 physical evidence items"
+    assert "detective_briefing" in case, "Must have detective briefing"
+    print("   ✅ Case generator: OK")
+
+    # Consistency tracker — test real contradiction vs non-contradiction
+    ct = ConsistencyTracker()
+    r1 = ct.record_claim("Suspect_A", "alibi", "I was at the bar all evening", 1)
+    assert r1["contradicted"] == False, "First claim should not be contradicted"
+
+    r2 = ct.record_claim("Suspect_A", "alibi", "I was at the bar having drinks", 2)
+    assert r2["contradicted"] == False, "Same bar alibi should NOT be a contradiction"
+
+    r3 = ct.record_claim("Suspect_A", "alibi", "I was at home all evening", 3)
+    assert r3["contradicted"] == True, "bar vs home IS a real contradiction"
+
+    r4 = ct.record_claim("Suspect_A", "location", "east wing", 4)
+    r5 = ct.record_claim("Suspect_A", "location", "parking lot", 5)
+    assert r5["contradicted"] == True, "east wing vs parking lot is a contradiction"
+    print("   ✅ Consistency tracker: OK (semantic normalization working)")
+
+    # Reward calculator
+    rc = RewardCalculator()
+    rc.apply_event("correct_accusation")
+    rc.apply_event("per_turn_cost")
+    r = rc.get_rewards()
+    assert abs(r["detective"] - 9.7) < 0.01, f"Expected 9.7, got {r['detective']}"
+    print("   ✅ Reward calculator: OK")
+
+    # System prompts — verify differentiation
+    prompts = {
+        role: build_system_prompt(role, case)
+        for role in ["detective", "suspect_a", "suspect_b", "witness"]
+    }
+    assert "ACTION:" in prompts["detective"], "Detective prompt must include action format"
+    assert "fake alibi" not in prompts["detective"].lower(), "Detective must not know the fake alibi"
+    criminal = case["criminal"]
+    innocent = SUSPECT_B if criminal == SUSPECT_A else SUSPECT_A
+    assert "FAKE ALIBI" in prompts["suspect_a" if criminal == SUSPECT_A else "suspect_b"], \
+        "Criminal suspect prompt must have fake alibi"
+    assert "REAL ALIBI" in prompts["suspect_b" if criminal == SUSPECT_A else "suspect_a"], \
+        "Innocent suspect prompt must have real alibi"
+    assert case["agent_knowledge"][AGENT_NAME_TO_KEY[criminal]]["fake_alibi"] in prompts["suspect_a" if criminal == SUSPECT_A else "suspect_b"], \
+        "Criminal suspect prompt must contain case-specific fake alibi"
+    assert case["agent_knowledge"][AGENT_NAME_TO_KEY[innocent]]["real_alibi"] in prompts["suspect_b" if criminal == SUSPECT_A else "suspect_a"], \
+        "Innocent suspect prompt must contain case-specific real alibi"
+    print("   ✅ Agent prompts: OK (alibis differentiated)")
+
+    print("\n✅ All verification checks passed!")

train_colab.ipynb

@@ -0,0 +1,165 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "language": "markdown"
+      },
+      "source": [
+        "# AI Crime Investigation World - Colab Training\n",
+        "\n",
+        "This notebook runs a minimal PPO training loop using TRL and logs reward progress for demo/judging."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "language": "python"
+      },
+      "outputs": [],
+      "source": [
+        "# Install dependencies\n",
+        "!pip install -q -r requirements-train.txt"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "language": "python"
+      },
+      "outputs": [],
+      "source": [
+        "# Optional: clone repo if running in a fresh Colab runtime\n",
+        "# !git clone <YOUR_REPO_URL>\n",
+        "# %cd SST_Final\n",
+        "\n",
+        "import torch\n",
+        "print('CUDA available:', torch.cuda.is_available())\n",
+        "if torch.cuda.is_available():\n",
+        "    print('GPU:', torch.cuda.get_device_name(0))"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "language": "python"
+      },
+      "outputs": [],
+      "source": [
+        "# Smoke test run (fast)\n",
+        "import os\n",
+        "\n",
+        "os.environ['TEST_MODE'] = '1'\n",
+        "os.environ['NUM_EPISODES'] = '5'\n",
+        "os.environ['MAX_TURNS'] = '8'\n",
+        "os.environ['MODEL_NAME'] = 'Qwen/Qwen2.5-1.5B-Instruct'\n",
+        "os.environ['NPC_MODEL_NAME'] = 'Qwen/Qwen2.5-0.5B-Instruct'\n",
+        "\n",
+        "!python train_colab.py"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "language": "python"
+      },
+      "outputs": [],
+      "source": [
+        "# Full demo run (edit values as needed)\n",
+        "import os\n",
+        "\n",
+        "os.environ['TEST_MODE'] = '0'\n",
+        "os.environ['NUM_EPISODES'] = '50'\n",
+        "os.environ['MODEL_NAME'] = 'Qwen/Qwen2.5-3B-Instruct'\n",
+        "# os.environ['NPC_MODEL_NAME'] = 'Qwen/Qwen2.5-0.5B-Instruct'\n",
+        "\n",
+        "!python train_colab.py"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "language": "python"
+      },
+      "outputs": [],
+      "source": [
+        "# Display reward curve and summary metrics\n",
+        "import json\n",
+        "import matplotlib.pyplot as plt\n",
+        "\n",
+        "with open('rewards.json', 'r') as f:\n",
+        "    data = json.load(f)\n",
+        "\n",
+        "rewards = data['rewards']\n",
+        "results = data.get('results', [])\n",
+        "\n",
+        "print('Episodes:', data.get('num_episodes', len(rewards)))\n",
+        "print('Model:', data.get('model', 'unknown'))\n",
+        "print('Correct:', results.count('correct'))\n",
+        "print('Wrong:', results.count('wrong'))\n",
+        "print('Timeout:', results.count('timeout'))\n",
+        "\n",
+        "plt.figure(figsize=(10, 4))\n",
+        "plt.plot(rewards, alpha=0.35, label='Per episode')\n",
+        "if len(rewards) > 1:\n",
+        "    window = min(20, max(2, len(rewards) // 4))\n",
+        "    smooth = []\n",
+        "    for i in range(len(rewards)):\n",
+        "        start = max(0, i - window + 1)\n",
+        "        smooth.append(sum(rewards[start:i+1]) / (i - start + 1))\n",
+        "    plt.plot(smooth, linewidth=2, label=f'Moving avg ({window})')\n",
+        "\n",
+        "plt.axhline(0, color='gray', linestyle='--', alpha=0.5)\n",
+        "plt.title('Detective Reward Over Training')\n",
+        "plt.xlabel('Episode')\n",
+        "plt.ylabel('Reward')\n",
+        "plt.legend()\n",
+        "plt.grid(alpha=0.25)\n",
+        "plt.show()"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "language": "python"
+      },
+      "outputs": [],
+      "source": [
+        "# Show before/after transcript snapshots\n",
+        "import json\n",
+        "\n",
+        "with open('episode_transcripts.json', 'r') as f:\n",
+        "    t = json.load(f)\n",
+        "\n",
+        "print('Captured episodes:', t.get('episodes_captured', []))\n",
+        "for ep in t.get('transcripts', []):\n",
+        "    print('\\n' + '=' * 70)\n",
+        "    print(f\"Episode {ep['episode']} | Result: {ep['result']} | Reward: {ep['detective_reward']}\")\n",
+        "    for line in ep.get('conversation_history', [])[:10]:\n",
+        "        speaker = line.get('speaker', 'Unknown')\n",
+        "        content = line.get('content', '')\n",
+        "        print(f\"- {speaker}: {content}\")\n",
+        "    if len(ep.get('conversation_history', [])) > 10:\n",
+        "        print('...')"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 5
+}

train_colab.py

@@ -0,0 +1,594 @@
+#!/usr/bin/env python3
+"""Component 6 — Minimal Training Script (Colab-Ready).
+
+Trains the detective agent using PPO (HuggingFace TRL) while other agents
+use fixed prompt-based LLM calls. Designed for free-tier Colab GPU.
+"""
+
+import json
+import os
+import inspect
+import sys
+import time
+import warnings
+from typing import Optional
+
+import matplotlib.pyplot as plt
+import numpy as np
+import torch
+from transformers import (
+    AutoConfig,
+    AutoModelForCausalLM,
+    AutoTokenizer,
+    BitsAndBytesConfig,
+    pipeline,
+)
+from peft import LoraConfig, TaskType
+from trl import PPOConfig, PPOTrainer, AutoModelForCausalLMWithValueHead
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from crime_env.environment import CrimeInvestigationEnv
+from crime_env.agent_prompts import build_system_prompt
+
+
+# ── Configuration ───────────────────────────────────────────────────────────
+
+
+def _env_bool(name: str, default: bool = False) -> bool:
+    value = os.environ.get(name)
+    if value is None:
+        return default
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
+def _env_int(name: str, default: int) -> int:
+    value = os.environ.get(name)
+    if value is None:
+        return default
+    try:
+        return int(value)
+    except ValueError:
+        return default
+
+# Default to a stronger model while keeping env override support.
+TEST_MODE = _env_bool("TEST_MODE", False)
+MODEL_NAME = os.environ.get("MODEL_NAME", "Qwen/Qwen2.5-3B-Instruct")
+NPC_MODEL_NAME = os.environ.get(
+    "NPC_MODEL_NAME",
+    "Qwen/Qwen2.5-0.5B-Instruct" if TEST_MODE else MODEL_NAME,
+)
+NUM_EPISODES = _env_int("NUM_EPISODES", 5 if TEST_MODE else 300)
+MAX_TURNS = _env_int("MAX_TURNS", 8 if TEST_MODE else 15)
+DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
+OUTPUT_DIR = "./ppo_detective"
+REWARDS_FILE = "rewards.json"
+TRANSCRIPTS_FILE = "episode_transcripts.json"
+NPC_MAX_NEW_TOKENS = _env_int("NPC_MAX_NEW_TOKENS", 64 if TEST_MODE else 150)
+DETECTIVE_MAX_NEW_TOKENS = _env_int(
+    "DETECTIVE_MAX_NEW_TOKENS", 48 if TEST_MODE else 80
+)
+DETECTIVE_RETRY_MAX_NEW_TOKENS = _env_int(
+    "DETECTIVE_RETRY_MAX_NEW_TOKENS", 32 if TEST_MODE else 64
+)
+TRANSCRIPT_EPISODES = {
+    int(x.strip())
+    for x in os.environ.get("TRANSCRIPT_EPISODES", "1,25,50").split(",")
+    if x.strip().isdigit()
+}
+STRICT_FORMAT_FALLBACK_THRESHOLD = float(
+    os.environ.get("STRICT_FORMAT_FALLBACK_THRESHOLD", "0.35")
+)
+STRICT_FORMAT_WINDOW = _env_int("STRICT_FORMAT_WINDOW", 5)
+
+
+# ── Model Loading ───────────────────────────────────────────────────────────
+
+
+def load_models():
+    """Load the detective (trainable) and NPC (fixed) models."""
+    print(f"Loading model: {MODEL_NAME}")
+    print(f"Device: {DEVICE}")
+
+    tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, trust_remote_code=True)
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+
+    model_config = AutoConfig.from_pretrained(MODEL_NAME, trust_remote_code=True)
+    hidden_size = getattr(model_config, "hidden_size", None)
+    use_quantization = DEVICE == "cuda" and (hidden_size is None or hidden_size >= 64)
+
+    # Quantization for memory efficiency on free Colab
+    quant_config = None
+    if use_quantization:
+        try:
+            quant_config = BitsAndBytesConfig(
+                load_in_4bit=True,
+                bnb_4bit_compute_dtype=torch.float16,
+                bnb_4bit_use_double_quant=True,
+                bnb_4bit_quant_type="nf4",
+            )
+        except Exception:
+            print("BitsAndBytes not available, loading without quantization")
+    elif DEVICE == "cuda":
+        print("Skipping 4-bit quantization for small hidden-size model")
+
+    # LoRA config — enables PEFT so PPOTrainer reuses base weights as the
+    # reference model instead of deepcopy-ing the quantized model (OOM fix).
+    lora_config = LoraConfig(
+        r=8,
+        lora_alpha=16,
+        target_modules=["q_proj", "v_proj"],
+        lora_dropout=0.05,
+        bias="none",
+        task_type=TaskType.CAUSAL_LM,
+    ) if DEVICE == "cuda" else None
+
+    # Detective model (trainable with value head)
+    model_dtype = torch.float16 if DEVICE == "cuda" else torch.float32
+    detective_load_kwargs = {
+        "quantization_config": quant_config,
+        "peft_config": lora_config,
+        "dtype": model_dtype,
+        "device_map": "auto" if DEVICE == "cuda" else None,
+        "trust_remote_code": True,
+    }
+
+    try:
+        detective_model = AutoModelForCausalLMWithValueHead.from_pretrained(
+            MODEL_NAME,
+            **detective_load_kwargs,
+        )
+    except ValueError as e:
+        error_text = str(e)
+        if DEVICE == "cuda" and "dispatched on the CPU or the disk" in error_text:
+            print("Low VRAM detected, retrying with CPU offload enabled for quantized layers...")
+            offload_quant_config = BitsAndBytesConfig(
+                load_in_4bit=True,
+                bnb_4bit_compute_dtype=torch.float16,
+                bnb_4bit_use_double_quant=True,
+                bnb_4bit_quant_type="nf4",
+                llm_int8_enable_fp32_cpu_offload=True,
+            )
+            detective_load_kwargs["quantization_config"] = offload_quant_config
+            detective_load_kwargs["device_map"] = "auto"
+            detective_model = AutoModelForCausalLMWithValueHead.from_pretrained(
+                MODEL_NAME,
+                **detective_load_kwargs,
+            )
+        else:
+            raise
+
+    # NPC pipeline — load a SEPARATE frozen copy of the base model.
+    # Bug 9 fix: Using detective_model.pretrained_model would cause PPO
+    # gradient updates to drift the NPC's behavior every step.
+    npc_base_model = None
+    try:
+        npc_base_model = AutoModelForCausalLM.from_pretrained(
+            NPC_MODEL_NAME,
+            quantization_config=quant_config,
+            dtype=model_dtype,
+            device_map="auto" if DEVICE == "cuda" else None,
+            trust_remote_code=True,
+        )
+        npc_base_model.eval()  # Freeze: no gradient tracking
+        for param in npc_base_model.parameters():
+            param.requires_grad = False
+    except Exception as e:
+        print(f"NPC model load warning: {e}")
+        print("Falling back to detective base model for NPC responses to keep training running.")
+        npc_base_model = detective_model.pretrained_model
+
+    npc_pipeline = pipeline(
+        "text-generation",
+        model=npc_base_model,
+        tokenizer=tokenizer,
+        max_new_tokens=NPC_MAX_NEW_TOKENS,
+        do_sample=True,
+        temperature=0.7,
+        top_p=0.9,
+    )
+
+    return detective_model, npc_pipeline, tokenizer
+
+
+# ── NPC LLM Call ────────────────────────────────────────────────────────────
+
+
+def make_npc_call(npc_pipeline):
+    """Create a callable for NPC agent responses."""
+
+    def llm_call(system_prompt: str, conversation_history: list[dict]) -> str:
+        # Build a bounded prompt to avoid model context overflows.
+        messages = f"System: {system_prompt[:800]}\n\n"
+
+        # Keep only recent turns and constrain prompt length.
+        recent_history = conversation_history[-10:]
+        for entry in recent_history:
+            speaker = entry.get("speaker", "Unknown")
+            content = entry.get("content", "")
+            messages += f"{speaker}: {content[:180]}\n"
+
+        # Hard cap prompt size for small-context models.
+        messages = messages[-1800:]
+
+        messages += "\nYour response:"
+
+        try:
+            result = npc_pipeline(messages, return_full_text=False)
+            response = result[0]["generated_text"].strip()
+            # Clean up: take first sentence/paragraph
+            if "\n" in response:
+                response = response.split("\n")[0]
+            return response[:300] if response else "I have nothing to add."
+        except Exception as e:
+            print(f"  NPC call error: {e}")
+            return "I don't recall anything specific about that."
+
+    return llm_call
+
+
+# ── Detective Action Generation ─────────────────────────────────────────────
+
+
+def generate_detective_action(
+    detective_model,
+    tokenizer,
+    observation: dict,
+    env=None,
+    strict_action_format: bool = False,
+) -> tuple[str, torch.Tensor, torch.Tensor, bool]:
+    """Generate a detective action using the trainable model.
+
+    Returns:
+        (action_string, query_tensor, response_tensor, used_fallback)
+    """
+    # Build prompt from observation
+    prompt = f"""You are a detective. Based on the conversation so far, choose your next action.
+
+Briefing: {observation['briefing'][:300]}
+
+Turn: {observation['turn']}/{MAX_TURNS}
+
+Recent conversation:
+"""
+    history = observation.get("conversation_history", [])
+    for entry in history[-6:]:
+        prompt += f"  {entry['speaker']}: {entry['content'][:100]}\n"
+
+    prompt += """
+Choose ONE action using EXACTLY this format:
+ACTION: ask_question | TARGET: Suspect_A | CONTENT: <question>
+ACTION: request_evidence | ITEM: keycard_log
+ACTION: accuse | TARGET: Suspect_A
+
+Your action:
+"""
+
+    if strict_action_format:
+        prompt += (
+            "\nIMPORTANT: Output ONLY a single valid ACTION line. "
+            "No explanations, no extra text.\n"
+        )
+
+    # Tokenize
+    inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=1024)
+    query_tensor = inputs["input_ids"].squeeze()
+
+    if DEVICE == "cuda":
+        inputs = {k: v.to(DEVICE) for k, v in inputs.items()}
+
+    # Generate
+    with torch.no_grad():
+        output = detective_model.pretrained_model.generate(
+            **inputs,
+            max_new_tokens=DETECTIVE_MAX_NEW_TOKENS,
+            do_sample=True,
+            temperature=0.4 if strict_action_format else 0.8,
+            top_p=0.8 if strict_action_format else 0.9,
+            pad_token_id=tokenizer.pad_token_id,
+        )
+
+    response_tensor = output.squeeze()[len(query_tensor):]
+    action_text = tokenizer.decode(response_tensor, skip_special_tokens=True).strip()
+
+    def _is_valid_action(text: str) -> bool:
+        normalized = text.strip().upper()
+        return normalized.startswith("ACTION:")
+
+    # Retry once in strict mode before falling back to scripted actions.
+    if not _is_valid_action(action_text):
+        strict_prompt = prompt + (
+            "\nFINAL REMINDER: Reply with one line starting with 'ACTION:' only.\n"
+        )
+        strict_inputs = tokenizer(
+            strict_prompt, return_tensors="pt", truncation=True, max_length=1024
+        )
+        if DEVICE == "cuda":
+            strict_inputs = {k: v.to(DEVICE) for k, v in strict_inputs.items()}
+
+        with torch.no_grad():
+            retry_output = detective_model.pretrained_model.generate(
+                **strict_inputs,
+                max_new_tokens=DETECTIVE_RETRY_MAX_NEW_TOKENS,
+                do_sample=True,
+                temperature=0.25,
+                top_p=0.75,
+                pad_token_id=tokenizer.pad_token_id,
+            )
+
+        strict_query_tensor = strict_inputs["input_ids"].squeeze().detach().cpu()
+        retry_response_tensor = retry_output.squeeze()[len(strict_query_tensor):]
+        retry_action_text = tokenizer.decode(
+            retry_response_tensor, skip_special_tokens=True
+        ).strip()
+        if _is_valid_action(retry_action_text):
+            query_tensor = strict_query_tensor
+            response_tensor = retry_response_tensor
+            action_text = retry_action_text
+
+    used_fallback = False
+
+    # If model output doesn't parse, generate a deterministic fallback action.
+    if not _is_valid_action(action_text):
+        used_fallback = True
+        turn = observation.get("turn", 0)
+        if turn < 4:
+            action_text = "ACTION: ask_question | TARGET: Suspect_A | CONTENT: Where were you at the time of the crime?"
+        elif turn < 8:
+            action_text = "ACTION: ask_question | TARGET: Suspect_B | CONTENT: Can you describe your alibi?"
+        elif turn < 10:
+            action_text = "ACTION: request_evidence | ITEM: keycard_log"
+        else:
+            action_text = "ACTION: ask_question | TARGET: Witness_1 | CONTENT: What did you see?"
+
+    # Never train PPO on fallback actions: they are scripted and off-policy.
+    if used_fallback:
+        response_tensor = tokenizer(
+            action_text,
+            return_tensors="pt",
+            add_special_tokens=False,
+        )["input_ids"].squeeze()
+
+    return action_text, query_tensor, response_tensor, used_fallback
+
+
+# ── Training Loop ───────────────────────────────────────────────────────────
+
+
+def train():
+    """Main PPO training loop."""
+    print("=" * 60)
+    print("  AI Crime Investigation World — PPO Training")
+    print("=" * 60)
+    if TEST_MODE:
+        print("  Running in TEST_MODE (smoke-test settings enabled)")
+        print(
+            f"  MODEL_NAME={MODEL_NAME} | NPC_MODEL_NAME={NPC_MODEL_NAME} | "
+            f"NUM_EPISODES={NUM_EPISODES} | MAX_TURNS={MAX_TURNS}"
+        )
+        print("=" * 60)
+
+    # Load models
+    detective_model, npc_pipeline, tokenizer = load_models()
+
+    warnings.filterwarnings(
+        "ignore",
+        message="No dataset is provided.*",
+        category=UserWarning,
+    )
+
+    # Compatibility guard: this script targets the TRL 0.9 PPOTrainer API.
+    ppo_init_params = inspect.signature(PPOTrainer.__init__).parameters
+    if "config" not in ppo_init_params:
+        raise RuntimeError(
+            "Incompatible TRL version detected. "
+            "Use `trl>=0.9.0,<0.10.0` for this training script."
+        )
+
+    # PPO config (support TRL variants with either ppo_epochs or num_ppo_epochs)
+    ppo_cfg_kwargs = {
+        "learning_rate": 1e-5,
+        "batch_size": 1,
+        "mini_batch_size": 1,
+        "gradient_accumulation_steps": 1,
+    }
+    ppo_cfg_params = inspect.signature(PPOConfig.__init__).parameters
+    if "num_ppo_epochs" in ppo_cfg_params:
+        ppo_cfg_kwargs["num_ppo_epochs"] = 1  # single-sample stability
+    elif "ppo_epochs" in ppo_cfg_params:
+        ppo_cfg_kwargs["ppo_epochs"] = 1  # single-sample stability
+
+    ppo_config = PPOConfig(**ppo_cfg_kwargs)
+
+    # PPO Trainer
+    ppo_trainer = PPOTrainer(
+        config=ppo_config,
+        model=detective_model,
+        tokenizer=tokenizer,
+    )
+
+    # Environment
+    npc_call = make_npc_call(npc_pipeline)
+    env = CrimeInvestigationEnv(llm_call=npc_call)
+
+    # Tracking
+    reward_log = []
+    results_log = []
+    transcript_log = []
+
+    print(f"\nStarting training for {NUM_EPISODES} episodes...\n")
+
+    strict_action_format = False
+    fallback_rate_history: list[float] = []
+
+    for episode in range(NUM_EPISODES):
+        t0 = time.time()
+
+        # Reset environment
+        obs = env.reset()
+        done = False
+        fallback_steps = 0
+        total_steps = 0
+        ppo_updates = 0
+
+        while not done:
+            # Generate detective action
+            action_text, query_tensor, response_tensor, used_fallback = (
+                generate_detective_action(
+                    detective_model,
+                    tokenizer,
+                    obs,
+                    env=env,
+                    strict_action_format=strict_action_format,
+                )
+            )
+
+            # Step environment
+            obs, reward, done, info = env.step(action_text)
+            total_steps += 1
+
+            if not used_fallback:
+                # PPOConfig uses batch_size=1, so update on each valid step.
+                try:
+                    reward_tensor = torch.tensor([reward], dtype=torch.float32)
+                    ppo_trainer.step(
+                        [query_tensor],
+                        [response_tensor],
+                        [reward_tensor],
+                    )
+                    ppo_updates += 1
+                except Exception as e:
+                    print(f"  PPO update error (episode {episode}, step {total_steps}): {e}")
+            else:
+                fallback_steps += 1
+
+        # Get final rewards
+        final_rewards = env.reward_calc.get_rewards()
+        detective_reward = final_rewards["detective"]
+        reward_log.append(detective_reward)
+
+        # Determine result
+        last_info = info
+        if last_info.get("action") == "accuse":
+            result = "correct" if last_info.get("correct") else "wrong"
+        else:
+            result = "timeout"
+        results_log.append(result)
+
+        if (episode + 1) in TRANSCRIPT_EPISODES:
+            transcript_log.append(
+                {
+                    "episode": episode + 1,
+                    "result": result,
+                    "detective_reward": round(detective_reward, 4),
+                    "turns": env.turn,
+                    "criminal": env.case.get("criminal") if env.case else None,
+                    "crime": env.case.get("crime") if env.case else None,
+                    "location": env.case.get("location") if env.case else None,
+                    "conversation_history": list(env.conversation_history),
+                    "evidence_log": list(env.evidence_log),
+                }
+            )
+
+        fallback_rate = fallback_steps / max(1, total_steps)
+        fallback_rate_history.append(fallback_rate)
+        if len(fallback_rate_history) > max(1, STRICT_FORMAT_WINDOW):
+            fallback_rate_history.pop(0)
+        rolling_fallback_rate = sum(fallback_rate_history) / len(fallback_rate_history)
+        strict_action_format = rolling_fallback_rate > STRICT_FORMAT_FALLBACK_THRESHOLD
+
+        elapsed = time.time() - t0
+        print(
+            f"Episode {episode + 1:>3}/{NUM_EPISODES} | "
+            f"Result: {result:<7} | "
+            f"Reward: {detective_reward:>+7.2f} | "
+            f"Turns: {env.turn:>2} | "
+            f"PPO updates: {ppo_updates:>2} | "
+            f"Fallback: {fallback_steps:>2}/{max(1, total_steps):<2} "
+            f"({fallback_rate*100:>5.1f}%, avg={rolling_fallback_rate*100:>5.1f}%) | "
+            f"Time: {elapsed:.1f}s"
+        )
+
+    # ── Save results ────────────────────────────────────────────────────
+
+    # Save reward log
+    with open(REWARDS_FILE, "w") as f:
+        json.dump(
+            {
+                "rewards": reward_log,
+                "results": results_log,
+                "num_episodes": NUM_EPISODES,
+                "model": MODEL_NAME,
+            },
+            f,
+            indent=2,
+        )
+    print(f"\nReward log saved to {REWARDS_FILE}")
+
+    with open(TRANSCRIPTS_FILE, "w") as f:
+        json.dump(
+            {
+                "episodes_captured": sorted([t["episode"] for t in transcript_log]),
+                "transcripts": transcript_log,
+            },
+            f,
+            indent=2,
+        )
+    print(f"Episode transcripts saved to {TRANSCRIPTS_FILE}")
+
+    # Plot reward curve
+    _plot_reward_curve(reward_log)
+
+    # Summary statistics
+    print("\n" + "=" * 60)
+    print("  TRAINING SUMMARY")
+    print("=" * 60)
+    print(f"  Episodes:           {NUM_EPISODES}")
+    print(f"  Correct accusations: {results_log.count('correct')}")
+    print(f"  Wrong accusations:   {results_log.count('wrong')}")
+    print(f"  Timeouts:            {results_log.count('timeout')}")
+    print(f"  Mean reward (last 50): {np.mean(reward_log[-50:]):.2f}")
+    print(f"  Mean reward (first 50): {np.mean(reward_log[:50]):.2f}")
+    print("=" * 60)
+
+
+def _plot_reward_curve(reward_log: list[float]) -> None:
+    """Plot and save the reward curve."""
+    fig, ax = plt.subplots(figsize=(12, 5))
+
+    episodes = np.arange(1, len(reward_log) + 1)
+    ax.plot(episodes, reward_log, alpha=0.3, color="#4a90d9", label="Per-episode")
+
+    # Smoothed curve (rolling average)
+    window = min(20, len(reward_log) // 4)
+    if window > 1:
+        smoothed = np.convolve(
+            reward_log, np.ones(window) / window, mode="valid"
+        )
+        ax.plot(
+            episodes[window - 1:],
+            smoothed,
+            color="#e74c3c",
+            linewidth=2,
+            label=f"Rolling avg ({window} ep)",
+        )
+
+    ax.set_xlabel("Episode", fontsize=12)
+    ax.set_ylabel("Detective Reward", fontsize=12)
+    ax.set_title("AI Crime Investigation — Detective Reward Over Training", fontsize=14)
+    ax.legend()
+    ax.grid(True, alpha=0.3)
+    ax.axhline(y=0, color="grey", linestyle="--", alpha=0.5)
+
+    plt.tight_layout()
+    plt.savefig("reward_curve.png", dpi=150)
+    print("Reward curve saved to reward_curve.png")
+    plt.show()
+
+
+# ── Entry Point ─────────────────────────────────────────────────────────────
+
+if __name__ == "__main__":
+    train()