Upload folder using huggingface_hub

Blog.md

@@ -338,3 +338,6 @@ governance policy for its own cognition.
 
 And in a crisis world, that may be the difference between fast agreement and
 good judgment.
+
+# Results
+![Results](https://drive.google.com/uc?export=view&id=1n9uF3NZFy3FszypLIl3cGuzeORwy7NOL)

__init__.py

@@ -1,16 +1,16 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-"""Crisisworldcortex Environment."""
-
-from .client import CrisisworldcortexEnv
-from .models import CrisisworldcortexAction, CrisisworldcortexObservation
-
-__all__ = [
-    "CrisisworldcortexAction",
-    "CrisisworldcortexObservation",
-    "CrisisworldcortexEnv",
-]
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Crisisworldcortex Environment."""
+
+from .client import CrisisworldcortexEnv
+from .models import CrisisworldcortexAction, CrisisworldcortexObservation
+
+__all__ = [
+    "CrisisworldcortexAction",
+    "CrisisworldcortexObservation",
+    "CrisisworldcortexEnv",
+]

demo/CLAUDE.md

@@ -1,45 +1,45 @@
-# demo/CLAUDE.md
-
-Replay-only visualization. Live demos fail under judging pressure; ship canned scenarios.
-
-## Belongs here
-
-- `visualizer/trace_renderer.py` — renders a JSON trace as a "council in action" view.
-- `visualizer/reward_curve_plot.py` — plots reward curves from training logs.
-- `demo_scenarios/*.json` — pre-recorded trajectories for the pitch (e.g. `scenario_flat_fails.json`, `scenario_cortex_holds_dissent.json`).
-
-## Does not belong here
-
-Live agent execution (record offline, replay here). Training logic. Graders.
-
-## Allowed imports
-
-- `cortex.schemas` — typed parse of trace JSON. Types only, no logic.
-- stdlib + plotting libs (matplotlib / plotly).
-
-## Forbidden imports
-
-- `server/*`, `training/*`, `baselines/*`.
-- `cortex.council`, `cortex.routing_policy` — if you need to re-run the agent, do it offline and ship a new JSON.
-
-## Binding contracts
-
-- Every JSON scenario conforms to `cortex.schemas.Trajectory`.
-- Rendering is deterministic: same JSON → same output, modulo timestamps.
-- The pitch-demo scenario must showcase B2 overcommit/misallocate vs Cortex dissent-preservation (design §27).
-- A pre-recorded demo video (MP4) lives alongside the JSON scenarios as the live-demo fallback.
-
-## Public APIs (owned here)
-
-- `render_trace(json_path: str, out_path: str) -> None`
-- `plot_reward_curves(log_paths: list[str], out_path: str) -> None`
-
-## Testing requirements
-
-- Each committed JSON scenario parses into a `Trajectory` without error.
-- `render_trace` produces a non-empty output file for each scenario.
-
-## Common failure modes
-
-- Live re-run during the demo — network/Colab flakiness kills the pitch. Replay only.
-- Renderer depending on a `cortex.council` instance — import breaks when Cortex API shifts. Keep read-only on types.
+# demo/CLAUDE.md
+
+Replay-only visualization. Live demos fail under judging pressure; ship canned scenarios.
+
+## Belongs here
+
+- `visualizer/trace_renderer.py` — renders a JSON trace as a "council in action" view.
+- `visualizer/reward_curve_plot.py` — plots reward curves from training logs.
+- `demo_scenarios/*.json` — pre-recorded trajectories for the pitch (e.g. `scenario_flat_fails.json`, `scenario_cortex_holds_dissent.json`).
+
+## Does not belong here
+
+Live agent execution (record offline, replay here). Training logic. Graders.
+
+## Allowed imports
+
+- `cortex.schemas` — typed parse of trace JSON. Types only, no logic.
+- stdlib + plotting libs (matplotlib / plotly).
+
+## Forbidden imports
+
+- `server/*`, `training/*`, `baselines/*`.
+- `cortex.council`, `cortex.routing_policy` — if you need to re-run the agent, do it offline and ship a new JSON.
+
+## Binding contracts
+
+- Every JSON scenario conforms to `cortex.schemas.Trajectory`.
+- Rendering is deterministic: same JSON → same output, modulo timestamps.
+- The pitch-demo scenario must showcase B2 overcommit/misallocate vs Cortex dissent-preservation (design §27).
+- A pre-recorded demo video (MP4) lives alongside the JSON scenarios as the live-demo fallback.
+
+## Public APIs (owned here)
+
+- `render_trace(json_path: str, out_path: str) -> None`
+- `plot_reward_curves(log_paths: list[str], out_path: str) -> None`
+
+## Testing requirements
+
+- Each committed JSON scenario parses into a `Trajectory` without error.
+- `render_trace` produces a non-empty output file for each scenario.
+
+## Common failure modes
+
+- Live re-run during the demo — network/Colab flakiness kills the pitch. Replay only.
+- Renderer depending on a `cortex.council` instance — import breaks when Cortex API shifts. Keep read-only on types.

inference.py

@@ -42,7 +42,11 @@ import os
 import sys
 from dataclasses import dataclass
 from typing import Any, Dict, List, Literal, Optional
-
+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except ImportError:
+    pass
 from baselines.flat_agent import B1FlatAgent, B1StepEvent
 from cortex.llm_client import LLMClient
 from CrisisWorldCortex.models import OuterActionPayload

manual_testing.py

@@ -0,0 +1,48 @@
+from .models import NoOp, DeployResource, RestrictMovement, CrisisworldcortexAction, CrisisworldcortexObservation, PublicCommunication
+from .server import CrisisWorld as CrisisWorldEnvironment
+
+def logger(count, obs: CrisisworldcortexObservation, action: CrisisworldcortexAction | None) -> None:
+    if action:
+        print(f"[Action Taken]: {action.model_dump(exclude_none=True)}")
+    else:
+        print("Initial Observation:")
+    print(f"turn={count}, done={obs.done}, reward={obs.reward}")
+    for reg in obs.regions:
+        print(
+            f"{reg.region_id} pop={reg.population} "
+            f"inf={reg.infected} rec={reg.recovered} "
+            f"dec={reg.deceased}"
+        )
+    print("-" * 40)
+    print()
+
+env = CrisisWorldEnvironment(config=EnvConfig(max_turns=6, num_regions=4))
+obs = env.reset(seed=42, episode_id="manual-test")
+logger(0, obs, None)
+
+# Turn 1 (NoOp)
+action = DeployResource(resource="medical", region_id="r0", amount=10)
+# action = NoOp()
+obs = env.step(action)
+logger(1, obs, action)
+
+# Turn 2 (NoOp)
+action = NoOp()
+obs = env.step(action)
+logger(2, obs, action)
+
+# Turn 3 (DeployResource to r0)
+action = RestrictMovement(region_id="r0", level=2)
+obs = env.step(action)
+logger(3, obs, action)
+
+# Turn 4 (PublicCommunication)
+action = PublicCommunication(audience="public", message="Stay safe! Wear masks and maintain social distancing. Don't move between regions, stay home if you can, and seek medical care if you have symptoms.")
+obs = env.step(action)
+logger(4, obs, action)
+
+# Turn 5 (NoOp)
+action = DeployResource(resource="medical", region_id="r0", amount=40)
+# action = NoOp()
+obs = env.step(action)
+logger(5, obs, action)

notebooks/colab_demo_unsloth_smollm.ipynb

@@ -0,0 +1,538 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# CrisisWorldCortex — Colab Mini-Demo (Unsloth + SmolLM2-360M)\n",
+    "\n",
+    "**Purpose.** Stand-alone Colab notebook that demonstrates the GRPO training pipeline end-to-end on free Colab resources. Mirrors the production pipeline in `training/scripts/train_b1_grpo.py` but with:\n",
+    "\n",
+    "- **Model:** `unsloth/SmolLM2-360M-Instruct` (~360M params, well under the 500M ceiling for free Colab).\n",
+    "- **Steps:** `MAX_TRAIN_STEPS = 20` (vs. 300 in production). Just enough to show learning signal flowing.\n",
+    "- **Dataset:** 2 tasks × 5 seeds = 10 prompts (vs. 3 tasks × 50 seeds = 150 in production).\n",
+    "- **Group size:** 2 (vs. 4 in production).\n",
+    "\n",
+    "Wall-clock: ~5–10 minutes on a free Colab T4. CPU-only fallback also works (slower).\n",
+    "\n",
+    "**Compatibility note.** The user's production HF Jobs runs use `train_b1_grpo.py` with Qwen3-7B / Llama-3.1-8B. This notebook does **not** replace those; it is a demo-grade smoke run for showing the loop turning. The training contract — single-step GRPO against the Phase-1-fixed `outer_reward`, parse-failure → marker action — is identical.\n",
+    "\n",
+    "**Prereqs.**\n",
+    "1. Colab Secrets has `HF_TOKEN` set (Tools → Secrets, name = `HF_TOKEN`, value = `hf_xxx` with write access). Skip if you don't want to push to Hub.\n",
+    "2. The HF Space `Angshuman28/CrisisWorldCortex` is running and reachable. (You can override `ENV_URL` below if you've forked it.)\n",
+    "\n",
+    "**One-shot run:** `Runtime → Run all`. The notebook prints a clear status line at the end indicating whether the loop completed, and prints reward statistics so you can see signal."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Install dependencies\n",
+    "\n",
+    "Mirrors the install cell in `notebooks/train_b1_grpo.ipynb`. Unsloth ships its own torch / vLLM / xformers stack tuned for Colab T4."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install --upgrade pip\n",
+    "!pip install unsloth vllm\n",
+    "!pip install --upgrade --no-deps \"trl>=0.14\" peft accelerate bitsandbytes\n",
+    "!pip install pydantic openenv huggingface_hub matplotlib datasets"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Authenticate with Hugging Face (optional)\n",
+    "\n",
+    "Reads `HF_TOKEN` from Colab Secrets. If unavailable, the notebook still runs end-to-end — the final Hub push step will be skipped."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "HF_TOKEN = \"\"\n",
+    "try:\n",
+    "    from google.colab import userdata\n",
+    "    HF_TOKEN = userdata.get(\"HF_TOKEN\") or \"\"\n",
+    "except Exception:\n",
+    "    HF_TOKEN = os.environ.get(\"HF_TOKEN\", \"\")\n",
+    "\n",
+    "if HF_TOKEN:\n",
+    "    os.environ[\"HF_TOKEN\"] = HF_TOKEN\n",
+    "    print(\"HF auth OK — Hub push enabled at the end.\")\n",
+    "else:\n",
+    "    print(\"No HF_TOKEN found. Notebook will still run; Hub push step will be skipped.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Clone CrisisWorldCortex and install\n",
+    "\n",
+    "Pulls the deployed HF Space repo and installs it locally. This gives the notebook the same `CrisisworldcortexEnv` HTTP client, `baselines.flat_agent` system prompt, and parser used by the production pipeline. **Nothing in the repo is modified — this is a read-only consumer.**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!rm -rf /content/CrisisWorldCortex\n",
+    "!git clone https://huggingface.co/spaces/Angshuman28/CrisisWorldCortex /content/CrisisWorldCortex\n",
+    "%cd /content/CrisisWorldCortex\n",
+    "!pip install -e ."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "sys.path.insert(0, \"/content/CrisisWorldCortex\")\n",
+    "\n",
+    "from baselines.flat_agent import (\n",
+    "    build_system_prompt,\n",
+    "    parse_action,\n",
+    "    parse_failure_marker,\n",
+    "    serialize_observation,\n",
+    ")\n",
+    "from CrisisWorldCortex import CrisisworldcortexAction, CrisisworldcortexObservation\n",
+    "from CrisisWorldCortex.client import CrisisworldcortexEnv\n",
+    "\n",
+    "print(\"CrisisWorld imports OK\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Load SmolLM2-360M with Unsloth + LoRA\n",
+    "\n",
+    "SmolLM2-360M is ~360M parameters — well under the user-requested 500M ceiling and comfortable on free Colab T4 (or CPU as fallback). LoRA rank 16 is plenty for a smoke demo.\n",
+    "\n",
+    "Alternative tiny models, all Unsloth-supported, all under 500M:\n",
+    "- `unsloth/SmolLM2-135M-Instruct` (135M — fastest)\n",
+    "- `unsloth/SmolLM2-360M-Instruct` (360M — current default, best signal-to-cost)\n",
+    "- `unsloth/Qwen2.5-0.5B-Instruct` (~500M — closest in family to the production Qwen3 baseline)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import torch\n",
+    "from unsloth import FastLanguageModel\n",
+    "\n",
+    "MODEL_NAME = \"unsloth/SmolLM2-360M-Instruct\"\n",
+    "MAX_SEQ_LEN = 2048\n",
+    "LORA_RANK = 16\n",
+    "\n",
+    "model, tokenizer = FastLanguageModel.from_pretrained(\n",
+    "    model_name=MODEL_NAME,\n",
+    "    max_seq_length=MAX_SEQ_LEN,\n",
+    "    load_in_4bit=True,\n",
+    "    fast_inference=True,  # vLLM-backed generate, required by GRPOTrainer\n",
+    "    max_lora_rank=LORA_RANK,\n",
+    "    gpu_memory_utilization=0.5,\n",
+    ")\n",
+    "\n",
+    "model = FastLanguageModel.get_peft_model(\n",
+    "    model,\n",
+    "    r=LORA_RANK,\n",
+    "    target_modules=[\n",
+    "        \"q_proj\",\n",
+    "        \"k_proj\",\n",
+    "        \"v_proj\",\n",
+    "        \"o_proj\",\n",
+    "        \"gate_proj\",\n",
+    "        \"up_proj\",\n",
+    "        \"down_proj\",\n",
+    "    ],\n",
+    "    lora_alpha=LORA_RANK * 2,\n",
+    "    use_gradient_checkpointing=\"unsloth\",\n",
+    "    random_state=42,\n",
+    ")\n",
+    "print(f\"Model + LoRA loaded — {MODEL_NAME}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Connect to the deployed CrisisWorld env\n",
+    "\n",
+    "Same HTTP client + URL as the production notebook. Each rollout = one `env.reset()` + one `env.step()` (single-step GRPO; matches the binding contract in `training/CLAUDE.md`)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ENV_URL = \"https://angshuman28-crisisworldcortex.hf.space\"\n",
+    "TASKS = (\"outbreak_easy\", \"outbreak_medium\")  # 2 tasks for the mini-demo\n",
+    "EPISODE_TICKS = 12\n",
+    "SEEDS_PER_TASK = 5\n",
+    "\n",
+    "\n",
+    "def make_env() -> CrisisworldcortexEnv:\n",
+    "    return CrisisworldcortexEnv(base_url=ENV_URL)\n",
+    "\n",
+    "\n",
+    "_test_env = make_env()\n",
+    "_obs = _test_env.reset(task_name=\"outbreak_easy\", seed=0, max_ticks=EPISODE_TICKS)\n",
+    "print(f\"Env OK. Initial tick={_obs.tick}, regions={[r.region for r in _obs.regions]}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Build the prompt dataset\n",
+    "\n",
+    "10 prompts total (2 tasks × 5 seeds). Production uses 150."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import random\n",
+    "from datasets import Dataset\n",
+    "\n",
+    "SYSTEM_PROMPT = build_system_prompt()\n",
+    "\n",
+    "\n",
+    "def make_chat_prompt(obs: CrisisworldcortexObservation) -> str:\n",
+    "    return tokenizer.apply_chat_template(\n",
+    "        [\n",
+    "            {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
+    "            {\"role\": \"user\", \"content\": serialize_observation(obs)},\n",
+    "        ],\n",
+    "        tokenize=False,\n",
+    "        add_generation_prompt=True,\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "rng = random.Random(0)\n",
+    "_seed_pool = [\n",
+    "    {\"task\": task, \"seed\": seed}\n",
+    "    for task in TASKS\n",
+    "    for seed in range(SEEDS_PER_TASK)\n",
+    "]\n",
+    "rng.shuffle(_seed_pool)\n",
+    "\n",
+    "_prompts: list[str] = []\n",
+    "_meta: list[dict] = []\n",
+    "for entry in _seed_pool:\n",
+    "    env = make_env()\n",
+    "    obs = env.reset(\n",
+    "        task_name=entry[\"task\"], seed=entry[\"seed\"], max_ticks=EPISODE_TICKS\n",
+    "    )\n",
+    "    _prompts.append(make_chat_prompt(obs))\n",
+    "    _meta.append(entry)\n",
+    "\n",
+    "train_dataset = Dataset.from_dict(\n",
+    "    {\n",
+    "        \"prompt\": _prompts,\n",
+    "        \"task\": [m[\"task\"] for m in _meta],\n",
+    "        \"seed\": [m[\"seed\"] for m in _meta],\n",
+    "    }\n",
+    ")\n",
+    "print(f\"Dataset built: {len(train_dataset)} examples\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Reward function\n",
+    "\n",
+    "Identical contract to the production trainer: parse the completion → submit one env step → return `obs.reward` ∈ `[-1.0, 1.0]`. Parse failures hit the §19 marker-action path (`-1.0` + terminate)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def crisisworld_reward(\n",
+    "    prompts: list[str],\n",
+    "    completions: list[str],\n",
+    "    task: list[str],\n",
+    "    seed: list[int],\n",
+    "    **_kwargs: object,\n",
+    ") -> list[float]:\n",
+    "    rewards: list[float] = []\n",
+    "    for completion, t, s in zip(completions, task, seed):\n",
+    "        env = make_env()\n",
+    "        env.reset(task_name=t, seed=int(s), max_ticks=EPISODE_TICKS)\n",
+    "        action_payload = parse_action(completion) or parse_failure_marker()\n",
+    "        try:\n",
+    "            result = env.step(CrisisworldcortexAction(action=action_payload))\n",
+    "            reward = (\n",
+    "                result.observation.reward\n",
+    "                if hasattr(result, \"observation\")\n",
+    "                else result.reward\n",
+    "            )\n",
+    "            rewards.append(float(reward) if reward is not None else 0.0)\n",
+    "        except Exception as exc:\n",
+    "            print(f\"[WARN] env.step failed task={t} seed={s}: {exc}\")\n",
+    "            rewards.append(-1.0)\n",
+    "    return rewards"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 8. GRPO training (mini)\n",
+    "\n",
+    "20 steps × group size 2 = 40 rollouts. Wall-clock ~5–10 minutes on T4. The point of this notebook is to **show the loop running and producing rewards**, not to converge."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from trl import GRPOConfig, GRPOTrainer\n",
+    "\n",
+    "MAX_TRAIN_STEPS = 20\n",
+    "GROUP_SIZE = 2\n",
+    "MAX_PROMPT_LEN = 1536\n",
+    "MAX_COMPLETION_LEN = 256\n",
+    "\n",
+    "training_args = GRPOConfig(\n",
+    "    output_dir=\"/content/colab_demo_lora\",\n",
+    "    learning_rate=5e-6,\n",
+    "    per_device_train_batch_size=GROUP_SIZE,\n",
+    "    gradient_accumulation_steps=1,\n",
+    "    num_generations=GROUP_SIZE,\n",
+    "    max_prompt_length=MAX_PROMPT_LEN,\n",
+    "    max_completion_length=MAX_COMPLETION_LEN,\n",
+    "    max_steps=MAX_TRAIN_STEPS,\n",
+    "    save_steps=MAX_TRAIN_STEPS,  # save once at the end\n",
+    "    logging_steps=1,\n",
+    "    report_to=\"none\",\n",
+    "    bf16=True,\n",
+    "    optim=\"adamw_8bit\",\n",
+    "    temperature=0.8,\n",
+    "    use_vllm=True,\n",
+    "    vllm_mode=\"colocate\",\n",
+    "    seed=42,\n",
+    ")\n",
+    "\n",
+    "trainer = GRPOTrainer(\n",
+    "    model=model,\n",
+    "    processing_class=tokenizer,\n",
+    "    reward_funcs=[crisisworld_reward],\n",
+    "    args=training_args,\n",
+    "    train_dataset=train_dataset,\n",
+    ")\n",
+    "print(\"GRPOTrainer constructed; starting train()...\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "trainer.train()\n",
+    "print(\"\\n=== Training finished ===\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 9. Reward summary\n",
+    "\n",
+    "Pulls the per-step rewards from the trainer log and plots them. With only 20 steps you should not expect convergence — the goal is to confirm rewards are non-trivially distributed and the gradient pipeline is alive."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "log_history = trainer.state.log_history\n",
+    "reward_steps = [\n",
+    "    (entry.get(\"step\", i), entry[\"reward\"])\n",
+    "    for i, entry in enumerate(log_history)\n",
+    "    if \"reward\" in entry\n",
+    "]\n",
+    "\n",
+    "if reward_steps:\n",
+    "    steps, rewards = zip(*reward_steps)\n",
+    "    print(f\"Logged {len(rewards)} reward points.\")\n",
+    "    print(f\"  mean={sum(rewards)/len(rewards):.3f}  min={min(rewards):.3f}  max={max(rewards):.3f}\")\n",
+    "\n",
+    "    plt.figure(figsize=(8, 4))\n",
+    "    plt.plot(steps, rewards, marker=\"o\", linewidth=1)\n",
+    "    plt.xlabel(\"GRPO step\")\n",
+    "    plt.ylabel(\"Mean group reward\")\n",
+    "    plt.title(\"CrisisWorldCortex mini-demo (SmolLM2-360M)\")\n",
+    "    plt.axhline(0.0, linestyle=\":\", color=\"grey\")\n",
+    "    plt.grid(alpha=0.3)\n",
+    "    plt.tight_layout()\n",
+    "    plt.show()\n",
+    "else:\n",
+    "    print(\"No 'reward' entries in log_history — check trainer logging_steps.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 10. (Optional) Push LoRA adapter to HF Hub\n",
+    "\n",
+    "Skipped if no `HF_TOKEN` was found in step 2. Change `HUB_REPO` to your namespace if you want to push."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "OUTPUT_DIR = \"/content/colab_demo_lora\"\n",
+    "HUB_REPO = \"Angshuman28/crisisworld-colab-demo-smollm2-360m\"\n",
+    "\n",
+    "model.save_pretrained(OUTPUT_DIR)\n",
+    "tokenizer.save_pretrained(OUTPUT_DIR)\n",
+    "print(f\"LoRA adapter saved locally to {OUTPUT_DIR}\")\n",
+    "\n",
+    "if HF_TOKEN:\n",
+    "    from huggingface_hub import HfApi\n",
+    "\n",
+    "    api = HfApi()\n",
+    "    api.create_repo(\n",
+    "        HUB_REPO, exist_ok=True, repo_type=\"model\", private=False, token=HF_TOKEN\n",
+    "    )\n",
+    "    api.upload_folder(\n",
+    "        folder_path=OUTPUT_DIR,\n",
+    "        repo_id=HUB_REPO,\n",
+    "        repo_type=\"model\",\n",
+    "        token=HF_TOKEN,\n",
+    "    )\n",
+    "    print(f\"Pushed to https://huggingface.co/{HUB_REPO}\")\n",
+    "else:\n",
+    "    print(\"HF_TOKEN unset — skipping Hub push. Adapter is still saved locally.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 11. Quick eval — one episode per task\n",
+    "\n",
+    "Runs a single 12-tick episode with the trained adapter on each task at `seed=0` and reports cumulative reward. With only 20 training steps this is a sanity check that the trained model still emits parseable actions, not an evaluation of policy quality."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "FastLanguageModel.for_inference(model)\n",
+    "\n",
+    "\n",
+    "def _hf_chat(system: str, user: str, max_new_tokens: int = 192) -> str:\n",
+    "    prompt = tokenizer.apply_chat_template(\n",
+    "        [\n",
+    "            {\"role\": \"system\", \"content\": system},\n",
+    "            {\"role\": \"user\", \"content\": user},\n",
+    "        ],\n",
+    "        tokenize=False,\n",
+    "        add_generation_prompt=True,\n",
+    "    )\n",
+    "    inputs = tokenizer(prompt, return_tensors=\"pt\").to(model.device)\n",
+    "    with torch.no_grad():\n",
+    "        out = model.generate(\n",
+    "            **inputs,\n",
+    "            max_new_tokens=max_new_tokens,\n",
+    "            do_sample=False,\n",
+    "            temperature=0.0,\n",
+    "        )\n",
+    "    return tokenizer.decode(\n",
+    "        out[0][inputs[\"input_ids\"].shape[1] :], skip_special_tokens=True\n",
+    "    )\n",
+    "\n",
+    "\n",
+    "def run_one_episode(task: str, seed: int) -> tuple[float, int, int]:\n",
+    "    env = make_env()\n",
+    "    obs = env.reset(task_name=task, seed=seed, max_ticks=EPISODE_TICKS)\n",
+    "    cumulative = 0.0\n",
+    "    parsed = 0\n",
+    "    failed = 0\n",
+    "    for _ in range(EPISODE_TICKS):\n",
+    "        completion = _hf_chat(SYSTEM_PROMPT, serialize_observation(obs))\n",
+    "        action = parse_action(completion)\n",
+    "        if action is None:\n",
+    "            failed += 1\n",
+    "            action = parse_failure_marker()\n",
+    "        else:\n",
+    "            parsed += 1\n",
+    "        result = env.step(CrisisworldcortexAction(action=action))\n",
+    "        obs = result.observation if hasattr(result, \"observation\") else result\n",
+    "        cumulative += obs.reward if obs.reward is not None else 0.0\n",
+    "        if obs.done:\n",
+    "            break\n",
+    "    return cumulative, parsed, failed\n",
+    "\n",
+    "\n",
+    "for task in TASKS:\n",
+    "    cum, parsed, failed = run_one_episode(task, seed=0)\n",
+    "    print(\n",
+    "        f\"task={task:<18} cum_reward={cum:+.3f}  parsed={parsed:2d}  parse_failures={failed:2d}\"\n",
+    "    )\n",
+    "\n",
+    "print(\"\\nMini-demo complete.\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

openenv.yaml

@@ -1,7 +1,7 @@
-spec_version: 1
-name: CrisisWorldCortex
-type: space
-runtime: fastapi
-app: server.app:app
-port: 8000
-
+spec_version: 1
+name: CrisisWorldCortex
+type: space
+runtime: fastapi
+app: server.app:app
+port: 8000
+

openenv_CrisisWorldCortex.egg-info/PKG-INFO

@@ -5,8 +5,7 @@ Summary: Crisisworldcortex environment for OpenEnv
 Requires-Python: >=3.10
 Requires-Dist: openenv-core[core]==0.2.3
 Requires-Dist: openai<3.0,>=2.0
-Requires-Dist: datasets>=4.0.0
-Requires-Dist: huggingface-hub>=1.0.0
+Requires-Dist: python-dotenv>=1.0.0
 Provides-Extra: dev
 Requires-Dist: pre-commit>=4.0.0; extra == "dev"
 Requires-Dist: pytest>=8.0.0; extra == "dev"

openenv_CrisisWorldCortex.egg-info/SOURCES.txt

@@ -7,7 +7,6 @@ pyproject.toml
 ./__init__.py
 ./client.py
 ./inference.py
-./mm.py
 ./models.py
 openenv_CrisisWorldCortex.egg-info/PKG-INFO
 openenv_CrisisWorldCortex.egg-info/SOURCES.txt
@@ -21,20 +20,14 @@ server/app.py
 tests/test_actions_round_trip.py
 tests/test_baseline_b1.py
 tests/test_baseline_b2.py
-tests/test_baseline_b3.py
 tests/test_cortex_brain_executive.py
 tests/test_cortex_brain_smoke.py
-tests/test_cortex_council.py
-tests/test_cortex_council_smoke.py
 tests/test_cortex_lenses.py
-tests/test_cortex_metacognition.py
 tests/test_cortex_perception.py
-tests/test_cortex_routing_policy.py
 tests/test_cortex_subagents.py
 tests/test_env_reset_kwargs.py
 tests/test_env_step_reward_wiring.py
 tests/test_import_graph.py
-tests/test_inference_agent_dispatch.py
 tests/test_legal_constraint_enforcement.py
 tests/test_llm_client.py
 tests/test_observation_no_latent_leak.py
@@ -51,9 +44,5 @@ tests/test_smoke_env.py
 tests/test_stdout_format.py
 tests/test_synthetic_rejection_payload.py
 tests/test_training_eval_metrics.py
-tests/test_training_multi_model_skeleton.py
 tests/test_training_reward_shaping.py
-tests/test_training_rollout_buffer.py
-tests/test_training_scripts_b1.py
-tests/test_training_scripts_collect_sft.py
-tests/test_training_scripts_sft.py
+tests/test_training_rollout_buffer.py

openenv_CrisisWorldCortex.egg-info/requires.txt

@@ -1,7 +1,6 @@
 openenv-core[core]==0.2.3
 openai<3.0,>=2.0
-datasets>=4.0.0
-huggingface-hub>=1.0.0
+python-dotenv>=1.0.0
 
 [dev]
 pre-commit>=4.0.0

server/__init__.py

@@ -1,11 +1,11 @@
-# Copyright (c) Meta Platforms, Inc. and affiliates.
-# All rights reserved.
-#
-# This source code is licensed under the BSD-style license found in the
-# LICENSE file in the root directory of this source tree.
-
-"""Crisisworldcortex environment server components."""
-
-from .CrisisWorldCortex_environment import CrisisworldcortexEnvironment
-
-__all__ = ["CrisisworldcortexEnvironment"]
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Crisisworldcortex environment server components."""
+
+from .CrisisWorldCortex_environment import CrisisworldcortexEnvironment
+
+__all__ = ["CrisisworldcortexEnvironment"]

server/requirements.txt

@@ -1,6 +1,6 @@
-openenv[core]>=0.2.0
-fastapi>=0.115.0
-uvicorn>=0.24.0
-
-
-
+openenv[core]>=0.2.0
+fastapi>=0.115.0
+uvicorn>=0.24.0
+
+
+

tests/CLAUDE.md

@@ -1,48 +1,48 @@
-# tests/CLAUDE.md
-
-Test surface per subsystem. Smoke bar first, boundary tests next, coverage last.
-
-## Belongs here
-
-- `conftest.py` — repo root on `sys.path` for bare-name imports.
-- One test module per subsystem boundary (table below).
-
-## Does not belong here
-
-Helpers that mutate real graders, simulator state, or disk. Fixtures that hit the live HF Space — mock or `pytest.skip`.
-
-## Run commands
-
-```bash
-uv run python -m pytest tests/ -v                                                               # all
-uv run python -m pytest tests/test_smoke_env.py::test_reset_returns_valid_observation -v        # one
-uv run python -m pytest --cov tests/                                                            # coverage
-```
-
-## Required tests — each maps to exactly one subsystem contract
-
-| File | Scope | Asserts |
-|---|---|---|
-| `test_package_exports.py` | wire package | Root `__init__` re-exports `CrisisworldcortexAction/Observation/Env`. |
-| `test_smoke_env.py` | `server/` env | `reset()` / `step()` return a valid `CrisisworldcortexObservation`. |
-| `test_actions_round_trip.py` | `server/` env | 6 MVP outer actions round-trip; `public_communication` is rejected at runtime. |
-| `test_reward_shape.py` | `server/graders/` | Every grader returns values in `[0.0, 1.0]`. |
-| `test_reward_non_constancy.py` | `server/graders/` | Grader output varies across ≥ 2 synthetic episodes. |
-| `test_anti_hivemind_protocol.py` | `cortex/` | 5 protocol steps fire in order; caps enforced (2 rounds, 1 cross-brain challenge, 1 Critic/brain/tick). |
-| `test_collapse_detector.py` | `cortex/` | Metacognition flags when all brains recommend the same action. |
-| `test_import_graph.py` | repo-wide | No `import server` under `cortex/**`; no `import cortex` under `server/**`; no `import server.simulator` under `training/**`. |
-| `test_baselines_smoke.py` | `baselines/` | B1 / B2 / B3 each run one episode on `outbreak_easy`. |
-| `test_training_smoke.py` | `training/` | `train_router.main()` runs one episode against a mocked env under 5 s. |
-
-## Binding rules
-
-- Every public API in a subsystem's CLAUDE.md has ≥ 1 test here.
-- Coverage target: 80% per subsystem; 100% for `server/graders/` and `cortex/anti_hivemind.py`.
-- No test may take > 10 s unless marked `@pytest.mark.slow` and gated behind `--runslow`.
-- `test_import_graph.py` uses a fresh subprocess import, not `sys.modules` monkey-patching — the latter passes under contamination.
-
-## Common failure modes
-
-- Smoke test asserting on current-echo values — breaks when real env logic lands. Assert on shape, not value.
-- Module-scope env instantiation in tests — slows collection and hides init errors until runtime.
-- Tests that hit the HF Space without a skip guard — CI flakes on rate limits.
+# tests/CLAUDE.md
+
+Test surface per subsystem. Smoke bar first, boundary tests next, coverage last.
+
+## Belongs here
+
+- `conftest.py` — repo root on `sys.path` for bare-name imports.
+- One test module per subsystem boundary (table below).
+
+## Does not belong here
+
+Helpers that mutate real graders, simulator state, or disk. Fixtures that hit the live HF Space — mock or `pytest.skip`.
+
+## Run commands
+
+```bash
+uv run python -m pytest tests/ -v                                                               # all
+uv run python -m pytest tests/test_smoke_env.py::test_reset_returns_valid_observation -v        # one
+uv run python -m pytest --cov tests/                                                            # coverage
+```
+
+## Required tests — each maps to exactly one subsystem contract
+
+| File | Scope | Asserts |
+|---|---|---|
+| `test_package_exports.py` | wire package | Root `__init__` re-exports `CrisisworldcortexAction/Observation/Env`. |
+| `test_smoke_env.py` | `server/` env | `reset()` / `step()` return a valid `CrisisworldcortexObservation`. |
+| `test_actions_round_trip.py` | `server/` env | 6 MVP outer actions round-trip; `public_communication` is rejected at runtime. |
+| `test_reward_shape.py` | `server/graders/` | Every grader returns values in `[0.0, 1.0]`. |
+| `test_reward_non_constancy.py` | `server/graders/` | Grader output varies across ≥ 2 synthetic episodes. |
+| `test_anti_hivemind_protocol.py` | `cortex/` | 5 protocol steps fire in order; caps enforced (2 rounds, 1 cross-brain challenge, 1 Critic/brain/tick). |
+| `test_collapse_detector.py` | `cortex/` | Metacognition flags when all brains recommend the same action. |
+| `test_import_graph.py` | repo-wide | No `import server` under `cortex/**`; no `import cortex` under `server/**`; no `import server.simulator` under `training/**`. |
+| `test_baselines_smoke.py` | `baselines/` | B1 / B2 / B3 each run one episode on `outbreak_easy`. |
+| `test_training_smoke.py` | `training/` | `train_router.main()` runs one episode against a mocked env under 5 s. |
+
+## Binding rules
+
+- Every public API in a subsystem's CLAUDE.md has ≥ 1 test here.
+- Coverage target: 80% per subsystem; 100% for `server/graders/` and `cortex/anti_hivemind.py`.
+- No test may take > 10 s unless marked `@pytest.mark.slow` and gated behind `--runslow`.
+- `test_import_graph.py` uses a fresh subprocess import, not `sys.modules` monkey-patching — the latter passes under contamination.
+
+## Common failure modes
+
+- Smoke test asserting on current-echo values — breaks when real env logic lands. Assert on shape, not value.
+- Module-scope env instantiation in tests — slows collection and hides init errors until runtime.
+- Tests that hit the HF Space without a skip guard — CI flakes on rate limits.

training/CLAUDE.md

@@ -1,64 +1,64 @@
-# training/CLAUDE.md
-
-GRPO training for the routing policy. Two Colab notebooks required for hackathon compliance.
-
-## Belongs here
-
-- `train_router.py` — manual PyTorch GRPO loop for the MLP router (Option B primary).
-- `rollout_buffer.py` — trajectory collection + router-step serialization.
-- `reward_shaping.py` — composes the 4-term training reward from `server.graders` outputs.
-- `train_router_colab.ipynb` — Colab-runnable notebook for the MLP router. **Required.**
-- `train_flat_agent_trl.ipynb` — TRL `GRPOTrainer` + Unsloth LoRA on a flat agent against `outbreak_easy`. **Required** for finale Unsloth/TRL compliance (design §E.3).
-- `configs/grpo_config.yaml` — GRPO hyperparameters.
-- `configs/tasks.yaml` — task-curriculum config (easy → medium → hard).
-
-## Does not belong here
-
-Baseline agents (→ `baselines/`). Plotting / reward curves (→ `demo/`, `scripts/`). SEIR dynamics (→ `server/simulator/`).
-
-## Allowed imports
-
-- `models`, `client`.
-- `cortex.routing_policy`, `cortex.council`, `cortex.schemas`, `cortex.metacognition`.
-- `server.graders` — **reward-name constants only** (e.g. the `training_reward` dict keys). Do not import `server.simulator`. Do not instantiate the env in-process.
-- Torch, TRL, Unsloth (compliance notebook only).
-
-## Forbidden imports
-
-- `server.simulator/*` — training hits the env over HTTP like production.
-- `baselines/*`, `demo/*`.
-
-## Binding contracts
-
-- **Training-data rows = router steps**, not ticks or rounds. One row per `RoutingAction` emission.
-- **Training reward = exactly the 4 terms returned by `server.graders.training_reward`** (see `server/CLAUDE.md` for the dict schema). Never mix in eval-only metrics.
-- Training episode length = 10–12 ticks. Eval episode length = 20 ticks (only if training is stable; otherwise eval also runs at 12).
-- Temperature > 0 on LLM subagents during rollouts (exploration); temperature = 0 during eval (reproducibility).
-- Pin the OpenEnv version in `pyproject.toml` before training runs — finale requires "latest release" at submission.
-
-## Colab notebook contracts
-
-- `train_router_colab.ipynb`: imports CrisisWorld as a local Python module (no Docker in Colab). Runs end-to-end on a fresh Colab T4.
-- `train_flat_agent_trl.ipynb`: uses `trl.GRPOTrainer(environment_factory=CrisisworldcortexEnv, ...)` + `unsloth.FastLanguageModel` LoRA wrapper. Runs end-to-end on a fresh Colab T4. **This notebook's absence disqualifies the submission.**
-
-## Public APIs (owned here)
-
-- `train_router.main(config_path: str) -> None`
-- `RolloutBuffer.add(router_step: RouterStep) -> None`
-- `RolloutBuffer.sample(batch_size: int) -> list[RouterStep]`
-- `shape_reward(trajectory: Trajectory) -> float` — weighted combination of the 4 training terms.
-
-## Testing requirements
-
-- `shape_reward` returns a scalar in `[0.0, 1.0]`.
-- `RolloutBuffer` round-trips synthetic router steps without data loss.
-- `train_router.py` runs 1 episode end-to-end against a mocked env in under 5 seconds (CI smoke).
-- Both Colab notebooks execute to completion for ≥ a few hundred training steps on Colab T4 pre-onsite.
-
-## Common failure modes
-
-- Logging ticks as training rows — collapses router's action granularity; GRPO credit assignment breaks.
-- Mixing eval metrics into the training reward — inflates the headline curve for reasons the paper can't defend.
-- Widening training episodes past 12 ticks — rollouts stop fitting in the GRPO update window; wall-clock explodes.
-- Training-reward dict keys drifting from `server.graders.training_reward` — shape-only tests miss this; trainer silently optimizes the wrong signal.
-- Missing the TRL compliance notebook at submission — automatic finale failure.
+# training/CLAUDE.md
+
+GRPO training for the routing policy. Two Colab notebooks required for hackathon compliance.
+
+## Belongs here
+
+- `train_router.py` — manual PyTorch GRPO loop for the MLP router (Option B primary).
+- `rollout_buffer.py` — trajectory collection + router-step serialization.
+- `reward_shaping.py` — composes the 4-term training reward from `server.graders` outputs.
+- `train_router_colab.ipynb` — Colab-runnable notebook for the MLP router. **Required.**
+- `train_flat_agent_trl.ipynb` — TRL `GRPOTrainer` + Unsloth LoRA on a flat agent against `outbreak_easy`. **Required** for finale Unsloth/TRL compliance (design §E.3).
+- `configs/grpo_config.yaml` — GRPO hyperparameters.
+- `configs/tasks.yaml` — task-curriculum config (easy → medium → hard).
+
+## Does not belong here
+
+Baseline agents (→ `baselines/`). Plotting / reward curves (→ `demo/`, `scripts/`). SEIR dynamics (→ `server/simulator/`).
+
+## Allowed imports
+
+- `models`, `client`.
+- `cortex.routing_policy`, `cortex.council`, `cortex.schemas`, `cortex.metacognition`.
+- `server.graders` — **reward-name constants only** (e.g. the `training_reward` dict keys). Do not import `server.simulator`. Do not instantiate the env in-process.
+- Torch, TRL, Unsloth (compliance notebook only).
+
+## Forbidden imports
+
+- `server.simulator/*` — training hits the env over HTTP like production.
+- `baselines/*`, `demo/*`.
+
+## Binding contracts
+
+- **Training-data rows = router steps**, not ticks or rounds. One row per `RoutingAction` emission.
+- **Training reward = exactly the 4 terms returned by `server.graders.training_reward`** (see `server/CLAUDE.md` for the dict schema). Never mix in eval-only metrics.
+- Training episode length = 10–12 ticks. Eval episode length = 20 ticks (only if training is stable; otherwise eval also runs at 12).
+- Temperature > 0 on LLM subagents during rollouts (exploration); temperature = 0 during eval (reproducibility).
+- Pin the OpenEnv version in `pyproject.toml` before training runs — finale requires "latest release" at submission.
+
+## Colab notebook contracts
+
+- `train_router_colab.ipynb`: imports CrisisWorld as a local Python module (no Docker in Colab). Runs end-to-end on a fresh Colab T4.
+- `train_flat_agent_trl.ipynb`: uses `trl.GRPOTrainer(environment_factory=CrisisworldcortexEnv, ...)` + `unsloth.FastLanguageModel` LoRA wrapper. Runs end-to-end on a fresh Colab T4. **This notebook's absence disqualifies the submission.**
+
+## Public APIs (owned here)
+
+- `train_router.main(config_path: str) -> None`
+- `RolloutBuffer.add(router_step: RouterStep) -> None`
+- `RolloutBuffer.sample(batch_size: int) -> list[RouterStep]`
+- `shape_reward(trajectory: Trajectory) -> float` — weighted combination of the 4 training terms.
+
+## Testing requirements
+
+- `shape_reward` returns a scalar in `[0.0, 1.0]`.
+- `RolloutBuffer` round-trips synthetic router steps without data loss.
+- `train_router.py` runs 1 episode end-to-end against a mocked env in under 5 seconds (CI smoke).
+- Both Colab notebooks execute to completion for ≥ a few hundred training steps on Colab T4 pre-onsite.
+
+## Common failure modes
+
+- Logging ticks as training rows — collapses router's action granularity; GRPO credit assignment breaks.
+- Mixing eval metrics into the training reward — inflates the headline curve for reasons the paper can't defend.
+- Widening training episodes past 12 ticks — rollouts stop fitting in the GRPO update window; wall-clock explodes.
+- Training-reward dict keys drifting from `server.graders.training_reward` — shape-only tests miss this; trainer silently optimizes the wrong signal.
+- Missing the TRL compliance notebook at submission — automatic finale failure.