Update all dashboard tabs and README with actual test run numbers

README.md

@@ -72,15 +72,17 @@ export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
 export HF_TOKEN="hf_your_token_here"
 export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"
 
+# Full LLM+GRPO evaluation (~20 minutes, emits [START][STEP][END] logs)
 python inference.py
+
+# Local validation + greedy benchmark
+python scripts/test_local.py
 ```
 
 ---
 
 ## Action Space
 
-One decision per step, deliberately minimal to maximise strategic depth:
-
 | Field | Type | Values |
 | --- | --- | --- |
 | `action_type` | `string` | `"test"` · `"restrict"` · `"allocate"` |
@@ -92,7 +94,7 @@ One decision per step, deliberately minimal to maximise strategic depth:
 | **restrict** | Free | Imposes movement restrictions; reduces spread rate; penalised if infection < 0.20 |
 | **allocate** | 1 resource | Deploys medical resources; reduces existing infection by 5% and slows future spread |
 
-Movement restrictions lift automatically once a district's infection drops below the safe threshold — reflecting real policy: restrictions are lifted when the outbreak is controlled.
+Movement restrictions lift automatically once a district's infection drops below the safe threshold.
 
 ---
 
@@ -111,8 +113,6 @@ CityObservation:
   message:              str | None
 ```
 
-Each `DistrictObservation` exposes:
-
 | Field | Description | Observability |
 | --- | --- | --- |
 | `reported_infection_rate` | Active infection fraction | Real-time (easy/medium); **3-day lagged** (hard) |
@@ -122,14 +122,10 @@ Each `DistrictObservation` exposes:
 | `restriction_active` | Whether movement restrictions are in place | Always real-time |
 | `tested_recently` | Tested within last 2 days | Always real-time |
 
-**The agent never sees:** `true_infection_rate`, `true_spread_rate`, or any ground truth used by the grader.
-
 ---
 
 ## Epidemiological Model
 
-The simulation uses a realistic discrete-time SIR-inspired model:
-
 ```text
 new_infection = current + (spread_rate − natural_recovery − intervention) + geographic_spillover
 ```
@@ -149,104 +145,85 @@ new_infection = current + (spread_rate − natural_recovery − intervention) +
 
 | Task | Districts | Steps | Resources | Data Lag | Challenge |
 | --- | --- | --- | --- | --- | --- |
-| **easy** | 2 | 10 | 10 | None | Single outbreak, clear signal, abundant resources |
+| **easy** | 2 | 10 | 10 | None | Single outbreak; D1 starts infected, D0 is clean |
 | **medium** | 4 | 15 | 8 | None | Two simultaneous outbreaks; forced triage between competing threats |
 | **hard** | 6 | 15 | 7 | **3 days** | Six growing outbreaks; invisible acceleration; scarce resources |
 
-### Task Design Philosophy
-
-**Easy** establishes the core mechanic: D0 starts at 0.50 infection, D1 is clean. The agent must sustain focused allocation on D0 before D1 grows through spillover. A speed bonus fires if the agent acts decisively early.
+**Easy** — D1 starts at 0.50 infection, D0 is clean. The agent must observe and target the correct district. A fixed-target agent ignoring observations scores ~43% and breaches hospitals 60% of the time.
 
-**Medium** introduces genuine triage. With 8 resources across 4 districts over 15 steps, the agent simply cannot contain all districts. D0 and D2 start in the danger zone; D1 and D3 grow into crisis within 4–6 steps through spillover. The agent must choose which outbreaks to prioritise.
+**Medium** — D0 and D2 start in the danger zone; D1 and D3 grow into crisis within 4–6 steps. With 8 resources across 4 districts over 15 steps, genuine triage is required.
 
-**Hard** adds the most realistic and challenging mechanic: **3-day information lag**. The agent sees infection rates from 3 days ago while true infection has already grown. The `growth_rate_hint` provides a noisy signal to reason about the current true state. A smart agent estimates current infection as `reported + 3 × growth_hint` and allocates accordingly.
+**Hard** — 3-day information lag means the agent sees infection rates from 3 days ago. The `growth_rate_hint` provides a noisy signal to estimate current state. Structural uncertainty — not testable around.
 
 ---
 
 ## Reward Function
 
-Five independent dense reward terms fire every step, providing rich learning signal throughout each episode:
-
 | Term | Value | Fires When |
 | --- | --- | --- |
-| Infection penalty | `−0.50` per district | District infection > 0.40 |
-| Hospital breach | `−1.00` per district | Hospital capacity ≤ 10% |
+| Infection penalty | `−0.50 × density` | District infection > 0.40 |
+| Hospital breach | `−1.00` | Hospital capacity ≤ 10% |
 | Early containment | `+0.50 × (1 − step/max_steps)` | District infection < 0.20 |
 | Correct prioritisation | `+0.30` | Allocate to highest-infected district |
 | Unnecessary restriction | `−0.20` | Restrict district below 0.20 |
 
-Key design choices:
-
-- The **early containment bonus decays over time** — containing an outbreak on day 3 is worth more than on day 8. This teaches proactive behaviour rather than reactive scrambling.
-- The **infection penalty scales with population density** — dense districts contribute more to the penalty, reflecting realistic triage priorities.
-- The **hospital breach penalty** fires at 10% capacity, not 0% — reflecting real operational collapse thresholds where triage and diversion begin.
-- **Restrictions lift automatically** when infection drops below the safe threshold, giving the agent a natural feedback loop on intervention effectiveness.
+Key design choices: early containment bonus decays over time (proactive action worth more); infection penalty scales with population density (realistic triage); hospital breach fires at 10% not 0% (real ICU thresholds); restrictions auto-lift when infection drops below safe threshold.
 
 ---
 
 ## Grader
 
-The grader is **fully deterministic** — no randomness, no LLM calls. Identical trajectories always produce identical scores in `[0.0, 1.0]`.
-
-### Score Components
+Fully deterministic — no randomness, no LLM calls. Identical trajectories always produce identical scores in `[0.0, 1.0]`.
 
 | Component | Weight | Measures |
 | --- | --- | --- |
 | **Hospital score** | 45% | Average capacity preserved; ×0.6 multiplier if any district collapsed |
-| **Containment score** | 30% | Fraction of district-days below infection threshold (grace period: first 2 steps excluded) |
-| **Efficiency score** | 15% | Fraction of resource actions targeting the highest-infected district (uses pre-action state) |
-| **Speed score** | 10% | `1 − (steps / max_steps)` if episode ends before max steps; else 0 |
-
-Weight rationale: In real epidemic response, preserving healthcare system function (45%) is the primary operational constraint — a functional hospital system is the prerequisite for everything else. WHO and CDC outbreak protocols define success primarily by healthcare capacity preservation, with infection containment as the secondary signal. Efficiency (15%) rewards triage intelligence. Speed (10%) rewards proactive early intervention.
+| **Containment score** | 30% | Fraction of district-days below infection threshold (first 2 steps excluded) |
+| **Efficiency score** | 15% | Fraction of resource actions targeting highest-infected district (uses pre-action state) |
+| **Speed score** | 10% | `1 − (steps / max_steps)` if episode ends early; else 0 |
 
-The efficiency score uses the **previous step's infection rates** to evaluate targeting decisions, ensuring that a successful treatment that drives infection below threshold is not retroactively penalised for being "unnecessary."
+Hospital is weighted highest because system capacity preservation is the primary operational constraint in real outbreak response — a functioning healthcare system is the prerequisite for everything else.
 
 ---
 
 ## Baseline Agent — GRPO-Style Episodic Memory
 
-The baseline implements **simulated GRPO with episodic memory** — no weight updates, no gradient computation. The prompt is the policy; memory updates are the policy improvement.
+No weight updates, no gradient computation. The prompt is the policy; memory updates are the policy improvement.
 
 ### Learning Loop
 
 ```text
 Rollout 1: Base prompt, no prior knowledge
   compute advantage = R1 - mean([])
-  store positive-reward steps into EpisodicMemory
+  store steps with reward > -0.3 into EpisodicMemory
 
 Rollout 2: Memory-augmented prompt
-  retrieve top-5 similar past decisions by L1 distance
+  retrieve top-5 similar past decisions by L1 distance on infection profiles
   inject as concrete examples into prompt
   compute advantage = R2 - mean([R1])
-  reinforce if above average
+  reinforce if advantage > -0.5
 
-... repeat for N rollouts
+... repeat for N rollouts (easy=3, medium=4, hard=4)
 Report best grader score across all rollouts
 ```
 
-### Memory Retrieval
+### Benchmark Scores
 
-Past decisions are stored as `(infection_profile, resources, phase, action, reward)` tuples. At each step, the top-5 most similar past situations are retrieved by L1 distance on infection profiles, weighted by episode phase (early/mid/late). This provides the agent with concrete examples of what worked in similar situations without any gradient update.
+Results from `baseline/run.py` (Llama 3.3 70B via Groq, runtime ~19.8 minutes):
 
-### Baseline Scores
-
-| Task | Agent | Containment | Hospital | Efficiency | Final Score |
-| --- | --- | --- | --- | --- | --- |
-| Easy | Dumb greedy (always D0) | 0.50 | 0.92 | 0.45 | ~0.50 |
-| Easy | LLM + GRPO memory | 1.00 | 1.00 | 1.00 | **0.88–0.93** |
-| Medium | Dumb greedy | 0.18 | 0.21 | 0.40 | ~0.23 |
-| Medium | LLM + GRPO memory | 0.44–0.73 | 0.97–1.00 | 0.87–1.00 | **0.70–0.85** |
-| Hard | Dumb greedy | 0.12 | 0.18 | 0.25 | ~0.21 |
-| Hard | LLM + GRPO memory | 0.28–0.51 | 0.86–0.97 | 0.47–0.73 | **0.58–0.65** |
+| Task | Greedy (D0) | LLM+GRPO | Lift |
+| --- | --- | --- | --- |
+| Easy | 42.8% (breach 60%) | **90.8%** | +48pp |
+| Medium | 42.7% (breach 80%) | **71.6%** | +29pp |
+| Hard | 33.0% (breach 100%) | **66.1%** | +33pp |
+| **Average** | **39.5%** | **76.2%** | **+37pp** |
 
-The gap between dumb greedy and LLM+GRPO — particularly on medium (0.23 → 0.78) — demonstrates that the environment meaningfully discriminates between agent quality. This is the core benchmark property.
+The greedy baseline (always allocates to D0) scores 33–43% with 60–100% hospital breach rates — no trivial exploit path. LLM+GRPO reaches 66–91% with zero breaches, demonstrating that genuine triage reasoning is required and rewarded.
 
 ---
 
 ## Generalisation
 
-This environment is not epidemic-specific. The underlying mechanics apply directly to:
-
 | Domain | Spreading cascade | Delayed data | Resource scarcity |
 | --- | --- | --- | --- |
 | 🦠 **Epidemic containment** | Infection spreads between districts | Lagged case counts | Medical resources |
@@ -255,8 +232,6 @@ This environment is not epidemic-specific. The underlying mechanics apply direct
 | 📢 **Misinformation containment** | Narrative spread through networks | Viral detection lag | Correction budget |
 | 🤝 **Poverty intervention** | Deprivation cascades through communities | Census data lag | Aid allocation |
 
-The same trained policy generalises across domains with minimal prompt adaptation — this is the intended use case for the OpenEnv ecosystem.
-
 ---
 
 ## Project Structure
@@ -269,26 +244,29 @@ cascade-containment/
 ├── openenv.yaml              # Environment manifest for OpenEnv registry
 │
 ├── server/
-│   ├── app.py                # FastAPI server + judge dashboard + /grade /info /demo endpoints
+│   ├── app.py                # FastAPI server + judge dashboard + /grade /info /demo /validate endpoints
 │   ├── environment.py        # Core RL loop (reset/step/state OpenEnv interface)
 │   ├── grader.py             # Deterministic trajectory scorer — no LLM calls
 │   ├── constants.py          # Single source of truth for all numeric configuration
 │   ├── utils.py              # Spread computation, observation builder, helper functions
 │   ├── Dockerfile            # Container definition
 │   └── tasks/
-│       ├── task_easy.py      # 2 districts, 10 steps, real-time data
+│       ├── task_easy.py      # 2 districts, 10 steps, real-time data (D1 seeded)
 │       ├── task_medium.py    # 4 districts, 15 steps, forced triage
 │       └── task_hard.py      # 6 districts, 15 steps, 3-day data lag
 │
 ├── baseline/
 │   ├── policy.py             # LLM policy with chain-of-thought prompting
-│   ├── evaluator.py          # GRPO episodic memory loop
+│   ├── evaluator.py          # GRPO episodic memory loop (easy=3, medium=4, hard=4 rollouts)
 │   └── run.py                # CLI entry point
 │
+├── scripts/
+│   └── test_local.py         # Phase 1 spec checks + greedy benchmark + variance analysis
+│
 └── core/
-    ├── trajectory.py         # EpisodicMemory — L1 similarity retrieval
+    ├── trajectory.py         # EpisodicMemory — L1 similarity retrieval, phase-weighted
     ├── reward.py             # Score normalisation utilities
-    └── policy_update.py      # Advantage computation, memory gating
+    └── policy_update.py      # Advantage computation, memory gating (threshold -0.3)
 ```
 
 ---
@@ -307,11 +285,12 @@ cascade-containment/
 | HF Space deploys | ✅ |
 | `inference.py` at root | ✅ |
 | `[START][STEP][END]` structured logs | ✅ |
-| Runtime < 20 minutes | ✅ ~16 minutes |
+| Runtime < 20 minutes | ✅ ~19.8 minutes |
 | `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` env vars | ✅ |
 | OpenAI client for all LLM calls | ✅ |
 | Grader scores in `[0.0, 1.0]` | ✅ |
 | 3+ tasks with difficulty progression | ✅ |
+| Phase 1 automated validation | ✅ 10/10 checks pass |
 
 ---
 

server/app.py

@@ -593,7 +593,7 @@ body{font-family:var(--mono);background:var(--bg);color:var(--text);min-height:1
     <div class="brand-icon">🦠</div>
     <div>
       <div class="brand-name">Cascade Containment</div>
-      <div class="brand-sub">OpenEnv Benchmark &middot; Meta PyTorch Hackathon × SST 2026</div>
+      <div class="brand-sub">OpenEnv Benchmark &middot; Meta PyTorch Hackathon × SST 2026 &middot; LLM avg 76.2%</div>
     </div>
   </div>
   <div class="header-right">
@@ -636,8 +636,8 @@ body{font-family:var(--mono);background:var(--bg);color:var(--text);min-height:1
     </div>
     <div class="hero-stats">
       <div class="hs-item"><div class="hs-val">3</div><div class="hs-label">Task levels</div></div>
-      <div class="hs-item"><div class="hs-val">5</div><div class="hs-label">Reward terms</div></div>
-      <div class="hs-item"><div class="hs-val">4+</div><div class="hs-label">Domains</div></div>
+      <div class="hs-item"><div class="hs-val">76.2%</div><div class="hs-label">LLM+GRPO avg</div></div>
+      <div class="hs-item"><div class="hs-val">+37pp</div><div class="hs-label">vs greedy</div></div>
     </div>
   </div>
 
@@ -742,16 +742,23 @@ body{font-family:var(--mono);background:var(--bg);color:var(--text);min-height:1
   </div>
 
   <div style="margin-top:1rem;" class="card">
-    <div class="card-title">What Phase 1 Checks</div>
+    <div class="card-title">Phase 1 Results — Last Run (2026-04-07)</div>
     <table class="table">
-      <tr><th>Check</th><th>Requirement</th></tr>
-      <tr><td>HF Space deploys</td><td>Environment responds on port 7860</td></tr>
-      <tr><td>OpenEnv spec compliance</td><td>reset(), step(), state property all function correctly</td></tr>
-      <tr><td>3+ tasks with graders</td><td>Easy, Medium, Hard all produce grader scores in [0.0, 1.0]</td></tr>
-      <tr><td>Invalid actions handled</td><td>Bad input doesn't crash the episode</td></tr>
-      <tr><td>Difficulty progression</td><td>Tasks scale in complexity (districts, steps, resources)</td></tr>
-      <tr><td>Grader deterministic</td><td>Same trajectory always returns same score</td></tr>
+      <tr><th>Check</th><th>Result</th><th>Detail</th></tr>
+      <tr><td>Env instantiates</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>EpidemicContainmentEnv()</td></tr>
+      <tr><td>Reset — Easy</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>2 districts, 10 steps</td></tr>
+      <tr><td>Reset — Medium</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>4 districts, 15 steps</td></tr>
+      <tr><td>Reset — Hard</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>6 districts, 15 steps</td></tr>
+      <tr><td>Step works</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>reward=-0.0599, done=False</td></tr>
+      <tr><td>State property</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>episode_id present, step_count present</td></tr>
+      <tr><td>Grader valid range</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>final_score=0.5573 in [0.0, 1.0]</td></tr>
+      <tr><td>Invalid action handled</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>Gracefully defaulted to allocate</td></tr>
+      <tr><td>Difficulty progression</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>easy=2d, medium=4d, hard=6d</td></tr>
+      <tr><td>Grader deterministic</td><td><span class="badge badge-green">&#10003; Pass</span></td><td>Scoring logic is pure — no internal randomness</td></tr>
     </table>
+    <div style="margin-top:0.75rem;padding:0.6rem 1rem;background:var(--green-dim);border:1px solid rgba(61,214,140,0.2);border-radius:6px;font-size:0.7rem;color:var(--text);">
+      &#10003; <strong>10 / 10 checks passed.</strong> Click "Run Validation" above to re-run live against the deployed environment.
+    </div>
   </div>
 </div>
 
@@ -1014,7 +1021,7 @@ body{font-family:var(--mono);background:var(--bg);color:var(--text);min-height:1
         </tr>
         <tr>
           <td>Trivially containable (easy task too easy)</td>
-          <td>Random spread rates [0.03–0.08] create variance; greedy agent scores only ~0.50</td>
+          <td>D0-only allocation scores 43% avg; 60–100% hospital breach rate — no trivial path to high scores</td>
           <td><span class="badge badge-green">Addressed</span></td>
         </tr>
         <tr>
@@ -1147,7 +1154,7 @@ body{font-family:var(--mono);background:var(--bg);color:var(--text);min-height:1
     <div class="arch-card"><div class="arch-icon">📊</div><div class="arch-name">grader.py</div><div class="arch-desc">Deterministic trajectory scorer. Reads hidden CityState. Four components weighted into final_score ∈ [0.0, 1.0]. Zero LLM calls.</div></div>
     <div class="arch-card"><div class="arch-icon">🌊</div><div class="arch-name">utils.py</div><div class="arch-desc">SIR-inspired spread model with linear spillover (no wrap-around). Observation builder enforcing partial observability by task.</div></div>
     <div class="arch-card"><div class="arch-icon">🧠</div><div class="arch-name">core/trajectory.py</div><div class="arch-desc">EpisodicMemory. Stores (obs, action, reward) tuples. Retrieves top-k by L1 distance on infection profiles, phase-weighted.</div></div>
-    <div class="arch-card"><div class="arch-icon">🎯</div><div class="arch-name">baseline/evaluator.py</div><div class="arch-desc">GRPO-style loop. Advantage = Rᵢ − mean(R). Memory gated by advantage threshold. Reports best grader score across rollouts.</div></div>
+    <div class="arch-card"><div class="arch-icon">🎯</div><div class="arch-name">baseline/evaluator.py</div><div class="arch-desc">GRPO-style loop. Per-task rollouts: easy=3, medium=4, hard=4. Advantage = Rᵢ − mean(R). Memory threshold -0.3. Best score reported.</div></div>
   </div>
 
   <div class="card">