README.md

---
title: Cascade Containment
emoji: 🦠
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
---

## 🦠 An RL Benchmark for Sequential Resource Allocation Under Spreading Cascade Dynamics

[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compliant-blue?style=flat-square)](https://github.com/meta-pytorch/OpenEnv)
[![Python](https://img.shields.io/badge/Python-3.10%2B-blue?style=flat-square)](https://python.org)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue?style=flat-square)](https://hub.docker.com)
[![HF Space](https://img.shields.io/badge/HF%20Space-Live-green?style=flat-square)](https://huggingface.co/spaces/TheRubberDuckDebuggers/cascade-containment)
[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](LICENSE)

Meta PyTorch OpenEnv Hackathon × SST 2026 — [Live Demo](https://therubberduckdebuggers-cascade-containment.hf.space) · [GitHub](https://github.com/Rohitchandramouli/cascade-containment)

---

## The Problem

Sequential resource allocation under uncertainty is one of the most consequential decision problems in the real world. Whether containing an epidemic, deploying firefighting crews, isolating a cyberattack, or routing aid — the agent faces the same fundamental challenge:

- **Resources are scarce** — you cannot cover every district simultaneously
- **Data is delayed** — by the time a crisis is visible, it has already grown
- **Interventions cascade** — actions in one district affect adjacent ones
- **Acting too late is catastrophic** — hospital collapse ends the episode; proactive containment is rewarded exponentially more than reactive response

No existing OpenEnv benchmark formalises this problem class. Cascade Containment does.

---

## Environment Overview

A city health authority must allocate limited medical resources across districts to contain a spreading outbreak. Each step, the agent observes district infection rates (possibly lagged), hospital capacity levels, and growth signals — then decides where to deploy resources, impose restrictions, or gather data.

The environment is **not epidemic-specific**. The underlying mechanics — spreading cascade, delayed observation, resource scarcity, geographic spillover — are structurally identical across multiple real-world domains.

---

## Quick Start

```python
from client import CascadeContainmentEnv
from models import ContainmentAction

with CascadeContainmentEnv(
    base_url="https://therubberduckdebuggers-cascade-containment.hf.space"
).sync() as env:
    result = env.reset(task_name="medium")
    obs    = result.observation

    while not result.done:
        most_infected = max(obs.districts, key=lambda d: d.reported_infection_rate)
        action = ContainmentAction(
            action_type="allocate",
            district_id=most_infected.district_id
        )
        result = env.step(action)
        obs    = result.observation
        print(f"Step {obs.current_step}: reward={result.reward:.3f}")
```

## Running the Full Baseline Evaluation

```bash
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
export HF_TOKEN="hf_your_token_here"
export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"

# Full LLM+GRPO evaluation (~10 minutes, emits [START][STEP][END] logs)
python inference.py

# Local validation + greedy benchmark
python scripts/test_local.py
```

---

## Action Space

| Field | Type | Values |
| --- | --- | --- |
| `action_type` | `string` | `"test"` · `"restrict"` · `"allocate"` |
| `district_id` | `int` | 0-indexed district target |

| Action | Cost | Effect |
| --- | --- | --- |
| **test** | 1 resource | Reveals accurate current infection data for district |
| **restrict** | Free | Imposes movement restrictions; reduces spread rate; penalised if infection < 0.20 |
| **allocate** | 1 resource | Deploys medical resources; reduces existing infection by 5% and slows future spread |

Movement restrictions lift automatically once a district's infection drops below the safe threshold.

---

## Observation Space

The agent receives a filtered, potentially lagged view of the world — **never the full ground truth**:

```python
CityObservation:
  districts:            List[DistrictObservation]  # per-district visible state
  available_resources:  int                         # budget remaining this step
  current_step:         int
  max_steps:            int
  done:                 bool
  reward:               float | None
  message:              str | None
```

| Field | Description | Observability |
| --- | --- | --- |
| `reported_infection_rate` | Active infection fraction | Real-time (easy/medium); **3-day lagged** (hard) |
| `growth_rate_hint` | Noisy signal of true spread rate | Always real-time ± noise |
| `hospital_capacity_remaining` | ICU/ward capacity fraction | Always real-time |
| `population_density` | District's share of city population | Always real-time |
| `restriction_active` | Whether movement restrictions are in place | Always real-time |
| `tested_recently` | Tested within last 2 days | Always real-time |

---

## Epidemiological Model

```text
new_infection = current + (spread_rate − natural_recovery − intervention) + geographic_spillover
```

| Parameter | Value | Rationale |
| --- | --- | --- |
| Spread rate | 3–8% per day | Realistic for respiratory outbreaks (seasonal flu: 5–10%) |
| Natural recovery | 1% per day | Background case resolution without medical intervention |
| Treatment effect | −5% existing infection | Medical deployment (antivirals, PPE, rapid response) |
| Spread reduction | −10% per allocation | Resource-driven suppression of transmission |
| Geographic spillover | 1% to adjacent districts | Linear topology — no wrap-around (geographically realistic) |
| Hospital breach threshold | ≤10% capacity | Real ICU overflow and triage failure threshold |

---

## Three Tasks

| Task | Districts | Steps | Resources | Data Lag | Challenge |
| --- | --- | --- | --- | --- | --- |
| **easy** | 2 | 10 | 10 | None | Single outbreak; D1 starts infected, D0 is clean |
| **medium** | 4 | 15 | 8 | None | Two simultaneous outbreaks; forced triage between competing threats |
| **hard** | 6 | 15 | 7 | **3 days** | Six seeded infections (only D2 and D4 above safe threshold); 3-day data lag; scarce resources |

**Easy** — D1 starts at 0.50 infection, D0 is clean. The agent must observe and target the correct district. A fixed-target agent ignoring observations scores ~43% and breaches hospitals 60% of the time.

**Medium** — D0 starts above the infection threshold (0.42); D2 is in the warning zone (0.38, below the 0.40 threshold). D1 and D3 start low but grow into crisis within 4–6 steps via spillover. With 8 resources across 4 districts over 15 steps, genuine triage is required.

**Hard** — 3-day information lag means the agent sees infection rates from 3 days ago. The `growth_rate_hint` provides a noisy signal to estimate current state. Structural uncertainty — not testable around.

---

## Reward Function

| Term | Value | Fires When |
| --- | --- | --- |
| Infection penalty | `−0.50 × density` | District infection > 0.40 |
| Hospital breach | `−1.00` | Hospital capacity ≤ 10% |
| Early containment | `+0.50 × (1 − step/max_steps)` | District infection < 0.20 |
| Correct prioritisation | `+0.30` | Allocate to highest-infected district |
| Unnecessary restriction | `−0.20` | Restrict district below 0.20 |

Key design choices: early containment bonus decays over time (proactive action worth more); infection penalty scales with population density (realistic triage); hospital breach fires at 10% not 0% (real ICU thresholds); restrictions auto-lift when infection drops below safe threshold.

---

## Grader

Fully deterministic — no randomness, no LLM calls. Identical trajectories always produce identical scores in `[0.0, 1.0]`.

| Component | Weight | Measures |
| --- | --- | --- |
| **Hospital score** | 45% | Average capacity preserved; ×0.6 multiplier if any district collapsed |
| **Containment score** | 30% | Fraction of district-days below infection threshold (first 2 steps excluded) |
| **Efficiency score** | 15% | Fraction of resource actions targeting highest-infected district (grader uses pre-action state to avoid penalising successful treatments) |
| **Speed score** | 10% | `1 − (steps / max_steps)` if episode ends early; else 0 |

Hospital is weighted highest because system capacity preservation is the primary operational constraint in real outbreak response — a functioning healthcare system is the prerequisite for everything else.

---

## Baseline Agent — GRPO-Style Episodic Memory

No weight updates, no gradient computation. The prompt is the policy; memory updates are the policy improvement.

### Learning Loop

```text
Rollout 1: Base prompt, no prior knowledge
  compute advantage = R1 - mean([])
  store steps with reward > -0.3 into EpisodicMemory

Rollout 2: Memory-augmented prompt
  retrieve top-5 similar past decisions by L1 distance on infection profiles
  inject as concrete examples into prompt
  compute advantage = R2 - mean([R1])
  reinforce if advantage > -0.5

... repeat for N rollouts (easy=2, medium=3, hard=3)
Report best grader score across all rollouts
```

### Benchmark Scores

Results from `baseline/run.py` (Llama 3.1 8B Instant via Groq, runtime ~10 minutes):

| Task | Greedy (D0) | LLM+GRPO | Lift |
| --- | --- | --- | --- |
| Easy | 42.8% (breach 60%) | **90.8%** | +48pp |
| Medium | 39.6% (breach 100%) | **78.0%** | +38pp |
| Hard | 35.3% (breach 100%) | **61.1%** | +26pp |
| **Average** | **39.2%** | **75.9%** | **+37pp** |

The greedy baseline (always allocates to D0) scores 33–43% with 60–100% hospital breach rates — no trivial exploit path. LLM+GRPO reaches 66–91% with zero breaches, demonstrating that genuine triage reasoning is required and rewarded.

---

## Generalisation

| Domain | Spreading cascade | Delayed data | Resource scarcity |
| --- | --- | --- | --- |
| 🦠 **Epidemic containment** | Infection spreads between districts | Lagged case counts | Medical resources |
| 🔥 **Wildfire deployment** | Fire spreads across terrain | Satellite update delay | Firefighting crews |
| 🛡️ **Cyberattack isolation** | Lateral movement between systems | Detection lag | Security team hours |
| 📢 **Misinformation containment** | Narrative spread through networks | Viral detection lag | Correction budget |
| 🤝 **Poverty intervention** | Deprivation cascades through communities | Census data lag | Aid allocation |

---

## Project Structure

```text
cascade-containment/
├── inference.py              # Evaluation entry point (mandatory [START][STEP][END] logs)
├── models.py                 # Typed data contracts: Action, Observation, State
├── client.py                 # OpenEnv client interface
├── openenv.yaml              # Environment manifest for OpenEnv registry
│
├── server/
│   ├── app.py                # FastAPI server + judge dashboard + /grade /info /demo /validate endpoints
│   ├── environment.py        # Core RL loop (reset/step/state OpenEnv interface)
│   ├── grader.py             # Deterministic trajectory scorer — no LLM calls
│   ├── constants.py          # Single source of truth for all numeric configuration
│   ├── utils.py              # Spread computation, observation builder, helper functions
│   ├── Dockerfile            # Container definition
│   └── tasks/
│       ├── task_easy.py      # 2 districts, 10 steps, real-time data (D1 seeded)
│       ├── task_medium.py    # 4 districts, 15 steps, forced triage
│       └── task_hard.py      # 6 districts, 15 steps, 3-day data lag
│
├── baseline/
│   ├── policy.py             # LLM policy with chain-of-thought prompting
│   ├── evaluator.py          # GRPO episodic memory loop (easy=2, medium=3, hard=3 rollouts)
│   └── run.py                # CLI entry point
│
├── scripts/
│   └── test_local.py         # Phase 1 spec checks + greedy benchmark + variance analysis
│
└── core/
    ├── trajectory.py         # EpisodicMemory — L1 similarity retrieval, phase-weighted
    ├── reward.py             # Score normalisation utilities
    └── policy_update.py      # Advantage computation, memory gating (threshold -0.3)
```

---

## OpenEnv Compliance

| Requirement | Status |
| --- | --- |
| `reset()` returns `CityObservation` | ✅ |
| `step(action)` returns `CityObservation` | ✅ |
| `state` property returns `State` | ✅ |
| Typed `Action` subclass | ✅ `ContainmentAction(Action)` |
| Typed `Observation` subclass | ✅ `CityObservation(Observation)` |
| `openenv.yaml` manifest | ✅ |
| Dockerfile builds | ✅ |
| HF Space deploys | ✅ |
| `inference.py` at root | ✅ |
| `[START][STEP][END]` structured logs | ✅ |
| Runtime < 20 minutes | ✅ ~~10 minutes |
| `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` env vars | ✅ |
| OpenAI client for all LLM calls | ✅ |
| Grader scores in `[0.0, 1.0]` | ✅ |
| 3+ tasks with difficulty progression | ✅ |
| Phase 1 automated validation | ✅ 10/10 checks pass |

---

## Setup and Local Development

### Local Server

```bash
pip install -r requirements.txt
uvicorn server.app:app --host 0.0.0.0 --port 7860

export ENV_BASE_URL=http://localhost:7860
python baseline/run.py
```

### Docker

```bash
docker build -t cascade-containment .
docker run -p 7860:7860 cascade-containment
```

---

## Tags

`reinforcement-learning` · `resource-allocation` · `sequential-decision-making` · `partial-observability` · `cascade-dynamics` · `epidemic-response` · `openenv` · `llm-agent` · `grpo` · `episodic-memory` · `triage` · `multi-district` · `docker` · `fastapi`