title: Cascade Containment
emoji: 🦠
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
🦠 An RL Benchmark for Sequential Resource Allocation Under Spreading Cascade Dynamics
Meta PyTorch OpenEnv Hackathon × SST 2026 — Live Demo · GitHub
The Problem
Sequential resource allocation under uncertainty is one of the most consequential decision problems in the real world. Whether containing an epidemic, deploying firefighting crews, isolating a cyberattack, or routing aid — the agent faces the same fundamental challenge:
- Resources are scarce — you cannot cover every district simultaneously
- Data is delayed — by the time a crisis is visible, it has already grown
- Interventions cascade — actions in one district affect adjacent ones
- Acting too late is catastrophic — hospital collapse ends the episode; proactive containment is rewarded exponentially more than reactive response
No existing OpenEnv benchmark formalises this problem class. Cascade Containment does.
Environment Overview
A city health authority must allocate limited medical resources across districts to contain a spreading outbreak. Each step, the agent observes district infection rates (possibly lagged), hospital capacity levels, and growth signals — then decides where to deploy resources, impose restrictions, or gather data.
The environment is not epidemic-specific. The underlying mechanics — spreading cascade, delayed observation, resource scarcity, geographic spillover — are structurally identical across multiple real-world domains.
Quick Start
from client import CascadeContainmentEnv
from models import ContainmentAction
with CascadeContainmentEnv(
base_url="https://therubberduckdebuggers-cascade-containment.hf.space"
).sync() as env:
result = env.reset(task_name="medium")
obs = result.observation
while not result.done:
most_infected = max(obs.districts, key=lambda d: d.reported_infection_rate)
action = ContainmentAction(
action_type="allocate",
district_id=most_infected.district_id
)
result = env.step(action)
obs = result.observation
print(f"Step {obs.current_step}: reward={result.reward:.3f}")
Running the Full Baseline Evaluation
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
export HF_TOKEN="hf_your_token_here"
export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"
# Full LLM+GRPO evaluation (~10 minutes, emits [START][STEP][END] logs)
python inference.py
# Local validation + greedy benchmark
python scripts/test_local.py
Action Space
| Field | Type | Values |
|---|---|---|
action_type |
string |
"test" · "restrict" · "allocate" |
district_id |
int |
0-indexed district target |
| Action | Cost | Effect |
|---|---|---|
| test | 1 resource | Reveals accurate current infection data for district |
| restrict | Free | Imposes movement restrictions; reduces spread rate; penalised if infection < 0.20 |
| allocate | 1 resource | Deploys medical resources; reduces existing infection by 5% and slows future spread |
Movement restrictions lift automatically once a district's infection drops below the safe threshold.
Observation Space
The agent receives a filtered, potentially lagged view of the world — never the full ground truth:
CityObservation:
districts: List[DistrictObservation] # per-district visible state
available_resources: int # budget remaining this step
current_step: int
max_steps: int
done: bool
reward: float | None
message: str | None
| Field | Description | Observability |
|---|---|---|
reported_infection_rate |
Active infection fraction | Real-time (easy/medium); 3-day lagged (hard) |
growth_rate_hint |
Noisy signal of true spread rate | Always real-time ± noise |
hospital_capacity_remaining |
ICU/ward capacity fraction | Always real-time |
population_density |
District's share of city population | Always real-time |
restriction_active |
Whether movement restrictions are in place | Always real-time |
tested_recently |
Tested within last 2 days | Always real-time |
Epidemiological Model
new_infection = current + (spread_rate − natural_recovery − intervention) + geographic_spillover
| Parameter | Value | Rationale |
|---|---|---|
| Spread rate | 3–8% per day | Realistic for respiratory outbreaks (seasonal flu: 5–10%) |
| Natural recovery | 1% per day | Background case resolution without medical intervention |
| Treatment effect | −5% existing infection | Medical deployment (antivirals, PPE, rapid response) |
| Spread reduction | −10% per allocation | Resource-driven suppression of transmission |
| Geographic spillover | 1% to adjacent districts | Linear topology — no wrap-around (geographically realistic) |
| Hospital breach threshold | ≤10% capacity | Real ICU overflow and triage failure threshold |
Three Tasks
| Task | Districts | Steps | Resources | Data Lag | Challenge |
|---|---|---|---|---|---|
| easy | 2 | 10 | 10 | None | Single outbreak; D1 starts infected, D0 is clean |
| medium | 4 | 15 | 8 | None | Two simultaneous outbreaks; forced triage between competing threats |
| hard | 6 | 15 | 7 | 3 days | Six seeded infections (only D2 and D4 above safe threshold); 3-day data lag; scarce resources |
Easy — D1 starts at 0.50 infection, D0 is clean. The agent must observe and target the correct district. A fixed-target agent ignoring observations scores ~43% and breaches hospitals 60% of the time.
Medium — D0 starts above the infection threshold (0.42); D2 is in the warning zone (0.38, below the 0.40 threshold). D1 and D3 start low but grow into crisis within 4–6 steps via spillover. With 8 resources across 4 districts over 15 steps, genuine triage is required.
Hard — 3-day information lag means the agent sees infection rates from 3 days ago. The growth_rate_hint provides a noisy signal to estimate current state. Structural uncertainty — not testable around.
Reward Function
| Term | Value | Fires When |
|---|---|---|
| Infection penalty | −0.50 × density |
District infection > 0.40 |
| Hospital breach | −1.00 |
Hospital capacity ≤ 10% |
| Early containment | +0.50 × (1 − step/max_steps) |
District infection < 0.20 |
| Correct prioritisation | +0.30 |
Allocate to highest-infected district |
| Unnecessary restriction | −0.20 |
Restrict district below 0.20 |
Key design choices: early containment bonus decays over time (proactive action worth more); infection penalty scales with population density (realistic triage); hospital breach fires at 10% not 0% (real ICU thresholds); restrictions auto-lift when infection drops below safe threshold.
Grader
Fully deterministic — no randomness, no LLM calls. Identical trajectories always produce identical scores in [0.0, 1.0].
| Component | Weight | Measures |
|---|---|---|
| Hospital score | 45% | Average capacity preserved; ×0.6 multiplier if any district collapsed |
| Containment score | 30% | Fraction of district-days below infection threshold (first 2 steps excluded) |
| Efficiency score | 15% | Fraction of resource actions targeting highest-infected district (grader uses pre-action state to avoid penalising successful treatments) |
| Speed score | 10% | 1 − (steps / max_steps) if episode ends early; else 0 |
Hospital is weighted highest because system capacity preservation is the primary operational constraint in real outbreak response — a functioning healthcare system is the prerequisite for everything else.
Baseline Agent — GRPO-Style Episodic Memory
No weight updates, no gradient computation. The prompt is the policy; memory updates are the policy improvement.
Learning Loop
Rollout 1: Base prompt, no prior knowledge
compute advantage = R1 - mean([])
store steps with reward > -0.3 into EpisodicMemory
Rollout 2: Memory-augmented prompt
retrieve top-5 similar past decisions by L1 distance on infection profiles
inject as concrete examples into prompt
compute advantage = R2 - mean([R1])
reinforce if advantage > -0.5
... repeat for N rollouts (easy=2, medium=3, hard=3)
Report best grader score across all rollouts
Benchmark Scores
Results from baseline/run.py (Llama 3.1 8B Instant via Groq, runtime ~10 minutes):
| Task | Greedy (D0) | LLM+GRPO | Lift |
|---|---|---|---|
| Easy | 42.8% (breach 60%) | 90.8% | +48pp |
| Medium | 39.6% (breach 100%) | 78.0% | +38pp |
| Hard | 35.3% (breach 100%) | 61.1% | +26pp |
| Average | 39.2% | 75.9% | +37pp |
The greedy baseline (always allocates to D0) scores 33–43% with 60–100% hospital breach rates — no trivial exploit path. LLM+GRPO reaches 66–91% with zero breaches, demonstrating that genuine triage reasoning is required and rewarded.
Generalisation
| Domain | Spreading cascade | Delayed data | Resource scarcity |
|---|---|---|---|
| 🦠 Epidemic containment | Infection spreads between districts | Lagged case counts | Medical resources |
| 🔥 Wildfire deployment | Fire spreads across terrain | Satellite update delay | Firefighting crews |
| 🛡️ Cyberattack isolation | Lateral movement between systems | Detection lag | Security team hours |
| 📢 Misinformation containment | Narrative spread through networks | Viral detection lag | Correction budget |
| 🤝 Poverty intervention | Deprivation cascades through communities | Census data lag | Aid allocation |
Project Structure
cascade-containment/
├── inference.py # Evaluation entry point (mandatory [START][STEP][END] logs)
├── models.py # Typed data contracts: Action, Observation, State
├── client.py # OpenEnv client interface
├── openenv.yaml # Environment manifest for OpenEnv registry
│
├── server/
│ ├── app.py # FastAPI server + judge dashboard + /grade /info /demo /validate endpoints
│ ├── environment.py # Core RL loop (reset/step/state OpenEnv interface)
│ ├── grader.py # Deterministic trajectory scorer — no LLM calls
│ ├── constants.py # Single source of truth for all numeric configuration
│ ├── utils.py # Spread computation, observation builder, helper functions
│ ├── Dockerfile # Container definition
│ └── tasks/
│ ├── task_easy.py # 2 districts, 10 steps, real-time data (D1 seeded)
│ ├── task_medium.py # 4 districts, 15 steps, forced triage
│ └── task_hard.py # 6 districts, 15 steps, 3-day data lag
│
├── baseline/
│ ├── policy.py # LLM policy with chain-of-thought prompting
│ ├── evaluator.py # GRPO episodic memory loop (easy=2, medium=3, hard=3 rollouts)
│ └── run.py # CLI entry point
│
├── scripts/
│ └── test_local.py # Phase 1 spec checks + greedy benchmark + variance analysis
│
└── core/
├── trajectory.py # EpisodicMemory — L1 similarity retrieval, phase-weighted
├── reward.py # Score normalisation utilities
└── policy_update.py # Advantage computation, memory gating (threshold -0.3)
OpenEnv Compliance
| Requirement | Status |
|---|---|
reset() returns CityObservation |
✅ |
step(action) returns CityObservation |
✅ |
state property returns State |
✅ |
Typed Action subclass |
✅ ContainmentAction(Action) |
Typed Observation subclass |
✅ CityObservation(Observation) |
openenv.yaml manifest |
✅ |
| Dockerfile builds | ✅ |
| HF Space deploys | ✅ |
inference.py at root |
✅ |
[START][STEP][END] structured logs |
✅ |
| Runtime < 20 minutes | ✅ ~~10 minutes |
API_BASE_URL, MODEL_NAME, HF_TOKEN env vars |
✅ |
| OpenAI client for all LLM calls | ✅ |
Grader scores in [0.0, 1.0] |
✅ |
| 3+ tasks with difficulty progression | ✅ |
| Phase 1 automated validation | ✅ 10/10 checks pass |
Setup and Local Development
Local Server
pip install -r requirements.txt
uvicorn server.app:app --host 0.0.0.0 --port 7860
export ENV_BASE_URL=http://localhost:7860
python baseline/run.py
Docker
docker build -t cascade-containment .
docker run -p 7860:7860 cascade-containment
Tags
reinforcement-learning · resource-allocation · sequential-decision-making · partial-observability · cascade-dynamics · epidemic-response · openenv · llm-agent · grpo · episodic-memory · triage · multi-district · docker · fastapi