RohitChandramouli6618 commited on
Commit
fedfdcb
Β·
1 Parent(s): a7b25d3

Fix README markdownlint errors

Browse files
Files changed (1) hide show
  1. README.md +46 -46
README.md CHANGED
@@ -1,4 +1,4 @@
1
- ο»Ώ---
2
  title: Cascade Containment
3
  emoji: 🦠
4
  colorFrom: red
@@ -8,7 +8,7 @@ app_port: 7860
8
  pinned: false
9
  ---
10
 
11
- # 🦠 Cascade Containment
12
 
13
  > An RL benchmark for sequential resource allocation under spreading cascade dynamics.
14
 
@@ -34,19 +34,22 @@ No existing OpenEnv benchmark formalizes this problem class. Cascade Containment
34
  ## Environment Design
35
 
36
  ### Action Space
 
37
  One decision per step β€” kept deliberately simple to maximize strategic depth:
38
 
39
- | Field | Type | Values |
40
- |-------|------|--------|
41
- | `action_type` | string | `"test"` Β· `"restrict"` Β· `"allocate"` |
42
- | `district_id` | int | 0-indexed district target |
43
 
44
  - **test** β€” spend 1 resource to get accurate infection data for a district
45
  - **restrict** β€” impose movement restriction (free, but penalised if infection is low)
46
  - **allocate** β€” deploy 1 resource unit to reduce spread rate this step
47
 
48
  ### Observation Space
 
49
  The agent receives a filtered, potentially lagged view of the world β€” never the full ground truth:
 
50
  ```python
51
  CityObservation:
52
  districts: List[DistrictObservation] # per-district visible state
@@ -59,23 +62,25 @@ CityObservation:
59
  ```
60
 
61
  Each `DistrictObservation` contains:
 
62
  - `reported_infection_rate` β€” real-time (easy/medium) or **3 days lagged** (hard)
63
  - `growth_rate_hint` β€” noisy signal of true spread rate
64
  - `hospital_capacity_remaining` β€” always accurate (hospitals report in real time)
65
  - `tested_recently`, `restriction_active`
66
 
67
- ### The Key Design Decision: Partial Observability
 
68
  The hard task exposes infection rates from **3 days ago**. The agent must learn to act on noisy forward signals (`growth_rate_hint`) rather than react to confirmed data β€” exactly the challenge real public health officials face. This single mechanic is what separates a thoughtful agent from a reactive one.
69
 
70
  ---
71
 
72
  ## Three Tasks
73
 
74
- | Task | Districts | Steps | Resources | Data Lag | Challenge |
75
- |------|-----------|-------|-----------|----------|-----------|
76
- | `easy` | 2 | 10 | 10 | None | Single outbreak, clear signal |
77
- | `medium` | 4 | 15 | 8 | None | Two simultaneous outbreaks, forced triage |
78
- | `hard` | 6 | 20 | 7 | 3 days | Scarce resources, invisible acceleration |
79
 
80
  ---
81
 
@@ -83,15 +88,15 @@ The hard task exposes infection rates from **3 days ago**. The agent must learn
83
 
84
  Five shaped reward terms fire independently each step, providing dense feedback throughout the episode:
85
 
86
- | Term | Value | Purpose |
87
- |------|-------|---------|
88
- | Infection penalty | `-0.50` per district above threshold | Primary containment signal |
89
- | Hospital breach | `-1.00` per collapsed hospital | Catastrophic failure deterrent |
90
- | Early containment | `+0.50 Γ— (1 - step/max_steps)` | Teaches proactive behaviour |
91
- | Unnecessary restriction | `-0.20` | Prevents lazy blanket lockdowns |
92
- | Correct prioritisation | `+0.30` | Rewards triage intelligence |
93
 
94
- The early containment bonus decays over time β€” containing an outbreak on day 3 is worth more than on day 8. This single design decision is what teaches the agent to act before crises emerge rather than after.
95
 
96
  ---
97
 
@@ -99,12 +104,12 @@ The early containment bonus decays over time β€” containing an outbreak on day 3
99
 
100
  The grader is fully deterministic β€” no randomness, no LLM calls β€” producing a weighted composite score in `[0.0, 1.0]`:
101
 
102
- | Component | Weight | Measures |
103
- |-----------|--------|---------|
104
- | Containment score | 45% | District-days below infection threshold |
105
- | Hospital score | 30% | Capacity preserved across episode |
106
- | Efficiency score | 15% | Resources directed to high-need districts |
107
- | Speed score | 10% | Containment achieved faster than max steps |
108
 
109
  ---
110
 
@@ -126,17 +131,16 @@ The prompt is the policy. Memory updates are the policy improvement. This produc
126
 
127
  Dumb greedy policy (always allocates to district 0):
128
 
129
- | Task | Score | Hospital Breached |
130
- |------|-------|-------------------|
131
- | Easy | ~0.50 | No |
132
- | Medium | ~0.23 | Yes |
133
- | Hard | ~0.21 | Yes |
134
-
135
- A smart LLM agent using the episodic memory baseline consistently scores 0.65–0.80 on easy and shows meaningful improvement on medium across rollouts.
136
 
137
  ---
138
 
139
  ## Usage
 
140
  ```python
141
  from client import CascadeContainmentEnv
142
  from models import ContainmentAction
@@ -144,28 +148,22 @@ from models import ContainmentAction
144
  with CascadeContainmentEnv(
145
  base_url="https://therubberduckdebuggers-cascade-containment.hf.space"
146
  ).sync() as env:
147
- # Run easy task
148
  obs = env.reset(task_name="easy")
149
-
150
  while not obs.done:
151
- action = ContainmentAction(
152
- action_type="allocate",
153
- district_id=0
154
- )
155
- result = env.step(action)
156
  obs = result.observation
157
  print(f"Reward: {result.reward:.4f}")
158
  ```
159
 
160
  ### Running the Full Evaluation
 
161
  ```bash
162
- # Set required environment variables
163
  export API_BASE_URL="https://router.huggingface.co/v1"
164
  export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
165
  export HF_TOKEN="your_hf_token"
166
  export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"
167
 
168
- # Run inference
169
  python inference.py
170
  ```
171
 
@@ -173,19 +171,20 @@ python inference.py
173
 
174
  ## Generalisation
175
 
176
- This environment is not epidemic-specific. The core mechanics β€” spreading cascade, delayed data, resource scarcity, spatial spillover β€” are identical to:
177
 
178
  - **Wildfire deployment** β€” pre-position crews before fire reaches populated areas
179
  - **Cyberattack isolation** β€” quarantine systems before lateral movement completes
180
  - **Misinformation containment** β€” deploy corrections before false narratives entrench
181
- - **Poverty intervention** β€” allocate aid where need is growing, not just where it's visible
182
 
183
  The environment is designed to be a lasting benchmark for this general problem class, not a pandemic novelty.
184
 
185
  ---
186
 
187
  ## Project Structure
188
- ```
 
189
  epidemic_containment_env/
190
  β”œβ”€β”€ models.py # Data contracts (Action, Observation, State)
191
  β”œβ”€β”€ constants.py # All numeric configuration
@@ -212,4 +211,5 @@ epidemic_containment_env/
212
  ---
213
 
214
  ## Tags
215
- `reinforcement-learning` Β· `resource-allocation` Β· `sequential-decision-making` Β· `partial-observability` Β· `cascade-dynamics` Β· `openenv` Β· `llm-agent`
 
 
1
+ ---
2
  title: Cascade Containment
3
  emoji: 🦠
4
  colorFrom: red
 
8
  pinned: false
9
  ---
10
 
11
+ ## 🦠 Cascade Containment
12
 
13
  > An RL benchmark for sequential resource allocation under spreading cascade dynamics.
14
 
 
34
  ## Environment Design
35
 
36
  ### Action Space
37
+
38
  One decision per step β€” kept deliberately simple to maximize strategic depth:
39
 
40
+ | Field | Type | Values |
41
+ |--------------|--------|-------------------------------------|
42
+ | `action_type`| string | `"test"` Β· `"restrict"` Β· `"allocate"` |
43
+ | `district_id`| int | 0-indexed district target |
44
 
45
  - **test** β€” spend 1 resource to get accurate infection data for a district
46
  - **restrict** β€” impose movement restriction (free, but penalised if infection is low)
47
  - **allocate** β€” deploy 1 resource unit to reduce spread rate this step
48
 
49
  ### Observation Space
50
+
51
  The agent receives a filtered, potentially lagged view of the world β€” never the full ground truth:
52
+
53
  ```python
54
  CityObservation:
55
  districts: List[DistrictObservation] # per-district visible state
 
62
  ```
63
 
64
  Each `DistrictObservation` contains:
65
+
66
  - `reported_infection_rate` β€” real-time (easy/medium) or **3 days lagged** (hard)
67
  - `growth_rate_hint` β€” noisy signal of true spread rate
68
  - `hospital_capacity_remaining` β€” always accurate (hospitals report in real time)
69
  - `tested_recently`, `restriction_active`
70
 
71
+ ### Partial Observability
72
+
73
  The hard task exposes infection rates from **3 days ago**. The agent must learn to act on noisy forward signals (`growth_rate_hint`) rather than react to confirmed data β€” exactly the challenge real public health officials face. This single mechanic is what separates a thoughtful agent from a reactive one.
74
 
75
  ---
76
 
77
  ## Three Tasks
78
 
79
+ | Task | Districts | Steps | Resources | Data Lag | Challenge |
80
+ |----------|-----------|-------|-----------|----------|------------------------------------------|
81
+ | `easy` | 2 | 10 | 10 | None | Single outbreak, clear signal |
82
+ | `medium` | 4 | 15 | 8 | None | Two simultaneous outbreaks, forced triage|
83
+ | `hard` | 6 | 20 | 7 | 3 days | Scarce resources, invisible acceleration |
84
 
85
  ---
86
 
 
88
 
89
  Five shaped reward terms fire independently each step, providing dense feedback throughout the episode:
90
 
91
+ | Term | Value | Purpose |
92
+ |-------------------------|--------------------------------|----------------------------------|
93
+ | Infection penalty | `-0.50` per district above threshold | Primary containment signal |
94
+ | Hospital breach | `-1.00` per collapsed hospital | Catastrophic failure deterrent |
95
+ | Early containment | `+0.50 Γ— (1 - step/max_steps)` | Teaches proactive behaviour |
96
+ | Unnecessary restriction | `-0.20` | Prevents lazy blanket lockdowns |
97
+ | Correct prioritisation | `+0.30` | Rewards triage intelligence |
98
 
99
+ The early containment bonus decays over time β€” containing an outbreak on day 3 is worth more than on day 8. This design decision teaches the agent to act before crises emerge rather than after.
100
 
101
  ---
102
 
 
104
 
105
  The grader is fully deterministic β€” no randomness, no LLM calls β€” producing a weighted composite score in `[0.0, 1.0]`:
106
 
107
+ | Component | Weight | Measures |
108
+ |-------------------|--------|-------------------------------------------|
109
+ | Containment score | 45% | District-days below infection threshold |
110
+ | Hospital score | 30% | Capacity preserved across episode |
111
+ | Efficiency score | 15% | Resources directed to high-need districts |
112
+ | Speed score | 10% | Containment achieved faster than max steps|
113
 
114
  ---
115
 
 
131
 
132
  Dumb greedy policy (always allocates to district 0):
133
 
134
+ | Task | Score | Hospital Breached |
135
+ |--------|-------|-------------------|
136
+ | Easy | ~0.50 | No |
137
+ | Medium | ~0.23 | Yes |
138
+ | Hard | ~0.21 | Yes |
 
 
139
 
140
  ---
141
 
142
  ## Usage
143
+
144
  ```python
145
  from client import CascadeContainmentEnv
146
  from models import ContainmentAction
 
148
  with CascadeContainmentEnv(
149
  base_url="https://therubberduckdebuggers-cascade-containment.hf.space"
150
  ).sync() as env:
 
151
  obs = env.reset(task_name="easy")
152
+
153
  while not obs.done:
154
+ result = env.step(ContainmentAction(action_type="allocate", district_id=0))
 
 
 
 
155
  obs = result.observation
156
  print(f"Reward: {result.reward:.4f}")
157
  ```
158
 
159
  ### Running the Full Evaluation
160
+
161
  ```bash
 
162
  export API_BASE_URL="https://router.huggingface.co/v1"
163
  export MODEL_NAME="meta-llama/Llama-3.1-8B-Instruct"
164
  export HF_TOKEN="your_hf_token"
165
  export ENV_BASE_URL="https://therubberduckdebuggers-cascade-containment.hf.space"
166
 
 
167
  python inference.py
168
  ```
169
 
 
171
 
172
  ## Generalisation
173
 
174
+ This environment is not epidemic-specific. The core mechanics β€” spreading cascade, delayed data, resource scarcity, spatial spillover β€” apply to:
175
 
176
  - **Wildfire deployment** β€” pre-position crews before fire reaches populated areas
177
  - **Cyberattack isolation** β€” quarantine systems before lateral movement completes
178
  - **Misinformation containment** β€” deploy corrections before false narratives entrench
179
+ - **Poverty intervention** β€” allocate aid where need is growing, not just where it is visible
180
 
181
  The environment is designed to be a lasting benchmark for this general problem class, not a pandemic novelty.
182
 
183
  ---
184
 
185
  ## Project Structure
186
+
187
+ ```text
188
  epidemic_containment_env/
189
  β”œβ”€β”€ models.py # Data contracts (Action, Observation, State)
190
  β”œβ”€β”€ constants.py # All numeric configuration
 
211
  ---
212
 
213
  ## Tags
214
+
215
+ `reinforcement-learning` Β· `resource-allocation` Β· `sequential-decision-making` Β· `partial-observability` Β· `cascade-dynamics` Β· `openenv` Β· `llm-agent`