mm.md

mm.md — 6-step walkthrough on outbreak_easy

Run: uv run python mm.py

Setup

• Task: outbreak_easy, seed=0, max_ticks=12.
• Initial latent state (per server/simulator/tasks.py): R1 hot with I≈0.03 (30 cases / 1000 pop); R2 / R3 / R4 quiet with I≈0.001 (1 case / 1000 pop).
• Initial resources: 1000 test_kits, 500 hospital_beds, 20 mobile_units, 2000 vaccine_doses.
• Telemetry: delay = 1 tick, σ_cases = 0.02 (≈ ±20 cases of noise), σ_compliance = 0.05.

The cases field printed each tick is delayed and noisy — it's reported_cases_d_ago. The reward is computed on the latent ground truth, not the telemetry, so reward dynamics may not visibly match the printed cases.

Step-by-step intent

Step 1 — NoOp baseline

• Intent: see how the env evolves with no intervention.
• Expected: R1 grows slowly under R0=1.5 (within-region β ≈ 0.3); R2-R4 stay near zero. Reward should be high (most population still susceptible, low total infection).

Step 2 — DeployResource(R1, test_kits, 200)

• Intent: test_kits efficacy is 0.00002 / unit / tick; 200 units contribute -0.004 to R1's I per tick over 2 ticks.
• Expected: kits inventory drops 1000 → 800. Reward changes are too small to read off — this is mostly to demonstrate the deployment flow (accepted=True, inventory delta).

Step 3 — RestrictMovement(R1, moderate)

• Intent: severity multiplier = 0.25 → R0_eff on R1 drops 25%. Slows transmission inside R1.
• Expected: active_restrictions shows R1=moderate(4) (4-tick duration, decremented each tick). R1 case-growth slows. Compliance starts gentle decay under restriction.

Step 4 — DeployResource(R1, vaccine_doses, 500)

• Intent: vaccine efficacy 0.0001 / unit; 500 units → -0.05 ΔI on R1 plus equivalent S → R conversion.
• Expected: vax inventory 2000 → 1500. R1's hospital_load eases over the next 2 ticks; R1 compliance_proxy holds steady.

Step 5 — Escalate(national)

• Intent: unlocks the restrict_movement.strict rule via the L1 legal_constraints entry. SEIR is a no-op for this tick.
• Expected: accepted=True. Internally escalation_unlocked_strict=True. The legal_constraints list still contains L1 in the observation (it's the rule, not the lock state); only the lock has flipped.

Step 6 — RestrictMovement(R1, strict)

• Intent: severity multiplier = 0.5 → R0_eff on R1 drops 50%. Pre-step-5 this would have been accepted=False (legal-violation).
• Expected: accepted=True. R1's restriction flips moderate → strict. Compliance starts faster decay (-0.03 / tick under strict).

Reading the output

Each step prints:

Field	Meaning
action	kind of payload submitted
accepted	env's verdict (False = V2-illegal or legal-violation or insufficient resource)
reward	per-tick outer_reward ∈ [0, 1] (design §15 weighted sum)
regions	cases (delayed + noisy), hosp (current), comp (noisy)
resources	global inventory (test_kits / hospital_beds_free / mobile_units / vaccine_doses)
restricts	active restrictions per region with (ticks_remaining)
tick	current / max_ticks; done is True only at terminal

The reward has 6 components weighted (per design §15):

Component	Weight	What it measures
r_infect	0.35	1 - mean(I) — average infection across regions
r_time	0.18	1 - tick / max_ticks — early-tick bias
r_hosp	0.17	1 - mean(hospital_load)
r_casc	0.15	binary: 1 if no region exceeds I=0.30, else 0
r_policy	0.12	binary: 1 if last action accepted, else 0
r_fair	0.03	1 - pstdev(I) — equality across regions

Because outbreak_easy starts low-infection and the 3-consecutive- safe-ticks rule fires quickly, the env may report done=True before all 6 steps are exhausted. The script keeps stepping regardless so you see the full intended sequence; in production the agent would break on done.

Caveats

• Telemetry noise on tick 0: the printed R1: cases=48 at the initial state can exceed the true 30 cases per 1000 because of the Gaussian noise draw. Different seeds will print different numbers for the same latent state.
• Compliance proxy is similarly noised — small fluctuations don't reflect real compliance changes.
• Reward stays high throughout even though done=True flips early; the success-terminal +0.20 bonus is not included in obs.reward (per the env-step separation pin from Session 7d) — it's composed downstream by the trainer in reward_shaping.py.