mm.md

# mm.md — 6-step walkthrough on `outbreak_easy`

Run: `uv run python mm.py`

## Setup

- **Task:** `outbreak_easy`, **seed=0**, `max_ticks=12`.
- **Initial latent state** (per `server/simulator/tasks.py`): R1 hot
  with `I≈0.03` (~30 cases / 1000 pop); R2 / R3 / R4 quiet with
  `I≈0.001` (~1 case / 1000 pop).
- **Initial resources:** 1000 test_kits, 500 hospital_beds, 20 mobile_units,
  2000 vaccine_doses.
- **Telemetry:** delay = 1 tick, σ_cases = 0.02 (≈ ±20 cases of noise),
  σ_compliance = 0.05.

The `cases` field printed each tick is **delayed and noisy** — it's
`reported_cases_d_ago`. The reward is computed on the latent ground
truth, not the telemetry, so reward dynamics may not visibly match
the printed cases.

## Step-by-step intent

### Step 1 — `NoOp` baseline

- **Intent:** see how the env evolves with no intervention.
- **Expected:** R1 grows slowly under R0=1.5 (within-region β ≈ 0.3);
  R2-R4 stay near zero. Reward should be high (most population still
  susceptible, low total infection).

### Step 2 — `DeployResource(R1, test_kits, 200)`

- **Intent:** test_kits efficacy is `0.00002 / unit / tick`; 200 units
  contribute `-0.004` to R1's I per tick over 2 ticks.
- **Expected:** kits inventory drops 1000 → 800. Reward changes are
  too small to read off — this is mostly to demonstrate the deployment
  flow (`accepted=True`, inventory delta).

### Step 3 — `RestrictMovement(R1, moderate)`

- **Intent:** severity multiplier = 0.25 → R0_eff on R1 drops 25%.
  Slows transmission inside R1.
- **Expected:** `active_restrictions` shows `R1=moderate(4)` (4-tick
  duration, decremented each tick). R1 case-growth slows. Compliance
  starts gentle decay under restriction.

### Step 4 — `DeployResource(R1, vaccine_doses, 500)`

- **Intent:** vaccine efficacy `0.0001 / unit`; 500 units → `-0.05` ΔI
  on R1 plus equivalent S → R conversion.
- **Expected:** vax inventory 2000 → 1500. R1's hospital_load eases
  over the next 2 ticks; R1 compliance_proxy holds steady.

### Step 5 — `Escalate(national)`

- **Intent:** unlocks the `restrict_movement.strict` rule via the L1
  legal_constraints entry. SEIR is a no-op for this tick.
- **Expected:** `accepted=True`. Internally
  `escalation_unlocked_strict=True`. The `legal_constraints` list
  still contains L1 in the observation (it's the rule, not the lock
  state); only the lock has flipped.

### Step 6 — `RestrictMovement(R1, strict)`

- **Intent:** severity multiplier = 0.5 → R0_eff on R1 drops 50%.
  Pre-step-5 this would have been `accepted=False` (legal-violation).
- **Expected:** `accepted=True`. R1's restriction flips
  `moderate → strict`. Compliance starts faster decay
  (-0.03 / tick under strict).

## Reading the output

Each step prints:

| Field | Meaning |
|---|---|
| `action` | `kind` of payload submitted |
| `accepted` | env's verdict (False = V2-illegal **or** legal-violation **or** insufficient resource) |
| `reward` | per-tick `outer_reward` ∈ [0, 1] (design §15 weighted sum) |
| `regions` | `cases` (delayed + noisy), `hosp` (current), `comp` (noisy) |
| `resources` | global inventory (test_kits / hospital_beds_free / mobile_units / vaccine_doses) |
| `restricts` | active restrictions per region with `(ticks_remaining)` |
| `tick` | `current / max_ticks`; `done` is True only at terminal |

The reward has **6 components** weighted (per design §15):

| Component | Weight | What it measures |
|---|---|---|
| `r_infect` | 0.35 | `1 - mean(I)` — average infection across regions |
| `r_time` | 0.18 | `1 - tick / max_ticks` — early-tick bias |
| `r_hosp` | 0.17 | `1 - mean(hospital_load)` |
| `r_casc` | 0.15 | binary: 1 if no region exceeds I=0.30, else 0 |
| `r_policy` | 0.12 | binary: 1 if last action accepted, else 0 |
| `r_fair` | 0.03 | `1 - pstdev(I)` — equality across regions |

Because `outbreak_easy` starts low-infection and the 3-consecutive-
safe-ticks rule fires quickly, **the env may report `done=True`
before all 6 steps are exhausted**. The script keeps stepping
regardless so you see the full intended sequence; in production the
agent would break on `done`.

## Caveats

- **Telemetry noise on tick 0:** the printed `R1: cases=48` at the
  initial state can exceed the true 30 cases per 1000 because of the
  Gaussian noise draw. Different seeds will print different numbers
  for the same latent state.
- **Compliance proxy** is similarly noised — small fluctuations don't
  reflect real compliance changes.
- **Reward stays high throughout** even though `done=True` flips
  early; the success-terminal +0.20 bonus is **not** included in
  `obs.reward` (per the env-step separation pin from Session 7d) —
  it's composed downstream by the trainer in `reward_shaping.py`.