PROJECT_COMPLETE_GUIDE.md

911 Dispatch Project - Complete Beginner Guide

1. What this project is (in plain language)

This project is a simulator where an AI agent learns to behave like a city emergency dispatch supervisor.

Think of it like a strategy game:

• There are emergencies (incidents).
• There are responders (fire, police, EMS units).
• The agent must choose what to do each turn (dispatch, reassign, cancel, request mutual aid, etc.).
• The simulator gives a score for each decision and a final score for the whole run.

The goal is to train and evaluate decision-making quality under pressure.

2. What an RL environment means

RL means Reinforcement Learning.

In RL, four core ideas exist:

• Agent: the decision-maker (your model or baseline policy).
• Environment: the world that reacts to actions (this simulator).
• Reward: a number that says how good/bad the last action outcome was.
• Episode: one complete run from start to finish.

For this project:

• Agent picks an action.
• Environment updates city state.
• Environment returns:
  • updated observation,
  • reward,
  • done flag (whether run is over).

That loop repeats until the episode ends.

3. Important clarification: "scheme of electricity" vs "city schema"

There is no electricity scheme in this codebase.

What exists is a city schema.

City schema means a configuration blueprint for the simulation:

• city size (grid),
• districts,
• available units,
• unit speeds,
• default recommended unit types for each incident type.

The schema is loaded from data files and used to initialize deterministic, repeatable scenarios.

4. Project architecture (high level)

1. Scenario/task setup

• A task fixture builds initial units/incidents and metadata.

2. State machine update engine

• Validates actions.
• Applies action effects.
• Advances time by one tick.
• Updates incident statuses and unit statuses.

3. Reward + scoring

• Computes per-step reward components.
• Computes episode-level score using task-specific graders.

4. API server

• Exposes reset/step/state endpoints.

5. Dashboard

• Polls backend state repeatedly and renders units/incidents + reward bars.

5. What is the task?

A task is a scenario type with its own initial conditions, difficulty, and final grading logic.

This project has 4 tasks:

1. single_incident (easy)

• One incident, small unit pool.
• Focus: dispatch the right unit fast.

2. multi_incident (medium)

• Multiple incidents at the same time.
• Focus: triage/prioritization and handling P1 incidents.

3. mass_casualty (hard)

• Incident waves with severe emergencies and resource conflicts.
• Focus: survival outcomes under surge.

4. shift_surge (hard)

• New incidents arrive over time and some units go out of service.
• Focus: long-horizon operations and city coverage under degradation.

6. What is an episode?

An episode is one full run of a task from reset until terminal condition.

Episode starts when reset is called.

• step_count starts at 0.
• city_time starts at 0 seconds.
• units and incidents are loaded from selected task fixture.

Episode ends when any terminal condition is hit:

• max steps reached,
• at least one incident escalates,
• all incidents resolved.

7. What is a step?

A step is one action cycle:

1. Agent sends one action.
2. Validator checks if action is legal.
3. State machine applies action effects.
4. Time advances by 30 seconds.
5. Reward is computed.
6. Observation + reward + done are returned.

Important:

• step_count increases by 1 per step.
• city_time increases by 30 seconds per step.

8. At what step are we right now?

Snapshot from the live backend at the time this guide was generated:

• task_id: multi_incident
• episode_id: d2cd525e-2596-44cb-bbe3-af33236264a0
• step_count: 8
• city_time: 240.0 seconds
• cumulative_reward: 1.6
• episode_score: 0.0
• legal_actions currently available: 36

This is a live value, not a constant. If you reset again, step_count returns to 0.

9. Action space (what actions exist)

Current action types include:

• DISPATCH
• CANCEL
• REASSIGN
• STAGE
• MUTUAL_AID
• UPGRADE
• DOWNGRADE

Legal actions are generated from current state and filtered by protocol validation, so only valid actions appear in legal_actions.

10. How scoring works (complete detail)

There are two scoring layers:

1. Step reward (every action)
2. Episode score (whole run)

10.1 Step reward (RewardCalculator)

Step reward uses a weighted sum of 5 components:

• response_time: 30%
• triage: 25%
• survival: 25%
• coverage: 12%
• protocol: 8%

Total formula:

• total = 0.30 * response_time + 0.25 * triage + 0.25 * survival + 0.12 * coverage + 0.08 * protocol
• result is clamped to [0, 1]

Safety rule:

• If any Priority-1 incident existed and survival component is 0, total score is capped at 0.2.

Component details:

1. response_time

• Only meaningful for DISPATCH.
• For non-DISPATCH actions it returns neutral 0.5.
• For DISPATCH: compares ETA to severity benchmark.

2. triage

• Only meaningful for DISPATCH.
• Checks if dispatched unit type matches required unit types for incident type.
• Handles enum-qualified metadata keys safely.

3. survival

• Based on P1 incidents seen vs resolved without failure.
• Uses metadata lists: p1_seen, resolved_incidents, failed_incidents.

4. coverage

• Measures how many districts still have AVAILABLE coverage.

5. protocol

• If action invalid: 0.0.
• If valid and no phraseology text in Action.notes: neutral 0.5.
• If Action.notes provided: uses PhraseologyJudge score + readback correctness.

10.2 Episode score (whole run)

Episode score is task-specific via a central grade_episode router.

Why this matters:

• Different tasks need different definitions of success.
• Mean step reward alone is often too weak for real evaluation.

Task-specific episode graders:

1. single_incident

• +0.50 if incident resolved
• +0.30 if MEDIC dispatched correctly
• +0.20 if resolved within first 10 steps

2. multi_incident

• Uses P1 resolution, overall resolution ratio, and escalation penalty
• score = 0.5 * p1_score + 0.3 * resolution_score - 0.2 * failure_penalty

3. mass_casualty

• Emphasizes P1 survival with penalties for failures
• score = 0.6 * survival_score + 0.3 * mean_reward - failure_penalty

4. shift_surge (improved)

• Emphasizes long-horizon operational quality:
  • incident throughput (resolved ratio)
  • P1 survival
  • coverage
  • low backlog
  • mean reward
  • escalation penalty

11. Very important score semantics

In the OpenEnv wrapper:

• reward return value from step is per-step reward.
• observation.score is overwritten to episode score.

Also stored in metadata:

• cumulative_reward: running sum of step rewards.
• episode_rewards: list of per-step rewards.
• episode_score: current episode-level grade.

So if you compare values:

• reward = immediate local quality for this action
• observation.score = global task progress quality for the run

12. Is the dashboard connected to backend or just static?

It is connected to backend.

How we know:

• The dashboard JavaScript calls API endpoint http://localhost:8000/dashboard/state.
• It polls every 500 ms.
• It renders live units/incidents, step, and reward breakdown from backend response.

Connection behavior:

• If backend is unreachable, dashboard shows disconnected status.
• If backend is running and reset was called, dashboard updates live as step changes.

13. Why we used Docker

Docker is used to package the app and dependencies so it runs consistently everywhere.

Benefits:

• Same runtime on your machine, CI, and deployment platforms.
• No "works on my machine" package mismatch issues.
• Easy deployment with a single container image.
• Port compatibility: server reads PORT environment variable (important for hosted platforms).

In this project:

• Root Dockerfile runs uvicorn on 0.0.0.0 and PORT (default 8000).
• That makes it suitable for local run and hosted environments.

14. What API key are we using?

The project expects environment variables. Keys are not hardcoded in repository files.

Required for LLM mode:

• API_BASE_URL
• MODEL_NAME
• OPENAI_API_KEY

Compatibility fallback:

• HF_TOKEN is accepted if OPENAI_API_KEY is not set.

No-key mode:

• USE_RANDOM=true bypasses LLM and uses a deterministic random baseline agent.

Practical meaning:

• If USE_RANDOM=true, you can run without any API key.
• If USE_RANDOM is not true, OPENAI_API_KEY (or HF_TOKEN fallback) is needed.

15. Backend API endpoints (what each does)

• GET /health

  • health check

• GET /tasks

  • list available tasks

• POST /reset

  • start new episode for selected task

• POST /step

  • apply one action and move simulation one step

• GET /state

  • current state

• GET /dashboard/state

  • extended state for HTML dashboard (includes legal actions + last observation)

• GET /metadata and GET /schema

  • environment metadata and contracts

• POST /mcp

  • minimal JSON-RPC endpoint

16. What the dashboard shows vs what it does not show

Shows:

• Unit cards (status, assignment, ETA, location)
• Incident cards (type, severity, status, assigned units)
• Map view for units/incidents
• Last step reward component bars
• Header task/episode/step values

Nuance:

• Header "Score" currently uses metadata.cumulative_reward.
• Episode score is available too (metadata.episode_score), but not currently shown as the main header score.

17. Beginner glossary

• incident: emergency case to be handled
• unit: responder vehicle/team (EMS, fire, police, etc.)
• legal action: an action that passes protocol checks in current state
• reward: immediate feedback signal for one step
• episode score: overall quality of a full run
• terminal: episode is finished

18. Practical "how to think" summary

When you judge behavior quality in this project:

• Use step rewards to understand local tactical quality.
• Use episode score to understand mission success for the selected task.
• Use dashboard to observe live state transitions.
• Use task definitions to interpret what success means in each scenario.

If you remember one thing:

• This is not a generic chatbot app. It is a decision simulator where actions change a world state over time and are graded both step-by-step and across full episodes.