docs/iteration_plan.md

# Fleetmind Iteration Plan

This document captures the next disciplined improvement loop for Fleetmind.

## Iteration Principles

1. Change one thing at a time.
2. Evaluate on fixed train seeds and held-out eval seeds.
3. Judge progress by both score and behavior.
4. Keep the environment black-box from the agent's perspective.

## Seed Discipline

Use small repeated seed sets instead of one-off runs.

Suggested split:
- train seeds: `1, 2, 3`
- eval seeds: `101, 102, 103`

Use the same seeds before and after each change.

## Curriculum

Run experiments in this order:
- `low_demand` for API and policy sanity
- `high_demand` for strategy formation
- `hotspot_congestion` for dynamic robustness

The goal is not to learn only on the hardest task.

## Iteration 1: Better End-of-Run Learning Signal

Hypothesis:
- Agents need a clearer terminal summary to improve across repeated episodes.

Success criteria:
- agents can explain what caused score loss
- policy revisions become more targeted

Candidate additions:
- average lateness
- reward lost to expiry
- reward lost to rejection
- cumulative idle penalty incurred

## Iteration 2: Reduce Waiting / No-Op Exploits

Hypothesis:
- repeated empty steps with worthwhile visible work are still too attractive.

Success criteria:
- agents stop gaining from passive waiting loops
- short strategic holding remains viable

Candidate changes:
- track repeated no-op steps
- increase penalty only when worthwhile serviceable work exists
- reset the counter after meaningful assignments or rejections

## Iteration 3: Make Medium/Hard More Distinct Strategically

Hypothesis:
- `high_demand` and `hotspot_congestion` should differ not only in pressure, but in the kind of planning they demand.

Success criteria:
- medium rewards stable prioritization
- hard rewards adaptation to evolving demand and congestion

Candidate changes:
- keep `high_demand` steady but capacity-constrained
- make `hotspot_congestion` reward anticipation of shifting hotspot phases more strongly

## Evaluation Protocol

For each iteration:

1. Run baseline and target on train/eval seeds.
2. Run one black-box agent with the prompt in `docs/agent_eval_prompt.md`.
3. Compare:
- cumulative reward
- on-time rate
- expiries
- rejections
- qualitative strategy

Keep the change only if both behavior and metrics improve.