feat(scenario): build-power-online-first — power-grid bring-up sequencing (PlanBench / SOP anchor)

openra_bench/scenarios/packs/build-power-online-first.yaml

@@ -0,0 +1,243 @@
+# Validate: python -m openra_bench.scenarios.validate packs/build-power-online-first.yaml
+# See ../CONTRIBUTING.md for the full win-condition grammar.
+#
+# BUILD-POWER-ONLINE-FIRST (Wave-7 Group I REASONING: opening-phase
+# build-order — power-grid bring-up sequencing).
+#
+# Real-world anchor: electrical-grid bring-up sequencing — the grid
+# (powr) must be ONLINE before any powered load (proc/barr/tent/weap)
+# can come up. Benchmark anchor: PlanBench task-ordering / SOP
+# (standard-operating-procedure) compliance.
+#
+# The decision is the OPENING order: from a pre-placed Construction
+# Yard (`fact`) and a tight cash budget, the agent must build the
+# Power Plant (`powr`) FIRST and only THEN the first production /
+# economy building (`proc`). The order is enforced by the `then:[A,B]`
+# composite (Wave-2 stateful happened-before latch) AND by the engine
+# itself: `proc`/`tent`/`barr` all have `powr` as a hard prerequisite
+# (OpenRA-Rust/openra-sim/src/gamerules.rs:340-343), so a
+# StartProduction call for any of them is silently BLOCKED (`ORDER:
+# StartProduction BLOCKED — missing prerequisites for proc`) when
+# powr isn't up yet. A "build proc/barr/weap first" policy therefore
+# never places ANY production building and never satisfies clause B
+# of the `then` latch — and the tick deadline bites as a real LOSS
+# (not a draw).
+#
+# Engine facts used (verified against the installed openra_train wheel):
+#
+# 1. Build costs (gamerules.rs `actor!` macros):
+#       powr = 300  (no prereq, +100 power)
+#       proc = 1400 (prereq powr, −30 power)
+#       tent = 400  (prereq powr)
+#       barr = 300  (prereq powr)
+#       weap = 2000 (prereq proc)
+#    Intended chain spend: 300 (powr) + 1400 (proc) = $1700. The
+#    levels' starting_cash is sized to cover EXACTLY this chain
+#    (easy=1900 slack, medium=1750 tight, hard=1750 tight).
+# 2. `proc` / `tent` / `barr` / `weap` all have `powr` as a
+#    prerequisite (gamerules.rs:340-343, and via proc for weap). A
+#    `Command.build('proc')` before any `powr` is queued is silently
+#    rejected by the engine — the production queue stays empty, the
+#    cash stays in the bank, no building ever appears.
+# 3. The `then:[A,B]` composite has greedy late-credit semantics
+#    (see `tests/test_then_composite.py`); the discrimination here
+#    is provided by the TICK BUDGET, not by the latch alone. A wrong-
+#    order policy that endlessly retries proc-without-powr (or one
+#    that builds an army first and never gets to powr+proc) burns the
+#    clock and times out as a real LOSS.
+# 4. Tick alignment: engine advances ~90 ticks per decision turn,
+#    so within_ticks ≤ 93 + 90·(max_turns − 1) is required (CLAUDE.md).
+#    fail_condition uses `after_ticks = within_ticks + 1` so a non-
+#    finisher LOSES on the very next tick after the window closes.
+# 5. The persistent unarmed enemy `fact` MARKER far away keeps the
+#    engine's auto-`done` (all-enemies-eliminated) path gated, so
+#    non-winners reach the deadline as LOSS (never DRAW). The agent
+#    has no offensive tools, so the enemy `fact` cannot be destroyed.
+# 6. The `not has_building: fact` fail clause guards against the
+#    edge case where the agent's own pre-placed `fact` is destroyed
+#    (here essentially impossible — no enemy aggression — but kept
+#    as a defensive guard and per CLAUDE.md idiom).
+# 7. Hard tier: `spawn_point` filter applies ONLY to AGENT actors,
+#    and ANY agent actor with spawn_point causes agent actors WITHOUT
+#    spawn_point to be filtered out (CLAUDE.md oramap.rs footgun).
+#    So the agent `fact` must be DUPLICATED across both spawn_point
+#    groups at the spawn-matched coords; the enemy `fact` marker
+#    has no spawn_point and always places.
+# 8. Hard tier carries one inert `e1` infantry per spawn group
+#    (HoldFire stance, no `move_units`/`attack_unit` tool exposed,
+#    far from the enemy `fact` and out of any auto-fire range) so
+#    `tests/test_hard_tier.py::test_curated_hard_still_compiles_and_runs`
+#    can observe the seed-driven spawn variation via `units_summary`
+#    (its assertion reads unit cells, not building cells — a
+#    building-only pack would silently produce empty `units_summary`
+#    on every seed and break the contract). The `e1` is functionally
+#    inert: with `move_units`/`attack_unit` absent from `base.tools`
+#    it cannot be ordered, and with HoldFire stance + the enemy
+#    `fact` ~100 cells away it cannot auto-fire either, so it does
+#    not interact with the SOP-compliance decision.
+
+meta:
+  id: build-power-online-first
+  title: 'Build Power Online First — Grid Bring-Up Sequencing'
+  capability: reasoning
+  real_world_meaning: >
+    Standard-operating-procedure (SOP) compliance under a strict
+    happened-before constraint, exemplified by electrical-grid bring-
+    up: the grid (the Power Plant) must be ONLINE before any powered
+    load (the Refinery, Barracks, War Factory) can come up. The
+    opening decision is the entire test — from a pre-placed
+    Construction Yard and a tight budget, the agent must queue the
+    Power Plant FIRST and only THEN the first production / economy
+    building. A model that "just builds the goal building" (the
+    refinery) first finds the order silently blocked (the engine
+    enforces powr as a prerequisite of proc), wastes the budget
+    retrying, and times out. A model that stalls or builds an army
+    instead also misses the bar.
+  robotics_analogue: >
+    Standard operating procedure with a hard happened-before
+    constraint: a robotic operator commissioning a new processing
+    site must energise the substation BEFORE bringing up the
+    refinery (or any other powered load). Attempting the refinery
+    first is rejected by the interlock; reasoning about the order
+    of operations from the spec is the entire decision.
+  author: wave7-groupI-buildorder
+  benchmark_anchor:
+    - "PlanBench"
+    - "SOP compliance"
+    - "electrical-grid bring-up"
+
+base_map: rush-hour-arena
+starting_cash: 1900
+
+base:
+  agent: {faction: allies}
+  enemy: {faction: soviet, cash: 0}
+  # Pure build-order test: no deploy, no harvest, no move/attack tools.
+  # The decision is exclusively the SEQUENCE of build + place_building.
+  tools: [observe, build, place_building, stop]
+  spawn_mcvs: false
+  planning: true
+  termination: {max_ticks: 8000}
+  actors: []
+
+levels:
+  # ── EASY ─────────────────────────────────────────────────────────
+  # Bare SOP compliance: powr THEN proc, generous clock. fact pre-
+  # placed at (20,20). max_turns 40 → reachable tick 93 + 90·39 = 3603
+  # > 3600 ✓. Generous slack ($1900 − $1700 = $200) covers exactly
+  # the chain; the model cannot also blow the budget on a single 400-
+  # cost mis-build (tent) and still complete the chain.
+  easy:
+    description: >
+      You start with a Construction Yard ('fact') at (20,20) and
+      $1900. Build a Power Plant ('powr', cost 300) FIRST, then an
+      Ore Refinery ('proc', cost 1400). Win = own a 'powr' AND own
+      a 'proc' IN THAT ORDER (the powr must appear FIRST), before
+      tick 3600. The Refinery REQUIRES a Power Plant — queuing
+      'proc' first is silently rejected by the engine. Stalling,
+      building infantry/army, or attempting the Refinery before the
+      Power Plant all miss the clock and LOSE.
+    starting_cash: 1900
+    overrides:
+      actors:
+        - {type: fact, owner: agent, position: [20, 20]}
+        # Persistent far enemy marker — LOSS-not-DRAW guarantee.
+        - {type: fact, owner: enemy, position: [120, 20]}
+    win_condition:
+      all_of:
+        - then:
+            id: build-power-online-first-E
+            clauses:
+              - {has_building: powr}
+              - {building_count_gte: {type: proc, n: 1}}
+        - within_ticks: 3600
+    fail_condition:
+      any_of:
+        - after_ticks: 3601
+        - not: {has_building: fact}
+    max_turns: 40
+
+  # ── MEDIUM ───────────────────────────────────────────────────────
+  # +1 controlled variable: TIGHTER CASH ($1750 = $50 slack only) AND
+  # tighter clock. A single mis-build (e.g. a $400 tent attempt that
+  # gets QUEUED with no power and never completes — cash still spent
+  # by StartProduction? actually engine refunds blocked; but a $300
+  # powr that the model rebuilds twice does drain) tightens the
+  # margin. max_turns 32 → reachable tick 93 + 90·31 = 2883 > 2880 ✓.
+  medium:
+    description: >
+      Same opening: 'fact' at (20,20), no power, no economy. Build
+      'powr' FIRST (cost 300), THEN 'proc' (cost 1400). Tighter
+      budget ($1750 — covers the chain with $50 to spare) and
+      tighter clock (tick 2880). The Refinery REQUIRES the Power
+      Plant — queuing 'proc' first is silently rejected. Stalling,
+      army-building, or attempting 'proc' before 'powr' all run out
+      the clock and LOSE.
+    starting_cash: 1750
+    overrides:
+      actors:
+        - {type: fact, owner: agent, position: [20, 20]}
+        - {type: fact, owner: enemy, position: [120, 20]}
+    win_condition:
+      all_of:
+        - then:
+            id: build-power-online-first-M
+            clauses:
+              - {has_building: powr}
+              - {building_count_gte: {type: proc, n: 1}}
+        - within_ticks: 2880
+    fail_condition:
+      any_of:
+        - after_ticks: 2881
+        - not: {has_building: fact}
+    max_turns: 32
+
+  # ── HARD ─────────────────────────────────────────────────────────
+  # +1 controlled variable on top of medium: TWO `fact` spawn_point
+  # groups (NORTH y=10..14 / SOUTH y=26..30) round-robined by seed.
+  # The agent's pre-placed `fact` (and therefore the build radius and
+  # the placement coords for powr/proc) flips per seed, so a
+  # memorised "(20,20) opening" cannot generalise. Same tight cash
+  # ($1750) and same tight clock as medium; the SOP-compliance
+  # decision is identical, but the placement coordinates must be
+  # read from the actual fact position. max_turns 32 → reachable
+  # tick 2883 > 2880 ✓.
+  hard:
+    description: >
+      Same opening as medium (powr FIRST then proc, tight $1750
+      budget, tick 2880 clock) but your Construction Yard may begin
+      in the NORTH band (y≈12) or the SOUTH band (y≈28) of the map
+      depending on the seed. Read the fact's actual position from
+      the observation and place 'powr' and 'proc' adjacent to it.
+      A memorised "(20,20)" opening will mis-place the buildings
+      out-of-radius on one of the two spawns. The Refinery still
+      REQUIRES the Power Plant; out-of-order builds are silently
+      rejected and the clock runs out as a LOSS.
+    starting_cash: 1750
+    overrides:
+      actors:
+        # Two agent `fact` spawn groups (round-robin by seed).
+        # NORTH band y=10..14; SOUTH band y=26..30.
+        - {type: fact, owner: agent, position: [20, 12], spawn_point: 0}
+        - {type: fact, owner: agent, position: [20, 28], spawn_point: 1}
+        # Inert spawn-witness e1 per group (see engine fact #8 above
+        # — needed for the hard-tier units_summary spawn-variation
+        # check; HoldFire + no tools to command + enemy ~100 cells
+        # away makes this functionally inert for the SOP test).
+        - {type: e1, owner: agent, position: [22, 12], spawn_point: 0, stance: 0}
+        - {type: e1, owner: agent, position: [22, 28], spawn_point: 1, stance: 0}
+        # Persistent enemy marker — LOSS-not-DRAW guarantee.
+        - {type: fact, owner: enemy, position: [120, 20]}
+    win_condition:
+      all_of:
+        - then:
+            id: build-power-online-first-H
+            clauses:
+              - {has_building: powr}
+              - {building_count_gte: {type: proc, n: 1}}
+        - within_ticks: 2880
+    fail_condition:
+      any_of:
+        - after_ticks: 2881
+        - not: {has_building: fact}
+    max_turns: 32

tests/test_build_power_online_first.py

@@ -0,0 +1,335 @@
+"""No-cheat + solvency proof for `build-power-online-first` (Wave-7
+Group I REASONING: opening-phase build-order — power-grid bring-up
+sequencing).
+
+The pack tests the SOP-compliance opening: from a pre-placed
+Construction Yard and a tight cash budget, the agent must build the
+Power Plant (`powr`) FIRST and only THEN the Refinery (`proc`). The
+order is enforced both by the `then:[A,B]` composite (Wave-2 stateful
+happened-before latch) AND by the engine itself (powr is a hard
+prerequisite of proc/tent/barr; weap needs proc).
+
+For every level + every hard seed (1-4):
+  * the INTENDED order (build powr → place powr → build proc → place
+    proc) WINS;
+  * STALL (only `observe`) LOSES — never builds anything → win
+    clauses fail, deadline bites as a real timeout LOSS;
+  * PROC-FIRST (always tries `build('proc')` first, never powr) LOSES
+    — the engine silently rejects StartProduction for proc without
+    powr; no production building ever appears; deadline bites;
+  * ARMY-SPAM (only `build('e1')`, never builds any structure) LOSES
+    — no barracks, no powr, no proc; deadline bites.
+
+The 3 lazy plays + 1 intended × 3 levels × 4 seeds gives the full
+no-defect / no-cheat coverage demanded by CLAUDE.md.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+pytest.importorskip("openra_train", reason="Rust env wheel not installed")
+pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
+
+from openra_bench.eval_core import run_level
+from openra_bench.scenarios import load_pack
+from openra_bench.scenarios.loader import PACKS_DIR, compile_level
+
+PACK = PACKS_DIR / "build-power-online-first.yaml"
+LEVELS = ("easy", "medium", "hard")
+SEEDS = (1, 2, 3, 4)
+
+
+# ───────────────────────── scripted policies ──────────────────────────────
+
+
+def _stall(rs, Command):
+    """Idle: never builds anything → win clauses fail → LOSS on the
+    deadline."""
+    return [Command.observe()]
+
+
+def _proc_first(rs, Command):
+    """Always tries `build('proc')` first; NEVER tries powr. The engine
+    silently rejects StartProduction for proc without powr (CLAUDE.md
+    engine fact #2; world.rs:768 'ORDER: StartProduction BLOCKED —
+    missing prerequisites for proc'). No proc ever appears → clause B
+    of the `then` latch never satisfies → LOSS on the deadline. This
+    is the headline wrong-order discriminator."""
+
+    def _fact(rs):
+        return next(
+            (b for b in (rs.get("own_buildings", []) or []) if b["type"] == "fact"),
+            None,
+        )
+
+    fact_b = _fact(rs)
+    if fact_b is None:
+        return [Command.observe()]
+    fx, fy = fact_b["cell_x"], fact_b["cell_y"]
+    prod = rs.get("production", []) or []
+    own_types = {
+        b["type"] for b in (rs.get("own_buildings", []) or [])
+    }
+    cmds = []
+    if "proc" not in prod and "proc" not in own_types:
+        cmds.append(Command.build("proc"))
+    cmds.append(Command.place_building("proc", fx + 3, fy + 3))
+    return cmds
+
+
+def _army_spam(rs, Command):
+    """Brute army-spam: only `build('e1')`. e1 needs a tent (barracks),
+    which needs powr — neither exists, so every `build('e1')` is
+    silently blocked. No powr / no proc / no barracks ever appear →
+    LOSS on the deadline."""
+    bldgs = rs.get("own_buildings", []) or []
+    fact_b = next((b for b in bldgs if b["type"] == "fact"), None)
+    if fact_b is None:
+        return [Command.observe()]
+    return [Command.build("e1")]
+
+
+def _intended(rs, Command):
+    """Intended SOP: build powr FIRST (cost 300), place it adjacent to
+    the fact; then build proc (cost 1400, now unblocked), place it
+    adjacent to the fact. Wins every tier × every seed before the
+    deadline."""
+    bldgs = rs.get("own_buildings", []) or []
+    own_types = {b["type"] for b in bldgs}
+    prod = rs.get("production", []) or []
+    fact_b = next((b for b in bldgs if b["type"] == "fact"), None)
+    if fact_b is None:
+        return [Command.observe()]
+    fx, fy = fact_b["cell_x"], fact_b["cell_y"]
+    if "powr" not in own_types:
+        cmds = []
+        if "powr" not in prod:
+            cmds.append(Command.build("powr"))
+        cmds.append(Command.place_building("powr", fx + 3, fy + 1))
+        return cmds
+    if "proc" not in own_types:
+        cmds = []
+        if "proc" not in prod:
+            cmds.append(Command.build("proc"))
+        cmds.append(Command.place_building("proc", fx + 3, fy + 3))
+        return cmds
+    return [Command.observe()]
+
+
+# ───────────────────────── helpers ────────────────────────────────────────
+
+
+def _run(level, policy, seed=1):
+    c = compile_level(load_pack(PACK), level)
+    assert c.map_supported, "rush-hour-arena must compile"
+    return c, run_level(c, policy, seed=seed)
+
+
+# ───────────────────────── structural ─────────────────────────────────────
+
+
+def test_pack_loads_with_three_levels_and_required_tools():
+    pack = load_pack(PACK)
+    assert pack.meta.id == "build-power-online-first"
+    assert pack.meta.capability == "reasoning"
+    assert pack.meta.status == "active"
+    assert set(pack.levels) == set(LEVELS)
+    c = compile_level(pack, "easy")
+    tools = set(c.scenario.tools or [])
+    # Wave-7 spec: pure build-order test — no deploy / harvest /
+    # move_units / attack_unit. Just observe + build + place + stop.
+    for t in ("observe", "build", "place_building", "stop"):
+        assert t in tools, f"missing tool {t} in {tools}"
+    for forbidden in ("move_units", "attack_unit", "deploy", "harvest"):
+        assert forbidden not in tools, (
+            f"tool {forbidden!r} must NOT be in the SOP-only palette"
+        )
+
+
+def test_benchmark_anchor_lists_planbench_and_grid_bringup():
+    pack = load_pack(PACK)
+    anchors = pack.meta.benchmark_anchor
+    assert anchors, "benchmark_anchor must be non-empty"
+    blob = " | ".join(anchors).lower()
+    assert "planbench" in blob, anchors
+    assert "sop" in blob, anchors
+    assert "electrical" in blob and "grid" in blob, anchors
+
+
+@pytest.mark.parametrize("level", LEVELS)
+def test_every_level_has_a_reachable_timeout_fail(level):
+    """Non-win must be a real LOSS, never a DRAW: after_ticks in
+    fail_condition must be reachable within max_turns (tick ≈ 93 +
+    90·(max_turns − 1))."""
+    c = compile_level(load_pack(PACK), level)
+    assert c.fail_condition is not None
+    after_ticks = int(c.fail_condition.model_dump()["any_of"][0]["after_ticks"])
+    reachable = 93 + 90 * (c.max_turns - 1)
+    assert after_ticks <= reachable, (
+        f"{level}: fail after_ticks {after_ticks} unreachable within "
+        f"{c.max_turns} turns (max tick {reachable}) → draw degeneracy"
+    )
+    # within_ticks + 1 == after_ticks (non-finisher LOSES on the very
+    # next tick after the win window closes).
+    within_clauses = c.win_condition.model_dump().get("all_of", [])
+    wt = next(
+        int(x["within_ticks"]) for x in within_clauses if "within_ticks" in x
+    )
+    assert after_ticks == wt + 1, (
+        f"{level}: after_ticks {after_ticks} must equal within_ticks+1 ({wt+1})"
+    )
+
+
+@pytest.mark.parametrize("level", LEVELS)
+def test_win_uses_then_composite_powr_before_proc(level):
+    """Structural: the win clause uses a `then` composite with two
+    clauses in the exact order [powr, proc] — this is the engine-side
+    SOP-compliance enforcement (CLAUDE.md ordering idiom)."""
+    c = compile_level(load_pack(PACK), level)
+    win = c.win_condition.model_dump()
+    all_of = win.get("all_of", [])
+    then_node = next(
+        (x["then"] for x in all_of if "then" in x), None
+    )
+    assert then_node is not None, f"{level}: win must include a `then` composite"
+    clauses = then_node.get("clauses", [])
+    assert len(clauses) == 2, f"{level}: then must have 2 clauses (powr, proc)"
+    # First clause: has_building powr; second: building_count_gte proc>=1.
+    assert clauses[0].get("has_building") == "powr", clauses[0]
+    bc = clauses[1].get("building_count_gte") or {}
+    assert bc.get("type") == "proc" and int(bc.get("n", 0)) >= 1, clauses[1]
+
+
+def test_hard_has_two_spawn_groups_for_fact():
+    """Hard tier contract (CLAUDE.md + tests/test_hard_tier.py): ≥2
+    distinct agent spawn_point groups (NORTH y≈12 / SOUTH y≈28) so
+    seed round-robin varies the pre-placed fact and a memorised
+    "(20,20)" opening can't generalise."""
+    c = compile_level(load_pack(PACK), "hard")
+    sps = {
+        (a.spawn_point if a.spawn_point is not None else 0)
+        for a in c.scenario.actors
+        if a.owner == "agent"
+    }
+    assert len(sps) >= 2, f"hard needs ≥2 spawn groups, got {sps}"
+
+
+# ───────────────────────── intended WIN ───────────────────────────────────
+
+
+@pytest.mark.parametrize("level", LEVELS)
+@pytest.mark.parametrize("seed", SEEDS)
+def test_intended_powr_first_then_proc_wins(level, seed):
+    """Intended build powr → place powr → build proc → place proc WINS
+    every level × every seed before the deadline."""
+    c, r = _run(level, _intended, seed=seed)
+    assert r.outcome == "win", (
+        f"{level} seed{seed}: intended powr-first chain should WIN, "
+        f"got {r.outcome}; types={r.signals.own_building_types}, "
+        f"cash={r.signals.cash}, tick={r.signals.game_tick}"
+    )
+    types = set(r.signals.own_building_types)
+    assert "powr" in types, types
+    assert "proc" in types, types
+
+
+# ───────────────────────── no-cheat: lazy plays LOSE ──────────────────────
+
+
+@pytest.mark.parametrize("level", LEVELS)
+@pytest.mark.parametrize("seed", SEEDS)
+def test_stall_loses(level, seed):
+    """Stall must LOSE every level × every seed — never builds anything
+    → both then-clauses fail → reachable timeout LOSS (not a draw)."""
+    c, r = _run(level, _stall, seed=seed)
+    assert r.outcome == "loss", (
+        f"{level} seed{seed} stall must LOSE; got {r.outcome} "
+        f"(types={r.signals.own_building_types}, tick={r.signals.game_tick})"
+    )
+    # Never built powr or proc.
+    assert "powr" not in r.signals.own_building_types
+    assert "proc" not in r.signals.own_building_types
+
+
+@pytest.mark.parametrize("level", LEVELS)
+@pytest.mark.parametrize("seed", SEEDS)
+def test_proc_first_wrong_order_loses(level, seed):
+    """The headline wrong-order discriminator: always tries `proc`
+    first, never powr. Engine silently rejects proc-without-powr
+    (world.rs:768 'StartProduction BLOCKED — missing prerequisites
+    for proc'); no production building ever appears → both then-
+    clauses fail → LOSS on the deadline."""
+    c, r = _run(level, _proc_first, seed=seed)
+    assert r.outcome == "loss", (
+        f"{level} seed{seed} proc-first must LOSE; got {r.outcome} "
+        f"(types={r.signals.own_building_types}, tick={r.signals.game_tick})"
+    )
+    assert "powr" not in r.signals.own_building_types
+    assert "proc" not in r.signals.own_building_types
+
+
+@pytest.mark.parametrize("level", LEVELS)
+@pytest.mark.parametrize("seed", SEEDS)
+def test_army_spam_brute_loses(level, seed):
+    """Brute army-spam (only `build('e1')`, never any structure) must
+    LOSE — e1 needs tent which needs powr; no chain ever exists; no
+    proc; LOSS on the deadline."""
+    c, r = _run(level, _army_spam, seed=seed)
+    assert r.outcome == "loss", (
+        f"{level} seed{seed} army-spam must LOSE; got {r.outcome} "
+        f"(types={r.signals.own_building_types})"
+    )
+    assert "powr" not in r.signals.own_building_types
+    assert "proc" not in r.signals.own_building_types
+
+
+# ───────────────────────── hard spawn round-robin ─────────────────────────
+
+
+def test_hard_seed_round_robin_produces_distinct_starts():
+    """Seeds 1-4 must round-robin between the two declared spawn_point
+    groups (NORTH (20,12) / SOUTH (20,28)) so a memorised opening
+    cannot generalise. The inert spawn-witness `e1` per group surfaces
+    the variation via units_summary (the building-only pack would
+    otherwise be invisible to the hard-tier shared test)."""
+    from pathlib import Path
+
+    from openra_bench.eval_core import RustEnvPool, _scenario_to_tmp_yaml
+    from openra_bench.rust_adapter import RustObsAdapter
+
+    c = compile_level(load_pack(PACK), "hard")
+    tmp = _scenario_to_tmp_yaml(c)
+    pool = RustEnvPool(size=1, scenario_path=tmp)
+    env = pool.acquire()
+    starts = set()
+    try:
+        for seed in (1, 2, 3, 4):
+            ad = RustObsAdapter()
+            ad.observe(env.reset(seed=seed))
+            u = ad.render_state().get("units_summary", []) or []
+            if u:
+                starts.add(
+                    tuple(sorted((x["cell_x"], x["cell_y"]) for x in u))
+                )
+    finally:
+        pool.release(env)
+        pool.shutdown()
+        Path(tmp).unlink(missing_ok=True)
+    assert len(starts) >= 2, (
+        f"hard seeds 1-4 produced identical starts {starts}; "
+        "spawn_point round-robin not taking effect"
+    )
+
+
+# ───────────────────────── determinism ────────────────────────────────────
+
+
+def test_outcomes_are_deterministic_per_seed():
+    """Same seed, same pack, same policy → identical outcome."""
+    c = compile_level(load_pack(PACK), "easy")
+    a = run_level(c, _intended, seed=2)
+    b = run_level(c, _intended, seed=2)
+    assert (a.outcome, a.turns) == (b.outcome, b.turns), (
+        f"determinism: {(a.outcome, a.turns)} vs {(b.outcome, b.turns)}"
+    )