feat(scenario): def-position-revealed-direction — scout-then-fortify adaptive defense (PlanBench replanning anchor)

openra_bench/scenarios/packs/def-position-revealed-direction.yaml

@@ -0,0 +1,457 @@
+# def-position-revealed-direction.yaml
+#
+# REASONING capability — INTEL-DRIVEN ADAPTIVE DEFENSE. Unlike its
+# sibling `def-position-expected-direction` (where the threat axis is
+# STATED in the brief and the agent merely commits a budget to the
+# named lane), this pack hides the axis: the agent must FIRST SCOUT to
+# discover where the enemy's forward outpost actually is, THEN fortify
+# THAT lane with pillboxes, THEN engage and kill the incoming rush.
+# Pre-fortifying any one direction on faith LOSES on seeds where the
+# enemy comes from the opposite side; only the scout-then-fortify
+# discipline generalises.
+#
+# Win idiom (the load-bearing chain):
+#   then:
+#     - {buildings_discovered_gte: 1}    # FIRST — scout finds the
+#                                        # enemy outpost on the
+#                                        # approach axis (outposts are
+#                                        # OUTSIDE pbox/base vision; a
+#                                        # scout MUST move on medium/
+#                                        # hard).
+#     - {building_count_gte: {type: pbox, n: 3}}  # THEN — commit 3
+#                                        # pillboxes (defensive
+#                                        # posture).
+#     - {units_killed_gte: K}            # THEN — actually engage and
+#                                        # break the incoming rush.
+#   AND building_count_gte:{type: fact, n: 1} AND within_ticks: T.
+# Fail: after_ticks: T+1 OR not building_count_gte:{type: fact, n: 1}
+#       OR not own_units_gte: 1.
+# (We use `building_count_gte` rather than `has_building` because the
+# latter checks `own_building_types`, which ACCUMULATES across the
+# episode — once `fact` has been owned, `has_building:fact` stays true
+# even after the fact is razed, so it is structurally useless as a
+# "still standing" fail clause. `building_count_gte` reads the live
+# `own_buildings` list each frame and bites correctly when the fact
+# is destroyed.)
+#
+# Pre-fortify-N-without-scout fails: clause 1 of the then-chain never
+# latches because the pbox vision radius (~6) cannot reach the enemy
+# outpost (placed 10+ cells away from the base on the approach axis).
+# Pre-fortify-N-with-scout (memorised NORTH on hard) fails on the
+# SOUTH-spawn seeds: the defenses sit silent on the wrong axis, the
+# south-band rushers walk straight to the south-spawn fact,
+# units_killed never reaches K, and the fact is razed → double LOSS
+# path (chain clause 3 + the fact-loss fail clause). Stall LOSES on
+# the deadline (or earlier on the fact-loss fail clause when the rush
+# closes the home fact).
+#
+# Real-world / benchmark anchors:
+#   - PlanBench replanning: model must update its plan based on
+#     observation (which axis is the actual threat) before committing.
+#   - Military reactive defense / intel-driven facility hardening:
+#     reconnaissance precedes fortification; pre-committing without
+#     intel is the classical "Maginot Line" failure mode (defenses on
+#     the wrong axis are dead weight).
+#   - Intel-driven defense: convert reconnaissance into committed
+#     defensive posture; a generalist policy MUST drive the scout
+#     loop, not memorise a fixed axis.
+#
+# Distinct from siblings:
+#   * def-position-expected-direction — intel is GIVEN; the decision
+#     is purely WHICH axis given the textual cue. THIS pack hides the
+#     axis; the decision is whether to SCOUT before committing.
+#   * def-pre-position-mobile-reserve — direction unknown; mobile
+#     reserve commits on detection (no static defenses). THIS pack
+#     requires STATIC fortification AFTER scouting.
+#   * def-surprise-flank-react — intel is FALSE (announced N, actual
+#     S); the trap is trusting the brief. THIS pack has NO false
+#     intel; the brief states "you don't know — scout to find out".
+#   * lh-scout-react-counter — scout→react→counter on a FIXED enemy
+#     posture; the chain is the same idiom but the capability is
+#     offensive (raze the construction yard) rather than defensive
+#     (fortify the lane).
+#
+# ENGINE FACTS (per CLAUDE.md / SCENARIO_QUALITY.md):
+#   - `pbox` costs 600cr; needs tent (which needs powr). Pre-place
+#     fact + proc + powr + tent so the pbox queue is hot from turn 1.
+#     Starting cash 2400 = 3 × 600 + ~600 buffer.
+#   - Enemy actors with NO spawn_point ALWAYS place regardless of the
+#     agent's spawn group (CLAUDE.md). So on hard we declare BOTH a
+#     NORTH band and a SOUTH band; the seed-driven agent spawn picks
+#     which band is the "near" (immediate) one. Each band has its
+#     own outpost building so scouting either side reveals exactly
+#     one outpost.
+#   - `then:` is the happened-before composite: clauses latch IN
+#     ORDER, so a pre-fortify-no-scout play that places 3 pbox first
+#     and then idles cannot satisfy clause 1 (scout) after the fact
+#     UNLESS a scout actually drives close enough to see the
+#     outpost — which is the load-bearing capability.
+#   - `building_count_gte:{pbox, 3}` (NOT building_in_region) — the
+#     scout latch already enforces axis awareness via clause 1; we
+#     don't double-encode the region. Placement is purely about the
+#     agent committing to defense after intel; lane choice is enforced
+#     by the units_killed clause (3 pbox on the wrong axis sit silent,
+#     the rushers walk past, kills don't accrue, fact dies).
+#   - Inert enemy `fact` marker far east — anti-DRAW per CLAUDE.md;
+#     without a persistent enemy actor the engine auto-`done`s on
+#     enemy-elim before the win/fail evaluator runs.
+#   - `enemy_unit_spotted` interrupt — re-decide the instant a rusher
+#     unit is spotted (the actual axis becomes confirmed by the rush
+#     itself, in case the scout missed the outpost).
+#   - `enemy_building_spotted` interrupt — re-decide the instant the
+#     outpost is discovered (the FIRST clause latches).
+#   - Tick budget: max_turns 66 → reachable tick 93 + 90·65 = 5943 ≥
+#     5402 (deadline 5400 + 1 = 5401), so a non-finisher is a real
+#     reachable timeout LOSS, never a draw.
+#
+# Validate (no model / no network):
+#   python3 -m openra_bench.scenarios.validate \
+#     openra_bench/scenarios/packs/def-position-revealed-direction.yaml
+
+meta:
+  id: def-position-revealed-direction
+  title: 'Scout the Axis, Then Fortify — Intel-Driven Adaptive Defense'
+  capability: reasoning
+  real_world_meaning: >
+    Adaptive defense under uncertainty: the threat axis is NOT known
+    a priori; the agent must drive its scouts to discover the
+    enemy's forward outpost (where the build-up actually is), then
+    commit pillboxes on THAT lane, then engage. Pre-committing to a
+    single direction without scouting is the classical Maginot-Line
+    failure mode — defenses on the wrong axis are dead weight while
+    the unguarded lane is overrun. The capability under test is the
+    intel-action loop: observation FIRST, commitment SECOND.
+  robotics_analogue: >
+    Intel-driven facility hardening / military reactive defense: a
+    perimeter operator does not know which gate will be breached;
+    they must dispatch reconnaissance, confirm the threat axis from
+    sensor returns, and then deploy barriers / responders on the
+    confirmed lane. Skipping the reconnaissance step and pre-staging
+    on a guessed lane is correct half the time and catastrophically
+    wrong the other half. The robust doctrine is scout-then-fortify,
+    not commit-on-faith.
+  benchmark_anchor:
+    - 'PlanBench replanning'
+    - 'military reactive defense'
+    - 'intel-driven defense'
+  author: openra-bench
+
+# rush-hour-arena (128×40, playable x∈[2..126], y∈[2..38]). Agent
+# base at (20, 20) on easy/medium. The fact + proc + powr + tent let
+# the pbox queue fire from turn 1. Two jeep scouts at the base — the
+# agent must drive them to confirm the enemy outpost's lane before
+# committing pillboxes (medium/hard; on easy the jeeps are pre-staged
+# forward and passively see the outpost from turn 1).
+base_map: rush-hour-arena
+
+# Pack-wide budget — funds the intended 3-pbox triple (3 × 600 =
+# 1800cr) with a small buffer (~600cr) for replacement defenders or
+# an extra pbox if the agent wants. Spamming pbox on the wrong axis
+# burns the same budget without satisfying the kill clause (wrong-axis
+# pbox sit silent, the rushers walk past).
+starting_cash: 2400
+
+base:
+  agent:
+    faction: allies
+  # Concentrated charge at the agent centroid every ~8 ticks — the
+  # canonical rush-defense opponent. The bot's SPAWN POSITION is the
+  # hidden information the model must discover by scouting.
+  enemy:
+    faction: soviet
+    bot_type: rusher
+  tools:
+    - observe
+    - build
+    - place_building
+    - move_units
+    - attack_unit
+    - attack_move
+    - stop
+  planning: true
+  # Re-decide the instant the scout discovers an enemy outpost
+  # (the first latch in the then-chain) or a rusher unit is spotted
+  # (confirms the actual axis) or a defender is lost (defense
+  # commitment is reactive).
+  interrupts:
+    enemy_building_spotted: true
+    enemy_unit_spotted: true
+    own_unit_destroyed: true
+  termination:
+    max_ticks: 12000
+  actors:
+    # Base-pack defaults; each level overrides with the spawn-specific
+    # layout. (Anti-empty-actors sanity for the loader.)
+    - {type: fact, owner: agent, position: [20, 20]}
+
+levels:
+  # ── EASY ─────────────────────────────────────────────────────────
+  # The bare scout-then-fortify chain. Axis is NORTH; the brief
+  # explicitly tells the model the chain (scout → fortify → engage)
+  # and the threat axis. The two jeeps are pre-positioned FORWARD
+  # (y=8) so a single passive observe on turn 1 already has line-of-
+  # sight to the enemy outpost (a `tent` at (20, 2), within jeep
+  # vision range of ~7). The first then-clause latches immediately;
+  # the agent only needs to build 3 pbox and engage the incoming
+  # light rush (3 e1) — a 3-kill bar matched by the screen + pbox.
+  # max_turns 66 → reachable tick 5943 ≥ 5401.
+  easy:
+    description: >
+      A light enemy rush is incoming from the NORTH (your scouts can
+      already see the enemy outpost up the lane). The discipline
+      under test is the scout-then-fortify chain: confirm the threat
+      with intel, then commit defenses, then engage. You start with
+      a construction yard, refinery, power plant, and barracks at
+      (20, 20), four rifle defenders, and TWO scout jeeps pre-
+      positioned NORTH at (20, 8)/(22, 8) — already in line of
+      sight of the enemy outpost. Build at least THREE pillboxes
+      (pbox), then destroy ≥3 of the incoming rifle infantry. Keep
+      the construction yard standing and ≥1 of your own units
+      alive. Win before tick 5400. Stalling loses on the clock;
+      pure-build-no-fight loses on the kill bar; pure-fight-no-
+      build loses when the rush overruns the defenders.
+    overrides:
+      actors:
+        # Agent base — fact + proc + powr + tent so the pbox queue
+        # (needs powr + tent) is hot from turn 1. The tent is also
+        # the source of the four pre-placed rifle defenders.
+        - {type: fact, owner: agent, position: [20, 20]}
+        - {type: proc, owner: agent, position: [16, 20]}
+        - {type: powr, owner: agent, position: [20, 24]}
+        - {type: tent, owner: agent, position: [16, 24]}
+        # Two scout jeeps pre-positioned FORWARD-NORTH at y=8 —
+        # already within vision range of the enemy outpost at (20, 2)
+        # (distance 6, inside jeep sight ~7). A single observe on
+        # turn 1 latches `buildings_discovered_gte:1` — easy teaches
+        # the chain without requiring scout MOVEMENT.
+        - {type: jeep, owner: agent, position: [20, 8], stance: 1}
+        - {type: jeep, owner: agent, position: [22, 8], stance: 1}
+        # Four starter defenders SOUTH of the base (away from the
+        # NORTH lane mouth) — they survive a light push so
+        # own_units_gte:1 holds; positioning them away from the lane
+        # avoids auto-fire chasing the rush past the pbox line.
+        - {type: e1, owner: agent, position: [22, 24], stance: 2}
+        - {type: e1, owner: agent, position: [24, 24], stance: 2}
+        - {type: e1, owner: agent, position: [22, 26], stance: 2}
+        - {type: e1, owner: agent, position: [24, 26], stance: 2}
+        # Enemy forward outpost (NORTH): a `tent` at (20, 2) —
+        # placed at the playable edge so it is OUT of canonical pbox
+        # vision (a pbox at y=14 is 12 cells from (20,2), well past
+        # any pbox sight range). The forward jeeps at y=8 sit 6
+        # cells away and see it from turn 1 (jeep sight ~7).
+        - {type: tent, owner: enemy, position: [20, 2]}
+        # Light NORTH rusher push — 3 e1 at (20, 4). The rusher bot
+        # charges the agent centroid (~y=22 with the southern
+        # defenders), walking south through (20, 14) where the pbox
+        # line intercepts.
+        - {type: e1, owner: enemy, position: [20, 4], stance: 3, count: 3}
+        # Inert enemy `fact` marker far east — anti-DRAW per
+        # CLAUDE.md; without a persistent enemy actor the engine
+        # auto-`done`s on enemy-elim before the win/fail evaluator
+        # runs.
+        - {type: fact, owner: enemy, position: [120, 20]}
+    win_condition:
+      all_of:
+        - then:
+            id: scout-fortify-engage-easy
+            clauses:
+              - {buildings_discovered_gte: 1}
+              - {building_count_gte: {type: pbox, n: 3}}
+              - {units_killed_gte: 3}
+        - building_count_gte: {type: fact, n: 1}
+        - own_units_gte: 1
+        - within_ticks: 5400
+    fail_condition:
+      any_of:
+        - after_ticks: 5401
+        - not: {building_count_gte: {type: fact, n: 1}}
+        - not: {own_units_gte: 1}
+    max_turns: 66
+
+  # ── MEDIUM ───────────────────────────────────────────────────────
+  # +1 controlled variable: the AXIS IS FOGGED. Jeeps now start AT
+  # the base (20, 20), NOT forward — the outpost at (20, 2) is 18
+  # cells away, well outside passive vision (~7). A no-scout play
+  # cannot latch the first then-clause. The intended play is
+  # `move_units([jeep1, jeep2], 20, 8)` to enter outpost sight, THEN
+  # build 3 pbox NORTH, THEN engage.
+  # A pair of forward delayer e1 at y=10 absorbs the first wave so
+  # the pbox triple has time to come online (without them, even an
+  # intended-scout play loses because the rush razes the fact before
+  # 3 pbox finish building).
+  # max_turns 66 → reachable tick 5943 ≥ 5401.
+  medium:
+    description: >
+      A rush is incoming, but the threat axis is NOT given — you
+      must SCOUT to find it. You start with a construction yard,
+      refinery, power plant, and barracks at (20, 20), four rifle
+      defenders, and TWO scout jeeps at the base. The enemy has a
+      forward outpost on the approach axis; drive a jeep north (or
+      whichever lane your observation suggests) to register at least
+      ONE enemy building, THEN build at least THREE pillboxes (pbox),
+      THEN destroy ≥3 of the incoming rifle infantry. Keep the
+      construction yard standing and ≥1 of your own units alive.
+      Win before tick 5400. Skipping the scout phase fails the
+      then-chain at clause 1; pre-fortifying without scouting wastes
+      the budget on the wrong-axis intuition; stalling loses on the
+      clock.
+    overrides:
+      actors:
+        - {type: fact, owner: agent, position: [20, 20]}
+        - {type: proc, owner: agent, position: [16, 20]}
+        - {type: powr, owner: agent, position: [20, 24]}
+        - {type: tent, owner: agent, position: [16, 24]}
+        # Jeeps AT the base — must drive to scout. Distance from
+        # (24, 20) to outpost (20, 2) is √(16+324)≈18, far out of
+        # jeep vision (~7).
+        - {type: jeep, owner: agent, position: [24, 20], stance: 1}
+        - {type: jeep, owner: agent, position: [26, 20], stance: 1}
+        - {type: e1, owner: agent, position: [22, 24], stance: 2}
+        - {type: e1, owner: agent, position: [24, 24], stance: 2}
+        - {type: e1, owner: agent, position: [22, 26], stance: 2}
+        - {type: e1, owner: agent, position: [24, 26], stance: 2}
+        # Forward delayer pair NORTH of the base — stationed ahead of
+        # the pbox line so they soak the first wave while the pbox
+        # triple comes online (the rush from y=4 reaches y=14 before
+        # the 1st pbox is up; without absorbers, the wave punches
+        # through and razes the fact before clause 3 can latch).
+        - {type: e1, owner: agent, position: [20, 10], stance: 2}
+        - {type: e1, owner: agent, position: [22, 10], stance: 2}
+        # Enemy outpost (NORTH) at the playable edge (y=2) — well
+        # outside pbox vision (pbox at y=14 is 12 cells away). Scout
+        # MUST drive a jeep north to enter outpost sight range.
+        # Placing pbox WITHOUT scouting cannot retroactively satisfy
+        # clause 1 of the then-chain.
+        - {type: tent, owner: enemy, position: [20, 2]}
+        # NORTH rush — 3 e1 (matched to easy because the medium +1
+        # controlled variable is the SCOUT requirement (outpost no
+        # longer visible from starting jeep position), NOT raw
+        # defensive load — a heavier push would force the medium to
+        # discriminate on a SECOND axis (absorption) and break the
+        # clean tier attribution).
+        - {type: e1, owner: enemy, position: [20, 4], stance: 3, count: 3}
+        - {type: fact, owner: enemy, position: [120, 20]}
+    win_condition:
+      all_of:
+        - then:
+            id: scout-fortify-engage-medium
+            clauses:
+              - {buildings_discovered_gte: 1}
+              - {building_count_gte: {type: pbox, n: 3}}
+              - {units_killed_gte: 3}
+        - building_count_gte: {type: fact, n: 1}
+        - own_units_gte: 1
+        - within_ticks: 5400
+    fail_condition:
+      any_of:
+        - after_ticks: 5401
+        - not: {building_count_gte: {type: fact, n: 1}}
+        - not: {own_units_gte: 1}
+    max_turns: 66
+
+  # ── HARD ─────────────────────────────────────────────────────────
+  # +1 controlled variable: TWO agent spawn_point groups round-robin
+  # the AGENT BASE LATITUDE per seed (NORTH-staged base centred at
+  # (20, 16) vs SOUTH-staged base centred at (20, 24)). Enemy actors
+  # don't honour spawn_point (CLAUDE.md) so BOTH a north outpost
+  # (20, 4) and a south outpost (20, 36) always place; the MATCHING-
+  # side band is the immediate threat on each spawn (the rusher bot
+  # charges the agent centroid, which sits near each base's
+  # latitude, so the closer band has the shorter walk). The capability
+  # test is the same scout-then-fortify chain — but the agent must
+  # FIRST identify its OWN base latitude from observation (the
+  # threat is on the SIDE FACING the base's own latitude) and THEN
+  # commit defenses on that side. A memorised "always defend NORTH"
+  # opening cannot generalise: on the SOUTH spawn the north scout
+  # never reaches the south outpost (clause 1 never latches), the
+  # south-band rushers walk straight past the wrong-axis pbox line,
+  # and the south-spawn fact dies → double LOSS path.
+  # The rush is 4 e1 + 1 e3 per band (the e3 is the discriminator —
+  # rocket splash + AT damage means the base defenders alone cannot
+  # hold the matching lane, so the on-axis pbox triple is load-
+  # bearing; the agent base is staged farther from each rush band
+  # (14 cells instead of 10) to give the intended play time to land
+  # the 3 pbox before the rush arrives).
+  # max_turns 66 → reachable tick 5943 ≥ 5401.
+  hard:
+    description: >
+      A rush is incoming, but the threat axis is NOT given AND your
+      base could be in the NORTH half (fact at y=16) OR the SOUTH
+      half (fact at y=24) — chosen by seed. Identify your base's
+      latitude from observation; scout the lane FACING your base
+      (if your base is north of centre, the threat approaches from
+      the north edge; if south of centre, from the south edge), then
+      build at least THREE pillboxes (pbox) on that lane (between
+      your base and the threat — roughly y≈10 if your base is at
+      y=16, y≈30 if your base is at y=24), THEN destroy ≥3 of the
+      incoming rifle infantry. Keep the construction yard standing
+      and ≥1 of your own units alive. Win before tick 5400. A
+      memorised "always defend NORTH" opening loses on the SOUTH-
+      base seeds (rushers walk past your wrong-axis pbox line and
+      raze the fact); a no-scout commit fails the chain at clause 1.
+    overrides:
+      actors:
+        # ── NORTH spawn (spawn_point 0) — base centred at (20, 16).
+        # Threat: from the north band at y=2 (north outpost at y=4).
+        # Base-to-rush distance = 14 cells, gives the intended scout-
+        # then-fortify play time to land 3 pbox at y≈10 (between base
+        # at y=16 and band at y=2) before the rush arrives. Correct
+        # defense lane: pbox at y≈10. The defender ring + the 4
+        # starter e1 sit SOUTH of fact (y=18..20, away from the lane
+        # mouth) so they survive the first wave and satisfy
+        # own_units_gte:1.
+        - {type: fact, owner: agent, position: [20, 16], spawn_point: 0}
+        - {type: proc, owner: agent, position: [16, 16], spawn_point: 0}
+        - {type: powr, owner: agent, position: [20, 20], spawn_point: 0}
+        - {type: tent, owner: agent, position: [16, 20], spawn_point: 0}
+        - {type: jeep, owner: agent, position: [24, 16], stance: 1, spawn_point: 0}
+        - {type: jeep, owner: agent, position: [26, 16], stance: 1, spawn_point: 0}
+        - {type: e1, owner: agent, position: [22, 20], stance: 2, spawn_point: 0}
+        - {type: e1, owner: agent, position: [24, 20], stance: 2, spawn_point: 0}
+        - {type: e1, owner: agent, position: [22, 18], stance: 2, spawn_point: 0}
+        - {type: e1, owner: agent, position: [24, 18], stance: 2, spawn_point: 0}
+        # ── SOUTH spawn (spawn_point 1) — base centred at (20, 24).
+        # Mirror of the NORTH spawn (fact 8 cells south of map mid),
+        # threat from the south band at y=38 (south outpost at y=36).
+        # Correct defense lane: pbox at y≈30.
+        - {type: fact, owner: agent, position: [20, 24], spawn_point: 1}
+        - {type: proc, owner: agent, position: [16, 24], spawn_point: 1}
+        - {type: powr, owner: agent, position: [20, 20], spawn_point: 1}
+        - {type: tent, owner: agent, position: [16, 20], spawn_point: 1}
+        - {type: jeep, owner: agent, position: [24, 24], stance: 1, spawn_point: 1}
+        - {type: jeep, owner: agent, position: [26, 24], stance: 1, spawn_point: 1}
+        - {type: e1, owner: agent, position: [22, 20], stance: 2, spawn_point: 1}
+        - {type: e1, owner: agent, position: [24, 20], stance: 2, spawn_point: 1}
+        - {type: e1, owner: agent, position: [22, 22], stance: 2, spawn_point: 1}
+        - {type: e1, owner: agent, position: [24, 22], stance: 2, spawn_point: 1}
+        # Enemy outposts and rush bands — enemy actors don't honour
+        # spawn_point (CLAUDE.md), so BOTH north and south bands
+        # always place. Each spawn faces the MATCHING side as the
+        # near (immediate) threat — the opposite-side rush has the
+        # full map width to walk and is well outside the 5400-tick
+        # budget for arrival.
+        - {type: tent, owner: enemy, position: [20, 4]}
+        - {type: e1, owner: enemy, position: [20, 2], stance: 3, count: 4}
+        - {type: e3, owner: enemy, position: [22, 2], stance: 3, count: 1}
+        - {type: tent, owner: enemy, position: [20, 36]}
+        - {type: e1, owner: enemy, position: [20, 38], stance: 3, count: 4}
+        - {type: e3, owner: enemy, position: [22, 38], stance: 3, count: 1}
+        # Inert enemy `fact` marker far east — anti-DRAW.
+        - {type: fact, owner: enemy, position: [120, 20]}
+    win_condition:
+      all_of:
+        - then:
+            id: scout-fortify-engage-hard
+            clauses:
+              - {buildings_discovered_gte: 1}
+              - {building_count_gte: {type: pbox, n: 3}}
+              - {units_killed_gte: 3}
+        - building_count_gte: {type: fact, n: 1}
+        - own_units_gte: 1
+        - within_ticks: 5400
+    fail_condition:
+      any_of:
+        - after_ticks: 5401
+        - not: {building_count_gte: {type: fact, n: 1}}
+        - not: {own_units_gte: 1}
+    max_turns: 66

tests/test_def_position_revealed_direction.py

@@ -0,0 +1,475 @@
+"""def-position-revealed-direction scenario pack — full no-cheat
+validation on Rust.
+
+REASONING capability — INTEL-DRIVEN ADAPTIVE DEFENSE. The threat axis
+is HIDDEN from the brief (sibling `def-position-expected-direction`
+DECLARES the axis; this pack does not). The agent must FIRST drive its
+scout jeeps to register the enemy's forward outpost on the actual
+approach lane, THEN commit a 3-pbox triple between its base and that
+outpost, THEN engage and break the incoming rush. The win predicate is
+the Wave-2 `then:` happened-before composite:
+
+    then:
+      - {buildings_discovered_gte: 1}         # SCOUT first (axis intel)
+      - {building_count_gte: {type: pbox, n: 3}}  # FORTIFY second
+      - {units_killed_gte: 3}                 # ENGAGE third
+
+paired with `building_count_gte:{fact, 1}` (the home fact still stands)
+and `within_ticks: 5400` (a reachable deadline given max_turns=66, which
+caps ticks at 93+90·65=5943). Fail is `after_ticks: 5401` OR
+`not building_count_gte:{fact,1}` OR `not own_units_gte:1` — a non-win
+is always a real reachable LOSS, never a draw.
+
+`building_count_gte:{fact,1}` is used in place of the more common
+`has_building: fact` because the latter checks
+`own_building_types` which ACCUMULATES across the episode (once
+the agent ever owned a fact the set still contains "fact" after
+the fact is razed). The `building_count_gte` predicate reads the
+live `own_buildings` list each frame and correctly bites when
+the fact is destroyed — without this swap the stall policy on
+the hard NORTH spawn ended in DRAW (engine auto-`done` on player
+elimination before the deadline) rather than LOSS.
+
+These tests prove deterministically (no model / no network) that:
+
+* the intended scout-then-fortify-then-engage policy WINS every level
+  and every hard seed (1..4) — including BOTH the NORTH-base and the
+  SOUTH-base spawn groups on hard;
+* stall (observe-only) LOSES every level and every seed (real LOSS,
+  never draw);
+* no-scout-pre-fortify-NORTH (build 3 pbox at y=14 immediately and
+  never move the jeeps) WINS only on EASY (where the jeeps start at
+  y=8 already in sight of the outpost) and LOSES on every MEDIUM and
+  HARD seed (clause 1 of the then-chain never latches);
+* scout-then-fortify-WRONG-AXIS (memorised "always defend NORTH") LOSES
+  on every hard SOUTH-spawn seed — the south rush walks past the
+  wrong-axis pbox line and razes the SOUTH-spawn fact, AND the
+  buildings-discovered clause cannot latch on the SOUTH-spawn jeep
+  because the north outpost is out of sight;
+* the hard tier defines ≥2 spawn_point groups (NORTH base y=16 /
+  SOUTH base y=24) so the base latitude rotates by seed (no single-
+  cell memorised opening can generalise);
+* every level's `after_ticks` is reachable inside `max_turns` so a
+  non-finisher is a real timeout LOSS, never a draw.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+pytest.importorskip("openra_train", reason="Rust env wheel not installed")
+pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
+
+from openra_bench.eval_core import run_level
+from openra_bench.scenarios import load_pack
+from openra_bench.scenarios.loader import PACKS_DIR, compile_level
+
+PACK = PACKS_DIR / "def-position-revealed-direction.yaml"
+LEVELS = ("easy", "medium", "hard")
+SEEDS = (1, 2, 3, 4)
+
+
+# ── scripted policies ────────────────────────────────────────────────
+
+
+def stall(rs, C):
+    """Observe-only. Never scouts, never builds, never engages.
+    Reachable timeout LOSS (or fact-loss LOSS, which is the same
+    fail-clause family)."""
+    return [C.observe()]
+
+
+def make_intended():
+    """The intended capability play: identify own base latitude
+    (NORTH-of-mid → threat-from-NORTH; SOUTH-of-mid → threat-from-
+    SOUTH; mid → easy preset NORTH lane), DRIVE the jeeps to the
+    matching outpost band, BUILD 3 pbox between base and that band,
+    and let the on-axis pbox + base defenders rack up ≥3 kills on
+    the incoming rush."""
+
+    state = {"scout_dispatched": False, "queued": 0}
+
+    def policy(rs, C):
+        units = rs.get("units_summary") or []
+        own_b = rs.get("own_buildings") or []
+        fact = next(
+            (b for b in own_b if b.get("type") == "fact"), None,
+        )
+        if fact is None:
+            return [C.observe()]
+        fx, fy = int(fact["cell_x"]), int(fact["cell_y"])
+        # Pick the threat side from base latitude. Hard: y=16 ⇒
+        # north lane (scout to y=8, pbox at y=10); y=24 ⇒ south
+        # lane (scout to y=32, pbox at y=30). Easy/medium: y=20 ⇒
+        # north preset.
+        if fy < 20:
+            scout_y = max(2, fy - 8)
+            pbox_y = max(4, fy - 6)
+        elif fy > 20:
+            scout_y = min(38, fy + 8)
+            pbox_y = min(34, fy + 6)
+        else:
+            scout_y = 8
+            pbox_y = 14
+
+        types = [b.get("type") for b in own_b]
+        pbox_count = sum(1 for t in types if t == "pbox")
+
+        cmds = []
+        jeeps = [u for u in units if u.get("type") == "jeep"]
+        if jeeps and not state["scout_dispatched"]:
+            cmds.append(
+                C.move_units([j["id"] for j in jeeps], fx, scout_y),
+            )
+            state["scout_dispatched"] = True
+        # Queue the 3 pbox up front; one StartProduction per turn
+        # (the Defense queue is one-at-a-time, so queueing more
+        # than 3 wastes only the StartProduction order).
+        if state["queued"] < 3:
+            cmds.append(C.build("pbox"))
+            state["queued"] += 1
+        # Always emit the NEXT slot's place_building — the engine
+        # buffers blocked PLACEs until production finishes, then
+        # places at the requested cell.
+        if pbox_count < 3:
+            dx = -2 + 2 * pbox_count
+            cmds.append(C.place_building("pbox", fx + dx, pbox_y))
+        if not cmds:
+            cmds.append(C.observe())
+        return cmds
+
+    return policy
+
+
+def make_no_scout_north_pbox():
+    """Pre-fortify NORTH-lane WITHOUT ever moving the jeeps. On EASY
+    this WINS (the jeeps are pre-staged at y=8, passively in sight of
+    the outpost at y=2 — clause 1 latches at turn 1 even without a
+    move command). On MEDIUM and HARD it LOSES every seed — the
+    jeeps sit at the base, outside outpost vision, so the then-chain
+    NEVER reaches clause 1, and the deadline expires."""
+
+    state = {"queued": 0}
+
+    def policy(rs, C):
+        own_b = rs.get("own_buildings") or []
+        fact = next(
+            (b for b in own_b if b.get("type") == "fact"), None,
+        )
+        if fact is None:
+            return [C.observe()]
+        fx = int(fact["cell_x"])
+        # Memorised: always build pbox at y=14 (the easy/medium
+        # canonical NORTH lane). NEVER moves the jeeps.
+        types = [b.get("type") for b in own_b]
+        pbox_count = sum(1 for t in types if t == "pbox")
+        cmds = []
+        if state["queued"] < 3:
+            cmds.append(C.build("pbox"))
+            state["queued"] += 1
+        if pbox_count < 3:
+            dx = -2 + 2 * pbox_count
+            cmds.append(C.place_building("pbox", fx + dx, 14))
+        if not cmds:
+            cmds.append(C.observe())
+        return cmds
+
+    return policy
+
+
+def make_scout_then_fortify_always_north():
+    """Drives the scout NORTH and builds pbox on the NORTH lane —
+    the 'memorised: always defend NORTH' policy. On HARD SOUTH-spawn
+    seeds the north scout never reaches the south outpost (the
+    matching threat), so clause 1 stays unlatched AND the south rush
+    razes the SOUTH-spawn fact (the y=18 pbox line on the SOUTH
+    spawn is on the wrong axis). LOSS on every hard SOUTH-spawn
+    seed; WIN on every hard NORTH-spawn seed (where it accidentally
+    happens to be the correct lane)."""
+
+    state = {"scout_dispatched": False, "queued": 0}
+
+    def policy(rs, C):
+        units = rs.get("units_summary") or []
+        own_b = rs.get("own_buildings") or []
+        fact = next(
+            (b for b in own_b if b.get("type") == "fact"), None,
+        )
+        if fact is None:
+            return [C.observe()]
+        fx, fy = int(fact["cell_x"]), int(fact["cell_y"])
+        # ALWAYS scout NORTH (whatever the base latitude); ALWAYS
+        # build pbox on the NORTH side of base.
+        scout_y = max(2, fy - 8)
+        pbox_y = max(4, fy - 6)
+
+        types = [b.get("type") for b in own_b]
+        pbox_count = sum(1 for t in types if t == "pbox")
+        cmds = []
+        jeeps = [u for u in units if u.get("type") == "jeep"]
+        if jeeps and not state["scout_dispatched"]:
+            cmds.append(
+                C.move_units([j["id"] for j in jeeps], fx, scout_y),
+            )
+            state["scout_dispatched"] = True
+        if state["queued"] < 3:
+            cmds.append(C.build("pbox"))
+            state["queued"] += 1
+        if pbox_count < 3:
+            dx = -2 + 2 * pbox_count
+            cmds.append(C.place_building("pbox", fx + dx, pbox_y))
+        if not cmds:
+            cmds.append(C.observe())
+        return cmds
+
+    return policy
+
+
+# ── scenario-shape invariants ────────────────────────────────────────
+
+
+def test_pack_compiles_with_three_levels_and_rusher_bot():
+    pack = load_pack(PACK)
+    assert pack.meta.id == "def-position-revealed-direction"
+    assert pack.meta.capability == "reasoning"
+    assert set(pack.levels) == {"easy", "medium", "hard"}
+    # Required-by-spec benchmark anchors (per the seed taxonomy).
+    anchors = pack.meta.benchmark_anchor or []
+    assert any("PlanBench" in a for a in anchors), anchors
+    assert any("military" in a.lower() for a in anchors), anchors
+    assert any("intel" in a.lower() for a in anchors), anchors
+    # Rusher bot wired through to the engine for every level.
+    for lvl in LEVELS:
+        c = compile_level(pack, lvl)
+        assert c.map_supported
+        enemy = c.scenario.enemy
+        bot = (
+            getattr(enemy, "bot_type", None)
+            or getattr(enemy, "bot", None)
+        )
+        assert str(bot).lower() == "rusher", (lvl, bot)
+
+
+@pytest.mark.parametrize("level", LEVELS)
+def test_every_level_has_a_reachable_timeout_fail(level):
+    """Non-win must be a real LOSS: the `after_ticks` fail clause must
+    be strictly below the tick reachable at max_turns (≤90 ticks per
+    step in interrupt mode)."""
+    c = compile_level(load_pack(PACK), level)
+    assert c.fail_condition is not None
+    fc = c.fail_condition.model_dump(exclude_none=True)
+    deadline = None
+    for clause in fc.get("any_of", []) or []:
+        if "after_ticks" in clause:
+            deadline = int(clause["after_ticks"])
+    assert deadline is not None, f"{level}: no after_ticks fail clause"
+    reachable = 93 + 90 * (c.max_turns - 1)
+    assert deadline < reachable, (
+        f"{level}: deadline {deadline} unreachable within "
+        f"{c.max_turns} turns (max tick {reachable}) → draw degeneracy"
+    )
+
+
+@pytest.mark.parametrize("level", LEVELS)
+def test_every_level_uses_then_chain_with_scout_fortify_engage(level):
+    """The 3-phase scout→fortify→engage chain MUST be wired through
+    as a Wave-2 `then:` happened-before composite (the whole point of
+    the pack — ordered intel-driven defense, not a degenerate
+    permutation-tolerant `all_of`)."""
+    c = compile_level(load_pack(PACK), level)
+    win = c.win_condition.model_dump(exclude_none=True)
+    ao = win.get("all_of") or []
+    then_branches = [b for b in ao if "then" in b]
+    assert len(then_branches) == 1, (
+        f"{level}: expected exactly one then-chain in win; got {win}"
+    )
+    clauses = then_branches[0]["then"]["clauses"]
+    assert len(clauses) == 3, (
+        f"{level}: then-chain must be 3 clauses (scout, fortify, "
+        f"engage); got {len(clauses)}: {clauses}"
+    )
+    # Clause 1: scout latch — ≥1 enemy building discovered (the
+    # forward outpost the agent's jeep MUST reach by movement on
+    # medium/hard; passively visible on easy).
+    assert clauses[0].get("buildings_discovered_gte") == 1, clauses[0]
+    # Clause 2: fortify — exactly 3 pbox (not a token pbox).
+    bcg = clauses[1].get("building_count_gte") or {}
+    assert bcg.get("type") == "pbox" and int(bcg.get("n", 0)) == 3, (
+        clauses[1]
+    )
+    # Clause 3: engage — at least 3 kills.
+    assert clauses[2].get("units_killed_gte") == 3, clauses[2]
+
+
+@pytest.mark.parametrize("level", LEVELS)
+def test_win_predicate_uses_building_count_gte_for_fact_persistence(
+    level,
+):
+    """The fact-persistence clause MUST be `building_count_gte:
+    {type:fact, n:1}` (live count) and NOT `has_building:fact`
+    (accumulating set). See module docstring for the footgun."""
+    c = compile_level(load_pack(PACK), level)
+    win = c.win_condition.model_dump(exclude_none=True)
+    flat = str(win)
+    assert "has_building" not in flat, (
+        f"{level}: win must avoid the accumulating `has_building` "
+        f"footgun; use `building_count_gte:{{type:fact,n:1}}` "
+        f"instead. win={win}"
+    )
+    # And the live count clause IS present:
+    ao = win.get("all_of") or []
+    found = False
+    for clause in ao:
+        bcg = (
+            clause.get("building_count_gte")
+            if isinstance(clause, dict) else None
+        )
+        if isinstance(bcg, dict) and bcg.get("type") == "fact":
+            assert int(bcg.get("n", 0)) >= 1
+            found = True
+    assert found, f"{level}: missing fact-alive clause; got {win}"
+
+
+def test_hard_has_two_spawn_point_groups():
+    """Hard-tier contract: ≥2 distinct seed-driven spawn_point groups
+    so the threat axis rotates by seed (anti-memorisation). Both
+    bands of enemy outposts always place (CLAUDE.md: enemy actors
+    don't honour spawn_point) so the SAME scout-then-fortify
+    discipline is tested from a flipped base latitude per seed."""
+    c = compile_level(load_pack(PACK), "hard")
+    groups = {
+        a.spawn_point for a in c.scenario.actors
+        if a.owner == "agent" and a.spawn_point is not None
+    }
+    assert groups == {0, 1}, groups
+    # In-bounds check (rush-hour-arena playable y ≈ 2..38).
+    for a in c.scenario.actors:
+        x, y = a.position
+        assert 2 <= x <= 126 and 2 <= y <= 38, (a.type, a.position)
+
+
+def test_tools_list_matches_spec():
+    pack = load_pack(PACK)
+    expected = {
+        "observe", "build", "place_building", "move_units",
+        "attack_unit", "attack_move", "stop",
+    }
+    tools = (
+        pack.base.get("tools")
+        if isinstance(pack.base, dict)
+        else pack.base.tools
+    )
+    assert set(tools) == expected, tools
+
+
+# ── solvency: intended WINS every level + every hard seed ────────────
+
+
+@pytest.mark.parametrize("seed", SEEDS)
+@pytest.mark.parametrize("level", LEVELS)
+def test_intended_scout_fortify_engage_wins_every_level_and_seed(
+    level, seed,
+):
+    """The intended capability play (identify base latitude → scout
+    matching outpost → fortify matching lane → engage) MUST WIN on
+    every (level, seed). This is the load-bearing solvency test."""
+    c = compile_level(load_pack(PACK), level)
+    r = run_level(c, make_intended(), seed=seed)
+    assert r.outcome == "win", (
+        f"{level} seed{seed}: intended scout-fortify-engage must WIN; "
+        f"got {r.outcome} (tick={r.signals.game_tick}, "
+        f"kills={r.signals.units_killed}, "
+        f"then_progress={getattr(r.signals, 'then_progress', {})}, "
+        f"buildings_seen="
+        f"{len(r.signals.enemy_buildings_seen_ids)})"
+    )
+
+
+# ── no-cheat: every wrong / lazy policy LOSES (real LOSS, not draw) ──
+
+
+@pytest.mark.parametrize("seed", SEEDS)
+@pytest.mark.parametrize("level", LEVELS)
+def test_stall_loses_every_level_and_seed(level, seed):
+    """Stall (observe-only) MUST LOSE on every (level, seed). The
+    fail clause family (after_ticks: 5401 OR
+    not building_count_gte:{fact,1} OR not own_units_gte:1) bites
+    deterministically — never a draw."""
+    c = compile_level(load_pack(PACK), level)
+    r = run_level(c, stall, seed=seed)
+    assert r.outcome == "loss", (
+        f"{level} seed{seed}: stall must LOSE (real fail, not a "
+        f"draw); got {r.outcome} (tick={r.signals.game_tick})"
+    )
+
+
+@pytest.mark.parametrize("seed", SEEDS)
+def test_no_scout_pre_fortify_loses_on_medium(seed):
+    """Build 3 pbox at the canonical NORTH lane without ever moving
+    the jeeps. The jeeps sit at the base, outside outpost vision —
+    so clause 1 (`buildings_discovered_gte:1`) NEVER latches and the
+    deadline expires. (On EASY the jeeps START forward at y=8 and
+    passively see the outpost from turn 1, so this play wins easy —
+    that's by design, easy explicitly teaches the chain without
+    requiring scout movement.)"""
+    c = compile_level(load_pack(PACK), "medium")
+    r = run_level(c, make_no_scout_north_pbox(), seed=seed)
+    assert r.outcome == "loss", (
+        f"medium s={seed}: no-scout pre-fortify must LOSE; got "
+        f"{r.outcome} (tick={r.signals.game_tick}, "
+        f"then_progress={getattr(r.signals, 'then_progress', {})})"
+    )
+
+
+@pytest.mark.parametrize("seed", SEEDS)
+def test_no_scout_pre_fortify_loses_on_hard(seed):
+    """Same no-scout pre-fortify play on hard. Loses for two
+    independent reasons depending on spawn: (i) on NORTH spawn the
+    jeeps at y=16 can't see the y=4 outpost without moving — clause
+    1 never latches; (ii) on SOUTH spawn the pbox at y=14 is on the
+    WRONG axis (south rush comes from y=38), the south rush razes
+    the south-spawn fact AND clause 1 never latches."""
+    c = compile_level(load_pack(PACK), "hard")
+    r = run_level(c, make_no_scout_north_pbox(), seed=seed)
+    assert r.outcome == "loss", (
+        f"hard s={seed}: no-scout pre-fortify must LOSE; got "
+        f"{r.outcome} (tick={r.signals.game_tick}, "
+        f"then_progress={getattr(r.signals, 'then_progress', {})})"
+    )
+
+
+def test_scout_then_always_north_fortify_loses_on_hard_south_spawn():
+    """The memorised 'always scout/fortify NORTH' policy LOSES on
+    every hard SOUTH-spawn seed (s=1, s=3 in the engine's spawn_idx
+    round-robin): the north scout doesn't reach the south outpost
+    (clause 1 never latches) AND the south-spawn fact at y=24 is
+    razed by the y=38 south rush which the NORTH-side pbox line at
+    y=18 can't intercept. Demonstrates the spawn-variation
+    discrimination — a fixed-axis opening cannot generalise across
+    seeds."""
+    c = compile_level(load_pack(PACK), "hard")
+    # Verified empirically: SOUTH spawn corresponds to seeds 1 & 3
+    # in the engine's spawn_idx round-robin; NORTH to 2 & 4.
+    south_spawn_seeds = (1, 3)
+    for seed in south_spawn_seeds:
+        r = run_level(
+            c, make_scout_then_fortify_always_north(), seed=seed,
+        )
+        assert r.outcome == "loss", (
+            f"hard SOUTH-spawn s={seed}: scout-and-fortify-always-"
+            f"NORTH must LOSE; got {r.outcome} "
+            f"(tick={r.signals.game_tick}, "
+            f"then_progress={getattr(r.signals, 'then_progress', {})})"
+        )
+
+
+# ── determinism ──────────────────────────────────────────────────────
+
+
+def test_intended_run_is_deterministic_on_easy():
+    c = compile_level(load_pack(PACK), "easy")
+    a = run_level(c, make_intended(), seed=3)
+    b = run_level(c, make_intended(), seed=3)
+    assert (a.outcome, a.turns) == (b.outcome, b.turns), (
+        "same seed must be deterministic"
+    )