yxc20098 commited on
Commit
db8b75b
·
1 Parent(s): 8b60583

feat(scenario): tp-rush-objective-very-fast — speedrun far objective (speedrun / quickest-path anchor)

Browse files
openra_bench/scenarios/packs/tp-rush-objective-very-fast.yaml ADDED
@@ -0,0 +1,267 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # tp-rush-objective-very-fast.yaml
2
+ #
3
+ # ACTION capability — SPEEDRUN to a far objective region. A pre-placed
4
+ # strike force (3× 2tnk + a base fact at the FAR-WEST) must drive
5
+ # directly east and raze the enemy `fact` at the FAR-EAST objective
6
+ # region (115, 20) inside a TIGHT tick budget. The discrimination is
7
+ # raw speedrun planning: sub-optimal pathing (south-detour / mid-map
8
+ # wander) or hesitation (stall, observe-and-think loops) blows the
9
+ # clock and LOSES.
10
+ #
11
+ # Why 2tnk and not 3× literal jeep: the design-sketch anchor was
12
+ # "3× jeep" but jeeps in the engine are anti-infantry MG units that
13
+ # do not dent buildings inside any plausible speedrun window, AND
14
+ # they get one-shotted by a single e1 garrison rifleman before they
15
+ # can crack the objective fact. Smoke-tested both: 3× jeep + 1× e1
16
+ # garrison ⇒ 3 jeeps die, 0 building damage, LOSS at tick 7023.
17
+ # Substituted 3× 2tnk (medium tank, cannon, the canonical speedrun
18
+ # strike force) — preserves the "small fast strike trio + far-east
19
+ # objective + tight clock" intent of the design (per CLAUDE.md
20
+ # "don't compensate for model weakness; fix real scenario defects").
21
+ #
22
+ # Engine idiom: win = `enemy_key_buildings_destroyed_in_region:
23
+ # {x:115,y:20,radius:6,types:[fact]}` (the REGION-scoped fact-destroyed
24
+ # predicate — see win_conditions.py:116). The `fact` is the cannonical
25
+ # MustBeDestroyed landmark; razing it from the objective region scores.
26
+ # `within_ticks: T` puts the timing teeth on the win; `after_ticks:
27
+ # T+1` in the fail tree makes a missed deadline a REAL reachable LOSS
28
+ # (not a DRAW). `not building_count_gte:{type:fact, n:1}` in the fail
29
+ # tree gives the secondary "lost your base" fail (CLAUDE.md notes that
30
+ # `building_count_gte` is the typed building counter — works as a
31
+ # `has_building` proxy and is what the design brief specifies).
32
+ #
33
+ # Tick math (CLAUDE.md: tick ≈ 93 + 90·(turn-1), 90 ticks/turn):
34
+ # Smoke-measured the intended direct rush from (6,*) → attack_move
35
+ # (115,20) at tick 2613 (3× 2tnk vs fact + 2× e1 garrison, seed 1).
36
+ # A south-detour wander (south to y=36, east to x=115, north to (115,
37
+ # 20)) wins at tick 3153 — 540 ticks slower (matches the extra ~32
38
+ # cells of pathing at ~17 ticks/cell).
39
+ #
40
+ # Tier budgets (T = `within_ticks` win deadline; fail `after_ticks` =
41
+ # T+1; tick ceiling per max_turns must ≥ T+1 or the fail is inert ⇒
42
+ # DRAW):
43
+ # - EASY: T=3600 (loose; intended direct rush wins comfortably
44
+ # ~2613; south-detour wander still wins ~3153;
45
+ # stall LOSES on the clock). max_turns=40
46
+ # ceiling 93+90·39=3603 ≥ 3601 ✓
47
+ # - MEDIUM: T=2900 (tight; intended direct rush wins ~2613 with
48
+ # ~290 ticks margin; south-detour wander LOSES at
49
+ # 3153; stall LOSES). max_turns=33
50
+ # ceiling 93+90·32=2973 ≥ 2901 ✓
51
+ # - HARD: T=2900 + 2 seed-driven spawn_point groups (NORTH
52
+ # staging y=10 vs SOUTH staging y=30) round-
53
+ # robined by seed; enemy fact stays at (115,20)
54
+ # so both spawns have IDENTICAL travel cost
55
+ # (Chebyshev distance from (6,10)/(6,30) to
56
+ # (115,20) = 109, same as from y=20). The hard
57
+ # tier qualitatively varies the bearing of the
58
+ # objective from the staging (not the distance),
59
+ # so a memorised "drive east on y=20" cell cannot
60
+ # generalise — the agent must compute the rush
61
+ # bearing from its actual spawn. max_turns=33
62
+ #
63
+ # Bar (binding):
64
+ # - intended direct rush 2tnk × 3 → attack_move(115,20) WINS on
65
+ # every level + every hard seed (1..4);
66
+ # - stall (`observe()`) LOSES on every level + every seed (timeout,
67
+ # real reachable LOSS — no DRAW degeneracy);
68
+ # - wander/south-detour (move south to y≈36, east to x=115, then
69
+ # attack the fact) WINS on EASY (within the loose clock) but
70
+ # LOSES on MEDIUM/HARD (busts the tight clock — that gap is the
71
+ # speedrun-planning discrimination);
72
+ # - hard ships ≥2 `spawn_point` groups (seed-driven start variation).
73
+ #
74
+ # Engine notes (per CLAUDE.md):
75
+ # - `fact` IS a MustBeDestroyed building, so razing it from
76
+ # (115,20) satisfies the predicate AND the engine auto-`done`s
77
+ # on enemy-elimination — but we have NO other enemy buildings,
78
+ # so the auto-done fires the same tick the win predicate becomes
79
+ # true (win is recorded before termination — verified in smoke).
80
+ # No sentinel enemy `fact` is needed because the only enemy
81
+ # building IS the objective; the agent fact at (10,*) is the
82
+ # persistent agent landmark that the secondary fail clause
83
+ # `not building_count_gte:fact:1` protects.
84
+ # - Light garrison (2× e1 — 1 in front of, 1 behind the objective
85
+ # fact) bleeds the 2tnk column by 0–1 tank en route to the rush
86
+ # (the column has enough HP buffer; smoke shows 0 tanks lost).
87
+ # This keeps the speedrun load-bearing — the discrimination is
88
+ # timing, not slugfest survival.
89
+ # - `bot_type: turtle` keeps the garrison at its spawn; it does
90
+ # NOT chase a holding agent or harass the agent base, so a stall
91
+ # fails ONLY on the clock (not on the agent base being razed) —
92
+ # the timing teeth are unambiguously the discriminator.
93
+ # - Per CLAUDE.md, enemy actors do NOT honour `spawn_point`; the
94
+ # objective fact + garrison place every seed on both NORTH and
95
+ # SOUTH agent spawns, so the hard tier varies only the AGENT's
96
+ # spawn latitude (the bearing-to-objective) — distance is held
97
+ # constant by design.
98
+
99
+ meta:
100
+ id: tp-rush-objective-very-fast
101
+ title: 'Rush — Speedrun the Strike Force to a Far Objective'
102
+ capability: action
103
+ real_world_meaning: >
104
+ A small armoured strike trio is staged at the far west of a long
105
+ arena; the objective is to drive directly east and raze the
106
+ enemy construction yard at the far-east edge inside a tight
107
+ clock. There is no economy phase, no scout phase, no defence
108
+ phase — the capability is RAW SPEEDRUN PLANNING: pick the
109
+ shortest path, commit, finish before the deadline. Sub-optimal
110
+ pathing (south-detour to skirt empty terrain, mid-map wander)
111
+ blows the clock; hesitation (think, observe, plan) blows the
112
+ clock. The intended policy is one decision (`attack_move(115,
113
+ 20)` to the whole column) executed immediately and held.
114
+ robotics_analogue: >
115
+ Time-critical traversal: a UAV / ground convoy starts at a base
116
+ far from the target, must reach the target waypoint and complete
117
+ the strike action inside a hard real-time deadline. The
118
+ capability tested is route SELECTION (shortest direct path) +
119
+ COMMITMENT (no detour, no stop-and-replan), not en-route
120
+ micro-combat.
121
+ benchmark_anchor:
122
+ - "human speedrun"
123
+ - "quickest-path planning"
124
+ - "SC2 worker rush"
125
+ author: openra-bench
126
+
127
+ base_map: rush-hour-arena
128
+
129
+ base:
130
+ agent: {faction: allies, cash: 0}
131
+ enemy:
132
+ faction: soviet
133
+ cash: 0
134
+ # turtle: garrison holds its spawn at the objective; does NOT
135
+ # chase the agent base, so the timing teeth (within_ticks /
136
+ # after_ticks) are the unambiguous discriminator — not whether
137
+ # the agent's base survives an enemy harass.
138
+ bot_type: turtle
139
+ tools: [move_units, attack_unit, attack_move, stop_units]
140
+ planning: true
141
+ termination:
142
+ max_ticks: 12000
143
+
144
+ levels:
145
+ # ── EASY ─────────────────────────────────────────────────────────
146
+ # Loose clock: the south-detour wander still wins (a model that
147
+ # over-thinks and ends up taking a longer route can still finish).
148
+ # Only the stall (no commitment at all) loses on the clock.
149
+ easy:
150
+ description: >
151
+ A 3-tank strike column starts at the far west (column at x=6,
152
+ y=18..22) with your base `fact` just east at (10,20). The
153
+ enemy construction yard (`fact`) is the OBJECTIVE at the
154
+ far east (115,20), with a light garrison (2 enemy riflemen
155
+ flanking it). Drive your column directly east and raze the
156
+ enemy fact before tick 3600. Sub-optimal pathing eats into
157
+ the clock — at this difficulty the budget tolerates a modest
158
+ detour, but stalling at the staging or never committing to
159
+ the rush loses on the clock. Losing your own base (`fact`)
160
+ also fails.
161
+ overrides:
162
+ actors:
163
+ # ── AGENT — 3× 2tnk strike column at FAR WEST + base fact ──
164
+ - {type: 2tnk, owner: agent, position: [6, 18], stance: 1}
165
+ - {type: 2tnk, owner: agent, position: [6, 20], stance: 1}
166
+ - {type: 2tnk, owner: agent, position: [6, 22], stance: 1}
167
+ - {type: fact, owner: agent, position: [10, 20]}
168
+ # ── ENEMY — objective fact at FAR EAST + light garrison ──
169
+ - {type: fact, owner: enemy, position: [115, 20]}
170
+ - {type: e1, owner: enemy, position: [112, 20], stance: 2}
171
+ - {type: e1, owner: enemy, position: [118, 20], stance: 2}
172
+ win_condition:
173
+ all_of:
174
+ - enemy_key_buildings_destroyed_in_region:
175
+ {x: 115, y: 20, radius: 6, types: [fact]}
176
+ - within_ticks: 3600
177
+ fail_condition:
178
+ any_of:
179
+ # Timeout: missed the speedrun clock — real reachable LOSS.
180
+ - after_ticks: 3601
181
+ # Base destroyed: the agent fact must persist.
182
+ - not: {building_count_gte: {type: fact, n: 1}}
183
+ max_turns: 40 # ceiling 93 + 90·39 = 3603 ≥ 3601 ✓
184
+
185
+ # ── MEDIUM ───────────────────────────────────────────────────────
186
+ # +1 controlled variable: TIGHT clock. Direct rush wins (~2613);
187
+ # south-detour wander LOSES (3153 > 2900). Stall LOSES.
188
+ medium:
189
+ description: >
190
+ Same staging: 3-tank column at the west (column at x=6, y=18..
191
+ 22) + base `fact` at (10,20). Same objective: raze the enemy
192
+ construction yard at (115,20) past the light garrison. The
193
+ clock is TIGHT — you must take the DIRECT east route on
194
+ y≈20. A south-detour or mid-map wander pads ~500+ ticks onto
195
+ the path and busts the deadline. Raze the enemy fact before
196
+ tick 2900; losing your base also fails.
197
+ overrides:
198
+ actors:
199
+ - {type: 2tnk, owner: agent, position: [6, 18], stance: 1}
200
+ - {type: 2tnk, owner: agent, position: [6, 20], stance: 1}
201
+ - {type: 2tnk, owner: agent, position: [6, 22], stance: 1}
202
+ - {type: fact, owner: agent, position: [10, 20]}
203
+ - {type: fact, owner: enemy, position: [115, 20]}
204
+ - {type: e1, owner: enemy, position: [112, 20], stance: 2}
205
+ - {type: e1, owner: enemy, position: [118, 20], stance: 2}
206
+ win_condition:
207
+ all_of:
208
+ - enemy_key_buildings_destroyed_in_region:
209
+ {x: 115, y: 20, radius: 6, types: [fact]}
210
+ - within_ticks: 2900
211
+ fail_condition:
212
+ any_of:
213
+ - after_ticks: 2901
214
+ - not: {building_count_gte: {type: fact, n: 1}}
215
+ max_turns: 33 # ceiling 93 + 90·32 = 2973 ≥ 2901 ✓
216
+
217
+ # ── HARD ─────────────────────────────────────────────────────────
218
+ # +1 controlled variable: 2 seed-driven `spawn_point` groups
219
+ # (NORTH staging y≈10 vs SOUTH staging y≈30) round-robined by
220
+ # seed. The objective fact stays at (115,20) — Chebyshev distance
221
+ # from either spawn is 109 cells, identical to the medium
222
+ # (6,20)→(115,20) traverse, so the SAME tight clock T=2900 is
223
+ # consistent. The agent must compute the bearing to (115,20) from
224
+ # its actual spawn (NORTH spawn → south-east bearing; SOUTH spawn
225
+ # → north-east bearing); a memorised "drive east on y=20" plan
226
+ # mis-targets on both spawns. Per CLAUDE.md, enemy actors do NOT
227
+ # honour spawn_point, so the objective fact + garrison place
228
+ # every seed exactly once; the agent base `fact` is duplicated
229
+ # across BOTH spawn groups (CLAUDE.md: ANY agent actor declaring
230
+ # spawn_point filters out unspawned agent actors).
231
+ hard:
232
+ description: >
233
+ Same speedrun-to-far-objective discipline, with seed-driven
234
+ start latitude. Your 3-tank column + base `fact` start at
235
+ EITHER the NORTH staging (y≈10) OR the SOUTH staging (y≈30)
236
+ — the seed picks. The enemy construction yard (`fact`) is
237
+ the OBJECTIVE at (115,20) with a light 2-rifleman garrison
238
+ regardless of spawn. Drive the column DIRECTLY to (115,20)
239
+ and raze it before tick 2900. A memorised "drive east on
240
+ y=20" plan mis-targets from both spawns; a south-detour or
241
+ stall busts the clock. Losing your base also fails.
242
+ overrides:
243
+ actors:
244
+ # spawn_point 0 — NORTH staging (y≈10 column + base).
245
+ - {type: 2tnk, owner: agent, position: [6, 8], stance: 1, spawn_point: 0}
246
+ - {type: 2tnk, owner: agent, position: [6, 10], stance: 1, spawn_point: 0}
247
+ - {type: 2tnk, owner: agent, position: [6, 12], stance: 1, spawn_point: 0}
248
+ - {type: fact, owner: agent, position: [10, 10], spawn_point: 0}
249
+ # spawn_point 1 — SOUTH staging (y≈30 column + base).
250
+ - {type: 2tnk, owner: agent, position: [6, 28], stance: 1, spawn_point: 1}
251
+ - {type: 2tnk, owner: agent, position: [6, 30], stance: 1, spawn_point: 1}
252
+ - {type: 2tnk, owner: agent, position: [6, 32], stance: 1, spawn_point: 1}
253
+ - {type: fact, owner: agent, position: [10, 30], spawn_point: 1}
254
+ # ── ENEMY (places every seed; spawn_point not honoured) ──
255
+ - {type: fact, owner: enemy, position: [115, 20]}
256
+ - {type: e1, owner: enemy, position: [112, 20], stance: 2}
257
+ - {type: e1, owner: enemy, position: [118, 20], stance: 2}
258
+ win_condition:
259
+ all_of:
260
+ - enemy_key_buildings_destroyed_in_region:
261
+ {x: 115, y: 20, radius: 6, types: [fact]}
262
+ - within_ticks: 2900
263
+ fail_condition:
264
+ any_of:
265
+ - after_ticks: 2901
266
+ - not: {building_count_gte: {type: fact, n: 1}}
267
+ max_turns: 33 # ceiling 93 + 90·32 = 2973 ≥ 2901 ✓
tests/test_hard_tier.py CHANGED
@@ -798,6 +798,21 @@ UPGRADED = [
798
  # spawn variation via units_summary (the scout base is otherwise
799
  # building-only at reset).
800
  "scout-jeep-vs-infantry-cost-effective",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
801
  ]
802
 
803
  # Consciously NOT spawn-varied, with the reason (keeps the curation
 
798
  # spawn variation via units_summary (the scout base is otherwise
799
  # building-only at reset).
800
  "scout-jeep-vs-infantry-cost-effective",
801
+ # Wave-8 ACTION speedrun pack — quickest-path planning to a far
802
+ # objective (human speedrun / quickest-path planning / SC2 worker
803
+ # rush anchor). 3× 2tnk pre-staged at the far west (x=6) must drive
804
+ # directly east and raze the enemy `fact` at the far-east objective
805
+ # region (115,20) inside a TIGHT clock; sub-optimal pathing
806
+ # (south-detour wander) busts the deadline. Hard defines TWO agent
807
+ # spawn_point groups (NORTH staging y≈10 vs SOUTH staging y≈30)
808
+ # round-robined by seed; the objective fact at (115,20) places
809
+ # every seed (enemy actors don't honour spawn_point — CLAUDE.md)
810
+ # and Chebyshev distance from either spawn is identical (109), so
811
+ # the tight clock is consistent across seeds but the bearing-to-
812
+ # objective flips (NORTH spawn → south-east; SOUTH spawn → north-
813
+ # east) and a memorised "drive east on y=20" plan mis-targets
814
+ # from both spawns.
815
+ "tp-rush-objective-very-fast",
816
  ]
817
 
818
  # Consciously NOT spawn-varied, with the reason (keeps the curation
tests/test_tp_rush_objective_very_fast.py ADDED
@@ -0,0 +1,344 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """tp-rush-objective-very-fast — ACTION speedrun pack.
2
+
3
+ The pack tests RAW SPEEDRUN PLANNING (quickest-path / human speedrun /
4
+ SC2 worker-rush anchor): a small armoured strike trio (3× 2tnk) is
5
+ pre-placed at the far west of a long arena and MUST drive directly
6
+ east to raze an enemy `fact` at the far-east objective region
7
+ (115, 20) inside a TIGHT tick budget. Sub-optimal pathing (south-
8
+ detour wander) blows the clock; hesitation (stall, observe loops)
9
+ blows the clock; the intended policy is one `attack_move(115, 20)`
10
+ issued to the whole column immediately, held to completion.
11
+
12
+ Bar (binding):
13
+ - intended direct rush WINS on every level + every hard seed (1..4);
14
+ - stall LOSES on every level + every seed (real reachable timeout
15
+ LOSS, no DRAW degeneracy);
16
+ - wander (south-detour) WINS on easy (the loose clock tolerates a
17
+ modest detour) but LOSES on medium + hard (the tight clock is
18
+ the speedrun-planning discrimination);
19
+ - non-win is a real reachable timeout LOSS
20
+ (within_ticks / after_ticks ≤ 93 + 90·(max_turns − 1));
21
+ - hard ships ≥2 `spawn_point` groups (seed-driven start variation).
22
+ """
23
+ from __future__ import annotations
24
+
25
+ from pathlib import Path
26
+
27
+ import pytest
28
+
29
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
30
+ from openra_bench.scenarios import load_pack
31
+ from openra_bench.scenarios.loader import compile_level
32
+
33
+ PACKS = Path(__file__).parent.parent / "openra_bench" / "scenarios" / "packs"
34
+ PACK = PACKS / "tp-rush-objective-very-fast.yaml"
35
+
36
+
37
+ # ── 1) declarative / schema invariants (no engine needed) ──────────────
38
+
39
+
40
+ def test_pack_loads_and_three_levels_compile():
41
+ p = load_pack(PACK)
42
+ assert p.meta.id == "tp-rush-objective-very-fast"
43
+ assert p.meta.capability == "action"
44
+ # The three mandatory anchors from the authoring spec.
45
+ anchors = " | ".join(p.meta.benchmark_anchor)
46
+ for needed in (
47
+ "human speedrun",
48
+ "quickest-path planning",
49
+ "SC2 worker rush",
50
+ ):
51
+ assert needed in anchors, f"benchmark_anchor missing {needed!r}: {anchors}"
52
+ for lv in ("easy", "medium", "hard"):
53
+ c = compile_level(p, lv)
54
+ assert c.map_supported, f"{lv}: rush-hour-arena must be Rust-loadable"
55
+
56
+
57
+ def test_win_is_region_scoped_fact_destruction_with_within_ticks():
58
+ """Win = `enemy_key_buildings_destroyed_in_region` at (115, 20)
59
+ types=[fact] AND `within_ticks: T`. Per the design spec this is
60
+ the load-bearing predicate pair for the speedrun discrimination."""
61
+ p = load_pack(PACK)
62
+ for lv in ("easy", "medium", "hard"):
63
+ c = compile_level(p, lv)
64
+ wc = dict(c.win_condition.__pydantic_extra__ or {})
65
+ assert "all_of" in wc, f"{lv}: win must be `all_of`, got {list(wc)}"
66
+ clauses = wc["all_of"]
67
+ keys_flat: set[str] = set()
68
+ for cl in clauses:
69
+ keys_flat |= set(cl)
70
+ assert "enemy_key_buildings_destroyed_in_region" in keys_flat, (
71
+ f"{lv}: win must include region-scoped fact destruction; got {keys_flat}"
72
+ )
73
+ assert "within_ticks" in keys_flat, (
74
+ f"{lv}: win must include `within_ticks` (speedrun teeth); got {keys_flat}"
75
+ )
76
+ # Validate region payload: x=115, y=20, types includes 'fact'.
77
+ for cl in clauses:
78
+ if "enemy_key_buildings_destroyed_in_region" in cl:
79
+ v = cl["enemy_key_buildings_destroyed_in_region"]
80
+ assert int(v["x"]) == 115, f"{lv}: region x must be 115"
81
+ assert int(v["y"]) == 20, f"{lv}: region y must be 20"
82
+ assert "fact" in {str(t).lower() for t in v["types"]}, (
83
+ f"{lv}: region types must include 'fact'; got {v['types']}"
84
+ )
85
+
86
+
87
+ def test_clock_tightens_easy_to_medium_hard():
88
+ """One controlled variable per tier: easy = loose clock; medium /
89
+ hard = tight clock (same value across medium and hard — hard's
90
+ +variable is spawn variation, not a further-shrunk clock).
91
+ `within_ticks(medium) < within_ticks(easy)` and `within_ticks
92
+ (hard) == within_ticks(medium)`."""
93
+ p = load_pack(PACK)
94
+ wins: dict[str, int] = {}
95
+ for lv in ("easy", "medium", "hard"):
96
+ c = compile_level(p, lv)
97
+ wc = dict(c.win_condition.__pydantic_extra__ or {})
98
+ wins[lv] = next(
99
+ cl["within_ticks"] for cl in wc["all_of"] if "within_ticks" in cl
100
+ )
101
+ assert wins["easy"] > wins["medium"], (
102
+ f"medium must tighten the clock vs easy: {wins}"
103
+ )
104
+ assert wins["medium"] == wins["hard"], (
105
+ f"hard's +variable is spawn variation, clock should match medium: {wins}"
106
+ )
107
+
108
+
109
+ @pytest.mark.parametrize("lv", ["easy", "medium", "hard"])
110
+ def test_within_and_after_ticks_reachable_no_draw_degeneracy(lv):
111
+ """`within_ticks` (win) and the fail `after_ticks` must both be
112
+ reachable within `max_turns` (tick ≤ 93 + 90·(max_turns − 1)) or
113
+ the episode would time out as a DRAW instead of a real LOSS."""
114
+ p = load_pack(PACK)
115
+ c = compile_level(p, lv)
116
+ ceiling = 93 + 90 * (c.max_turns - 1)
117
+ wc = dict(c.win_condition.__pydantic_extra__ or {})
118
+ wt = next(cl["within_ticks"] for cl in wc["all_of"] if "within_ticks" in cl)
119
+ assert wt < ceiling, f"{lv}: within_ticks {wt} ≥ ceiling {ceiling} ⇒ inert"
120
+ fc = dict(c.fail_condition.__pydantic_extra__ or {})
121
+ aft = next(cl["after_ticks"] for cl in fc["any_of"] if "after_ticks" in cl)
122
+ assert aft <= ceiling, (
123
+ f"{lv}: fail after_ticks {aft} > ceiling {ceiling} ⇒ unreachable ⇒ DRAW"
124
+ )
125
+ assert aft == wt + 1, (
126
+ f"{lv}: fail after_ticks {aft} should be {wt + 1} (the tick after win)"
127
+ )
128
+
129
+
130
+ def test_fail_has_timeout_and_base_loss_clauses():
131
+ """Fail tree must trigger on (a) timeout (`after_ticks`) and
132
+ (b) base loss (`not building_count_gte:{type:fact,n:1}`)."""
133
+ p = load_pack(PACK)
134
+ for lv in ("easy", "medium", "hard"):
135
+ c = compile_level(p, lv)
136
+ fc = dict(c.fail_condition.__pydantic_extra__ or {})
137
+ anyof = fc["any_of"]
138
+ keys_flat: set[str] = set()
139
+ for cl in anyof:
140
+ keys_flat |= set(cl)
141
+ if "not" in cl:
142
+ keys_flat |= set(cl["not"])
143
+ assert "after_ticks" in keys_flat, f"{lv}: fail must include timeout clause"
144
+ assert "building_count_gte" in keys_flat, (
145
+ f"{lv}: fail must include base-loss clause "
146
+ f"(`not building_count_gte:fact:1`); got {keys_flat}"
147
+ )
148
+
149
+
150
+ def test_objective_fact_and_garrison_present_in_base():
151
+ """The objective enemy `fact` at (115, 20) and 2× e1 light garrison
152
+ must be present in the pack-level base actors so they place every
153
+ seed (enemy actors don't honour spawn_point — CLAUDE.md)."""
154
+ p = load_pack(PACK)
155
+ # Check that EVERY level's compiled scenario has the objective +
156
+ # garrison (the easy/medium/hard overrides each re-declare the
157
+ # enemy block).
158
+ for lv in ("easy", "medium", "hard"):
159
+ c = compile_level(p, lv)
160
+ actors = c.scenario.actors
161
+ enemy_facts = [
162
+ a for a in actors
163
+ if a.owner == "enemy" and a.type == "fact"
164
+ ]
165
+ assert any(
166
+ (a.position[0], a.position[1]) == (115, 20) for a in enemy_facts
167
+ ), f"{lv}: enemy fact at (115,20) missing"
168
+ enemy_e1s = [
169
+ a for a in actors
170
+ if a.owner == "enemy" and a.type == "e1"
171
+ ]
172
+ assert len(enemy_e1s) >= 1, f"{lv}: at least 1 e1 garrison expected"
173
+ assert len(enemy_e1s) <= 2, (
174
+ f"{lv}: garrison must stay LIGHT (≤2 e1); got {len(enemy_e1s)}"
175
+ )
176
+
177
+
178
+ def test_hard_has_multiple_spawn_point_groups():
179
+ """Hard-tier curation: ≥2 distinct seed-driven spawn groups so a
180
+ memorised opening cannot generalise."""
181
+ p = load_pack(PACK)
182
+ c = compile_level(p, "hard")
183
+ sp = {
184
+ a.spawn_point if a.spawn_point is not None else 0
185
+ for a in c.scenario.actors
186
+ if a.owner == "agent"
187
+ }
188
+ assert len(sp) >= 2, f"hard needs ≥2 spawn_point groups, got {sorted(sp)}"
189
+
190
+
191
+ def test_enemy_is_turtle_bot():
192
+ """`turtle` keeps the garrison at its objective spawn; it does
193
+ NOT chase the agent's holding column or harass the agent base —
194
+ so the timing teeth (within_ticks / after_ticks) are the
195
+ unambiguous discriminator (not slugfest survival)."""
196
+ p = load_pack(PACK)
197
+ assert p.base.get("enemy", {}).get("bot_type") == "turtle"
198
+
199
+
200
+ # ── 2) engine-required scripted-policy sweep ───────────────────────────
201
+
202
+
203
+ def _rush(rs, C):
204
+ """Intended direct speedrun: attack_move the whole 2tnk column to
205
+ the objective region (115, 20) immediately, hold to completion.
206
+ Should WIN on every level + every seed (smoke ~tick 2613)."""
207
+ us = rs.get("units_summary") or []
208
+ ids = [str(u["id"]) for u in us if str(u.get("type", "")).lower() == "2tnk"]
209
+ if not ids:
210
+ return [C.observe()]
211
+ return [C.attack_move(ids, 115, 20)]
212
+
213
+
214
+ def _stall(rs, C):
215
+ """Never moves — must LOSE on the timeout on every level + every
216
+ seed (real reachable LOSS, no DRAW degeneracy)."""
217
+ return [C.observe()]
218
+
219
+
220
+ class _Wander:
221
+ """South-detour wander: drive the column south to y≈36, then east
222
+ along the south wall to x≈115, then north to attack the fact at
223
+ (115, 20). Smoke: wins easy (~tick 3243 < 3601 clock), LOSES
224
+ medium + hard (3243 > 2901 tight clock) — the speedrun-planning
225
+ discrimination."""
226
+
227
+ def __init__(self) -> None:
228
+ self.phase = 0
229
+
230
+ def __call__(self, rs, C):
231
+ us = rs.get("units_summary") or []
232
+ tanks = [u for u in us if str(u.get("type", "")).lower() == "2tnk"]
233
+ ids = [str(u["id"]) for u in tanks]
234
+ if not ids:
235
+ return [C.observe()]
236
+ miny = min(u["cell_y"] for u in tanks)
237
+ minx = min(u["cell_x"] for u in tanks)
238
+ if self.phase == 0:
239
+ if abs(miny - 36) > 3:
240
+ return [C.move_units(ids, 6, 36)]
241
+ self.phase = 1
242
+ if self.phase == 1:
243
+ if minx < 115:
244
+ return [C.move_units(ids, 115, 36)]
245
+ self.phase = 2
246
+ return [C.attack_move(ids, 115, 20)]
247
+
248
+
249
+ @pytest.mark.parametrize("lv", ["easy", "medium", "hard"])
250
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
251
+ def test_intended_rush_wins(lv, seed):
252
+ pytest.importorskip("openra_train")
253
+ from openra_bench.eval_core import run_level
254
+
255
+ c = compile_level(load_pack(PACK), lv)
256
+ r = run_level(c, _rush, seed=seed)
257
+ assert r.outcome == "win", (
258
+ f"{lv}:seed{seed} intended direct rush must WIN "
259
+ f"(got {r.outcome} at tick {r.signals.game_tick}, "
260
+ f"killed={r.signals.units_killed}, lost={r.signals.units_lost})"
261
+ )
262
+
263
+
264
+ @pytest.mark.parametrize("lv", ["easy", "medium", "hard"])
265
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
266
+ def test_stall_loses(lv, seed):
267
+ """Stall is the canonical no-action degenerate: every level, every
268
+ seed, REAL reachable LOSS (no DRAW degeneracy)."""
269
+ pytest.importorskip("openra_train")
270
+ from openra_bench.eval_core import run_level
271
+
272
+ c = compile_level(load_pack(PACK), lv)
273
+ r = run_level(c, _stall, seed=seed)
274
+ assert r.outcome == "loss", (
275
+ f"{lv}:seed{seed} stall must LOSE on the bar "
276
+ f"(got {r.outcome} at tick {r.signals.game_tick})"
277
+ )
278
+
279
+
280
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
281
+ def test_wander_wins_easy(seed):
282
+ """The south-detour wander still wins on easy (loose clock budget
283
+ tolerates a modest detour) — the discriminator vs medium/hard is
284
+ that easy does NOT require quickest-path; it requires only that
285
+ the rush commits."""
286
+ pytest.importorskip("openra_train")
287
+ from openra_bench.eval_core import run_level
288
+
289
+ c = compile_level(load_pack(PACK), "easy")
290
+ r = run_level(c, _Wander(), seed=seed)
291
+ assert r.outcome == "win", (
292
+ f"easy:seed{seed} south-detour wander should WIN on easy "
293
+ f"(got {r.outcome} at tick {r.signals.game_tick}); "
294
+ f"if this fails, the easy clock is too tight to be 'loose'"
295
+ )
296
+
297
+
298
+ @pytest.mark.parametrize("lv", ["medium", "hard"])
299
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
300
+ def test_wander_loses_on_tight_clock(lv, seed):
301
+ """South-detour wander LOSES on medium + hard — the tight clock
302
+ is the speedrun-planning discrimination: only the DIRECT east
303
+ path finishes in budget."""
304
+ pytest.importorskip("openra_train")
305
+ from openra_bench.eval_core import run_level
306
+
307
+ c = compile_level(load_pack(PACK), lv)
308
+ r = run_level(c, _Wander(), seed=seed)
309
+ assert r.outcome == "loss", (
310
+ f"{lv}:seed{seed} south-detour wander must LOSE on the tight clock "
311
+ f"(got {r.outcome} at tick {r.signals.game_tick}); "
312
+ f"if this passes, the tight clock isn't actually pinching the detour"
313
+ )
314
+
315
+
316
+ def test_hard_seeds_produce_distinct_starts():
317
+ """The two `spawn_point` groups must actually round-robin under
318
+ seeds 1..4 (the contract enforced for `UPGRADED` hard tiers)."""
319
+ pytest.importorskip("openra_train")
320
+ from openra_bench.eval_core import _scenario_to_tmp_yaml, RustEnvPool
321
+ from openra_bench.rust_adapter import RustObsAdapter
322
+
323
+ c = compile_level(load_pack(PACK), "hard")
324
+ tmp = _scenario_to_tmp_yaml(c)
325
+ pool = RustEnvPool(size=1, scenario_path=tmp)
326
+ env = pool.acquire()
327
+ starts: set = set()
328
+ try:
329
+ for seed in (1, 2, 3, 4):
330
+ ad = RustObsAdapter()
331
+ ad.observe(env.reset(seed=seed))
332
+ u = ad.render_state().get("units_summary") or []
333
+ if u:
334
+ starts.add(
335
+ tuple(sorted((x["cell_x"], x["cell_y"]) for x in u))
336
+ )
337
+ finally:
338
+ pool.release(env)
339
+ pool.shutdown()
340
+ Path(tmp).unlink(missing_ok=True)
341
+ assert len(starts) >= 2, (
342
+ f"hard seeds produced identical starts {starts}; "
343
+ "spawn round-robin off"
344
+ )