yxc20098 commited on
Commit
21036c2
·
1 Parent(s): 771d9fe

feat(scenario): build-defensive-tower-line — pbox-line choke defense (ERQA / MicroRTS anchor)

Browse files
openra_bench/scenarios/packs/build-defensive-tower-line.yaml ADDED
@@ -0,0 +1,258 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ meta:
2
+ id: build-defensive-tower-line
3
+ title: 'Build a Defensive Tower LINE Across the Choke (Not a Cluster, Not a Scatter)'
4
+ capability: reasoning
5
+ real_world_meaning: >
6
+ Where do you commit your defensive structures when the threat is
7
+ funnelled through a known corridor whose width matters? Military
8
+ perimeter doctrine and firewall rule design both say: spread the
9
+ coverage across the full width of the corridor so no enemy unit
10
+ can slip past on an unguarded row. A single dense cluster wastes
11
+ overlapping fire on one cell while the corridor edges stay open;
12
+ a scatter across the map leaves the corridor itself uncovered.
13
+ The win predicate makes the LINE topology load-bearing — total
14
+ pillbox count alone is not enough; ≥1 pillbox must sit on EACH of
15
+ the corridor's vertical rungs (the four narrow sub-regions that
16
+ span y=18..22 at the choke column).
17
+ robotics_analogue: >
18
+ Network firewall / Web Application Firewall rule placement: when
19
+ every connection MUST traverse a known ingress (the public WAN
20
+ edge / the only API gateway), the right architecture is one rule
21
+ per protocol/port across the FULL inspection surface, not three
22
+ duplicated rules on one port while the rest stay open. Likewise a
23
+ physical perimeter patrol covers the WHOLE corridor width — a
24
+ cluster at one waypoint or a scatter across unrelated nodes both
25
+ leave the actual approach lane traversable.
26
+ benchmark_anchor:
27
+ - "ERQA"
28
+ - "MicroRTS defense"
29
+ - "military perimeter"
30
+ author: openra-bench
31
+
32
+ # rush-hour-arena (128×40). The map has a narrow lane around y≈18..22
33
+ # at mid-map (x≈60); the `rusher` scripted bot charges the agent's
34
+ # centroid (agent fact on the west), so its path is forced THROUGH
35
+ # that lane on every seed. Agent base pre-placed at the west (fact +
36
+ # tent + powr) so the pbox queue is ready turn 1.
37
+ # Pack-level cash is overridden per-level (4/4/4 pbox at 600cr each;
38
+ # medium/hard get tighter constraints, not more pbox — see below).
39
+ # An unarmed high-HP enemy `fact` far east keeps the engine alive
40
+ # past rusher annihilation so the win/fail check actually runs.
41
+ # SISTER PACK: def-tower-line-vs-cluster inverts the topology bar to
42
+ # enforce CLUSTER (graph min-cut doctrine); this pack enforces LINE
43
+ # (corridor-width perimeter doctrine). The two together discriminate
44
+ # whether the model understands the FORCING GEOMETRY: a chokepoint
45
+ # (single cell on a wide approach → cluster) vs a corridor (full
46
+ # vertical width that any one row can leak through → line).
47
+ base_map: rush-hour-arena
48
+ starting_cash: 2400
49
+
50
+ base:
51
+ agent:
52
+ faction: allies
53
+ enemy:
54
+ faction: soviet
55
+ bot_type: rusher
56
+ tools:
57
+ - observe
58
+ - build
59
+ - place_building
60
+ - move_units
61
+ - attack_unit
62
+ - attack_move
63
+ - stop
64
+ planning: true
65
+ # No interrupts — perimeter design is a STATIC up-front decision
66
+ # (the corridor is known a priori, the rush composition is fixed).
67
+ # Dropping interrupts also makes the tick budget deterministic
68
+ # (each step is exactly 90 ticks ⇒ max_turns is a hard tick
69
+ # budget that the `after_ticks` fail clause reliably bites in).
70
+ termination:
71
+ max_ticks: 12000
72
+ actors:
73
+ # Pre-placed agent base on the WEST so rusher path is forced
74
+ # through the mid-map corridor on the way to the fact centroid.
75
+ - {type: fact, owner: agent, position: [10, 20]}
76
+ - {type: tent, owner: agent, position: [14, 18]}
77
+ - {type: powr, owner: agent, position: [14, 22]}
78
+
79
+ levels:
80
+ # ── EASY ── bare LINE skill. Budget covers exactly 4 pbox (2400cr).
81
+ # Win requires ≥1 pbox in EACH of the 4 corridor rungs (y=18,19,21,22
82
+ # at x=60, radius 0.5 — only the exact rung cell counts, so a
83
+ # and a cluster on y=20 misses ALL FOUR). The intended LINE places
84
+ # one pbox per rung (4 rungs × 1 pbox = budget exhausted). A cluster
85
+ # at (60,20) satisfies the count clause but FAILS each rung; a
86
+ # random scatter (e.g. 4 pbox near the base) misses every rung; a
87
+ # stall loses on the count clause AND the fact razed by the rush.
88
+ # max_turns 60 ⇒ reachable tick 93+90·59 = 5403; deadline 5400.
89
+ easy:
90
+ description: >
91
+ A rusher band must traverse the narrow corridor at x=60,
92
+ y=18..22 to reach your base on the west. Build 4 pillboxes
93
+ (pbox — 600cr each, budget exactly 2400) AND place ONE on each
94
+ of the four corridor rungs (at (60,18), (60,19),
95
+ (60,21), (60,22)) so the rusher cannot slip past on any row.
96
+ A cluster on the middle of the corridor satisfies the count
97
+ but FAILS every rung; a random scatter near the base fails
98
+ every rung; a pure-army layout (no pbox) fails the count and
99
+ lets the rush raze the fact. Your fact must survive.
100
+ starting_cash: 2400
101
+ overrides:
102
+ actors:
103
+ - {type: fact, owner: agent, position: [10, 20]}
104
+ - {type: tent, owner: agent, position: [14, 18]}
105
+ - {type: powr, owner: agent, position: [14, 22]}
106
+ # 4 mobile defenders pre-placed (light wave).
107
+ - {type: e1, owner: agent, position: [12, 19], stance: 2}
108
+ - {type: e1, owner: agent, position: [12, 21], stance: 2}
109
+ - {type: e1, owner: agent, position: [16, 19], stance: 2}
110
+ - {type: e1, owner: agent, position: [16, 21], stance: 2}
111
+ # Light rusher band at far east — forced through the corridor
112
+ # on the way to agent fact centroid (10, 20).
113
+ - {type: e1, owner: enemy, position: [100, 20], stance: 3, count: 4}
114
+ - {type: e3, owner: enemy, position: [102, 20], stance: 3, count: 1}
115
+ # Unarmed high-HP marker (anti-DRAW): keeps the episode alive
116
+ # past full rusher elimination so the win/fail check fires.
117
+ - {type: fact, owner: enemy, position: [120, 20]}
118
+ win_condition:
119
+ all_of:
120
+ - building_count_gte: {type: pbox, n: 4}
121
+ - building_in_region: {type: pbox, x: 60, y: 18, radius: 0.5, count: 1}
122
+ - building_in_region: {type: pbox, x: 60, y: 19, radius: 0.5, count: 1}
123
+ - building_in_region: {type: pbox, x: 60, y: 21, radius: 0.5, count: 1}
124
+ - building_in_region: {type: pbox, x: 60, y: 22, radius: 0.5, count: 1}
125
+ - building_count_gte: {type: fact, n: 1}
126
+ - within_ticks: 5400
127
+ fail_condition:
128
+ any_of:
129
+ - after_ticks: 5401
130
+ - not: {building_count_gte: {type: fact, n: 1}}
131
+ max_turns: 60
132
+
133
+ # ── MEDIUM ── +1 axis: HEAVY wave (rusher band ~doubled) with
134
+ # the SAME tight 4-pbox budget. The cluster, scatter, and pure-army
135
+ # layouts still all lose the same way as easy; the difference is
136
+ # that the LINE has to assemble FASTER before the heavier rush
137
+ # breaks through the pre-placed defenders. max_turns 60 ⇒
138
+ # reachable tick 5403; deadline 5400.
139
+ medium:
140
+ description: >
141
+ Same forced rusher corridor at x=60, y=18..22. Build 4 pillboxes
142
+ (budget 2400cr = exactly 4 pbox at 600 each) AND place ONE on
143
+ each of the four corridor rungs (at (60,18),
144
+ (60,19), (60,21), (60,22)). The rush wave is heavier than easy
145
+ — the LINE must assemble fast. A cluster, a scatter, and a
146
+ pure-army layout all lose; the fact must survive.
147
+ starting_cash: 2400
148
+ overrides:
149
+ actors:
150
+ - {type: fact, owner: agent, position: [10, 20]}
151
+ - {type: tent, owner: agent, position: [14, 18]}
152
+ - {type: powr, owner: agent, position: [14, 22]}
153
+ # 5 mobile base defenders + a 4-rifle corridor screen at the
154
+ # choke entrance (x=58) so the LINE has time to assemble
155
+ # behind the screen against the heavier wave. The screen
156
+ # WILL be ground down by the rush; only the LINE topology
157
+ # finishes the survivors before the fact dies.
158
+ - {type: e1, owner: agent, position: [12, 19], stance: 2}
159
+ - {type: e1, owner: agent, position: [12, 21], stance: 2}
160
+ - {type: e1, owner: agent, position: [16, 19], stance: 2}
161
+ - {type: e1, owner: agent, position: [16, 21], stance: 2}
162
+ - {type: e1, owner: agent, position: [14, 20], stance: 2}
163
+ - {type: e1, owner: agent, position: [58, 18], stance: 2}
164
+ - {type: e1, owner: agent, position: [58, 19], stance: 2}
165
+ - {type: e1, owner: agent, position: [58, 21], stance: 2}
166
+ - {type: e1, owner: agent, position: [58, 22], stance: 2}
167
+ # Heavier rusher band (one more e1 than easy, same e3
168
+ # grenadier). The tighter timing pressure — not more cash
169
+ # for more pbox — is what makes medium harder than easy.
170
+ # Band staged at x=115 (vs easy x=100) so the LINE has time
171
+ # to fully assemble behind the corridor screen before the
172
+ # heavier band engages it.
173
+ - {type: e1, owner: enemy, position: [115, 20], stance: 3, count: 5}
174
+ - {type: e3, owner: enemy, position: [117, 20], stance: 3, count: 1}
175
+ # Anti-DRAW marker.
176
+ - {type: fact, owner: enemy, position: [120, 20]}
177
+ win_condition:
178
+ all_of:
179
+ - building_count_gte: {type: pbox, n: 4}
180
+ - building_in_region: {type: pbox, x: 60, y: 18, radius: 0.5, count: 1}
181
+ - building_in_region: {type: pbox, x: 60, y: 19, radius: 0.5, count: 1}
182
+ - building_in_region: {type: pbox, x: 60, y: 21, radius: 0.5, count: 1}
183
+ - building_in_region: {type: pbox, x: 60, y: 22, radius: 0.5, count: 1}
184
+ - building_count_gte: {type: fact, n: 1}
185
+ - within_ticks: 5400
186
+ fail_condition:
187
+ any_of:
188
+ - after_ticks: 5401
189
+ - not: {building_count_gte: {type: fact, n: 1}}
190
+ max_turns: 60
191
+
192
+ # ── HARD ── +1 axis: TWO spawn_point groups so the agent base
193
+ # latitude flips by seed (NORTH (10,12) vs SOUTH (10,28)). The
194
+ # rusher band is symmetric across y=20 and ALWAYS places (enemy
195
+ # actors don't honour spawn_point — CLAUDE.md), so the corridor
196
+ # column at x=60 remains the choke for both seeds, but the rush
197
+ # geometry approaches each base from a different bearing. The
198
+ # corridor is still y=18..22 at x=60 (it's a fixed map feature) so
199
+ # the LINE topology is identical across seeds — what flips is the
200
+ # agent's interpretation of "which corridor" to defend (the NORTH
201
+ # spawn could be tempted to cover y=14..18, the SOUTH spawn to
202
+ # cover y=22..26; both are WRONG — the corridor itself is at
203
+ # y=18..22 regardless of base latitude). max_turns 70 ⇒ reachable
204
+ # tick 93+90·69 = 6303; deadline 6300.
205
+ hard:
206
+ description: >
207
+ Agent base latitude flips between NORTH (y=12) and SOUTH (y=28)
208
+ by seed. Build 4 pillboxes (budget 2400cr = exactly 4 pbox at
209
+ 600 each) AND place ONE on each of the four corridor rungs
210
+ (at (60,18), (60,19), (60,21), (60,22)). The
211
+ corridor at x=60 y=18..22 is a fixed map feature — covering
212
+ the rows next to your base instead (y=14..18 for NORTH, y=22..26
213
+ for SOUTH) FAILS the rung clauses. The fact must survive.
214
+ starting_cash: 2400
215
+ overrides:
216
+ actors:
217
+ # spawn_point 0 — NORTH base at y=12. Fact at (10, 12);
218
+ # tent/powr offset west so they aren't directly on the
219
+ # rusher path.
220
+ - {type: fact, owner: agent, position: [10, 12], spawn_point: 0}
221
+ - {type: tent, owner: agent, position: [6, 12], spawn_point: 0}
222
+ - {type: powr, owner: agent, position: [6, 14], spawn_point: 0}
223
+ - {type: e1, owner: agent, position: [12, 12], stance: 2, spawn_point: 0}
224
+ - {type: e1, owner: agent, position: [12, 13], stance: 2, spawn_point: 0}
225
+ - {type: e1, owner: agent, position: [12, 11], stance: 2, spawn_point: 0}
226
+ - {type: e1, owner: agent, position: [14, 12], stance: 2, spawn_point: 0}
227
+ - {type: e1, owner: agent, position: [8, 12], stance: 2, spawn_point: 0}
228
+ # spawn_point 1 — SOUTH base at y=28 (mirror across y=20).
229
+ - {type: fact, owner: agent, position: [10, 28], spawn_point: 1}
230
+ - {type: tent, owner: agent, position: [6, 28], spawn_point: 1}
231
+ - {type: powr, owner: agent, position: [6, 26], spawn_point: 1}
232
+ - {type: e1, owner: agent, position: [12, 28], stance: 2, spawn_point: 1}
233
+ - {type: e1, owner: agent, position: [12, 27], stance: 2, spawn_point: 1}
234
+ - {type: e1, owner: agent, position: [12, 29], stance: 2, spawn_point: 1}
235
+ - {type: e1, owner: agent, position: [14, 28], stance: 2, spawn_point: 1}
236
+ - {type: e1, owner: agent, position: [8, 28], stance: 2, spawn_point: 1}
237
+ # Enemies don't honour spawn_point (engine: oramap.rs).
238
+ # Symmetric rusher band at y=20 (mid-latitude); charges agent
239
+ # centroid so its path crosses the x=60 corridor on every
240
+ # seed regardless of which base latitude was picked.
241
+ - {type: e1, owner: enemy, position: [100, 20], stance: 3, count: 6}
242
+ - {type: e3, owner: enemy, position: [102, 20], stance: 3, count: 2}
243
+ # Anti-DRAW marker.
244
+ - {type: fact, owner: enemy, position: [120, 20]}
245
+ win_condition:
246
+ all_of:
247
+ - building_count_gte: {type: pbox, n: 4}
248
+ - building_in_region: {type: pbox, x: 60, y: 18, radius: 0.5, count: 1}
249
+ - building_in_region: {type: pbox, x: 60, y: 19, radius: 0.5, count: 1}
250
+ - building_in_region: {type: pbox, x: 60, y: 21, radius: 0.5, count: 1}
251
+ - building_in_region: {type: pbox, x: 60, y: 22, radius: 0.5, count: 1}
252
+ - building_count_gte: {type: fact, n: 1}
253
+ - within_ticks: 6300
254
+ fail_condition:
255
+ any_of:
256
+ - after_ticks: 6301
257
+ - not: {building_count_gte: {type: fact, n: 1}}
258
+ max_turns: 70
openra_bench/scenarios/packs/econ-harvester-pathing-optimization.yaml ADDED
@@ -0,0 +1,279 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ meta:
2
+ id: econ-harvester-pathing-optimization
3
+ title: Economy — Harvester Pathing Optimization (Split-Routing M/M/c)
4
+ capability: reasoning
5
+ real_world_meaning: >
6
+ Split-routing under heterogeneous round-trip cost. The agent owns
7
+ TWO harvesters and faces TWO ore patches: A — a NEAR small patch
8
+ (rich per-cell but quickly exhausted by two collectors stacking on
9
+ it) — and B — a FAR larger patch (slow per-cycle but doesn't
10
+ saturate). The optimal policy sends ONE harvester to A and ONE to
11
+ B (the "split route"), so the proc receives a steady alternating
12
+ delivery stream from two short and one long round-trip cycle. The
13
+ LAZY/BRUTE alternatives both fail: piling BOTH harvs on A pumps
14
+ early cash but never enters B's region (the routing constraint in
15
+ the win predicate goes unmet); piling BOTH on B earns barely any
16
+ income within the deadline (long round-trip dominates). Stalling
17
+ earns zero. The capability under test is the OR vehicle-routing /
18
+ SC2 worker-distribution insight: distribute collectors across
19
+ heterogeneous sources to balance round-trip throughput, not to
20
+ monocrop the best-looking node.
21
+ robotics_analogue: >
22
+ Fleet vehicle-routing across a multi-server queueing system
23
+ (M/M/c): two pickup stations at different distances from a depot,
24
+ each modelled as a server with its own service rate; the optimal
25
+ fleet allocation puts exactly one vehicle per server so the depot
26
+ queue alternates arrivals from heterogeneous cycle times rather
27
+ than stacking all vehicles on the lowest-latency server (which
28
+ saturates the server and starves throughput) or the
29
+ highest-throughput server (which dominates the cycle and starves
30
+ the depot in early windows).
31
+ author: wave7-fleet
32
+ benchmark_anchor:
33
+ - "SC2 worker distribution"
34
+ - "OR vehicle routing"
35
+ - "M/M/c queueing"
36
+
37
+ # ENGINE NOTE (verified 2026-05-20 against installed openra_train wheel,
38
+ # post-S0/S1 harvest income — Task #14, scripted run_level seeds 1-4):
39
+ #
40
+ # 1. Per-harvester yields over 4500 ticks (50 turns) on rush-hour-arena,
41
+ # proc at (12,18), 2 harvs at (14,18)/(14,20):
42
+ # A=(16,18) NEAR — ~9000 cr/harv/4500t (1 harv: 8000; 2 harvs: 18000)
43
+ # B=(60,18) MID — ~1500 cr/harv/4500t (1 harv: 1000-2000)
44
+ # B=(80,18) FAR — ~1000 cr/harv/4500t (1 harv: 1000)
45
+ # The 2-harv yield on A is HIGHER than 1A+1B in raw credits, BUT
46
+ # the win predicate REQUIRES a harv inside BOTH region A AND region
47
+ # B (`units_of_type_in_region_gte` clauses). The "stack on A"
48
+ # policy never enters B's region (the harvest order persists, the
49
+ # harv shuttles between A and proc), so the routing clause fails
50
+ # even at 18000 cr. This is the discrimination teeth: the win
51
+ # predicate enforces SPLIT routing as a hard constraint, not as a
52
+ # cash-output preference.
53
+ #
54
+ # 2. Empirical (seed 1, scripted policies):
55
+ # stall 0 cr — LOSS every tier (no cash, no region cover)
56
+ # 2-on-A 18000 cr but 0 harvs in B — LOSS every tier
57
+ # 2-on-B 2000 cr (FAR) — LOSS every tier (cash < bar)
58
+ # 1A+1B (split) 4500-6000 cr + both region clauses fire — WIN
59
+ # every tier; WIN fires at ~turn 25 (easy, B nearer)
60
+ # or turn 35 (medium/hard, B farther) when the
61
+ # loaded harv from B passes through B's radius
62
+ # while the A harv is at A.
63
+ #
64
+ # 3. The `harvest` order with an explicit target cell PERSISTS — the
65
+ # harv shuttles patch ↔ proc on that route. To pivot to a different
66
+ # patch the agent must re-issue `harvest` with new coords. This is
67
+ # the routing knob the model uses.
68
+ #
69
+ # 4. Tick budget: engine advances ~90 ticks per decision turn.
70
+ # Easy/medium use max_turns=50 → ceiling 4503 → within_ticks =
71
+ # after_ticks = 4500 (non-finisher LOSES, not draws).
72
+ # Hard uses max_turns=50 → ceiling 4503 → within_ticks =
73
+ # after_ticks = 4500 (same alignment).
74
+ #
75
+ # 5. The pre-placed proc + fact + harvs trigger ConquestVictoryConditions
76
+ # — without a persistent enemy actor the engine auto-`done`s on
77
+ # "all enemies dead". An unarmed e1 at (120,36) stance:0 sits well
78
+ # outside the harvest envelope so it can't be killed; the win/fail
79
+ # predicate evaluates cleanly to the bar.
80
+ #
81
+ # 6. Hard tier spawn round-robin (verified seeds 1-4): seeds 1,3
82
+ # pick spawn_point 1 (SOUTH base, proc at y=28); seeds 2,4 pick
83
+ # spawn_point 0 (NORTH base, proc at y=14). The neutral mines
84
+ # place at the SAME four cells across both spawns (CLAUDE.md:
85
+ # "spawn_point filter applies ONLY to AGENT actors"), but the
86
+ # SPAWN-MATCHED A/B pair flips per seed: NORTH → A=(16,14)
87
+ # B=(80,14); SOUTH → A=(16,28) B=(80,28). A memorised "always
88
+ # split (16,14)+(80,14)" policy LOSES on the SOUTH seeds (1,3)
89
+ # because the harvs head north and never enter the SOUTH-matched
90
+ # regions of the active win-predicate disjunct. The win predicate
91
+ # is `any_of` over the two spawn-matched (A,B) pairs.
92
+
93
+ base_map: rush-hour-arena
94
+ starting_cash: 0
95
+
96
+ base:
97
+ agent:
98
+ faction: allies
99
+ enemy:
100
+ faction: soviet
101
+ tools:
102
+ - observe
103
+ - harvest
104
+ - move_units
105
+ - stop
106
+ planning: true
107
+ termination:
108
+ max_ticks: 40000
109
+ actors:
110
+ # Pre-placed agent base + 2 harvs centred on row y=18..20.
111
+ - {type: fact, owner: agent, position: [10, 22]}
112
+ - {type: proc, owner: agent, position: [12, 18]}
113
+ - {type: harv, owner: agent, position: [14, 18]}
114
+ - {type: harv, owner: agent, position: [14, 20]}
115
+ # Default patches (overridden per level for tier-specific geometry).
116
+ - {type: mine, owner: neutral, position: [16, 18]}
117
+ - {type: mine, owner: neutral, position: [80, 18]}
118
+ # Inert enemy marker far from the harvest envelope keeps the
119
+ # episode from auto-terminating on "all enemies dead" before the
120
+ # win/fail predicate is evaluated.
121
+ - {type: e1, owner: enemy, position: [120, 36], stance: 0}
122
+
123
+ levels:
124
+ easy:
125
+ description: >
126
+ Two patches and two harvesters. A — NEAR patch at (16,18) — is
127
+ rich per-cell; B — MID patch at (60,18) — is slower per round-
128
+ trip. The win predicate requires economy_value ≥ 4000 AND at
129
+ least one harvester in EACH patch's region (radius 6). Stacking
130
+ both harvs on A pumps 18000 cr but never enters B's region —
131
+ LOSS. Stacking both on B earns only ~2000 cr — LOSS. The split
132
+ (1 harv to A, 1 to B) clears both routing clauses AND the modest
133
+ cash bar — WIN. The capability asked is "use BOTH patches with
134
+ one harv each, don't monocrop the best-looking one".
135
+ starting_cash: 0
136
+ overrides:
137
+ actors:
138
+ - {type: fact, owner: agent, position: [10, 22]}
139
+ - {type: proc, owner: agent, position: [12, 18]}
140
+ - {type: harv, owner: agent, position: [14, 18]}
141
+ - {type: harv, owner: agent, position: [14, 20]}
142
+ - {type: mine, owner: neutral, position: [16, 18]}
143
+ - {type: mine, owner: neutral, position: [60, 18]}
144
+ - {type: e1, owner: enemy, position: [120, 36], stance: 0}
145
+ win_condition:
146
+ all_of:
147
+ - economy_value_gte: 4000
148
+ - has_building: fact
149
+ - units_of_type_in_region_gte:
150
+ type: harv
151
+ x: 16
152
+ y: 18
153
+ radius: 6
154
+ n: 1
155
+ - units_of_type_in_region_gte:
156
+ type: harv
157
+ x: 60
158
+ y: 18
159
+ radius: 6
160
+ n: 1
161
+ - within_ticks: 4500
162
+ # ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES.
163
+ fail_condition:
164
+ any_of:
165
+ - after_ticks: 4500
166
+ - not: {has_building: fact}
167
+ - not: {has_building: proc}
168
+ max_turns: 50
169
+ medium:
170
+ description: >
171
+ Same split-routing capability as easy, but B is pushed out to
172
+ (80,18) (FAR) so 2-on-B yields only ~2000 cr/4500t and the bar
173
+ is tightened to 5000 cr. The capability asked is identical —
174
+ one harv per patch — but the deadline now bites harder on a
175
+ monocrop policy: 2-on-A still earns the most credits (18000)
176
+ but the B-region clause is unsatisfied so the win never fires;
177
+ 2-on-B clears the routing clause but only earns ~2000 cr; the
178
+ split (1 each) clears both at ~6000 cr around turn 35.
179
+ starting_cash: 0
180
+ win_condition:
181
+ all_of:
182
+ - economy_value_gte: 5000
183
+ - has_building: fact
184
+ - units_of_type_in_region_gte:
185
+ type: harv
186
+ x: 16
187
+ y: 18
188
+ radius: 6
189
+ n: 1
190
+ - units_of_type_in_region_gte:
191
+ type: harv
192
+ x: 80
193
+ y: 18
194
+ radius: 6
195
+ n: 1
196
+ - within_ticks: 4500
197
+ # ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES.
198
+ fail_condition:
199
+ any_of:
200
+ - after_ticks: 4500
201
+ - not: {has_building: fact}
202
+ - not: {has_building: proc}
203
+ max_turns: 50
204
+ hard:
205
+ description: >
206
+ Two spawn groups round-robined per seed: NORTH base (y=14) and
207
+ SOUTH base (y=28). Four neutral mines stay fixed at
208
+ (16,14)/(16,28)/(80,14)/(80,28); the SPAWN-MATCHED (A,B) pair
209
+ flips per seed — NORTH → A=(16,14), B=(80,14); SOUTH →
210
+ A=(16,28), B=(80,28). The win predicate is `any_of` over the
211
+ two spawn-matched split clauses, so a memorised "always send to
212
+ (16,14)+(80,14)" policy LOSES on the SOUTH seeds because the
213
+ harvs never enter the SOUTH-matched regions of the satisfied
214
+ disjunct. The capability asked is "identify your own spawn,
215
+ then split-route to the MATCHED A and B regions" — a route-
216
+ planning task that cannot be memorised from a single seed.
217
+ starting_cash: 0
218
+ overrides:
219
+ actors:
220
+ # spawn_point 0 — base NORTH (proc + harvs around y=14).
221
+ - {type: fact, owner: agent, position: [10, 14], spawn_point: 0}
222
+ - {type: proc, owner: agent, position: [12, 14], spawn_point: 0}
223
+ - {type: harv, owner: agent, position: [14, 14], spawn_point: 0}
224
+ - {type: harv, owner: agent, position: [14, 15], spawn_point: 0}
225
+ # spawn_point 1 — base SOUTH (proc + harvs around y=28).
226
+ - {type: fact, owner: agent, position: [10, 28], spawn_point: 1}
227
+ - {type: proc, owner: agent, position: [12, 28], spawn_point: 1}
228
+ - {type: harv, owner: agent, position: [14, 28], spawn_point: 1}
229
+ - {type: harv, owner: agent, position: [14, 29], spawn_point: 1}
230
+ # Four shared patches — identical to both spawns; the MATCHED
231
+ # (A,B) pair flips per seed (NORTH → (16,14)+(80,14); SOUTH →
232
+ # (16,28)+(80,28)).
233
+ - {type: mine, owner: neutral, position: [16, 14]}
234
+ - {type: mine, owner: neutral, position: [80, 14]}
235
+ - {type: mine, owner: neutral, position: [16, 28]}
236
+ - {type: mine, owner: neutral, position: [80, 28]}
237
+ - {type: e1, owner: enemy, position: [120, 36], stance: 0}
238
+ win_condition:
239
+ all_of:
240
+ - has_building: fact
241
+ - within_ticks: 4500
242
+ - any_of:
243
+ # NORTH-matched split.
244
+ - all_of:
245
+ - economy_value_gte: 5000
246
+ - units_of_type_in_region_gte:
247
+ type: harv
248
+ x: 16
249
+ y: 14
250
+ radius: 6
251
+ n: 1
252
+ - units_of_type_in_region_gte:
253
+ type: harv
254
+ x: 80
255
+ y: 14
256
+ radius: 6
257
+ n: 1
258
+ # SOUTH-matched split.
259
+ - all_of:
260
+ - economy_value_gte: 5000
261
+ - units_of_type_in_region_gte:
262
+ type: harv
263
+ x: 16
264
+ y: 28
265
+ radius: 6
266
+ n: 1
267
+ - units_of_type_in_region_gte:
268
+ type: harv
269
+ x: 80
270
+ y: 28
271
+ radius: 6
272
+ n: 1
273
+ # ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES.
274
+ fail_condition:
275
+ any_of:
276
+ - after_ticks: 4500
277
+ - not: {has_building: fact}
278
+ - not: {has_building: proc}
279
+ max_turns: 50
tests/test_build_defensive_tower_line.py ADDED
@@ -0,0 +1,297 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """build-defensive-tower-line scenario family, full loop on Rust.
2
+
3
+ The pack tests DEFENSIVE PERIMETER TOPOLOGY: when the threat is funnelled
4
+ through a known corridor whose WIDTH matters (y=18..22 at x=60), the right
5
+ architecture is one pbox per row across the FULL corridor width (a LINE),
6
+ NOT a cluster on the centre row and NOT a scatter near the base. This is
7
+ the sibling/inverse of `def-tower-line-vs-cluster` (which forces a
8
+ CLUSTER at a single bottleneck cell); together the two packs discriminate
9
+ whether the model understands the FORCING GEOMETRY (single-cell chokepoint
10
+ vs corridor-width approach).
11
+
12
+ Anchors: ERQA spatial commit / MicroRTS defense placement / military
13
+ perimeter (firewall rule placement).
14
+
15
+ The win predicate makes the LINE topology load-bearing — total pbox
16
+ count alone is not enough:
17
+
18
+ * `building_count_gte:{pbox, n:4}` ⇒ the agent built the full budget;
19
+ * `building_in_region:{pbox, x:60, y:Y, radius:0.5, count:1}` for each
20
+ of the four corridor rungs Y ∈ {18,19,21,22} ⇒ exactly one pbox per
21
+ row across the corridor (a tiny radius 0.5 means only the exact cell
22
+ counts, so a cluster on (60,20) misses ALL FOUR rungs and a scatter
23
+ near the base misses all four);
24
+ * `building_count_gte:{fact,n:1}` (present-tense — `has_building` is
25
+ the one-shot "ever-seen" set, see CLAUDE.md footgun);
26
+ * `within_ticks` paired with `after_ticks` in the fail clause ⇒ a
27
+ non-finisher is a real reachable timeout LOSS (no interrupts on this
28
+ pack ⇒ each step is exactly 90 ticks, so max_turns is a hard tick
29
+ budget that the `after_ticks` deadline reliably bites in).
30
+
31
+ The scripted-policy validations prove deterministically that:
32
+
33
+ * the intended LINE policy (one pbox at each of the four corridor rung
34
+ cells) WINS every level + every hard seed (1..4);
35
+ * stall / random-4-pbox (4 pboxes placed near the base, away from the
36
+ corridor) both LOSE every level + every hard seed — a real LOSS,
37
+ not a draw;
38
+ * the hard tier defines ≥2 spawn_point groups (NORTH base y=12 / SOUTH
39
+ base y=28) so a memorised base-relative placement cannot generalise.
40
+ """
41
+
42
+ from __future__ import annotations
43
+
44
+ import pytest
45
+
46
+ pytest.importorskip("openra_train", reason="Rust env wheel not installed")
47
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
48
+
49
+ from openra_bench.eval_core import run_level
50
+ from openra_bench.scenarios import load_pack
51
+ from openra_bench.scenarios.loader import PACKS_DIR, compile_level
52
+
53
+ PACK = PACKS_DIR / "build-defensive-tower-line.yaml"
54
+ LEVELS = ("easy", "medium", "hard")
55
+ SEEDS = (1, 2, 3, 4)
56
+
57
+ # Required corridor rung cells (the four "rungs" spanning y=18..22 at
58
+ # the choke column x=60; y=20 is the middle that the LINE topology
59
+ # leaves open by construction — placing on y=20 won't satisfy any rung).
60
+ RUNGS = [(60, 18), (60, 19), (60, 21), (60, 22)]
61
+
62
+ # Cells used by the "random-4-pbox" wrong-topology policy: 4 pboxes
63
+ # clustered near the base rather than along the corridor. None of these
64
+ # lie inside ANY rung region (radius 0.5 around the rung cells), so the
65
+ # region clauses are all unsatisfied.
66
+ RANDOM_CELLS_NEAR_BASE = [(20, 18), (22, 20), (24, 22), (26, 19)]
67
+
68
+
69
+ # ── scripted policies ────────────────────────────────────────────────
70
+
71
+
72
+ def stall(rs, C):
73
+ """Observe-only — the agent never spends. Fact gets razed by the
74
+ rush AND the count/region clauses are never satisfied."""
75
+ return [C.observe()]
76
+
77
+
78
+ def make_line():
79
+ """Intended LINE topology: one pbox at EACH of the four corridor
80
+ rung cells (60,18) (60,19) (60,21) (60,22)."""
81
+
82
+ def policy(rs, C):
83
+ own_b = rs.get("own_buildings") or []
84
+ n = sum(1 for b in own_b if b.get("type") == "pbox")
85
+ prod = rs.get("production") or []
86
+ prod_items = [p.get("item") for p in prod if isinstance(p, dict)]
87
+ # Once 4 pboxes are up, idle (the win clause re-evaluates each turn).
88
+ if n >= len(RUNGS):
89
+ return [C.observe()]
90
+ cmds = []
91
+ if "pbox" not in prod_items:
92
+ cmds.append(C.build("pbox"))
93
+ cmds.append(C.place_building("pbox", RUNGS[n][0], RUNGS[n][1]))
94
+ return cmds
95
+
96
+ return policy
97
+
98
+
99
+ def make_random_4_pbox():
100
+ """WRONG TOPOLOGY: 4 pboxes placed near the base (not at the
101
+ corridor rungs). Satisfies `building_count_gte:{pbox,n:4}` but
102
+ FAILS every rung region (none of the cells lie in any rung's
103
+ radius-0.5 disk), so the win predicate cannot fire."""
104
+
105
+ def policy(rs, C):
106
+ own_b = rs.get("own_buildings") or []
107
+ n = sum(1 for b in own_b if b.get("type") == "pbox")
108
+ prod = rs.get("production") or []
109
+ prod_items = [p.get("item") for p in prod if isinstance(p, dict)]
110
+ if n >= len(RANDOM_CELLS_NEAR_BASE):
111
+ return [C.observe()]
112
+ cmds = []
113
+ if "pbox" not in prod_items:
114
+ cmds.append(C.build("pbox"))
115
+ cmds.append(
116
+ C.place_building(
117
+ "pbox",
118
+ RANDOM_CELLS_NEAR_BASE[n][0],
119
+ RANDOM_CELLS_NEAR_BASE[n][1],
120
+ )
121
+ )
122
+ return cmds
123
+
124
+ return policy
125
+
126
+
127
+ # ── scenario-shape invariants ────────────────────────────────────────
128
+
129
+
130
+ def test_pack_compiles_with_three_levels_and_rusher_bot():
131
+ pack = load_pack(PACK)
132
+ assert pack.meta.id == "build-defensive-tower-line"
133
+ assert pack.meta.capability == "reasoning"
134
+ assert set(pack.levels) == {"easy", "medium", "hard"}
135
+ # Required-by-spec benchmark anchors.
136
+ anchors = pack.meta.benchmark_anchor
137
+ assert "ERQA" in anchors, anchors
138
+ assert "MicroRTS defense" in anchors, anchors
139
+ assert "military perimeter" in anchors, anchors
140
+ # Rusher bot wired through (charges agent centroid → forces the
141
+ # rush path through the corridor on every seed).
142
+ for lvl in LEVELS:
143
+ c = compile_level(pack, lvl)
144
+ assert c.map_supported
145
+ bot = getattr(c.scenario.enemy, "bot_type", None) or getattr(
146
+ c.scenario.enemy, "bot", None
147
+ )
148
+ assert str(bot).lower() == "rusher", (lvl, bot)
149
+
150
+
151
+ def test_starting_cash_is_exact_pbox_budget():
152
+ """The cash is intentionally tight (4 pbox at 600 each = 2400 on
153
+ every level, zero slack). A model that spends on units OR extra
154
+ power runs out before the count clause is satisfied."""
155
+ pack = load_pack(PACK)
156
+ for lvl in LEVELS:
157
+ c = compile_level(pack, lvl)
158
+ assert c.starting_cash == 2400, (lvl, c.starting_cash)
159
+
160
+
161
+ @pytest.mark.parametrize("level", LEVELS)
162
+ def test_every_level_has_a_reachable_timeout_fail(level):
163
+ """Non-win must be a real LOSS: the `after_ticks` fail clause must
164
+ be strictly below the tick reachable at max_turns. No interrupts on
165
+ this pack ⇒ each step is exactly 90 ticks (max tick = 93+90·(N-1))."""
166
+ c = compile_level(load_pack(PACK), level)
167
+ assert c.fail_condition is not None
168
+ fc = c.fail_condition.model_dump(exclude_none=True)
169
+ deadline = None
170
+ for clause in fc.get("any_of", []) or []:
171
+ if "after_ticks" in clause:
172
+ deadline = int(clause["after_ticks"])
173
+ assert deadline is not None, f"{level}: no after_ticks fail clause"
174
+ reachable = 93 + 90 * (c.max_turns - 1)
175
+ assert deadline < reachable, (
176
+ f"{level}: deadline {deadline} unreachable within "
177
+ f"{c.max_turns} turns (max tick {reachable}) → draw degeneracy"
178
+ )
179
+
180
+
181
+ def test_fact_alive_clause_uses_present_tense_predicate():
182
+ """The fact-survival clause must use the PRESENT-TENSE predicate
183
+ (`building_count_gte:{type:fact,n:1}`) rather than `has_building`,
184
+ which is a one-shot "ever seen" set that stays true after the fact
185
+ is destroyed (a documented CLAUDE.md footgun). Otherwise the rush
186
+ razing the fact would not trigger a LOSS."""
187
+ for lvl in LEVELS:
188
+ c = compile_level(load_pack(PACK), lvl)
189
+ fc = c.fail_condition.model_dump(exclude_none=True)
190
+ fact_clauses = [
191
+ clause for clause in fc.get("any_of", []) or []
192
+ if isinstance(clause, dict)
193
+ and isinstance(clause.get("not"), dict)
194
+ and "building_count_gte" in (clause["not"] or {})
195
+ and (clause["not"]["building_count_gte"] or {}).get("type") == "fact"
196
+ ]
197
+ assert fact_clauses, f"{lvl}: missing present-tense fact-alive fail clause"
198
+
199
+
200
+ def test_win_requires_one_pbox_per_corridor_rung():
201
+ """The LINE-enforcement contract: every level's win clause requires
202
+ exactly one pbox in EACH of the four corridor rungs at x=60
203
+ y∈{18,19,21,22}. A cluster on the centre row (y=20) misses all four
204
+ rungs because each rung region has radius 0.5 (cell-exact)."""
205
+ for lvl in LEVELS:
206
+ c = compile_level(load_pack(PACK), lvl)
207
+ wc = c.win_condition.model_dump(exclude_none=True)
208
+ rungs_seen = set()
209
+ for clause in wc.get("all_of", []) or []:
210
+ br = clause.get("building_in_region")
211
+ if (
212
+ isinstance(br, dict)
213
+ and br.get("type") == "pbox"
214
+ and int(br.get("x", -1)) == 60
215
+ and int(br.get("count", 0)) == 1
216
+ and float(br.get("radius", 0)) <= 1.0
217
+ ):
218
+ rungs_seen.add(int(br["y"]))
219
+ assert rungs_seen == {18, 19, 21, 22}, (
220
+ f"{lvl}: corridor rungs y∈{{18,19,21,22}} required, got {sorted(rungs_seen)}"
221
+ )
222
+
223
+
224
+ def test_hard_has_two_spawn_point_groups():
225
+ """Hard-tier contract: ≥2 distinct agent spawn_point groups so a
226
+ memorised relative-to-base placement that lands in the same world
227
+ cell on every seed cannot generalise."""
228
+ c = compile_level(load_pack(PACK), "hard")
229
+ groups = {
230
+ a.spawn_point for a in c.scenario.actors
231
+ if a.owner == "agent" and a.spawn_point is not None
232
+ }
233
+ assert groups == {0, 1}, groups
234
+ # In-bounds check (rush-hour-arena playable y ≈ 2..38, x ≈ 2..126):
235
+ for a in c.scenario.actors:
236
+ x, y = a.position
237
+ assert 2 <= x <= 126 and 2 <= y <= 38, (a.type, a.position)
238
+
239
+
240
+ # ── solvency: intended LINE wins every level + every hard seed ───────
241
+
242
+
243
+ @pytest.mark.parametrize("level", LEVELS)
244
+ def test_intended_line_wins_every_level_and_seed(level):
245
+ c = compile_level(load_pack(PACK), level)
246
+ for seed in SEEDS:
247
+ r = run_level(c, make_line(), seed=seed)
248
+ assert r.outcome == "win", (
249
+ f"{level} seed{seed}: intended LINE topology must WIN; "
250
+ f"got {r.outcome} (tick={r.signals.game_tick}, "
251
+ f"kills={r.signals.units_killed}, "
252
+ f"lost={r.signals.units_lost}, "
253
+ f"buildings={r.signals.own_buildings})"
254
+ )
255
+
256
+
257
+ # ── no-cheat: every lazy / wrong-topology policy LOSES (not draws) ───
258
+
259
+
260
+ @pytest.mark.parametrize("level", LEVELS)
261
+ @pytest.mark.parametrize(
262
+ "policy_name,policy_factory",
263
+ [
264
+ ("stall", lambda: stall),
265
+ ("random_4_pbox", lambda: make_random_4_pbox()),
266
+ ],
267
+ )
268
+ def test_lazy_and_wrong_topology_policies_lose_every_level_and_seed(
269
+ level, policy_name, policy_factory
270
+ ):
271
+ """Stall (rush razes fact AND clock runs out with no pbox) and
272
+ random-4-pbox (count satisfied but every rung region unsatisfied,
273
+ so the win never fires and the clock runs out) must ALL LOSE on
274
+ every level + every seed — no draw."""
275
+ c = compile_level(load_pack(PACK), level)
276
+ fn = policy_factory()
277
+ for seed in SEEDS:
278
+ r = run_level(c, fn, seed=seed)
279
+ assert r.outcome == "loss", (
280
+ f"{level} seed{seed} {policy_name}: must LOSE (real fail, "
281
+ f"not a draw); got {r.outcome} (tick={r.signals.game_tick}, "
282
+ f"buildings={r.signals.own_buildings})"
283
+ )
284
+
285
+
286
+ # ── determinism ──────────────────────────────────────────────────────
287
+
288
+
289
+ def test_intended_run_is_deterministic_on_easy():
290
+ c = compile_level(load_pack(PACK), "easy")
291
+ a = run_level(c, make_line(), seed=3)
292
+ b = run_level(c, make_line(), seed=3)
293
+ assert (a.outcome, a.turns, a.signals.units_killed) == (
294
+ b.outcome,
295
+ b.turns,
296
+ b.signals.units_killed,
297
+ ), "same seed must be deterministic"
tests/test_econ_harvester_pathing_optimization.py ADDED
@@ -0,0 +1,358 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Full contributor-loop validation for econ-harvester-pathing-optimization.
2
+
3
+ The pack tests harvester SPLIT-routing under heterogeneous round-trip
4
+ cost (OR vehicle-routing / SC2 worker-distribution / M/M/c queueing).
5
+ TWO harvesters, TWO patches — A near and B far. The optimal policy
6
+ sends ONE harv to A and ONE to B; the win predicate encodes the
7
+ routing requirement as `units_of_type_in_region_gte` clauses on BOTH
8
+ patch regions plus a modest cash bar.
9
+
10
+ Bar (per CLAUDE.md "no defect, no cheat"):
11
+ - stall LOSES every tier (no income, no region cover).
12
+ - 2-on-A LOSES every tier — pumps more credits than the bar but never
13
+ enters B's region, so the routing clause fails.
14
+ - 2-on-B LOSES every tier — clears B's region but the FAR round-trip
15
+ earns ~2000 cr, well below the cash bar.
16
+ - 1A+1B (intended SPLIT) WINS every tier and every hard seed.
17
+ - Hard: a memorised single-pair "always send to (16,14)+(80,14)"
18
+ policy LOSES on the spawn-mismatched seeds (the `any_of` disjunct
19
+ for that spawn pair never matches).
20
+ - The capability-policy that IDENTIFIES the matched (A,B) pair from
21
+ the harvs' Y row and splits accordingly WINS every hard seed.
22
+ """
23
+
24
+ from __future__ import annotations
25
+
26
+ import pytest
27
+
28
+ pytest.importorskip("openra_train", reason="Rust env wheel not installed")
29
+
30
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
31
+ from openra_bench.eval_core import run_level
32
+ from openra_bench.scenarios import load_pack
33
+ from openra_bench.scenarios.loader import PACKS_DIR, compile_level
34
+
35
+ PACK = PACKS_DIR / "econ-harvester-pathing-optimization.yaml"
36
+
37
+ # Easy / medium geometry.
38
+ EASY_A = (16, 18)
39
+ EASY_B = (60, 18)
40
+ MED_A = (16, 18)
41
+ MED_B = (80, 18)
42
+
43
+ # Hard tier: four patches, NORTH and SOUTH spawn-matched pairs.
44
+ HARD_NA = (16, 14)
45
+ HARD_NB = (80, 14)
46
+ HARD_SA = (16, 28)
47
+ HARD_SB = (80, 28)
48
+
49
+
50
+ # ---------------------------------------------------------------- policies
51
+
52
+
53
+ def stall_policy(rs, Command):
54
+ return [Command.observe()]
55
+
56
+
57
+ def _make_alloc(targets):
58
+ """Send harv[i] (in id order) to targets[i] every turn. The
59
+ `harvest` order persists so re-issuing is idempotent; passing
60
+ `None` for a slot leaves that harv idle (used by the stall-ish
61
+ one-harv probes)."""
62
+ def f(rs, Command):
63
+ harvs = sorted(
64
+ (u for u in rs.get("units_summary", []) if u.get("type") == "harv"),
65
+ key=lambda u: u["id"],
66
+ )
67
+ cmds = []
68
+ for h, t in zip(harvs, targets):
69
+ if t is not None:
70
+ cmds.append(Command.harvest([str(h["id"])], *t))
71
+ return cmds or [Command.observe()]
72
+ return f
73
+
74
+
75
+ def _make_smart_hard():
76
+ """Hard-tier intended policy: identify the matched (A,B) pair from
77
+ the harvs' Y row (NORTH base → harvs at y=14..15 → split to
78
+ (16,14)+(80,14); SOUTH base → y=28..29 → split to (16,28)+(80,28))."""
79
+ def f(rs, Command):
80
+ harvs = sorted(
81
+ (u for u in rs.get("units_summary", []) if u.get("type") == "harv"),
82
+ key=lambda u: u["id"],
83
+ )
84
+ if not harvs:
85
+ return [Command.observe()]
86
+ y = harvs[0]["cell_y"]
87
+ if y < 21:
88
+ targets = [HARD_NA, HARD_NB]
89
+ else:
90
+ targets = [HARD_SA, HARD_SB]
91
+ return [
92
+ Command.harvest([str(h["id"])], *t)
93
+ for h, t in zip(harvs, targets)
94
+ ]
95
+ return f
96
+
97
+
98
+ # ---------------------------------------------------------------- helpers
99
+
100
+
101
+ def _run(level, policy_factory, seed=1):
102
+ c = compile_level(load_pack(PACK), level)
103
+ assert c.map_supported, "rush-hour-arena terrain must be present"
104
+ policy = policy_factory() if callable(policy_factory) else policy_factory
105
+ return c, run_level(c, policy, seed=seed)
106
+
107
+
108
+ def _ev(res):
109
+ return res.signals.cash + res.signals.resources
110
+
111
+
112
+ # ---------------------------------------------------------------- structural
113
+
114
+
115
+ def test_pack_loads_and_meta_active():
116
+ pack = load_pack(PACK)
117
+ assert pack.meta.status == "active"
118
+ assert pack.meta.id == "econ-harvester-pathing-optimization"
119
+ assert pack.meta.capability == "reasoning"
120
+ anchors = pack.meta.benchmark_anchor
121
+ # The task's three named real-world / benchmark anchors.
122
+ assert any("SC2" in a and "worker" in a for a in anchors), anchors
123
+ assert any("OR" in a and "routing" in a for a in anchors), anchors
124
+ assert any("M/M/c" in a for a in anchors), anchors
125
+
126
+
127
+ def test_all_tiers_have_reachable_deadlines():
128
+ """tick-alignment idiom: within_ticks ≤ ceiling AND
129
+ after_ticks ≤ ceiling AND within_ticks == after_ticks (so a
130
+ non-finisher LOSES, not draws)."""
131
+ pack = load_pack(PACK)
132
+
133
+ def _find_within(node):
134
+ """Recurse through nested all_of/any_of to find the
135
+ within_ticks leaf in the win condition."""
136
+ if isinstance(node, dict):
137
+ if "within_ticks" in node:
138
+ return int(node["within_ticks"])
139
+ for k in ("all_of", "any_of"):
140
+ if k in node:
141
+ for c in node[k]:
142
+ v = _find_within(c)
143
+ if v is not None:
144
+ return v
145
+ return None
146
+
147
+ for lvl in ("easy", "medium", "hard"):
148
+ L = pack.levels[lvl]
149
+ ceiling = 93 + 90 * (L.max_turns - 1)
150
+ wt = _find_within(L.win_condition.model_dump())
151
+ ft = next(
152
+ int(c["after_ticks"])
153
+ for c in L.fail_condition.model_dump()["any_of"]
154
+ if "after_ticks" in c
155
+ )
156
+ assert wt is not None, f"{lvl}: within_ticks missing from win"
157
+ assert wt <= ceiling, f"{lvl}: within_ticks {wt} > ceiling {ceiling}"
158
+ assert ft <= ceiling, f"{lvl}: after_ticks {ft} > ceiling {ceiling}"
159
+ assert wt == ft, (
160
+ f"{lvl}: within_ticks {wt} != after_ticks {ft} "
161
+ "(non-finisher must LOSE, not draw)"
162
+ )
163
+
164
+
165
+ def test_hard_has_two_seed_driven_spawn_groups():
166
+ """Hard tier must define ≥2 agent spawn_point groups (the
167
+ UPGRADED contract — a single memorised opening can't generalise)."""
168
+ c = compile_level(load_pack(PACK), "hard")
169
+ sp = {
170
+ (a.spawn_point if a.spawn_point is not None else 0)
171
+ for a in c.scenario.actors
172
+ if a.owner == "agent"
173
+ }
174
+ assert len(sp) >= 2, (
175
+ f"hard must define ≥2 agent spawn_point groups; got {sorted(sp)}"
176
+ )
177
+
178
+
179
+ # ---------------------------------------------------------------- EASY
180
+
181
+
182
+ def test_easy_stall_loses():
183
+ _, res = _run("easy", lambda: stall_policy)
184
+ assert res.outcome == "loss", (
185
+ f"stall must LOSE easy; got {res.outcome} ev={_ev(res)}"
186
+ )
187
+
188
+
189
+ def test_easy_both_to_a_loses_despite_high_cash():
190
+ """The crucial discrimination: 2-on-A earns ~18000 cr (well over
191
+ the 4000 bar) but never enters B's region — the routing clause
192
+ fails so the win predicate as a whole fails. LOSS, not WIN."""
193
+ _, res = _run("easy", lambda: _make_alloc([EASY_A, EASY_A]))
194
+ assert res.outcome == "loss", (
195
+ f"2-on-A must LOSE easy (no harv in B region); "
196
+ f"got {res.outcome} ev={_ev(res)}"
197
+ )
198
+ assert _ev(res) >= 8000, (
199
+ f"2-on-A should still HAVE earned a lot of cash "
200
+ f"(routing clause is the teeth, not cash); ev={_ev(res)}"
201
+ )
202
+
203
+
204
+ def test_easy_both_to_b_loses_on_cash():
205
+ """2-on-B clears B's region but the FAR round-trip earns ~2000 cr
206
+ over 4500 ticks — well below the 4000 bar. LOSS."""
207
+ _, res = _run("easy", lambda: _make_alloc([EASY_B, EASY_B]))
208
+ assert res.outcome == "loss", (
209
+ f"2-on-B must LOSE easy (~2000 ev < 4000 bar); "
210
+ f"got {res.outcome} ev={_ev(res)}"
211
+ )
212
+
213
+
214
+ def test_easy_split_wins():
215
+ """The intended split (1 harv to A, 1 harv to B) clears both
216
+ routing clauses AND the modest cash bar — WIN."""
217
+ _, res = _run("easy", lambda: _make_alloc([EASY_A, EASY_B]))
218
+ assert res.outcome == "win", (
219
+ f"1A+1B split must WIN easy; got {res.outcome} ev={_ev(res)}"
220
+ )
221
+
222
+
223
+ def test_easy_split_wins_either_assignment():
224
+ """The assignment of which harv-id goes where doesn't matter —
225
+ routing is symmetric. Sanity check the reversed assignment also
226
+ wins (catches a hidden id-ordering dependency)."""
227
+ _, res = _run("easy", lambda: _make_alloc([EASY_B, EASY_A]))
228
+ assert res.outcome == "win", (
229
+ f"1B+1A split must WIN easy; got {res.outcome} ev={_ev(res)}"
230
+ )
231
+
232
+
233
+ # ---------------------------------------------------------------- MEDIUM
234
+
235
+
236
+ def test_medium_stall_loses():
237
+ _, res = _run("medium", lambda: stall_policy)
238
+ assert res.outcome == "loss", (
239
+ f"stall must LOSE medium; got {res.outcome} ev={_ev(res)}"
240
+ )
241
+
242
+
243
+ def test_medium_both_to_a_loses_despite_high_cash():
244
+ """2-on-A still earns the most credits (~18000) but the B-region
245
+ clause is unsatisfied. LOSS."""
246
+ _, res = _run("medium", lambda: _make_alloc([MED_A, MED_A]))
247
+ assert res.outcome == "loss", (
248
+ f"2-on-A must LOSE medium (no harv in B region); "
249
+ f"got {res.outcome} ev={_ev(res)}"
250
+ )
251
+
252
+
253
+ def test_medium_both_to_b_loses_on_cash():
254
+ """B at (80,18) is FAR — 2 harvs there earn only ~2000 cr over
255
+ 4500 ticks, well below the 5000 bar."""
256
+ _, res = _run("medium", lambda: _make_alloc([MED_B, MED_B]))
257
+ assert res.outcome == "loss", (
258
+ f"2-on-B must LOSE medium (~2000 ev < 5000 bar); "
259
+ f"got {res.outcome} ev={_ev(res)}"
260
+ )
261
+
262
+
263
+ def test_medium_split_wins():
264
+ _, res = _run("medium", lambda: _make_alloc([MED_A, MED_B]))
265
+ assert res.outcome == "win", (
266
+ f"1A+1B split must WIN medium; got {res.outcome} ev={_ev(res)}"
267
+ )
268
+
269
+
270
+ def test_medium_split_wins_either_assignment():
271
+ _, res = _run("medium", lambda: _make_alloc([MED_B, MED_A]))
272
+ assert res.outcome == "win", (
273
+ f"1B+1A split must WIN medium; got {res.outcome} ev={_ev(res)}"
274
+ )
275
+
276
+
277
+ # ---------------------------------------------------------------- HARD
278
+
279
+
280
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
281
+ def test_hard_stall_loses_every_seed(seed):
282
+ _, res = _run("hard", lambda: stall_policy, seed=seed)
283
+ assert res.outcome == "loss", (
284
+ f"stall must LOSE hard/seed{seed}; got {res.outcome} ev={_ev(res)}"
285
+ )
286
+
287
+
288
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
289
+ def test_hard_both_to_matched_a_loses_every_seed(seed):
290
+ """Stacking both harvs on the NORTH-near patch (16,14) is a "use
291
+ only A" policy. On a NORTH spawn it clears the A-region clause
292
+ but the NORTH B-region is empty. On a SOUTH spawn it's outright
293
+ far from both matched regions. LOSS every seed."""
294
+ _, res = _run("hard", lambda: _make_alloc([HARD_NA, HARD_NA]), seed=seed)
295
+ assert res.outcome == "loss", (
296
+ f"2-on-A(north) must LOSE hard/seed{seed}; "
297
+ f"got {res.outcome} ev={_ev(res)}"
298
+ )
299
+
300
+
301
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
302
+ def test_hard_both_to_matched_b_loses_every_seed(seed):
303
+ """Mirror: 2-on-B(north) — far patch only, never enters A's
304
+ region (on either spawn) and earns ~1000 cr."""
305
+ _, res = _run("hard", lambda: _make_alloc([HARD_NB, HARD_NB]), seed=seed)
306
+ assert res.outcome == "loss", (
307
+ f"2-on-B(north) must LOSE hard/seed{seed}; "
308
+ f"got {res.outcome} ev={_ev(res)}"
309
+ )
310
+
311
+
312
+ def test_hard_memorised_north_split_loses_on_south_spawn_seeds():
313
+ """A model that memorises "always split (16,14)+(80,14)" loses on
314
+ the SOUTH-spawn seeds (1 and 3 per round-robin) because the harvs
315
+ head to NORTH-matched cells and never enter the SOUTH-matched
316
+ regions — the SOUTH disjunct fails (no harvs in its regions) and
317
+ the NORTH disjunct fails on cash (the harvs are too far from
318
+ their own proc to round-trip efficiently)."""
319
+ for seed in (1, 3):
320
+ _, res = _run("hard", lambda: _make_alloc([HARD_NA, HARD_NB]), seed=seed)
321
+ assert res.outcome == "loss", (
322
+ f"memorised-NORTH-split must LOSE hard/seed{seed} (SOUTH spawn); "
323
+ f"got {res.outcome} ev={_ev(res)}"
324
+ )
325
+
326
+
327
+ def test_hard_memorised_south_split_loses_on_north_spawn_seeds():
328
+ """Symmetric: memorising "always split (16,28)+(80,28)" loses on
329
+ NORTH-spawn seeds 2 and 4."""
330
+ for seed in (2, 4):
331
+ _, res = _run("hard", lambda: _make_alloc([HARD_SA, HARD_SB]), seed=seed)
332
+ assert res.outcome == "loss", (
333
+ f"memorised-SOUTH-split must LOSE hard/seed{seed} (NORTH spawn); "
334
+ f"got {res.outcome} ev={_ev(res)}"
335
+ )
336
+
337
+
338
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
339
+ def test_hard_smart_spawn_matched_split_wins_every_seed(seed):
340
+ """The intended capability — identify the spawn-matched (A,B)
341
+ pair from the agent's harv Y position, then split-route to BOTH
342
+ matched patches — WINS every seed cleanly."""
343
+ _, res = _run("hard", _make_smart_hard, seed=seed)
344
+ assert res.outcome == "win", (
345
+ f"SMART spawn-matched split must WIN hard/seed{seed}; "
346
+ f"got {res.outcome} ev={_ev(res)}"
347
+ )
348
+
349
+
350
+ # ---------------------------------------------------------------- determinism
351
+
352
+
353
+ def test_outcomes_are_deterministic_per_seed():
354
+ """Same seed, same policy → identical outcome and ev."""
355
+ c = compile_level(load_pack(PACK), "medium")
356
+ a = run_level(c, _make_alloc([MED_A, MED_B]), seed=2)
357
+ b = run_level(c, _make_alloc([MED_A, MED_B]), seed=2)
358
+ assert (a.outcome, a.turns, _ev(a)) == (b.outcome, b.turns, _ev(b))
tests/test_hard_tier.py CHANGED
@@ -117,6 +117,14 @@ UPGRADED = [
117
  # flips per seed ((16,14) for NORTH, (16,28) for SOUTH), so a
118
  # memorised "always send to (16,14)" cannot generalise.
119
  "econ-multi-patch-allocation",
 
 
 
 
 
 
 
 
120
  # Group F opening greenfield seed (Wave-4): cold-start from a
121
  # single MCV with no buildings / no harvester / no income. Hard
122
  # tier defines 2 agent spawn_point groups (NORTH (20,14) / SOUTH
 
117
  # flips per seed ((16,14) for NORTH, (16,28) for SOUTH), so a
118
  # memorised "always send to (16,14)" cannot generalise.
119
  "econ-multi-patch-allocation",
120
+ # Wave-7 econ reasoning: split-routing under heterogeneous
121
+ # round-trip cost (OR vehicle-routing / SC2 worker-distribution /
122
+ # M/M/c). Hard defines two agent spawn_point groups (NORTH base
123
+ # y=14 / SOUTH base y=28) round-robined by seed; four neutral
124
+ # mines stay fixed but the SPAWN-MATCHED (A,B) pair flips per
125
+ # seed (NORTH → (16,14)+(80,14); SOUTH → (16,28)+(80,28)), so a
126
+ # memorised single-pair split cannot generalise.
127
+ "econ-harvester-pathing-optimization",
128
  # Group F opening greenfield seed (Wave-4): cold-start from a
129
  # single MCV with no buildings / no harvester / no income. Hard
130
  # tier defines 2 agent spawn_point groups (NORTH (20,14) / SOUTH