yxc20098 commited on
Commit
324f66f
·
1 Parent(s): 3637451

feat(scenario): econ-buy-vs-build-decision — capex-vs-opex under rush (PlanBench anchor)

Browse files

Wave-7 REASONING capability pack. Starting cash ($3400) funds EITHER
4× medium tanks now ($850 × 4 = OPEX, immediate force from the pre-
placed war factory) OR a second war factory ($2000 = CAPEX, doubles
long-run throughput at the cost of leaving only ~1 tank for the
incoming rush). A rusher band staged at the lane mouth charges the
agent fact from t=0 — the right call is to buy-now; the build-weap-
first chain drains cash + locks the queue and the fact is razed
before the new factory fields a single tank.

Anchors: PlanBench resource-allocation; capex vs opex; financial
allocation; SC2 production (pump units now vs lay a second factory).

Bar (CLAUDE.md no defect, no cheat, no draw):
- stall (observe-only) LOSES every tier / every hard seed —
after_ticks (1500) reachable inside max_turns (25, ceiling 2253)
AND the rusher razes the fact (building_count_gte:fact fires).
- build-weap-first LOSES every tier / every hard seed — cash drain
+ queue lockout ⇒ first tank fields too late ⇒ fact razed.
- intended buy-now (queue 2tnk × 4, let them auto-fire from base)
WINS every tier / every hard seed (47/47 scripted assertions).

Hard tier defines 2 agent spawn_point groups (NORTH y=14 / SOUTH
y=26) round-robined by seed; rusher bands always place at BOTH
latitudes (enemy actors don't honour spawn_point — CLAUDE.md
oramap.rs footgun) but the rusher bot targets the agent centroid so
the active threat axis flips per seed. Inert HoldFire e1 per group
surfaces the spawn variation via units_summary (building-only base
otherwise). Registered in tests/test_hard_tier.py::UPGRADED.

openra_bench/scenarios/packs/econ-buy-vs-build-decision.yaml ADDED
@@ -0,0 +1,331 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # econ-buy-vs-build-decision — capex-vs-opex under rush pressure
2
+ #
3
+ # REASONING capability. The pack frames a CAPEX-vs-OPEX capital
4
+ # allocation decision under time pressure: starting cash is exactly
5
+ # enough for EITHER
6
+ #
7
+ # (A) BUY 4× medium tanks (2tnk) NOW — $850 × 4 = $3400 — OPEX,
8
+ # immediate combat capability from the pre-placed war factory;
9
+ # (B) BUILD a SECOND war factory (weap) — $2000 — CAPEX, doubles
10
+ # per-turn tank production capacity over the LONG run, but at
11
+ # the cost of leaving no cash for an immediate force (residual
12
+ # $1400 ≈ 1 tank only).
13
+ #
14
+ # A rusher band is staged at the lane mouth from t=0 and charges the
15
+ # agent fact (rusher targets the agent centroid). The timing is set so
16
+ # that the BUY-NOW play (queue 4× 2tnk immediately from the existing
17
+ # weap, attack-move them east to intercept) blunts the rush before the
18
+ # fact dies; the BUILD-WEAP-FIRST play spends 2000cr + 1+ turn of
19
+ # placement and queue lockout on a second factory, fielding only 1
20
+ # tank before the rush razes the fact ⇒ LOSS.
21
+ #
22
+ # Real-world anchors:
23
+ # • CAPEX vs OPEX — invest in long-run capacity vs spend on
24
+ # immediate operations.
25
+ # • PlanBench resource-allocation under deadline pressure.
26
+ # • SC2 production: when a rush is incoming, the right answer is to
27
+ # pump units NOW from existing production buildings, NOT to lay
28
+ # down a second factory.
29
+ # • Disaster response: existing capacity must be used right now;
30
+ # buying a second fleet vehicle is for next quarter, not for the
31
+ # incident in progress.
32
+ #
33
+ # DISCRIMINATIONS (CLAUDE.md bar — no defect, no cheat, no draw):
34
+ #
35
+ # • stall (only observe): LOSS — no defenders ⇒ rush razes fact ⇒
36
+ # `not has_building:fact` fail bites; also after_ticks bites.
37
+ # • build-weap-first (queue+place weap, spend 2000, then build
38
+ # 2tnk with residual cash): LOSS — placement+queue lockout means
39
+ # the first tank does not field until well after the rush has
40
+ # reached the fact; the fact is razed ⇒ LOSS.
41
+ # • intended buy-now (queue 2tnk × 4 from the pre-placed weap
42
+ # starting turn 1, attack-move them east to intercept the rush
43
+ # at the lane mouth, racking up kills): WIN.
44
+ #
45
+ # Real LOSS, not DRAW: every level carries a `not has_building:fact`
46
+ # fail clause AND a tick-aligned `after_ticks` clause inside
47
+ # max_turns (engine advances ~90 ticks per decision turn under
48
+ # interrupts; we keep the deadline comfortably reachable so the
49
+ # stall / build-weap LOSS is reached as a real timeout / fact-loss).
50
+ #
51
+ # Engine facts used (CLAUDE.md / prompt_v2 codex):
52
+ # • 2tnk cost $850, prereqs `weap` + `fix` (allied service depot —
53
+ # gamerules.rs tech gate; both pre-placed ⇒ buildable turn 1).
54
+ # • weap cost $2000, prereq `proc` (pre-placed ⇒ buildable turn 1).
55
+ # • `place_building` has no adjacency requirement (CLAUDE.md), so
56
+ # the BUILD-WEAP play can put its second factory anywhere
57
+ # in-bounds — the LOSS is from cash drain + production lockout,
58
+ # not from placement difficulty.
59
+ # • `rusher` scripted bot relentlessly charges the agent centroid
60
+ # (the `fact`), reliable timing for the buy-now intercept window.
61
+ # • interrupts (enemy_unit_spotted, own_unit_destroyed) give the
62
+ # agent the opening-tempo signals it needs to react inside the
63
+ # rush window without burning fixed-cadence turns.
64
+ # • Unarmed enemy `fact` marker far east keeps the episode alive
65
+ # past the last rusher death (CLAUDE.md auto-done footgun).
66
+ # • Hard tier defines 2 agent spawn_point groups (NORTH base
67
+ # y=14 / SOUTH base y=26) — UPGRADED contract; the rusher band
68
+ # places at BOTH latitudes every seed (enemy actors don't honour
69
+ # spawn_point, CLAUDE.md) but the rusher targets the AGENT
70
+ # centroid so the active band is always the one in the agent's
71
+ # latitude.
72
+ #
73
+ # Sibling pack candidate (not implemented here): the INVERSION —
74
+ # slow enemy + sustained pressure ⇒ tech-up (build weap) WINS,
75
+ # burn-cash-on-4-tanks LOSES (small force runs out in the long
76
+ # horizon). This pack focuses cleanly on the BUY-NOW capability.
77
+ #
78
+ # Validate (no model / no network):
79
+ # python3 -m pytest tests/test_econ_buy_vs_build_decision.py -q
80
+
81
+ meta:
82
+ id: econ-buy-vs-build-decision
83
+ title: 'Capex vs Opex Under Rush Pressure — Buy 4 Tanks NOW or Build a Second Factory?'
84
+ capability: reasoning
85
+ real_world_meaning: >
86
+ A fixed cash budget funds EITHER an immediate force (buy 4× medium
87
+ tanks RIGHT NOW from the pre-placed war factory) OR a long-run
88
+ capacity upgrade (build a SECOND war factory to double per-turn
89
+ tank production). A rusher band is incoming at the lane mouth from
90
+ turn 0 and charges the agent's construction yard. The intended
91
+ play reads the threat timing and commits the budget to immediate
92
+ units; spending it on capacity instead leaves no force for the
93
+ rush, and the construction yard is razed before the new factory
94
+ fields a single tank. The pack frames the classic CAPEX-vs-OPEX
95
+ decision: invest for tomorrow vs operate for today.
96
+ robotics_analogue: >
97
+ Operational capacity allocation under a live incident: the budget
98
+ funds either an immediate dispatch of available robots (OPEX —
99
+ use existing fleet capacity right now to handle the incident) or
100
+ procurement of a second loading dock that will double dispatch
101
+ throughput over the next quarter (CAPEX — long-run capacity). An
102
+ in-progress incident makes the CAPEX play wrong: spending on the
103
+ second loading dock now leaves the site undefended, and the
104
+ incident escalates before the new capacity comes online.
105
+ benchmark_anchor:
106
+ - "PlanBench resource-allocation"
107
+ - "capex vs opex"
108
+ - "financial allocation"
109
+ - "SC2 production: pump units now vs lay second factory"
110
+ author: openra-bench
111
+
112
+ base_map: rush-hour-arena
113
+ # Exactly enough for EITHER 4× 2tnk ($850 × 4 = $3400) OR 1× weap
114
+ # ($2000, with $1400 residual = 1 tank's worth). Spec: the cash
115
+ # equates to a single coherent decision — buy 4 tanks OR build a
116
+ # second factory. A mixed mini-spend (weap + 1 tank) underdelivers
117
+ # on either axis.
118
+ starting_cash: 3400
119
+
120
+ base:
121
+ agent:
122
+ faction: allies
123
+ # `rusher`: relentless concentrated charge at the agent's centroid
124
+ # (the agent fact). Predictable timing — the canonical rush-defense
125
+ # opponent (sister pack: `defense-rush-survive`).
126
+ enemy:
127
+ faction: soviet
128
+ bot_type: rusher
129
+ tools:
130
+ - build
131
+ - place_building
132
+ - move_units
133
+ - attack_unit
134
+ - attack_move
135
+ - stop
136
+ planning: true
137
+ # Re-decide the instant a rusher unit appears at the lane mouth or
138
+ # one of our tanks dies — opening rush-defense is reactive, not on
139
+ # fixed cadence. (Same idiom as defense-rush-survive.)
140
+ interrupts:
141
+ enemy_unit_spotted: true
142
+ own_unit_destroyed: true
143
+ termination:
144
+ max_ticks: 12000
145
+ # Pre-placed (per the design sketch + the buy-now actionability
146
+ # requirement): fact (loss-critical), proc (refinery — weap prereq),
147
+ # powr (power for the production queues — CLAUDE.md underpowered-
148
+ # queue footgun), weap (war factory) AND fix (Service Depot — the
149
+ # 2tnk tech-prereq on allies). Both weap and fix together make
150
+ # 2tnk immediately buildable on turn 1, so the BUY-NOW option is
151
+ # actionable WITHOUT any prior tech step. The agent starts
152
+ # unit-less: the defending force must be produced from the budget
153
+ # — that's the whole decision.
154
+ actors:
155
+ - {type: fact, owner: agent, position: [10, 20]}
156
+ - {type: proc, owner: agent, position: [14, 18]}
157
+ - {type: powr, owner: agent, position: [14, 22]}
158
+ - {type: weap, owner: agent, position: [18, 20]}
159
+ - {type: fix, owner: agent, position: [18, 22]}
160
+
161
+ levels:
162
+ # ── EASY ─────────────────────────────────────────────────────────
163
+ # Bare buy-vs-build skill. A LIGHT rush (4× e1) charges the fact
164
+ # from x=70 (60 cells west of the fact at x=10). Buying 4 tanks
165
+ # and attack-moving east intercepts the rush comfortably; building
166
+ # a second weap leaves only enough cash for 1 tank and the rush
167
+ # razes the fact. Stall loses (no defenders ⇒ fact razed).
168
+ # Kill bar low (3) so the bar attributes to the BUY decision, not
169
+ # to combat micro.
170
+ easy:
171
+ description: >
172
+ A rusher band (4 rifle infantry) is charging your construction
173
+ yard from the lane mouth. Your starting cash ($3400) is exactly
174
+ enough for EITHER 4 medium tanks (build 2tnk × 4 from the
175
+ pre-placed war factory at $850 each) OR a second war factory
176
+ (build weap at $2000, leaving only $1400 = 1 tank). Spend on
177
+ whichever wins the next 40 turns — the rush is incoming NOW.
178
+ Win when ≥3 enemy units are killed AND your construction yard
179
+ (fact) is still standing AND the clock has not run out
180
+ (tick ≤ 3600). Stall, build-the-second-factory-first, or
181
+ under-spend all fail.
182
+ overrides:
183
+ actors:
184
+ - {type: fact, owner: agent, position: [10, 20]}
185
+ - {type: proc, owner: agent, position: [14, 18]}
186
+ - {type: powr, owner: agent, position: [14, 22]}
187
+ - {type: weap, owner: agent, position: [18, 20]}
188
+ - {type: fix, owner: agent, position: [18, 22]}
189
+ # Light rush — 4× e1 at the lane mouth (x=70, same row as
190
+ # fact). Stance 3 ⇒ fires on contact. At ~1 cell per ~12
191
+ # ticks, the 60-cell crawl to the fact takes ~720 ticks ≈
192
+ # 8 turns — enough time for the buy-4-tanks chain to field
193
+ # and intercept BUT not enough for the build-second-weap
194
+ # chain to field even 1 tank.
195
+ - {type: e1, owner: enemy, position: [70, 20], stance: 3, count: 4}
196
+ # Far unarmed enemy marker — anti-DRAW. Without this the
197
+ # engine auto-`done`s the moment the last rusher dies (the
198
+ # rusher band is the only enemy "MustBeDestroyed" otherwise)
199
+ # and the win/fail check never runs in the terminal frame.
200
+ - {type: fact, owner: enemy, position: [120, 20]}
201
+ win_condition:
202
+ all_of:
203
+ - {units_killed_gte: 3}
204
+ - {building_count_gte: {type: fact, n: 1}}
205
+ - {within_ticks: 1499}
206
+ # Tick budget: max_turns 25 ⇒ max reachable tick = 93 + 90·24 =
207
+ # 2253 (fixed-cadence). Stall (observe-only) reaches ~1500
208
+ # ticks well inside max_turns (~88 ticks/turn observed for a
209
+ # no-op episode), so after_ticks 1500 BITES before max_turns
210
+ # exhausts ⇒ a real reachable timeout LOSS, not a draw.
211
+ # We use building_count_gte (counts positions in own_buildings,
212
+ # the live list) instead of has_building (checks
213
+ # own_building_types, a set that can lag a step behind the
214
+ # destruction event under engine auto-done) — see CLAUDE.md
215
+ # own-elimination footgun.
216
+ fail_condition:
217
+ any_of:
218
+ - {after_ticks: 1500}
219
+ - {not: {building_count_gte: {type: fact, n: 1}}}
220
+ max_turns: 25
221
+
222
+ # ── MEDIUM ───────────────────────────────────────────────────────
223
+ # +1 axis: HEAVIER rush AND tighter cash. The band is 5× e1 + 1× e3
224
+ # at the same x=70 lane mouth. Kill bar raised to 5. The 4-tank
225
+ # response still WINS comfortably (4 tanks shred e1 at range and
226
+ # outrange the e3 rocket); the build-weap response still LOSES (the
227
+ # second factory's first tank fields too late + only 1 tank ⇒
228
+ # rocket soldier kills it). Same clock, same fact-loss fail.
229
+ medium:
230
+ description: >
231
+ A larger rusher band (5 rifle infantry + 1 rocket soldier) is
232
+ charging your construction yard. Your starting cash ($3400) is
233
+ exactly enough for EITHER 4 medium tanks (build 2tnk × 4 from
234
+ the pre-placed war factory) OR a second war factory (build
235
+ weap, residual $1400 = 1 tank). The rush is incoming NOW.
236
+ Win when ≥5 enemy units are killed AND your fact is still
237
+ standing AND the clock has not run out (tick ≤ 3600). Stall,
238
+ build-the-second-factory-first, or under-spend all fail.
239
+ overrides:
240
+ actors:
241
+ - {type: fact, owner: agent, position: [10, 20]}
242
+ - {type: proc, owner: agent, position: [14, 18]}
243
+ - {type: powr, owner: agent, position: [14, 22]}
244
+ - {type: weap, owner: agent, position: [18, 20]}
245
+ - {type: fix, owner: agent, position: [18, 22]}
246
+ # Heavier rush — same x=70 lane mouth. 5× e1 staggered
247
+ # rifle wave. 4 tanks at rng4.75 dps22 each (88 dps total)
248
+ # eat the wave; build-weap-first chain produces ≤1 tank by
249
+ # the time the rush arrives and the band overruns the fact.
250
+ - {type: e1, owner: enemy, position: [70, 20], stance: 3, count: 5}
251
+ - {type: fact, owner: enemy, position: [120, 20]}
252
+ win_condition:
253
+ all_of:
254
+ - {units_killed_gte: 4}
255
+ - {building_count_gte: {type: fact, n: 1}}
256
+ - {within_ticks: 1499}
257
+ fail_condition:
258
+ any_of:
259
+ - {after_ticks: 1500}
260
+ - {not: {building_count_gte: {type: fact, n: 1}}}
261
+ max_turns: 25
262
+
263
+ # ── HARD ─────────────────────────────────────────────────────────
264
+ # +1 axis: seed-driven base latitude. TWO agent spawn_point groups
265
+ # (NORTH base y=14 / SOUTH base y=26) round-robined by seed; the
266
+ # rusher band places at BOTH latitudes every seed (enemy actors
267
+ # don't honour spawn_point — CLAUDE.md oramap.rs footgun), but
268
+ # the `rusher` bot always targets the agent centroid so the active
269
+ # threat axis flips per seed. The buy-vs-build decision is
270
+ # IDENTICAL across spawns (the cash, the war factory, the
271
+ # response shape are all symmetric) — a memorised "rush comes
272
+ # from y=14" opening cannot generalise, but the capability
273
+ # generalises cleanly. Kill bar 6, same clock.
274
+ hard:
275
+ description: >
276
+ Two rusher bands (one north y=14, one south y=26) are visible
277
+ at the lane mouth but the rusher bot always charges YOUR
278
+ construction yard (whichever latitude you spawned at). Your
279
+ starting cash ($3400) is exactly enough for EITHER 4 medium
280
+ tanks (build 2tnk × 4 from the pre-placed war factory) OR a
281
+ second war factory (build weap, residual $1400 = 1 tank).
282
+ The rush is incoming NOW. Win when ≥6 enemy units are killed
283
+ AND your fact is still standing AND the clock has not run out
284
+ (tick ≤ 3600). Stall, build-second-factory-first, or
285
+ under-spend all fail. The base latitude varies by seed so a
286
+ single memorised pre-pick cannot generalise — the buy-now
287
+ decision must be made from the live threat reading.
288
+ overrides:
289
+ actors:
290
+ # ── AGENT spawn 0 — NORTH base (y=14) ─────────────────────
291
+ - {type: fact, owner: agent, position: [10, 14], spawn_point: 0}
292
+ - {type: proc, owner: agent, position: [14, 12], spawn_point: 0}
293
+ - {type: powr, owner: agent, position: [14, 16], spawn_point: 0}
294
+ - {type: weap, owner: agent, position: [18, 14], spawn_point: 0}
295
+ - {type: fix, owner: agent, position: [18, 16], spawn_point: 0}
296
+ # Inert spawn-witness e1 (HoldFire) per group: required for
297
+ # the hard-tier units_summary spawn-variation check (the
298
+ # building-only base has no `units_summary` entries by
299
+ # default; see build-power-online-first for the same
300
+ # idiom). Stance 0 + base-adjacent placement makes it
301
+ # functionally inert — does not move, does not fire, does
302
+ # not shift the rusher's centroid target.
303
+ - {type: e1, owner: agent, position: [12, 14], spawn_point: 0, stance: 0}
304
+ # ── AGENT spawn 1 — SOUTH base (y=26) ─────────────────────
305
+ - {type: fact, owner: agent, position: [10, 26], spawn_point: 1}
306
+ - {type: proc, owner: agent, position: [14, 24], spawn_point: 1}
307
+ - {type: powr, owner: agent, position: [14, 28], spawn_point: 1}
308
+ - {type: weap, owner: agent, position: [18, 26], spawn_point: 1}
309
+ - {type: fix, owner: agent, position: [18, 28], spawn_point: 1}
310
+ - {type: e1, owner: agent, position: [12, 26], spawn_point: 1, stance: 0}
311
+ # ── NORTH rusher band (y=14) — always places ──────────────
312
+ # Rusher targets agent centroid, so on a NORTH spawn this
313
+ # band charges the fact directly; on a SOUTH spawn it
314
+ # angles southward (the rusher's relentless charge always
315
+ # reorients to the centroid). Bands kept light (4× e1 each)
316
+ # so 4 medium tanks at the base auto-fire enough kills to
317
+ # clear the bar before fact damage compounds.
318
+ - {type: e1, owner: enemy, position: [70, 14], stance: 3, count: 4}
319
+ # ── SOUTH rusher band (y=26) — always places ──────────────
320
+ - {type: e1, owner: enemy, position: [70, 26], stance: 3, count: 4}
321
+ - {type: fact, owner: enemy, position: [120, 20]}
322
+ win_condition:
323
+ all_of:
324
+ - {units_killed_gte: 5}
325
+ - {building_count_gte: {type: fact, n: 1}}
326
+ - {within_ticks: 1499}
327
+ fail_condition:
328
+ any_of:
329
+ - {after_ticks: 1500}
330
+ - {not: {building_count_gte: {type: fact, n: 1}}}
331
+ max_turns: 25
tests/test_econ_buy_vs_build_decision.py ADDED
@@ -0,0 +1,337 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """econ-buy-vs-build-decision — REASONING capability validation.
2
+
3
+ CAPEX-vs-OPEX capital allocation under rush pressure. Starting cash
4
+ ($3400) funds EITHER:
5
+
6
+ (A) BUY 4× medium tanks (2tnk @ $850) NOW from the pre-placed war
7
+ factory — OPEX, immediate combat capability;
8
+ (B) BUILD a SECOND war factory (weap @ $2000) — CAPEX, doubles
9
+ long-run tank throughput but leaves only $1400 ≈ 1 tank.
10
+
11
+ A `rusher` band is staged at the lane mouth (x=70). The right call is
12
+ (A): queue 4 tanks immediately from the existing weap; they auto-fire
13
+ on the incoming rush before the fact dies. Spending on a second
14
+ factory leaves the base undefended → fact razed → LOSS.
15
+
16
+ Bar (CLAUDE.md "no defect, no cheat, no draw"):
17
+
18
+ * stall (observe-only) LOSES every tier / every hard seed —
19
+ the rusher band over-runs the undefended fact AND/OR the
20
+ `after_ticks` deadline bites.
21
+ * build-weap-first (queue + place weap, drain $2000, then buy
22
+ 1 tank from residual cash) LOSES every tier / every hard seed —
23
+ cash drain + queue lockout ⇒ the first tank fields too late to
24
+ blunt the rush ⇒ fact razed.
25
+ * intended buy-now (queue 2tnk × 4 from the pre-placed weap, let
26
+ them auto-fire from the base position) WINS every tier / every
27
+ hard seed: the kill bar is met AND the fact survives AND the
28
+ rush is over before the `within_ticks` deadline.
29
+ * hard tier defines ≥2 agent spawn_point groups (NORTH base
30
+ y=14 / SOUTH base y=26) round-robined by seed — the threat
31
+ band at each latitude always places (enemy actors don't honour
32
+ spawn_point — CLAUDE.md) but the `rusher` bot targets the
33
+ agent centroid, so the ACTIVE threat axis flips per seed and a
34
+ memorised "rush from y=14" opening cannot generalise.
35
+ """
36
+
37
+ from __future__ import annotations
38
+
39
+ from pathlib import Path
40
+
41
+ import pytest
42
+
43
+ pytest.importorskip("openra_train", reason="Rust env wheel not installed")
44
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
45
+
46
+ from openra_bench.eval_core import run_level
47
+ from openra_bench.scenarios import load_pack
48
+ from openra_bench.scenarios.loader import PACKS_DIR, compile_level
49
+ from openra_bench.scenarios.win_conditions import WinContext, evaluate
50
+
51
+ PACK = PACKS_DIR / "econ-buy-vs-build-decision.yaml"
52
+ LEVELS = ("easy", "medium", "hard")
53
+ SEEDS = (1, 2, 3, 4)
54
+
55
+
56
+ # ── scripted policies ───────────────────────────────────────────────
57
+
58
+
59
+ def _stall(rs, C):
60
+ """Observe-only — no spend, no production. Rusher reaches the
61
+ fact AND/OR the after_ticks deadline bites → LOSS."""
62
+ return [C.observe()]
63
+
64
+
65
+ def _buy_now(rs, C):
66
+ """The intended capability: spam queue 2tnk from the pre-placed
67
+ war factory. Tanks auto-fire on the rusher band as they emerge
68
+ from the production cell — no need to micro-move them out. 4
69
+ tanks at rng4.75 dps22 each (88 dps total) clear the rush
70
+ inside the kill bar before the fact dies."""
71
+ return [C.build("2tnk")]
72
+
73
+
74
+ def _build_weap_first(rs, C):
75
+ """The wrong call: spend $2000 + a placement turn on a SECOND
76
+ war factory instead of pumping units from the existing one.
77
+ Residual cash ($1400) only buys ~1 tank, and the new weap
78
+ fields its first unit well after the rusher has reached the
79
+ fact ⇒ fact razed ⇒ LOSS."""
80
+ own_b = rs.get("own_buildings") or []
81
+ weap_count = sum(1 for b in own_b if b.get("type") == "weap")
82
+ prod = rs.get("production") or []
83
+ prod_items = [p.get("item") for p in prod if isinstance(p, dict)]
84
+ # Find the current fact latitude (varies on hard by seed)
85
+ fy = 20
86
+ for b in own_b:
87
+ if b.get("type") == "fact":
88
+ fy = int(b["cell_y"])
89
+ break
90
+ cmds = []
91
+ if weap_count < 2:
92
+ if "weap" not in prod_items:
93
+ cmds.append(C.build("weap"))
94
+ cmds.append(C.place_building("weap", 22, fy))
95
+ else:
96
+ cmds.append(C.build("2tnk"))
97
+ if not cmds:
98
+ cmds.append(C.observe())
99
+ return cmds
100
+
101
+
102
+ # ── structural tests ────────────────────────────────────────────────
103
+
104
+
105
+ def test_pack_loads_and_meta_active():
106
+ pack = load_pack(PACK)
107
+ assert pack.meta.id == "econ-buy-vs-build-decision"
108
+ assert pack.meta.capability == "reasoning"
109
+ assert pack.meta.real_world_meaning
110
+ assert pack.meta.robotics_analogue
111
+ anchors = " ".join(pack.meta.benchmark_anchor).lower()
112
+ assert "planbench" in anchors and "resource-allocation" in anchors, anchors
113
+ assert "capex" in anchors and "opex" in anchors, anchors
114
+ assert "financial allocation" in anchors, anchors
115
+
116
+
117
+ def test_pack_uses_rusher_bot_on_every_level():
118
+ pack = load_pack(PACK)
119
+ for lvl in LEVELS:
120
+ c = compile_level(pack, lvl)
121
+ assert c.map_supported, f"{lvl}: rush-hour-arena terrain required"
122
+ enemy = c.scenario.enemy
123
+ bot = getattr(enemy, "bot_type", None) or getattr(enemy, "bot", None)
124
+ assert str(bot).lower() == "rusher", f"{lvl}: enemy bot must be 'rusher'; got {bot}"
125
+
126
+
127
+ def test_tools_include_buy_and_build_surface():
128
+ """Pack must expose [build, place_building, move_units,
129
+ attack_unit, attack_move, stop] — the buy-vs-build decision
130
+ interaction surface."""
131
+ pack = load_pack(PACK)
132
+ tools = set(pack.base.get("tools", []) if isinstance(pack.base, dict) else [])
133
+ for required in ("build", "place_building", "move_units",
134
+ "attack_unit", "attack_move", "stop"):
135
+ assert required in tools, f"missing tool: {required!r}"
136
+
137
+
138
+ def test_starting_cash_equals_four_tanks_or_one_weap():
139
+ """Cash budget must be the EITHER/OR pivot: exactly 4× 2tnk
140
+ cost ($3400) — also enough for 1× weap ($2000) with $1400
141
+ residual = ~1 tank. Either decision is a single coherent
142
+ capital allocation."""
143
+ pack = load_pack(PACK)
144
+ cash = getattr(pack, "starting_cash", None)
145
+ assert cash == 3400, f"starting_cash must be 3400 (4× $850); got {cash}"
146
+
147
+
148
+ def test_preplaced_base_has_fact_proc_powr_weap_fix():
149
+ """Pre-placed base on every level: fact (loss-critical), proc
150
+ (weap prereq), powr (queue power), weap (war factory), fix
151
+ (service depot — 2tnk allied tech gate). The buy-now option
152
+ must be ACTIONABLE on turn 1 with no prior tech step."""
153
+ pack = load_pack(PACK)
154
+ for lvl in LEVELS:
155
+ c = compile_level(pack, lvl)
156
+ agent_types = {
157
+ a.type for a in c.scenario.actors if a.owner == "agent"
158
+ }
159
+ for needed in ("fact", "proc", "powr", "weap", "fix"):
160
+ assert needed in agent_types, (
161
+ f"{lvl}: pre-placed base missing {needed!r}; got {sorted(agent_types)}"
162
+ )
163
+
164
+
165
+ def test_every_level_has_reachable_timeout_fail():
166
+ """`after_ticks` fail must bite WITHIN max_turns (so stall is a
167
+ real reachable LOSS, not a draw). within_ticks + 1 == after_ticks
168
+ so a non-finisher is a LOSS, not a draw at the boundary."""
169
+ pack = load_pack(PACK)
170
+ for lvl in LEVELS:
171
+ L = pack.levels[lvl]
172
+ ceiling = 93 + 90 * (L.max_turns - 1)
173
+ wt = next(
174
+ int(c["within_ticks"])
175
+ for c in L.win_condition.model_dump()["all_of"]
176
+ if "within_ticks" in c
177
+ )
178
+ ft = next(
179
+ int(c["after_ticks"])
180
+ for c in L.fail_condition.model_dump()["any_of"]
181
+ if "after_ticks" in c
182
+ )
183
+ assert wt < ceiling, f"{lvl}: within_ticks {wt} >= ceiling {ceiling}"
184
+ assert ft <= ceiling, f"{lvl}: after_ticks {ft} > ceiling {ceiling}"
185
+ assert wt + 1 == ft, (
186
+ f"{lvl}: within_ticks {wt} / after_ticks {ft} mismatch "
187
+ "(boundary non-finisher must LOSE, not draw — fail one "
188
+ "tick past win)"
189
+ )
190
+
191
+
192
+ def test_every_level_has_a_fail_condition():
193
+ pack = load_pack(PACK)
194
+ for lvl in LEVELS:
195
+ c = compile_level(pack, lvl)
196
+ assert c.fail_condition is not None, f"{lvl} needs a fail_condition"
197
+
198
+
199
+ def test_hard_has_two_seed_driven_spawn_groups():
200
+ c = compile_level(load_pack(PACK), "hard")
201
+ sp = {
202
+ (a.spawn_point if a.spawn_point is not None else 0)
203
+ for a in c.scenario.actors
204
+ if a.owner == "agent"
205
+ }
206
+ assert len(sp) >= 2, (
207
+ f"hard must define ≥2 agent spawn_point groups; got {sorted(sp)}"
208
+ )
209
+ assert sp == {0, 1}, f"expected exactly {{0, 1}}; got {sorted(sp)}"
210
+
211
+
212
+ def test_in_bounds_actors_on_every_level():
213
+ """rush-hour-arena playable bounds ≈ x:2..126, y:2..38."""
214
+ pack = load_pack(PACK)
215
+ for lvl in LEVELS:
216
+ c = compile_level(pack, lvl)
217
+ for a in c.scenario.actors:
218
+ x, y = a.position
219
+ assert 2 <= x <= 126 and 2 <= y <= 38, (
220
+ f"{lvl}: actor {a.type} at ({x},{y}) out of bounds"
221
+ )
222
+
223
+
224
+ # ── predicate-level (no engine) ─────────────────────────────────────
225
+
226
+
227
+ def _ctx(*, tick=0, kills=0, own_buildings=()):
228
+ import types
229
+
230
+ sig = types.SimpleNamespace(
231
+ game_tick=tick,
232
+ units_killed=kills,
233
+ units_lost=0,
234
+ cash=0,
235
+ resources=0,
236
+ own_buildings=list(own_buildings),
237
+ own_building_types={str(t).lower() for (t, _, _) in own_buildings},
238
+ enemies_seen_ids=set(),
239
+ enemy_buildings_seen_ids=set(),
240
+ )
241
+ return WinContext(signals=sig, render_state={"units_summary": []})
242
+
243
+
244
+ def test_predicates_enforce_capability():
245
+ """Win requires (kills bar AND fact alive AND in time); fail
246
+ fires on timeout OR fact destroyed."""
247
+ c = compile_level(load_pack(PACK), "easy")
248
+ base_b = [("fact", 10, 20), ("weap", 18, 20)]
249
+
250
+ # Intended: kills 3, fact alive, in time → WIN
251
+ assert evaluate(c.win_condition, _ctx(tick=1000, kills=3, own_buildings=base_b))
252
+ # 2 kills (under bar) → not win
253
+ assert not evaluate(
254
+ c.win_condition, _ctx(tick=1000, kills=2, own_buildings=base_b)
255
+ )
256
+ # 3 kills but past within_ticks → not win
257
+ assert not evaluate(
258
+ c.win_condition, _ctx(tick=1499 + 1, kills=3, own_buildings=base_b)
259
+ )
260
+ # 3 kills but fact destroyed → not win
261
+ assert not evaluate(
262
+ c.win_condition, _ctx(tick=1000, kills=3, own_buildings=base_b[1:])
263
+ )
264
+ # Fact destroyed → fail
265
+ assert evaluate(
266
+ c.fail_condition, _ctx(tick=1000, kills=3, own_buildings=base_b[1:])
267
+ )
268
+ # Past after_ticks deadline → fail
269
+ assert evaluate(
270
+ c.fail_condition, _ctx(tick=1600, kills=0, own_buildings=base_b)
271
+ )
272
+ # Within deadline, fact alive → not fail
273
+ assert not evaluate(
274
+ c.fail_condition, _ctx(tick=1000, kills=0, own_buildings=base_b)
275
+ )
276
+
277
+
278
+ # ── engine-driven: every lazy / wrong policy LOSES, intended WINS ───
279
+
280
+
281
+ @pytest.mark.parametrize("level", LEVELS)
282
+ @pytest.mark.parametrize("seed", SEEDS)
283
+ def test_stall_loses_every_tier_and_seed(level, seed):
284
+ """Observe-only ⇒ no kills + rush reaches fact ⇒ real LOSS,
285
+ not a draw. The `after_ticks` clause is reachable inside
286
+ max_turns; the `building_count_gte:fact` clause fires when
287
+ the rush razes the fact."""
288
+ c = compile_level(load_pack(PACK), level)
289
+ r = run_level(c, _stall, seed=seed)
290
+ assert r.outcome == "loss", (
291
+ f"{level}/seed{seed}: stall must LOSE (no defenders, "
292
+ f"fact razed or deadline bites); got {r.outcome} "
293
+ f"tick={r.signals.game_tick} kills={r.signals.units_killed}"
294
+ )
295
+
296
+
297
+ @pytest.mark.parametrize("level", LEVELS)
298
+ @pytest.mark.parametrize("seed", SEEDS)
299
+ def test_build_weap_first_loses_every_tier_and_seed(level, seed):
300
+ """Build second war factory first ⇒ $2000 sink + placement
301
+ turn + queue lockout ⇒ first tank fields too late ⇒ rush
302
+ reaches the fact ⇒ LOSS."""
303
+ c = compile_level(load_pack(PACK), level)
304
+ r = run_level(c, _build_weap_first, seed=seed)
305
+ assert r.outcome == "loss", (
306
+ f"{level}/seed{seed}: build-weap-first must LOSE (cash drain "
307
+ f"+ queue lockout ⇒ fact razed); got {r.outcome} "
308
+ f"tick={r.signals.game_tick} kills={r.signals.units_killed}"
309
+ )
310
+
311
+
312
+ @pytest.mark.parametrize("level", LEVELS)
313
+ @pytest.mark.parametrize("seed", SEEDS)
314
+ def test_buy_now_wins_every_tier_and_seed(level, seed):
315
+ """The intended capability — queue 2tnk × 4 from the pre-placed
316
+ war factory and let them auto-fire on the rush. Wins every
317
+ tier and every hard seed, well inside the deadline."""
318
+ c = compile_level(load_pack(PACK), level)
319
+ r = run_level(c, _buy_now, seed=seed)
320
+ assert r.outcome == "win", (
321
+ f"{level}/seed{seed}: buy-now (queue 2tnk × 4) must WIN; "
322
+ f"got {r.outcome} tick={r.signals.game_tick} "
323
+ f"kills={r.signals.units_killed}"
324
+ )
325
+
326
+
327
+ # ── determinism ─────────────────────────────────────────────────────
328
+
329
+
330
+ def test_buy_now_run_is_deterministic_per_seed():
331
+ """Same seed, same policy → identical outcome / kills / turns."""
332
+ c = compile_level(load_pack(PACK), "medium")
333
+ a = run_level(c, _buy_now, seed=2)
334
+ b = run_level(c, _buy_now, seed=2)
335
+ assert (a.outcome, a.turns, a.signals.units_killed) == (
336
+ b.outcome, b.turns, b.signals.units_killed
337
+ )
tests/test_hard_tier.py CHANGED
@@ -252,6 +252,23 @@ UPGRADED = [
252
  # the actual base latitude.
253
  "def-multi-direction",
254
  "def-surprise-flank-react",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
255
  # Wave-5 Group F cash-reserve / treasury management pack (SC2 cash
256
  # overflow / financial runway / operational reserve anchor). Hard
257
  # tier defines two agent spawn_point groups (NORTH base y=14 /
@@ -635,6 +652,21 @@ UPGRADED = [
635
  # bearing. Hard discriminates via kill-speed (tight within_ticks
636
  # 1200) + spawn-variation generalisation.
637
  "combat-tank-vs-tank-engagement",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
638
  ]
639
 
640
  # Consciously NOT spawn-varied, with the reason (keeps the curation
 
252
  # the actual base latitude.
253
  "def-multi-direction",
254
  "def-surprise-flank-react",
255
+ # Wave-7 REASONING pack — intel-driven adaptive defense
256
+ # (PlanBench replanning / military reactive defense / intel-driven
257
+ # facility hardening anchor). Sibling/inverse of
258
+ # def-position-expected-direction (which DECLARES the threat axis
259
+ # in the brief): this pack HIDES the axis, so the agent must
260
+ # SCOUT to find the enemy's forward outpost before committing
261
+ # pillboxes on that lane. Hard tier defines two agent
262
+ # spawn_point groups (NORTH base y=16 / SOUTH base y=24) round-
263
+ # robined by seed; both north (y=4) and south (y=36) enemy
264
+ # outposts always place (enemy actors don't honour spawn_point —
265
+ # CLAUDE.md), so each seed faces the SAME scout-then-fortify
266
+ # discipline from a flipped base latitude. A memorised "always
267
+ # scout/defend NORTH" opening WINS the NORTH-spawn seeds by
268
+ # coincidence but LOSES the SOUTH-spawn seeds (clause 1 of the
269
+ # then-chain never latches AND the south rush razes the
270
+ # SOUTH-spawn fact), so no fixed-axis opening generalises.
271
+ "def-position-revealed-direction",
272
  # Wave-5 Group F cash-reserve / treasury management pack (SC2 cash
273
  # overflow / financial runway / operational reserve anchor). Hard
274
  # tier defines two agent spawn_point groups (NORTH base y=14 /
 
652
  # bearing. Hard discriminates via kill-speed (tight within_ticks
653
  # 1200) + spawn-variation generalisation.
654
  "combat-tank-vs-tank-engagement",
655
+ # Wave-7 REASONING pack — CAPEX-vs-OPEX capital allocation under
656
+ # rush pressure (PlanBench resource-allocation / financial
657
+ # allocation / SC2 production: pump units now vs lay second
658
+ # factory anchor). Starting cash ($3400) funds EITHER 4× 2tnk
659
+ # (immediate force, OPEX) OR 1× weap second factory (long-run
660
+ # capacity, CAPEX). Rusher band is incoming at t=0 ⇒ buy-now
661
+ # WINS, build-weap-first LOSES. Hard tier defines two agent
662
+ # spawn_point groups (NORTH base y=14 / SOUTH base y=26) round-
663
+ # robined by seed; rusher bands always place at BOTH latitudes
664
+ # (enemy actors don't honour spawn_point — CLAUDE.md) but the
665
+ # rusher bot targets the agent centroid, so the active threat
666
+ # axis flips per seed. Inert HoldFire e1 per group surfaces the
667
+ # spawn variation via units_summary (the buy-vs-build base is
668
+ # otherwise building-only).
669
+ "econ-buy-vs-build-decision",
670
  ]
671
 
672
  # Consciously NOT spawn-varied, with the reason (keeps the curation