yxc20098 commited on
Commit
771d9fe
·
1 Parent(s): 897ed61

feat(scenario): econ-silo-vs-spend — overflow cap management (SC2 silo / working-capital anchor)

Browse files

Wave-7 REASONING pack: cash starts NEAR the proc-only storage cap
($1800 vs the 2000 cap) on a kit of fact + proc + powr + harv.
Income accrues; harvested ore beyond the cap is LOST on deposit.
Three legitimate any_of paths legitimately resolve the cap pressure:

* SILO — build silo (cost 150, prereq proc); cap → 5000
[has_building:silo]
* KILLS — build tent (400, prereq powr — pre-placed) + e1×K;
attack the pre-placed HoldFire garrison
[units_killed_gte:K]
* PBOX — build tent + 2× pbox (600 ea); fortify the working capital
[building_count_gte:{type:pbox,n:2}]

HOARD (only observe / only harvest) never latches the any_of clause
→ timeout LOSS on every tier and every seed.

Win = ALL OF (any_of paths) AND economy_value_gte AND has_building:fact
AND within_ticks.
Fail = any_of [after_ticks T+1, not has_building:fact, not has_building:proc].

Hard tier defines 2 agent spawn_point groups (NORTH base y=14 /
SOUTH base y=26) round-robined by seed; neutral mines and the enemy
garrison duplicate at both latitudes (CLAUDE.md: enemy/neutral
actors don't honour spawn_point). Registered in
tests/test_hard_tier.py UPGRADED. Pack passes the no-cheat bar
(scripted validation, no model needed):

stall LOSES every tier/seed (3 tiers × 4 seeds)
pure_hold LOSES every tier/seed
silo path WINS every tier/seed

Benchmark anchors: SC2 silo management; working-capital allocation;
inventory cap; financial cap management; FIFO perishable inventory cap.

openra_bench/scenarios/packs/econ-silo-vs-spend.yaml ADDED
@@ -0,0 +1,341 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Validate: python -m openra_bench.scenarios.validate packs/econ-silo-vs-spend.yaml
2
+ # See ../CONTRIBUTING.md for the full win-condition grammar.
3
+ #
4
+ # ECON-SILO-VS-SPEND (Wave-7 — capital-allocation decision under
5
+ # storage-cap pressure: either SPEND immediately on units / defenses
6
+ # that "use" the working capital, OR BUILD A SILO to enlarge the
7
+ # storage cap, but DO NOT HOARD).
8
+ #
9
+ # Real-world anchor: SC2 silo management; working-capital allocation;
10
+ # inventory cap; FIFO perishable inventory cap (capped storage means
11
+ # each new unit of income MUST be acted on or it spoils on deposit).
12
+ #
13
+ # Pack frame: the agent starts with a minimum revenue kit (1× fact,
14
+ # 1× proc, 1× powr, 1× harv, near a patch) and a starting cash buffer
15
+ # NEAR the storage cap. The storage cap on stored ore is set by
16
+ # buildings: proc adds 2000, silo adds 3000 (OpenRA-Rust/openra-sim/
17
+ # src/world.rs:3380-3392 `player_storage_capacity`). With proc only,
18
+ # cap = 2000; starting cash 1800 leaves only $200 of cash headroom
19
+ # before stored-ore overflow on deposit starts losing income.
20
+ #
21
+ # The model's decision each turn: act on the accumulating capital. The
22
+ # any_of clause has THREE legitimate paths (and one inadmissible play):
23
+ # * SILO: build('silo') (cost 150, prereq proc, no powr needed) →
24
+ # has_building:silo latches. The cheap conservative hedge —
25
+ # enlarges the storage cap so harvested ore doesn't spoil.
26
+ # * KILLS: build tent (400, prereq powr — already pre-placed!) +
27
+ # e1×K (100 each), engage the pre-placed enemy garrison at the
28
+ # central latitude → units_killed_gte:K latches. The aggressive
29
+ # deployment of capital into productive combat assets.
30
+ # * PBOX-LINE: build tent (400, prereq powr) + 2× pbox (600 each =
31
+ # 1200) → building_count_gte:{type:pbox,n:2} latches. The
32
+ # defensive-fixed-asset path: turn the working capital into
33
+ # fortifications. Same total spend (~1600) as the kills path.
34
+ # * HOARD = none of the above: only observe / only harvest leaves
35
+ # cash piling up; stored-ore saturates the proc cap and overflows;
36
+ # the any_of clause never latches → LOSS on the clock.
37
+ #
38
+ # The capability under test (REASONING): recognise that idle capital
39
+ # spoils at the cap — DO SOMETHING with it — and pick a path (silo OR
40
+ # spend on kills OR spend on pbox). All three paths are legitimate;
41
+ # the wrong answer is to hoard.
42
+ #
43
+ # ENGINE FACTS verified (CLAUDE.md):
44
+ # 1. `silo` cost 150, prereq `proc` (vendor/OpenRA/mods/ra/rules/
45
+ # structures.yaml SILO). Loaded into the ruleset at sim init by
46
+ # gamerules.rs::from_ruleset — buildable via Command.build("silo")
47
+ # + Command.place_building("silo", x, y) once `proc` is pre-placed.
48
+ # 2. Storage cap (`resources` cap; harvested ore beyond cap is LOST on
49
+ # deposit — world.rs:1706-1710 `(r + value).min(cap)`):
50
+ # proc = +2000
51
+ # silo = +3000
52
+ # Pre-placed proc → cap 2000. Adding silo → cap 5000.
53
+ # 3. Drain: 10 stored-ore units → cash per tick (world.rs:2937
54
+ # RESOURCE_DRAIN_PER_TICK). A single harv during unload deposits
55
+ # 25 ore/tick (resource_value Ore=25, world.rs:4122-4128, with
56
+ # BaleUnloadDelay=1). With ONE harv the `resources` pool oscillates
57
+ # 0..~300; with TWO harvs the pool climbs faster and approaches the
58
+ # 2000 proc cap under sustained throughput — at which point
59
+ # overflow loses income; the silo (cap +3000) is the cheap fix.
60
+ # 4. `units_killed_gte` increments when an enemy unit dies to an
61
+ # agent-owned weapon. The agent has NO weapons pre-placed (proc +
62
+ # harv + fact + powr are all unarmed) so killing requires building
63
+ # tent (400, prereq powr — pre-placed) + e1 (100, prereq tent)
64
+ # and engaging the placed enemy garrison.
65
+ # 5. `pbox` cost 600, prereq `tent` (CLAUDE.md — pbox cost 600 not
66
+ # 400). Two pbox = 1200 cash on top of tent 400 = 1600 total spend
67
+ # to latch the building_count_gte clause. Same order of magnitude
68
+ # as the kills path (the silo path is the cheap floor at 150).
69
+ # 6. Pre-placed enemy garrison is `stance: 0` (HoldFire) so the
70
+ # enemies DO NOT attack the agent's fact (no fail-by-fact-loss for
71
+ # the hoarder); but they CAN be killed by agent-built attackers
72
+ # for the KILLS path. The far inert enemy `fact` marker prevents
73
+ # engine auto-`done` via the "all enemies eliminated" path
74
+ # (CLAUDE.md auto-done footgun) so a non-finisher LOSES on the
75
+ # deadline (never DRAWS).
76
+ # 7. `has_building:silo` is safe as a non-key landmark predicate (we
77
+ # only key it as a WIN clause, not a destruction objective — the
78
+ # CLAUDE.md "silo is NOT MustBeDestroyed" footgun is about using
79
+ # silo as an *enemy* destruction landmark, not about owning one).
80
+ # 8. Tick alignment: engine advances ~90 ticks per decision turn so
81
+ # within_ticks ≤ 93 + 90·(max_turns − 1) (CLAUDE.md). fail uses
82
+ # after_ticks = within_ticks + 1 so a non-finisher LOSES.
83
+ # 9. spawn_point filter on agent actors only (CLAUDE.md) — neutral
84
+ # `mine` actors and enemy garrison `e1` actors always place; we
85
+ # duplicate them where needed for spawn-symmetric throughput on
86
+ # hard tier.
87
+ # 10. Fail clause includes `not has_building:proc` (per spec) — losing
88
+ # the refinery collapses both income AND storage cap to 0 (cap
89
+ # becomes the small per-player base), so a play that lets proc die
90
+ # loses regardless of cash on hand.
91
+
92
+ meta:
93
+ id: econ-silo-vs-spend
94
+ title: 'Silo vs Spend — Storage-Cap Working-Capital Allocation'
95
+ capability: reasoning
96
+ real_world_meaning: >
97
+ Capital-allocation under a perishable storage cap. The agent runs
98
+ a revenue kit (fact + proc + powr + harv on an ore patch) whose
99
+ income flows into a CAPPED storage pool (the refinery store):
100
+ once full, new harvested ore is lost on deposit. The decision
101
+ every turn is what to do with the working capital that's
102
+ accumulating: either SPEND it on productive combat assets (build
103
+ infantry, kill the enemy garrison), SPEND it on fixed defensive
104
+ assets (build a pbox line), OR BUILD A SILO so the storage cap
105
+ grows and ore keeps banking. All three paths legitimately resolve
106
+ the cap pressure; the only failure mode is HOARDING — observing
107
+ without building or killing, which leaves capacity to spoil and
108
+ never satisfies the "use the capital" mandate (the any_of clause).
109
+ robotics_analogue: >
110
+ Working-capital management for a robotic operations stack with a
111
+ FIFO inventory cap: a collector pipeline deposits perishable goods
112
+ into a fixed-capacity warehouse, and a continuous spend channel
113
+ converts warehouse stock into bank capital. The operator must
114
+ EITHER deploy the bank capital on productive assets (extra
115
+ collectors, defenders), build fixed infrastructure (fortified
116
+ pickets), OR expand the warehouse (build storage so each new
117
+ collection still has somewhere to go). Letting capital sit idle
118
+ while the warehouse fills loses goods at the cap — the
119
+ perishable-inventory / cash-conversion-cycle discipline.
120
+ author: wave7-econ-silo-seed
121
+ benchmark_anchor:
122
+ - SC2 silo management
123
+ - working-capital allocation
124
+ - inventory cap
125
+ - financial cap management
126
+ - "FIFO perishable inventory cap"
127
+
128
+ base_map: rush-hour-arena
129
+ starting_cash: 1800
130
+
131
+ base:
132
+ agent: {faction: allies}
133
+ enemy: {faction: soviet, cash: 0}
134
+ # Spec-required toolset for the silo-or-spend decision: observe (read
135
+ # cash + resources + own_buildings), build + place_building (queue
136
+ # silo OR tent+e1 OR tent+pbox), harvest + move_units (sustain
137
+ # income + commit attackers), attack_unit / attack_move (drive
138
+ # infantry into the garrison for the KILLS path), stop.
139
+ tools:
140
+ - observe
141
+ - build
142
+ - place_building
143
+ - harvest
144
+ - move_units
145
+ - attack_unit
146
+ - attack_move
147
+ - stop
148
+ spawn_mcvs: false
149
+ planning: true
150
+ termination: {max_ticks: 8000}
151
+ actors: []
152
+
153
+ levels:
154
+ # ── EASY ─────────────────────────────────────────────────────────
155
+ # Loose clock + low income (1 harv on 1 patch) + low bars. The
156
+ # decision is the bare skill: realise that doing NOTHING with the
157
+ # capital fails the any_of clause. The cheap silo (cost 150) is the
158
+ # canonical floor and is solvable with the starting buffer alone.
159
+ # KILLS and PBOX-LINE are also possible (build tent + e1×1, or tent
160
+ # + 2× pbox) but slower; the SILO path is the easy floor.
161
+ # max_turns 20 → ceiling 93 + 90·19 = 1803 > 1800 ✓.
162
+ easy:
163
+ description: >
164
+ You own a Construction Yard ('fact'), an Ore Refinery ('proc'),
165
+ a Power Plant ('powr'), and one Ore Truck ('harv') on an ore
166
+ patch. Starting cash is $1800 (near the storage cap) and income
167
+ flows from the harvester. Your STORAGE CAP for harvested ore is
168
+ 2000 (the proc alone); a SILO ('silo', cost 150, prereq proc)
169
+ adds another 3000 cap and is the cheap fix. Win = ALL OF
170
+ (((you own a 'silo') OR (you have killed ≥1 enemy unit) OR
171
+ (you own ≥2 'pbox')) AND (economy_value ≥ 1800 — cash + stored
172
+ ore) AND (you still own a 'fact') AND (current tick ≤ 1800)).
173
+ The decision is capital-allocation: build('silo') + place; OR
174
+ build tent (prereq powr — pre-placed!) then e1 and attack the
175
+ garrison at (40,18); OR build tent + 2× pbox. Doing NONE of
176
+ these (only `observe`, or only `harvest`) leaves you with cash
177
+ but no silo, no kills, no pbox → the any_of clause never
178
+ latches → LOSS on the clock.
179
+ starting_cash: 1800
180
+ overrides:
181
+ actors:
182
+ - {type: fact, owner: agent, position: [8, 18]}
183
+ - {type: proc, owner: agent, position: [12, 18]}
184
+ - {type: powr, owner: agent, position: [10, 20]}
185
+ - {type: harv, owner: agent, position: [14, 18]}
186
+ - {type: mine, owner: neutral, position: [22, 18]}
187
+ # Killable enemy garrison (HoldFire so they don't aggro the
188
+ # fact — the hoarder cannot lose by base destruction, the only
189
+ # fail vector is the clock). The agent must build attackers
190
+ # and engage to score kills for the KILLS path.
191
+ - {type: e1, owner: enemy, position: [40, 18], stance: 0}
192
+ - {type: e1, owner: enemy, position: [40, 19], stance: 0}
193
+ # Persistent far enemy marker prevents engine auto-`done` via
194
+ # the all-enemies-eliminated path (the garrison alone is a
195
+ # MustBeDestroyed enemy and a KILLS policy that wipes it could
196
+ # otherwise auto-end before the win/fail predicate evaluates).
197
+ - {type: fact, owner: enemy, position: [120, 20]}
198
+ win_condition:
199
+ all_of:
200
+ - any_of:
201
+ - has_building: silo
202
+ - units_killed_gte: 1
203
+ - building_count_gte: {type: pbox, n: 2}
204
+ - economy_value_gte: 1800
205
+ - has_building: fact
206
+ - within_ticks: 1800
207
+ fail_condition:
208
+ any_of:
209
+ - after_ticks: 1801
210
+ - not: {has_building: fact}
211
+ - not: {has_building: proc}
212
+ max_turns: 20
213
+
214
+ # ── MEDIUM ───────────────────────────────────────────────────────
215
+ # +1 controlled variable: HEAVY income (2 harvs on 2 patches) so
216
+ # the storage-cap pressure is real — sustained throughput pushes
217
+ # the `resources` pool toward the proc cap of 2000 and overflow
218
+ # starts losing income. The any_of bar bumps to K=2 kills (and
219
+ # pbox count stays at 2 — pbox is the steady defensive lane). The
220
+ # economy_value bar rises to 2200 so a hoarder cannot quietly
221
+ # satisfy it via cash alone within the tightened clock; either
222
+ # spend or silo is structurally required to CLEAR the any_of
223
+ # clause. max_turns 14 → ceiling 93 + 90·13 = 1263 > 1260 ✓.
224
+ medium:
225
+ description: >
226
+ Same kit as easy (fact + proc + powr + harv) but with TWO
227
+ harvesters on TWO ore patches — heavy throughput. Storage cap
228
+ is still 2000 (proc only); under sustained two-harv income the
229
+ stored-ore pool saturates the cap, at which point new harvested
230
+ ore is LOST on deposit. Decision: build a 'silo' (cost 150,
231
+ prereq proc) to raise the cap to 5000, OR build tent + ≥2 e1
232
+ and assault the garrison at (40,18) for ≥2 kills, OR build
233
+ tent + 2× pbox to latch the building_count_gte:pbox=2 clause.
234
+ Win = ALL OF (((silo) OR (≥2 kills) OR (≥2 pbox)) AND
235
+ economy_value ≥ 2200 AND own fact AND tick ≤ 1260). Stalling /
236
+ harvest-only never satisfies the any_of clause → LOSS on the
237
+ clock. After the silo build (150), $1800 starting cash + ~190
238
+ cr/turn from two harvs × 14 turns ≈ $4400 leaves ample
239
+ headroom above the 2200 EV bar; the KILLS chain (tent 400 +
240
+ 2× e1 100 = 600) and the PBOX-LINE chain (tent 400 + 2× pbox
241
+ 600 = 1600) likewise clear the bar from sustained income.
242
+ starting_cash: 1800
243
+ overrides:
244
+ actors:
245
+ - {type: fact, owner: agent, position: [8, 18]}
246
+ - {type: proc, owner: agent, position: [12, 18]}
247
+ - {type: powr, owner: agent, position: [10, 20]}
248
+ - {type: harv, owner: agent, position: [14, 18]}
249
+ - {type: harv, owner: agent, position: [14, 20]}
250
+ - {type: mine, owner: neutral, position: [22, 18]}
251
+ - {type: mine, owner: neutral, position: [22, 20]}
252
+ # Three e1s now (K=2 kills required); HoldFire so no base
253
+ # threat. KILLS path: ≥2 of these must die to agent attackers.
254
+ - {type: e1, owner: enemy, position: [40, 18], stance: 0}
255
+ - {type: e1, owner: enemy, position: [40, 19], stance: 0}
256
+ - {type: e1, owner: enemy, position: [40, 20], stance: 0}
257
+ - {type: fact, owner: enemy, position: [120, 20]}
258
+ win_condition:
259
+ all_of:
260
+ - any_of:
261
+ - has_building: silo
262
+ - units_killed_gte: 2
263
+ - building_count_gte: {type: pbox, n: 2}
264
+ - economy_value_gte: 2200
265
+ - has_building: fact
266
+ - within_ticks: 1260
267
+ fail_condition:
268
+ any_of:
269
+ - after_ticks: 1261
270
+ - not: {has_building: fact}
271
+ - not: {has_building: proc}
272
+ max_turns: 14
273
+
274
+ # ── HARD ─────────────────────────────────────────────────────────
275
+ # +1 controlled variable on top of medium: TWO agent spawn_point
276
+ # groups (NORTH base y=14 / SOUTH base y=26) round-robined by seed.
277
+ # Heavy income preserved (2 harvs / 2 patches per spawn); cap bar
278
+ # held at 2200 EV; the same any_of clause (silo OR ≥2 kills OR ≥2
279
+ # pbox). A memorised "place silo at (14,18)" opening cannot
280
+ # generalise — the placement cell must be near the *actual* fact
281
+ # each seed. Enemy e1 garrison + neutral mines duplicated at both
282
+ # spawn rows because neutral / enemy actors don't honour
283
+ # spawn_point filter (CLAUDE.md engine footgun). max_turns 14 →
284
+ # ceiling 1263 > 1260 ✓.
285
+ hard:
286
+ description: >
287
+ Same any_of decision (silo OR ≥2 kills OR ≥2 pbox) and the
288
+ same heavy income (2 harvs / 2 patches), but your base
289
+ orientation varies by seed: NORTH base (y=14) or SOUTH base
290
+ (y=26). Each spawn has its own near-patch pair on its row. The
291
+ silo (or pbox, or tent staging cell) must be placed near your
292
+ *actual* fact (a memorised cell from the other spawn misses on
293
+ the wrong seed). The garrison sits on the central latitude
294
+ (y≈19..21) so attackers travel a similar distance from either
295
+ spawn. Hoard (no silo, no army, no pbox, only observe /
296
+ harvest) loses on the any_of clause; intended silo or spend
297
+ WIN.
298
+ starting_cash: 1800
299
+ overrides:
300
+ actors:
301
+ # ── spawn_point 0: NORTH base (y=14) ───────────────────────
302
+ - {type: fact, owner: agent, position: [8, 14], spawn_point: 0}
303
+ - {type: proc, owner: agent, position: [12, 14], spawn_point: 0}
304
+ - {type: powr, owner: agent, position: [10, 12], spawn_point: 0}
305
+ - {type: harv, owner: agent, position: [14, 14], spawn_point: 0}
306
+ - {type: harv, owner: agent, position: [14, 16], spawn_point: 0}
307
+ # ── spawn_point 1: SOUTH base (y=26) ───────────────────────
308
+ - {type: fact, owner: agent, position: [8, 26], spawn_point: 1}
309
+ - {type: proc, owner: agent, position: [12, 26], spawn_point: 1}
310
+ - {type: powr, owner: agent, position: [10, 28], spawn_point: 1}
311
+ - {type: harv, owner: agent, position: [14, 26], spawn_point: 1}
312
+ - {type: harv, owner: agent, position: [14, 24], spawn_point: 1}
313
+ # Neutral mines and enemy actors always place regardless of
314
+ # the spawn_point filter (CLAUDE.md). Duplicate the patch
315
+ # pair at each spawn row so per-spawn throughput is symmetric.
316
+ - {type: mine, owner: neutral, position: [22, 14]}
317
+ - {type: mine, owner: neutral, position: [22, 16]}
318
+ - {type: mine, owner: neutral, position: [22, 26]}
319
+ - {type: mine, owner: neutral, position: [22, 24]}
320
+ # Killable enemy garrison on the centre latitude (y≈20) so
321
+ # the KILLS path's attack distance is symmetric per spawn.
322
+ # HoldFire so they don't aggro either base.
323
+ - {type: e1, owner: enemy, position: [40, 20], stance: 0}
324
+ - {type: e1, owner: enemy, position: [40, 21], stance: 0}
325
+ - {type: e1, owner: enemy, position: [40, 19], stance: 0}
326
+ - {type: fact, owner: enemy, position: [120, 20]}
327
+ win_condition:
328
+ all_of:
329
+ - any_of:
330
+ - has_building: silo
331
+ - units_killed_gte: 2
332
+ - building_count_gte: {type: pbox, n: 2}
333
+ - economy_value_gte: 2200
334
+ - has_building: fact
335
+ - within_ticks: 1260
336
+ fail_condition:
337
+ any_of:
338
+ - after_ticks: 1261
339
+ - not: {has_building: fact}
340
+ - not: {has_building: proc}
341
+ max_turns: 14
tests/test_econ_silo_vs_spend.py ADDED
@@ -0,0 +1,392 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """econ-silo-vs-spend — REASONING capability validation.
2
+
3
+ Real-world anchor: SC2 silo management; working-capital allocation;
4
+ FIFO perishable inventory cap. The agent starts with a minimum
5
+ revenue kit (fact + proc + powr + harv on an ore patch) and a starting
6
+ cash buffer NEAR the proc-only storage cap (1800 vs 2000). Income
7
+ accrues. The any_of clause has three legitimate spend-or-build paths:
8
+
9
+ * SILO: build('silo') (cost 150, prereq proc) — has_building:silo
10
+ * KILLS: build tent (400, prereq powr — pre-placed) + e1×K, attack
11
+ the pre-placed enemy garrison — units_killed_gte:K
12
+ * PBOX-LINE: build tent + 2× pbox (600 each) —
13
+ building_count_gte:{type:pbox,n:2}
14
+
15
+ Bar (CLAUDE.md "no defect, no cheat"):
16
+ - stall LOSES every tier / every hard seed (no silo, no kills, no
17
+ pbox → any_of unmet → timeout LOSS).
18
+ - pure-hold (harvest only, never build / never engage) LOSES every
19
+ tier / seed (same: any_of unmet, cash hoards into cap, ore
20
+ overflow spoils).
21
+ - intended silo policy WINS every tier / seed (silo is the cheap
22
+ floor at cost 150, latches has_building:silo).
23
+ - hard tier defines ≥2 agent spawn_point groups (NORTH base y=14 /
24
+ SOUTH base y=26) so a memorised opening cannot generalise.
25
+ """
26
+
27
+ from __future__ import annotations
28
+
29
+ from pathlib import Path
30
+
31
+ import pytest
32
+
33
+ pytest.importorskip("openra_train", reason="Rust env wheel not installed")
34
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
35
+
36
+ from openra_bench.eval_core import run_level
37
+ from openra_bench.scenarios import load_pack
38
+ from openra_bench.scenarios.loader import PACKS_DIR, compile_level
39
+ from openra_bench.scenarios.win_conditions import WinContext, evaluate
40
+
41
+ PACK = PACKS_DIR / "econ-silo-vs-spend.yaml"
42
+
43
+
44
+ # ── policies ────────────────────────────────────────────────────────
45
+
46
+
47
+ def _stall(rs, Command):
48
+ """No-op every turn. No silo, no kills, no pbox → any_of unmet →
49
+ timeout LOSS."""
50
+ return [Command.observe()]
51
+
52
+
53
+ def _pure_hold(rs, Command):
54
+ """Harvest only, never build / engage. Cash and stored ore
55
+ accumulate; cap overflow spoils harvested ore; the any_of clause
56
+ never latches (no silo, no kills, no pbox) → timeout LOSS."""
57
+ units = rs.get("units_summary", []) or []
58
+ harvs = [u for u in units if u.get("type") == "harv"]
59
+ if not harvs:
60
+ return [Command.observe()]
61
+ return [
62
+ Command.harvest([str(h["id"])], 22, int(h["cell_y"])) for h in harvs
63
+ ]
64
+
65
+
66
+ def _intended_silo(rs, Command):
67
+ """Build a silo (once) and keep harvesting. has_building:silo
68
+ latches the any_of clause; the silo doubles the cap to 5000 so
69
+ no income spoils; passive income clears the economy_value bar."""
70
+ units = rs.get("units_summary", []) or []
71
+ harvs = [u for u in units if u.get("type") == "harv"]
72
+ bldgs = rs.get("own_buildings") or []
73
+
74
+ def _btype(b):
75
+ if isinstance(b, dict):
76
+ return b.get("type")
77
+ if isinstance(b, (list, tuple)) and b:
78
+ return b[0]
79
+ return None
80
+
81
+ has_silo = any(_btype(b) == "silo" for b in bldgs)
82
+ facts = [b for b in bldgs if _btype(b) == "fact"]
83
+ cmds = []
84
+ for h in harvs:
85
+ cmds.append(Command.harvest([str(h["id"])], 22, int(h["cell_y"])))
86
+ if not has_silo:
87
+ cmds.append(Command.build("silo"))
88
+ if facts:
89
+ f = facts[0]
90
+ fx = f.get("cell_x") if isinstance(f, dict) else f[1]
91
+ fy = f.get("cell_y") if isinstance(f, dict) else f[2]
92
+ cmds.append(Command.place_building("silo", int(fx) + 2, int(fy) + 3))
93
+ else:
94
+ cmds.append(Command.place_building("silo", 10, 21))
95
+ return cmds
96
+
97
+
98
+ # ── helpers ─────────────────────────────────────────────────────────
99
+
100
+
101
+ def _run(level, policy, seed=1):
102
+ c = compile_level(load_pack(PACK), level)
103
+ assert c.map_supported, "rush-hour-arena terrain must be present"
104
+ return c, run_level(c, policy, seed=seed)
105
+
106
+
107
+ # ── structural ──────────────────────────────────────────────────────
108
+
109
+
110
+ def test_pack_loads_and_meta_active():
111
+ pack = load_pack(PACK)
112
+ assert pack.meta.id == "econ-silo-vs-spend"
113
+ assert pack.meta.capability == "reasoning"
114
+ assert pack.meta.real_world_meaning
115
+ assert pack.meta.robotics_analogue
116
+ anchors = " ".join(pack.meta.benchmark_anchor).lower()
117
+ # Anchor set required by Wave-7 spec.
118
+ assert "sc2 silo management" in anchors
119
+ assert "working-capital allocation" in anchors
120
+ assert "inventory cap" in anchors
121
+
122
+
123
+ def test_tools_include_required_set():
124
+ """Pack must declare the [observe, build, place_building, harvest,
125
+ move_units, attack_unit, attack_move, stop] toolset (the
126
+ silo-or-spend decision surface)."""
127
+ pack = load_pack(PACK)
128
+ tools = set(pack.base.get("tools", []) if isinstance(pack.base, dict) else [])
129
+ for required in (
130
+ "observe", "build", "place_building", "harvest",
131
+ "move_units", "attack_unit", "attack_move", "stop",
132
+ ):
133
+ assert required in tools, f"missing tool: {required!r}"
134
+
135
+
136
+ def test_starting_cash_near_storage_cap():
137
+ """Starting cash must be NEAR the proc-only storage cap (2000) so
138
+ overflow pressure bites quickly. The exact value is 1800."""
139
+ pack = load_pack(PACK)
140
+ # Pack-level default (overridden per-level to the same value).
141
+ assert pack.starting_cash == 1800
142
+ for lvl in ("easy", "medium", "hard"):
143
+ L = pack.levels[lvl]
144
+ # Cash is NEAR cap (within $300 of the 2000 proc cap).
145
+ assert L.starting_cash >= 1500
146
+ assert L.starting_cash <= 2000
147
+
148
+
149
+ def test_required_prebuild_kit_present_each_tier():
150
+ """Spec: pre-place 1× fact, 1× proc, 1× powr, 1× harv (no silos)."""
151
+ pack = load_pack(PACK)
152
+ for lvl in ("easy", "medium", "hard"):
153
+ c = compile_level(pack, lvl)
154
+ agent_actors = [a for a in c.scenario.actors if a.owner == "agent"]
155
+ types = [a.type for a in agent_actors]
156
+ for required in ("fact", "proc", "powr", "harv"):
157
+ assert required in types, (
158
+ f"{lvl}: missing pre-placed {required!r} (got types {types})"
159
+ )
160
+ # No silo pre-placed (the agent must build it).
161
+ assert "silo" not in types, f"{lvl}: silo must NOT be pre-placed"
162
+
163
+
164
+ def test_all_tiers_have_reachable_deadlines():
165
+ """tick-alignment idiom: within_ticks ≤ ceiling AND
166
+ after_ticks ≤ ceiling AND within_ticks + 1 == after_ticks (so a
167
+ non-finisher LOSES, not draws)."""
168
+ pack = load_pack(PACK)
169
+ for lvl in ("easy", "medium", "hard"):
170
+ L = pack.levels[lvl]
171
+ ceiling = 93 + 90 * (L.max_turns - 1)
172
+ wt = next(
173
+ int(c["within_ticks"])
174
+ for c in L.win_condition.model_dump()["all_of"]
175
+ if "within_ticks" in c
176
+ )
177
+ ft = next(
178
+ int(c["after_ticks"])
179
+ for c in L.fail_condition.model_dump()["any_of"]
180
+ if "after_ticks" in c
181
+ )
182
+ assert wt <= ceiling, f"{lvl}: within_ticks {wt} > ceiling {ceiling}"
183
+ assert ft <= ceiling, f"{lvl}: after_ticks {ft} > ceiling {ceiling}"
184
+ assert wt + 1 == ft, (
185
+ f"{lvl}: within_ticks {wt} / after_ticks {ft} mismatch "
186
+ "(non-finisher must LOSE one tick past the win deadline)"
187
+ )
188
+
189
+
190
+ def test_win_any_of_has_all_three_paths():
191
+ """Spec: win any_of must offer silo, kills, AND pbox paths."""
192
+ pack = load_pack(PACK)
193
+ for lvl in ("easy", "medium", "hard"):
194
+ L = pack.levels[lvl]
195
+ clauses = L.win_condition.model_dump()["all_of"]
196
+ any_of = next(c["any_of"] for c in clauses if "any_of" in c)
197
+ keys = set()
198
+ for clause in any_of:
199
+ keys |= set(clause.keys())
200
+ assert keys == {
201
+ "has_building", "units_killed_gte", "building_count_gte"
202
+ }, f"{lvl}: any_of must have silo / kills / pbox paths; got {keys}"
203
+
204
+
205
+ def test_fail_condition_present_on_every_tier():
206
+ pack = load_pack(PACK)
207
+ for lvl in ("easy", "medium", "hard"):
208
+ c = compile_level(pack, lvl)
209
+ assert c.fail_condition is not None, f"{lvl} needs a fail_condition"
210
+
211
+
212
+ def test_fail_clauses_include_proc_and_fact_and_timeout():
213
+ """Spec: fail = after_ticks T+1 OR not has_building:fact OR
214
+ not has_building:proc."""
215
+ pack = load_pack(PACK)
216
+ for lvl in ("easy", "medium", "hard"):
217
+ L = pack.levels[lvl]
218
+ clauses = L.fail_condition.model_dump()["any_of"]
219
+ has_after = any("after_ticks" in c for c in clauses)
220
+ has_not_fact = any(
221
+ isinstance(c.get("not"), dict)
222
+ and c["not"].get("has_building") == "fact"
223
+ for c in clauses
224
+ )
225
+ has_not_proc = any(
226
+ isinstance(c.get("not"), dict)
227
+ and c["not"].get("has_building") == "proc"
228
+ for c in clauses
229
+ )
230
+ assert has_after, f"{lvl}: fail missing after_ticks clause"
231
+ assert has_not_fact, f"{lvl}: fail missing not has_building:fact"
232
+ assert has_not_proc, f"{lvl}: fail missing not has_building:proc"
233
+
234
+
235
+ def test_hard_has_two_seed_driven_spawn_groups():
236
+ """Hard tier: ≥2 distinct agent spawn_point groups (per spec:
237
+ 'Hard ≥2 spawn_point groups')."""
238
+ c = compile_level(load_pack(PACK), "hard")
239
+ sp = {
240
+ (a.spawn_point if a.spawn_point is not None else 0)
241
+ for a in c.scenario.actors
242
+ if a.owner == "agent"
243
+ }
244
+ assert len(sp) >= 2, (
245
+ f"hard must define ≥2 agent spawn_point groups; got {sorted(sp)}"
246
+ )
247
+
248
+
249
+ # ── predicate-level (no engine) ─────────────────────────────────────
250
+
251
+
252
+ def _ctx(*, units=(), tick=500, cash=0, resources=0, own_buildings=(), killed=0):
253
+ import types
254
+ sig = types.SimpleNamespace(
255
+ game_tick=tick,
256
+ units_killed=killed,
257
+ units_lost=0,
258
+ cash=cash,
259
+ resources=resources,
260
+ own_buildings=list(own_buildings),
261
+ own_building_types={str(t).lower() for (t, _, _) in own_buildings},
262
+ enemies_seen_ids=set(),
263
+ enemy_buildings_seen_ids=set(),
264
+ )
265
+ return WinContext(signals=sig, render_state={"units_summary": list(units)})
266
+
267
+
268
+ def test_predicates_enforce_capability_silo_path():
269
+ """SILO path: has_building:silo latches the any_of clause."""
270
+ c = compile_level(load_pack(PACK), "medium")
271
+ kit_with_silo = [
272
+ ("fact", 8, 18), ("proc", 12, 18), ("powr", 10, 20),
273
+ ("silo", 10, 21),
274
+ ]
275
+ assert evaluate(
276
+ c.win_condition,
277
+ _ctx(tick=600, cash=2300, resources=0, own_buildings=kit_with_silo),
278
+ )
279
+
280
+
281
+ def test_predicates_enforce_capability_kills_path():
282
+ """KILLS path: units_killed_gte:2 latches the any_of clause."""
283
+ c = compile_level(load_pack(PACK), "medium")
284
+ kit_no_silo = [("fact", 8, 18), ("proc", 12, 18), ("powr", 10, 20)]
285
+ assert evaluate(
286
+ c.win_condition,
287
+ _ctx(
288
+ tick=600, cash=2300, resources=0,
289
+ own_buildings=kit_no_silo, killed=2,
290
+ ),
291
+ )
292
+
293
+
294
+ def test_predicates_enforce_capability_pbox_path():
295
+ """PBOX-LINE path: building_count_gte:{type:pbox,n:2} latches."""
296
+ c = compile_level(load_pack(PACK), "medium")
297
+ kit_with_pbox = [
298
+ ("fact", 8, 18), ("proc", 12, 18), ("powr", 10, 20),
299
+ ("tent", 14, 22), ("pbox", 16, 22), ("pbox", 16, 23),
300
+ ]
301
+ assert evaluate(
302
+ c.win_condition,
303
+ _ctx(tick=600, cash=2300, resources=0, own_buildings=kit_with_pbox),
304
+ )
305
+
306
+
307
+ def test_predicates_reject_hoard():
308
+ """HOARD (no silo, no kills, no pbox) — even with cash and EV
309
+ above the bar, the any_of clause is unmet → not a win."""
310
+ c = compile_level(load_pack(PACK), "medium")
311
+ kit_no_action = [("fact", 8, 18), ("proc", 12, 18), ("powr", 10, 20)]
312
+ assert not evaluate(
313
+ c.win_condition,
314
+ _ctx(tick=600, cash=5000, resources=0, own_buildings=kit_no_action),
315
+ )
316
+
317
+
318
+ def test_predicates_fail_on_proc_loss():
319
+ """proc destroyed → fail (per spec fail clause)."""
320
+ c = compile_level(load_pack(PACK), "medium")
321
+ no_proc = [("fact", 8, 18), ("powr", 10, 20), ("silo", 10, 21)]
322
+ assert evaluate(c.fail_condition, _ctx(tick=600, own_buildings=no_proc))
323
+
324
+
325
+ def test_predicates_fail_on_fact_loss():
326
+ """fact destroyed → fail (per spec fail clause)."""
327
+ c = compile_level(load_pack(PACK), "medium")
328
+ no_fact = [("proc", 12, 18), ("powr", 10, 20), ("silo", 10, 21)]
329
+ assert evaluate(c.fail_condition, _ctx(tick=600, own_buildings=no_fact))
330
+
331
+
332
+ def test_predicates_fail_on_timeout():
333
+ """tick past after_ticks → fail."""
334
+ c = compile_level(load_pack(PACK), "medium")
335
+ full_kit = [
336
+ ("fact", 8, 18), ("proc", 12, 18), ("powr", 10, 20),
337
+ ("silo", 10, 21),
338
+ ]
339
+ assert evaluate(
340
+ c.fail_condition,
341
+ _ctx(tick=1262, cash=3000, own_buildings=full_kit),
342
+ )
343
+
344
+
345
+ # ── engine-driven policies (the no-cheat bar) ───────────────────────
346
+
347
+
348
+ @pytest.mark.parametrize("level", ["easy", "medium", "hard"])
349
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
350
+ def test_stall_loses_every_tier_and_seed(level, seed):
351
+ """No silo, no kills, no pbox → any_of clause never latches →
352
+ timeout LOSS (hoarding policy must lose)."""
353
+ _, r = _run(level, _stall, seed=seed)
354
+ assert r.outcome == "loss", (
355
+ f"{level}/seed{seed}: stall must LOSE (hoarder); "
356
+ f"got {r.outcome} cash={r.signals.cash} turns={r.turns}"
357
+ )
358
+
359
+
360
+ @pytest.mark.parametrize("level", ["easy", "medium", "hard"])
361
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
362
+ def test_pure_hold_loses_every_tier_and_seed(level, seed):
363
+ """Harvest-only (no silo, no kills, no pbox) → any_of clause
364
+ never latches; stored ore overflows the proc cap as income piles
365
+ in; timeout LOSS."""
366
+ _, r = _run(level, _pure_hold, seed=seed)
367
+ assert r.outcome == "loss", (
368
+ f"{level}/seed{seed}: pure-hold must LOSE (still hoarding); "
369
+ f"got {r.outcome} cash={r.signals.cash} turns={r.turns}"
370
+ )
371
+
372
+
373
+ @pytest.mark.parametrize("level", ["easy", "medium", "hard"])
374
+ @pytest.mark.parametrize("seed", [1, 2, 3, 4])
375
+ def test_intended_silo_wins_every_tier_and_seed(level, seed):
376
+ """The intended capability — build a silo (cheap floor at $150)
377
+ while harvesting — WINS every tier and every hard seed."""
378
+ _, r = _run(level, _intended_silo, seed=seed)
379
+ assert r.outcome == "win", (
380
+ f"{level}/seed{seed}: silo discipline should WIN; "
381
+ f"got {r.outcome} cash={r.signals.cash} turns={r.turns}"
382
+ )
383
+
384
+
385
+ def test_outcomes_are_deterministic_per_seed():
386
+ """Same seed + same policy → identical outcome / cash / turn."""
387
+ c = compile_level(load_pack(PACK), "medium")
388
+ a = run_level(c, _intended_silo, seed=2)
389
+ b = run_level(c, _intended_silo, seed=2)
390
+ assert (a.outcome, a.turns, a.signals.cash) == (
391
+ b.outcome, b.turns, b.signals.cash
392
+ )
tests/test_hard_tier.py CHANGED
@@ -419,6 +419,21 @@ UPGRADED = [
419
  # y=20 so either spawn faces the same flank-vs-frontal decision
420
  # from a flipped bearing, and no memorised opening generalises.
421
  "combat-flanking-attack",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
422
  # Wave-7 combat-formation pack: military tank-wedge doctrine /
423
  # SC2 formation micro / combined-arms anchor. The agent commands
424
  # 5× 2tnk and must arrange them in a WEDGE (apex + 2 flankers
@@ -543,6 +558,19 @@ UPGRADED = [
543
  # (14,22)" opening cannot generalise — placement must be computed
544
  # relative to the actual fact each seed.
545
  "build-sequence-tech-fastest",
 
 
 
 
 
 
 
 
 
 
 
 
 
546
  ]
547
 
548
  # Consciously NOT spawn-varied, with the reason (keeps the curation
 
419
  # y=20 so either spawn faces the same flank-vs-frontal decision
420
  # from a flipped bearing, and no memorised opening generalises.
421
  "combat-flanking-attack",
422
+ # Wave-7 REASONING seed: military divide-and-conquer / defeat-in-
423
+ # detail / SMAC squad-isolation / CICERO splitting anchor. The
424
+ # agent commands 4× 2tnk vs TWO enemy clusters (N at y=15 / S at
425
+ # y=25, each 3× e3 + 1× 1tnk, stance:3). Engaging at the y=20
426
+ # midpoint puts the lead tank in range of BOTH clusters; the
427
+ # winning play is to flank well off-axis, isolate one cluster,
428
+ # destroy it, then pivot to the other. Hard tier defines two
429
+ # agent spawn_point groups (NORTH staging y=10..13 / SOUTH
430
+ # staging y=27..30) round-robined by seed; the two clusters are
431
+ # symmetric across y=20 so each spawn faces the same divide-and-
432
+ # conquer decision but the first-flank-target flips per seed
433
+ # (N spawn engages Cluster A first; S spawn engages Cluster B
434
+ # first) — a memorised "always flank Cluster A first" opening
435
+ # cannot generalise across seeds.
436
+ "combat-divide-and-conquer",
437
  # Wave-7 combat-formation pack: military tank-wedge doctrine /
438
  # SC2 formation micro / combined-arms anchor. The agent commands
439
  # 5× 2tnk and must arrange them in a WEDGE (apex + 2 flankers
 
558
  # (14,22)" opening cannot generalise — placement must be computed
559
  # relative to the actual fact each seed.
560
  "build-sequence-tech-fastest",
561
+ # Wave-7 REASONING econ pack — SC2 silo management / working-capital
562
+ # allocation / FIFO perishable inventory cap anchor. Cash starts NEAR
563
+ # the proc-only storage cap (1800 vs 2000); income accrues; the
564
+ # any_of clause is silo OR ≥K kills OR ≥2 pbox — three legitimate
565
+ # spend-or-build paths, none of which is "hoard". Hard tier defines
566
+ # two agent spawn_point groups (NORTH base y=14 / SOUTH base y=26)
567
+ # round-robined by seed; ore patches are duplicated at both
568
+ # latitudes so per-spawn throughput is symmetric (neutral / enemy
569
+ # actors don't honour spawn_point — CLAUDE.md). The garrison sits
570
+ # at y=19..21 so attack distance from either spawn is comparable,
571
+ # and a memorised "place silo at (14,18)" opening cannot generalise
572
+ # across seeds.
573
+ "econ-silo-vs-spend",
574
  ]
575
 
576
  # Consciously NOT spawn-varied, with the reason (keeps the curation