Spaces:
Running
Running
feat(scenario): econ-tech-vs-expand-decision — capital allocation tech building vs economic expansion (capex allocation)
Browse filesREASONING pack: one $2000 budget funds EITHER a second war factory
(TECH — a non-revenue capability building) OR a second harvester
(EXPAND — roughly doubles gathering income). The regime is fixed with
no near-term combat demand, so the win is an economy target by a
deadline: EXPAND compounds past the bar; sinking the budget into the
non-revenue tech building leaves the economy short. Stall and
build-tech LOSE; the early harvester buy WINS. Hard defines two
seed-driven spawn_point base groups.
openra_bench/scenarios/packs/econ-tech-vs-expand-decision.yaml
ADDED
|
@@ -0,0 +1,296 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# econ-tech-vs-expand-decision — capital allocation: tech building
|
| 2 |
+
# vs economic expansion (Wave-12).
|
| 3 |
+
#
|
| 4 |
+
# REASONING capability. One indivisible budget, two mutually
|
| 5 |
+
# exclusive ways to spend it:
|
| 6 |
+
#
|
| 7 |
+
# (A) TECH — build a war factory (weap, $2000): a capability
|
| 8 |
+
# investment. It unlocks the vehicle tech tier, but it is a
|
| 9 |
+
# NON-REVENUE structure — it generates ZERO income, and within
|
| 10 |
+
# the planning horizon of this regime it pays nothing back.
|
| 11 |
+
# (B) EXPAND — buy a SECOND ore harvester ($1100, the dominant
|
| 12 |
+
# slice of the budget) so the running refinery is fed by two
|
| 13 |
+
# collectors instead of one. A second harvester roughly DOUBLES
|
| 14 |
+
# gathering throughput; the extra income compounds over the
|
| 15 |
+
# whole episode.
|
| 16 |
+
#
|
| 17 |
+
# The regime is FIXED and the win predicate is an ECONOMY target
|
| 18 |
+
# (`economy_value_gte` — cash + stored ore — by a deadline). One
|
| 19 |
+
# answer is correct for this regime: there is no near-term combat
|
| 20 |
+
# demand the tech tier must satisfy (the only enemy is a token,
|
| 21 |
+
# far-off, passive garrison that never attacks inside the econ
|
| 22 |
+
# window), so the regime rewards COMPOUNDING the economy. EXPAND
|
| 23 |
+
# clears the EV bar; TECH sinks the budget into a structure that
|
| 24 |
+
# returns nothing and the economy never reaches the target → LOSS.
|
| 25 |
+
# This is the classic capex-allocation trap: do not buy capability
|
| 26 |
+
# the regime does not yet demand when the same budget, spent on
|
| 27 |
+
# capacity, compounds.
|
| 28 |
+
#
|
| 29 |
+
# Real-world anchors:
|
| 30 |
+
# • SC2 tech-vs-expand: the macro decision between teching up and
|
| 31 |
+
# taking another base — the right call is set by the opponent's
|
| 32 |
+
# timing. When no timing attack is incoming, expand.
|
| 33 |
+
# • PlanBench resource-allocation: pick the allocation that
|
| 34 |
+
# satisfies the goal, not merely some allocation.
|
| 35 |
+
# • Capex allocation / capital budgeting: a non-revenue capability
|
| 36 |
+
# purchase vs a capacity expansion whose throughput compounds —
|
| 37 |
+
# when the planning horizon has no demand the capability serves,
|
| 38 |
+
# the capacity investment dominates on NPV.
|
| 39 |
+
#
|
| 40 |
+
# Discrimination (four-policy bar — scripted, no model needed):
|
| 41 |
+
# • stall (observe-only): no income action — the economy idles at
|
| 42 |
+
# the starting reserve, far below the EV bar → the `after_ticks`
|
| 43 |
+
# deadline bites as a real reachable timeout LOSS.
|
| 44 |
+
# • build-tech (build + place the war factory, $2000): the budget
|
| 45 |
+
# is sunk into a non-revenue structure; with the reserve drained
|
| 46 |
+
# the single harvester's income alone cannot reach the EV bar →
|
| 47 |
+
# LOSS via `after_ticks`. A real reachable LOSS, not a draw.
|
| 48 |
+
# • under-spend / wrong-line (buy a non-revenue unit / partial
|
| 49 |
+
# spend): the economy still misses the bar → LOSS.
|
| 50 |
+
# • intended EXPAND (build('harv') early — a second harvester):
|
| 51 |
+
# gathering throughput roughly doubles; the compounded income
|
| 52 |
+
# clears the EV bar AND keeps the construction yard intact →
|
| 53 |
+
# WIN, comfortably inside the deadline.
|
| 54 |
+
#
|
| 55 |
+
# Engine facts used (CLAUDE.md / prompt_v2 codex / verified against
|
| 56 |
+
# the installed openra_train wheel, 2026-05-20):
|
| 57 |
+
# • harv ($1100): an ore harvester; build requires fact + powr +
|
| 58 |
+
# weap (the Vehicle queue host). The war factory is PRE-PLACED,
|
| 59 |
+
# so the harvester buy is actionable on turn 1 — the EXPAND
|
| 60 |
+
# option needs no prior tech step.
|
| 61 |
+
# • A SECOND weap is the TECH option: cost $2000, prereq proc
|
| 62 |
+
# (pre-placed). `place_building` has no adjacency requirement
|
| 63 |
+
# (CLAUDE.md) so the tech LOSS is from the cash sink, not from
|
| 64 |
+
# placement difficulty.
|
| 65 |
+
# • Income on rush-hour-arena from the near patches is ~1.1
|
| 66 |
+
# cash/tick per harvester after a ~450-tick warmup; a second
|
| 67 |
+
# harvester roughly doubles it. The EV bars are set ABOVE the
|
| 68 |
+
# one-harvester ceiling and ABOVE a tech-drained economy, but
|
| 69 |
+
# BELOW a two-harvester economy — so only EXPAND clears the bar.
|
| 70 |
+
# • `economy_value_gte` reads cash + stored ore; building the war
|
| 71 |
+
# factory deducts $2000 so the tech play visibly craters.
|
| 72 |
+
# • Tick alignment (~90 ticks/turn): within_ticks ≤ 93+90·
|
| 73 |
+
# (max_turns−1); fail after_ticks = within_ticks+1 ⇒ a
|
| 74 |
+
# non-finisher is a real reachable LOSS, never a DRAW.
|
| 75 |
+
# • A far inert passive enemy actor keeps the engine from
|
| 76 |
+
# auto-`done`ing on no-enemy-actors (CLAUDE.md). It is
|
| 77 |
+
# stance:0 and far from the base so it never influences the run
|
| 78 |
+
# — the "enemy-timing regime" is encoded by its irrelevance:
|
| 79 |
+
# no attack lands inside the econ window, so teching is
|
| 80 |
+
# premature.
|
| 81 |
+
# • Hard tier: two agent spawn_point groups (NORTH y=10..14 /
|
| 82 |
+
# SOUTH y=28..32), each carrying its own fact+proc+powr+weap+
|
| 83 |
+
# harv and a symmetric near-patch pair so the income economics
|
| 84 |
+
# are identical per spawn — a memorised "build at (x,y)"
|
| 85 |
+
# opening cannot generalise (spawn_point filters AGENT actors
|
| 86 |
+
# only; neutral mines place every seed).
|
| 87 |
+
#
|
| 88 |
+
# Validate (no model / no network):
|
| 89 |
+
# python3 -m pytest tests/test_econ_tech_vs_expand_decision.py -q
|
| 90 |
+
|
| 91 |
+
meta:
|
| 92 |
+
id: econ-tech-vs-expand-decision
|
| 93 |
+
title: 'Capital Allocation — Tech Building or Economic Expansion?'
|
| 94 |
+
capability: reasoning
|
| 95 |
+
real_world_meaning: >
|
| 96 |
+
One indivisible budget funds EITHER a tech building (a war
|
| 97 |
+
factory — a capability investment that generates no income) OR an
|
| 98 |
+
economic expansion (a second ore harvester that roughly doubles
|
| 99 |
+
gathering throughput). The regime is fixed: there is no near-term
|
| 100 |
+
combat demand the tech tier must satisfy, so the budget compounds
|
| 101 |
+
best as capacity. The win is an economy target by a deadline —
|
| 102 |
+
expanding clears it; sinking the budget into the non-revenue tech
|
| 103 |
+
building leaves the economy short. The pack frames the classic
|
| 104 |
+
tech-vs-expand decision: do not buy capability the regime does
|
| 105 |
+
not yet demand when the same budget, spent on capacity,
|
| 106 |
+
compounds.
|
| 107 |
+
robotics_analogue: >
|
| 108 |
+
Capital budgeting for an autonomous operation: the same budget
|
| 109 |
+
buys EITHER a new capability module that the current mission
|
| 110 |
+
profile does not call for OR an extra collector that compounds
|
| 111 |
+
throughput over the planning horizon. With no demand the new
|
| 112 |
+
capability serves inside the horizon, the capacity expansion
|
| 113 |
+
dominates on payback — invest in throughput, not in idle
|
| 114 |
+
capability.
|
| 115 |
+
benchmark_anchor:
|
| 116 |
+
- "SC2 tech-vs-expand"
|
| 117 |
+
- "PlanBench resource-allocation"
|
| 118 |
+
- "capex allocation"
|
| 119 |
+
- "capital budgeting"
|
| 120 |
+
author: openra-bench
|
| 121 |
+
|
| 122 |
+
base_map: rush-hour-arena
|
| 123 |
+
# EXACTLY enough for EITHER one harvester ($1100, EXPAND) OR one war
|
| 124 |
+
# factory ($2000, TECH) — never both. The budget equates to a single
|
| 125 |
+
# coherent capital allocation.
|
| 126 |
+
starting_cash: 2000
|
| 127 |
+
|
| 128 |
+
base:
|
| 129 |
+
agent:
|
| 130 |
+
faction: allies
|
| 131 |
+
enemy:
|
| 132 |
+
faction: soviet
|
| 133 |
+
tools:
|
| 134 |
+
- observe
|
| 135 |
+
- build
|
| 136 |
+
- place_building
|
| 137 |
+
- harvest
|
| 138 |
+
- move_units
|
| 139 |
+
- stop
|
| 140 |
+
planning: true
|
| 141 |
+
termination:
|
| 142 |
+
max_ticks: 40000
|
| 143 |
+
actors: [] # every level supplies its own actor list via overrides.
|
| 144 |
+
|
| 145 |
+
levels:
|
| 146 |
+
# ── EASY ────────────────────────────────────────────────────────
|
| 147 |
+
# The bare tech-vs-expand decision. A running economy (fact + proc
|
| 148 |
+
# + powr + weap + 1× harv + 2 near patches). The budget ($2000)
|
| 149 |
+
# buys EITHER a second harvester ($1100) OR a second war factory
|
| 150 |
+
# ($2000). Win when economy value reaches 5200 by tick 2700 with
|
| 151 |
+
# the construction yard intact. One harvester tops out near 4100 —
|
| 152 |
+
# short of the bar. Building the war factory sinks the whole
|
| 153 |
+
# budget into a non-revenue structure → the economy never reaches
|
| 154 |
+
# 5200. Only an early second-harvester buy clears the bar.
|
| 155 |
+
easy:
|
| 156 |
+
description: >
|
| 157 |
+
A running economy (1× harvester + refinery + war factory + 2
|
| 158 |
+
ore patches) and a budget of 2000 cr. That budget funds EITHER
|
| 159 |
+
a SECOND ore harvester (build harv, 1100 cr — roughly doubles
|
| 160 |
+
gathering income) OR a SECOND war factory (build weap, 2000 cr
|
| 161 |
+
— a tech building that generates no income). Win when economy
|
| 162 |
+
value (cash + stored ore) reaches 5200 AND your construction
|
| 163 |
+
yard is intact, before tick 2700. A single harvester tops out
|
| 164 |
+
near 4100 by the deadline — it cannot reach the bar. The war
|
| 165 |
+
factory returns nothing; sinking the budget into it leaves the
|
| 166 |
+
economy short. The intended play is to EXPAND — build('harv')
|
| 167 |
+
early so the doubled income compounds and clears the bar.
|
| 168 |
+
Stalling, building the war factory, or under-spending all
|
| 169 |
+
LOSE.
|
| 170 |
+
overrides:
|
| 171 |
+
actors:
|
| 172 |
+
- {type: fact, owner: agent, position: [8, 18]}
|
| 173 |
+
- {type: proc, owner: agent, position: [12, 18]}
|
| 174 |
+
- {type: powr, owner: agent, position: [8, 22]}
|
| 175 |
+
- {type: weap, owner: agent, position: [16, 22]}
|
| 176 |
+
- {type: harv, owner: agent, position: [14, 18]}
|
| 177 |
+
- {type: mine, owner: neutral, position: [22, 18]}
|
| 178 |
+
- {type: mine, owner: neutral, position: [22, 22]}
|
| 179 |
+
# Far inert passive enemy marker (anti-DRAW). stance:0 + far
|
| 180 |
+
# corner ⇒ it never attacks inside the econ window — the
|
| 181 |
+
# "enemy-timing regime" that makes teching premature.
|
| 182 |
+
- {type: e1, owner: enemy, position: [120, 36], stance: 0}
|
| 183 |
+
win_condition:
|
| 184 |
+
all_of:
|
| 185 |
+
- economy_value_gte: 5200
|
| 186 |
+
- building_count_gte: {type: fact, n: 1}
|
| 187 |
+
- within_ticks: 2700
|
| 188 |
+
# within_ticks ≤ 93+90·29 = 2703 ⇒ deadline bites inside
|
| 189 |
+
# max_turns; after_ticks = within_ticks+1 ⇒ a non-finisher is a
|
| 190 |
+
# real reachable LOSS, not a draw.
|
| 191 |
+
fail_condition:
|
| 192 |
+
any_of:
|
| 193 |
+
- after_ticks: 2701
|
| 194 |
+
- not: {building_count_gte: {type: fact, n: 1}}
|
| 195 |
+
- not: {building_count_gte: {type: proc, n: 1}}
|
| 196 |
+
max_turns: 30
|
| 197 |
+
|
| 198 |
+
# ── MEDIUM ──────────────────────────────────────────────────────
|
| 199 |
+
# +1 controlled variable: the EV bar tightens relative to the
|
| 200 |
+
# horizon. The bar (7300 EV by tick 3600) clears comfortably for an
|
| 201 |
+
# early second-harvester buy but not for a single harvester and not
|
| 202 |
+
# for a tech-drained economy. The decision is identical; the margin
|
| 203 |
+
# is thinner.
|
| 204 |
+
medium:
|
| 205 |
+
description: >
|
| 206 |
+
Same running economy and the same 2000 cr budget — EITHER a
|
| 207 |
+
second harvester (build harv, 1100 cr) OR a second war factory
|
| 208 |
+
(build weap, 2000 cr, no income). Win when economy value
|
| 209 |
+
reaches 7300 AND your construction yard is intact, before tick
|
| 210 |
+
3600. One harvester tops out near 5100 by the deadline — well
|
| 211 |
+
short. Sinking the budget into the war factory leaves the
|
| 212 |
+
economy short too. Only an early EXPAND — build('harv') — lets
|
| 213 |
+
the doubled income compound past the bar. Stalling, building
|
| 214 |
+
the war factory, or under-spending all LOSE.
|
| 215 |
+
overrides:
|
| 216 |
+
actors:
|
| 217 |
+
- {type: fact, owner: agent, position: [8, 18]}
|
| 218 |
+
- {type: proc, owner: agent, position: [12, 18]}
|
| 219 |
+
- {type: powr, owner: agent, position: [8, 22]}
|
| 220 |
+
- {type: weap, owner: agent, position: [16, 22]}
|
| 221 |
+
- {type: harv, owner: agent, position: [14, 18]}
|
| 222 |
+
- {type: mine, owner: neutral, position: [22, 18]}
|
| 223 |
+
- {type: mine, owner: neutral, position: [22, 22]}
|
| 224 |
+
- {type: e1, owner: enemy, position: [120, 36], stance: 0}
|
| 225 |
+
win_condition:
|
| 226 |
+
all_of:
|
| 227 |
+
- economy_value_gte: 7300
|
| 228 |
+
- building_count_gte: {type: fact, n: 1}
|
| 229 |
+
- within_ticks: 3600
|
| 230 |
+
# within_ticks ≤ 93+90·39 = 3603 ⇒ reachable in max_turns;
|
| 231 |
+
# after_ticks = within_ticks+1 ⇒ a non-finisher is a real LOSS.
|
| 232 |
+
fail_condition:
|
| 233 |
+
any_of:
|
| 234 |
+
- after_ticks: 3601
|
| 235 |
+
- not: {building_count_gte: {type: fact, n: 1}}
|
| 236 |
+
- not: {building_count_gte: {type: proc, n: 1}}
|
| 237 |
+
max_turns: 40
|
| 238 |
+
|
| 239 |
+
# ── HARD ────────────────────────────────────────────────────────
|
| 240 |
+
# +2 controlled variables vs medium:
|
| 241 |
+
# 1. The EV bar tightens further (10200 EV by tick 4500) — the
|
| 242 |
+
# tech-drained / single-harvester economies miss by a wide
|
| 243 |
+
# margin; only an early EXPAND clears it.
|
| 244 |
+
# 2. TWO seed-driven AGENT spawn_point groups (NORTH y=10..14 /
|
| 245 |
+
# SOUTH y=28..32), each carrying its own fact+proc+powr+weap+
|
| 246 |
+
# harv and a symmetric near-patch pair, so the income
|
| 247 |
+
# economics are identical per spawn — a memorised "build at
|
| 248 |
+
# (8,18)" opening cannot generalise. The neutral mines place
|
| 249 |
+
# every seed (spawn_point filters AGENT actors only); each
|
| 250 |
+
# spawn group has its own near-patch pair.
|
| 251 |
+
hard:
|
| 252 |
+
description: >
|
| 253 |
+
Same running economy and the same 2000 cr budget — EITHER a
|
| 254 |
+
second harvester (build harv, 1100 cr) OR a second war factory
|
| 255 |
+
(build weap, 2000 cr, no income). Win when economy value
|
| 256 |
+
reaches 10200 AND your construction yard is intact, before
|
| 257 |
+
tick 4500. One harvester and a tech-drained economy both miss
|
| 258 |
+
the bar by a wide margin. Only an early EXPAND — build('harv')
|
| 259 |
+
— compounds past it. The base latitude varies by seed so a
|
| 260 |
+
memorised opening cannot generalise; the tech-vs-expand
|
| 261 |
+
decision is identical at either spawn. Stalling, building the
|
| 262 |
+
war factory, or under-spending all LOSE.
|
| 263 |
+
overrides:
|
| 264 |
+
actors:
|
| 265 |
+
# ── spawn 0 — NORTH base (y=10..14) ───────────────────────
|
| 266 |
+
- {type: fact, owner: agent, position: [8, 10], spawn_point: 0}
|
| 267 |
+
- {type: proc, owner: agent, position: [12, 10], spawn_point: 0}
|
| 268 |
+
- {type: powr, owner: agent, position: [8, 14], spawn_point: 0}
|
| 269 |
+
- {type: weap, owner: agent, position: [16, 14], spawn_point: 0}
|
| 270 |
+
- {type: harv, owner: agent, position: [14, 10], spawn_point: 0}
|
| 271 |
+
- {type: mine, owner: neutral, position: [22, 10]}
|
| 272 |
+
- {type: mine, owner: neutral, position: [22, 14]}
|
| 273 |
+
# ── spawn 1 — SOUTH base (y=28..32) ───────────────────────
|
| 274 |
+
- {type: fact, owner: agent, position: [8, 28], spawn_point: 1}
|
| 275 |
+
- {type: proc, owner: agent, position: [12, 28], spawn_point: 1}
|
| 276 |
+
- {type: powr, owner: agent, position: [8, 32], spawn_point: 1}
|
| 277 |
+
- {type: weap, owner: agent, position: [16, 32], spawn_point: 1}
|
| 278 |
+
- {type: harv, owner: agent, position: [14, 28], spawn_point: 1}
|
| 279 |
+
- {type: mine, owner: neutral, position: [22, 28]}
|
| 280 |
+
- {type: mine, owner: neutral, position: [22, 32]}
|
| 281 |
+
# Far inert passive enemy marker (anti-DRAW) on the symmetry
|
| 282 |
+
# axis — never attacks inside the econ window.
|
| 283 |
+
- {type: e1, owner: enemy, position: [120, 20], stance: 0}
|
| 284 |
+
win_condition:
|
| 285 |
+
all_of:
|
| 286 |
+
- economy_value_gte: 10200
|
| 287 |
+
- building_count_gte: {type: fact, n: 1}
|
| 288 |
+
- within_ticks: 4500
|
| 289 |
+
# within_ticks ≤ 93+90·49 = 4503 ⇒ reachable in max_turns;
|
| 290 |
+
# after_ticks = within_ticks+1 ⇒ a non-finisher is a real LOSS.
|
| 291 |
+
fail_condition:
|
| 292 |
+
any_of:
|
| 293 |
+
- after_ticks: 4501
|
| 294 |
+
- not: {building_count_gte: {type: fact, n: 1}}
|
| 295 |
+
- not: {building_count_gte: {type: proc, n: 1}}
|
| 296 |
+
max_turns: 50
|
tests/test_econ_tech_vs_expand_decision.py
ADDED
|
@@ -0,0 +1,300 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""econ-tech-vs-expand-decision — REASONING capability validation.
|
| 2 |
+
|
| 3 |
+
Capital allocation: one indivisible budget ($2000) funds EITHER
|
| 4 |
+
|
| 5 |
+
(A) TECH — build a SECOND war factory (weap, $2000): a
|
| 6 |
+
non-revenue capability investment;
|
| 7 |
+
(B) EXPAND — buy a SECOND ore harvester (harv, $1100): roughly
|
| 8 |
+
doubles gathering throughput.
|
| 9 |
+
|
| 10 |
+
The regime is fixed (no near-term combat demand — the only enemy is
|
| 11 |
+
a token far-off passive marker) and the win is an ECONOMY target.
|
| 12 |
+
EXPAND clears the bar; sinking the budget into the non-revenue TECH
|
| 13 |
+
building leaves the economy short.
|
| 14 |
+
|
| 15 |
+
Bar (CLAUDE.md "no defect, no cheat, no draw"):
|
| 16 |
+
|
| 17 |
+
* stall (observe-only) LOSES every tier / every hard seed — the
|
| 18 |
+
economy idles at the starting reserve, far below the EV bar →
|
| 19 |
+
the `after_ticks` deadline bites.
|
| 20 |
+
* build-tech (build + place the war factory) LOSES every tier /
|
| 21 |
+
seed — the $2000 sink craters the economy → the EV bar is
|
| 22 |
+
never reached → real reachable timeout LOSS.
|
| 23 |
+
* intended EXPAND (build('harv') early) WINS every tier / every
|
| 24 |
+
hard seed — the doubled income compounds past the EV bar.
|
| 25 |
+
* hard tier defines ≥2 agent spawn_point groups (NORTH / SOUTH
|
| 26 |
+
base) round-robined by seed, each with a symmetric near-patch
|
| 27 |
+
pair so a memorised opening cannot generalise.
|
| 28 |
+
"""
|
| 29 |
+
|
| 30 |
+
from __future__ import annotations
|
| 31 |
+
|
| 32 |
+
from pathlib import Path
|
| 33 |
+
|
| 34 |
+
import pytest
|
| 35 |
+
|
| 36 |
+
pytest.importorskip("openra_train", reason="Rust env wheel not installed")
|
| 37 |
+
pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
|
| 38 |
+
|
| 39 |
+
from openra_bench.eval_core import run_level
|
| 40 |
+
from openra_bench.scenarios import load_pack
|
| 41 |
+
from openra_bench.scenarios.loader import PACKS_DIR, compile_level
|
| 42 |
+
from openra_bench.scenarios.win_conditions import WinContext, evaluate
|
| 43 |
+
|
| 44 |
+
PACK = PACKS_DIR / "econ-tech-vs-expand-decision.yaml"
|
| 45 |
+
LEVELS = ("easy", "medium", "hard")
|
| 46 |
+
SEEDS = (1, 2, 3, 4)
|
| 47 |
+
|
| 48 |
+
# Near ore patches per spawn (flat for easy/medium; both spawn pairs
|
| 49 |
+
# for hard — a harvester is routed to its nearest).
|
| 50 |
+
MINES_FLAT = [(22, 18), (22, 22)]
|
| 51 |
+
MINES_HARD = [(22, 10), (22, 14), (22, 28), (22, 32)]
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
# ── scripted policies ───────────────────────────────────────────────
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def _harvest_all(rs, C, mines):
|
| 58 |
+
cmds = []
|
| 59 |
+
for h in [u for u in (rs.get("units_summary") or []) if u.get("type") == "harv"]:
|
| 60 |
+
mx, my = min(
|
| 61 |
+
mines,
|
| 62 |
+
key=lambda m: (m[0] - h["cell_x"]) ** 2 + (m[1] - h["cell_y"]) ** 2,
|
| 63 |
+
)
|
| 64 |
+
cmds.append(C.harvest([str(h["id"])], mx, my))
|
| 65 |
+
return cmds
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
def _stall(rs, C):
|
| 69 |
+
"""Observe-only — no income action. The economy idles at the
|
| 70 |
+
starting reserve, far below the EV bar → LOSS."""
|
| 71 |
+
return [C.observe()]
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def _make_tech(mines):
|
| 75 |
+
"""Build the SECOND war factory — the non-revenue TECH option.
|
| 76 |
+
The $2000 sink craters the economy → the EV bar is never
|
| 77 |
+
reached → LOSS."""
|
| 78 |
+
|
| 79 |
+
def p(rs, C):
|
| 80 |
+
cmds = _harvest_all(rs, C, mines)
|
| 81 |
+
own = rs.get("own_buildings") or []
|
| 82 |
+
nweap = sum(1 for b in own if b.get("type") == "weap")
|
| 83 |
+
prod = [
|
| 84 |
+
x.get("item") for x in (rs.get("production") or []) if isinstance(x, dict)
|
| 85 |
+
]
|
| 86 |
+
if nweap < 2:
|
| 87 |
+
if "weap" not in prod:
|
| 88 |
+
cmds.append(C.build("weap"))
|
| 89 |
+
fy = 20
|
| 90 |
+
for b in own:
|
| 91 |
+
if b.get("type") == "fact":
|
| 92 |
+
fy = b["cell_y"]
|
| 93 |
+
cmds.append(C.place_building("weap", 24, fy))
|
| 94 |
+
return cmds if cmds else [C.observe()]
|
| 95 |
+
|
| 96 |
+
return p
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def _make_expand(mines):
|
| 100 |
+
"""The intended capability — buy the SECOND harvester EARLY so
|
| 101 |
+
the doubled gathering income compounds past the EV bar."""
|
| 102 |
+
|
| 103 |
+
def p(rs, C):
|
| 104 |
+
cmds = _harvest_all(rs, C, mines)
|
| 105 |
+
nharv = sum(
|
| 106 |
+
1 for u in (rs.get("units_summary") or []) if u.get("type") == "harv"
|
| 107 |
+
)
|
| 108 |
+
prod = [
|
| 109 |
+
x.get("item") for x in (rs.get("production") or []) if isinstance(x, dict)
|
| 110 |
+
]
|
| 111 |
+
if nharv < 2 and "harv" not in prod:
|
| 112 |
+
cmds.append(C.build("harv"))
|
| 113 |
+
return cmds if cmds else [C.observe()]
|
| 114 |
+
|
| 115 |
+
return p
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _mines_for(level):
|
| 119 |
+
return MINES_HARD if level == "hard" else MINES_FLAT
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
# ── structural tests ────────────────────────────────────────────────
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
def test_pack_loads_and_meta_reasoning():
|
| 126 |
+
pack = load_pack(PACK)
|
| 127 |
+
assert pack.meta.id == "econ-tech-vs-expand-decision"
|
| 128 |
+
assert pack.meta.capability == "reasoning"
|
| 129 |
+
assert pack.meta.real_world_meaning
|
| 130 |
+
assert pack.meta.robotics_analogue
|
| 131 |
+
anchors = " ".join(pack.meta.benchmark_anchor).lower()
|
| 132 |
+
assert "sc2 tech-vs-expand" in anchors, anchors
|
| 133 |
+
assert "planbench" in anchors and "resource-allocation" in anchors, anchors
|
| 134 |
+
assert "capex allocation" in anchors, anchors
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def test_starting_cash_is_the_either_or_pivot():
|
| 138 |
+
"""Budget $2000 — exactly one war factory (TECH) and enough for
|
| 139 |
+
one harvester (EXPAND, $1100). A single indivisible allocation."""
|
| 140 |
+
pack = load_pack(PACK)
|
| 141 |
+
assert getattr(pack, "starting_cash", None) == 2000
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def test_tools_include_build_and_harvest_surface():
|
| 145 |
+
pack = load_pack(PACK)
|
| 146 |
+
tools = set(pack.base.get("tools", []) if isinstance(pack.base, dict) else [])
|
| 147 |
+
for required in ("build", "place_building", "harvest"):
|
| 148 |
+
assert required in tools, f"missing tool: {required!r}"
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def test_preplaced_pipeline_on_every_level():
|
| 152 |
+
"""Every level pre-places fact + proc + powr + weap + harv so the
|
| 153 |
+
harvester buy (EXPAND) is actionable on turn 1 with no tech
|
| 154 |
+
step."""
|
| 155 |
+
pack = load_pack(PACK)
|
| 156 |
+
for lvl in LEVELS:
|
| 157 |
+
c = compile_level(pack, lvl)
|
| 158 |
+
agent_types = [a.type for a in c.scenario.actors if a.owner == "agent"]
|
| 159 |
+
for needed in ("fact", "proc", "powr", "weap", "harv"):
|
| 160 |
+
assert needed in agent_types, (
|
| 161 |
+
f"{lvl}: pipeline missing {needed!r}; got {sorted(set(agent_types))}"
|
| 162 |
+
)
|
| 163 |
+
# The pre-placed harvester proves EXPAND means a SECOND harv.
|
| 164 |
+
assert agent_types.count("harv") >= 1
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
def test_every_level_has_reachable_timeout_fail():
|
| 168 |
+
pack = load_pack(PACK)
|
| 169 |
+
for lvl in LEVELS:
|
| 170 |
+
L = pack.levels[lvl]
|
| 171 |
+
ceiling = 93 + 90 * (L.max_turns - 1)
|
| 172 |
+
wt = next(
|
| 173 |
+
int(c["within_ticks"])
|
| 174 |
+
for c in L.win_condition.model_dump()["all_of"]
|
| 175 |
+
if "within_ticks" in c
|
| 176 |
+
)
|
| 177 |
+
ft = next(
|
| 178 |
+
int(c["after_ticks"])
|
| 179 |
+
for c in L.fail_condition.model_dump()["any_of"]
|
| 180 |
+
if "after_ticks" in c
|
| 181 |
+
)
|
| 182 |
+
assert wt < ceiling, f"{lvl}: within_ticks {wt} >= ceiling {ceiling}"
|
| 183 |
+
assert ft <= ceiling, f"{lvl}: after_ticks {ft} > ceiling {ceiling}"
|
| 184 |
+
assert wt + 1 == ft, f"{lvl}: within/after mismatch {wt}/{ft}"
|
| 185 |
+
|
| 186 |
+
|
| 187 |
+
def test_every_level_has_a_fail_condition():
|
| 188 |
+
pack = load_pack(PACK)
|
| 189 |
+
for lvl in LEVELS:
|
| 190 |
+
c = compile_level(pack, lvl)
|
| 191 |
+
assert c.fail_condition is not None, f"{lvl} needs a fail_condition"
|
| 192 |
+
|
| 193 |
+
|
| 194 |
+
def test_hard_has_two_seed_driven_spawn_groups():
|
| 195 |
+
c = compile_level(load_pack(PACK), "hard")
|
| 196 |
+
sp = {
|
| 197 |
+
(a.spawn_point if a.spawn_point is not None else 0)
|
| 198 |
+
for a in c.scenario.actors
|
| 199 |
+
if a.owner == "agent"
|
| 200 |
+
}
|
| 201 |
+
assert sp == {0, 1}, f"hard must define spawn_point groups {{0,1}}; got {sorted(sp)}"
|
| 202 |
+
|
| 203 |
+
|
| 204 |
+
def test_in_bounds_actors_on_every_level():
|
| 205 |
+
pack = load_pack(PACK)
|
| 206 |
+
for lvl in LEVELS:
|
| 207 |
+
c = compile_level(pack, lvl)
|
| 208 |
+
for a in c.scenario.actors:
|
| 209 |
+
x, y = a.position
|
| 210 |
+
assert 2 <= x <= 126 and 2 <= y <= 38, (
|
| 211 |
+
f"{lvl}: actor {a.type} at ({x},{y}) out of bounds"
|
| 212 |
+
)
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
# ── predicate-level (no engine) ─────────────────────────────────────
|
| 216 |
+
|
| 217 |
+
|
| 218 |
+
def _ctx(*, tick=0, ev=0, has_fact=True, has_proc=True):
|
| 219 |
+
import types
|
| 220 |
+
|
| 221 |
+
own = []
|
| 222 |
+
if has_fact:
|
| 223 |
+
own.append(("fact", 8, 18))
|
| 224 |
+
if has_proc:
|
| 225 |
+
own.append(("proc", 12, 18))
|
| 226 |
+
sig = types.SimpleNamespace(
|
| 227 |
+
game_tick=tick,
|
| 228 |
+
units_killed=0,
|
| 229 |
+
units_lost=0,
|
| 230 |
+
cash=ev,
|
| 231 |
+
resources=0,
|
| 232 |
+
own_buildings=own,
|
| 233 |
+
own_building_types={t for (t, _, _) in own},
|
| 234 |
+
enemies_seen_ids=set(),
|
| 235 |
+
enemy_buildings_seen_ids=set(),
|
| 236 |
+
)
|
| 237 |
+
return WinContext(signals=sig, render_state={"units_summary": []})
|
| 238 |
+
|
| 239 |
+
|
| 240 |
+
def test_predicates_enforce_economy_target():
|
| 241 |
+
c = compile_level(load_pack(PACK), "easy")
|
| 242 |
+
# EV at bar, fact + proc alive, in time → WIN
|
| 243 |
+
assert evaluate(c.win_condition, _ctx(tick=2000, ev=5200))
|
| 244 |
+
# EV under bar → not win
|
| 245 |
+
assert not evaluate(c.win_condition, _ctx(tick=2000, ev=5199))
|
| 246 |
+
# EV at bar but past deadline → not win
|
| 247 |
+
assert not evaluate(c.win_condition, _ctx(tick=2702, ev=6000))
|
| 248 |
+
# Past after_ticks → fail
|
| 249 |
+
assert evaluate(c.fail_condition, _ctx(tick=2701, ev=0))
|
| 250 |
+
# fact destroyed → fail
|
| 251 |
+
assert evaluate(c.fail_condition, _ctx(tick=1000, ev=6000, has_fact=False))
|
| 252 |
+
# healthy in-window economy → not fail
|
| 253 |
+
assert not evaluate(c.fail_condition, _ctx(tick=1000, ev=3000))
|
| 254 |
+
|
| 255 |
+
|
| 256 |
+
# ── engine-driven: every lazy/wrong policy LOSES, intended WINS ──────
|
| 257 |
+
|
| 258 |
+
|
| 259 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 260 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 261 |
+
def test_stall_loses_every_tier_and_seed(level, seed):
|
| 262 |
+
c = compile_level(load_pack(PACK), level)
|
| 263 |
+
r = run_level(c, _stall, seed=seed)
|
| 264 |
+
assert r.outcome == "loss", (
|
| 265 |
+
f"{level}/seed{seed}: stall must LOSE; got {r.outcome} "
|
| 266 |
+
f"EV={r.signals.cash + r.signals.resources} tick={r.signals.game_tick}"
|
| 267 |
+
)
|
| 268 |
+
|
| 269 |
+
|
| 270 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 271 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 272 |
+
def test_build_tech_loses_every_tier_and_seed(level, seed):
|
| 273 |
+
"""Sinking the budget into the second war factory craters the
|
| 274 |
+
economy → the EV bar is never reached → LOSS."""
|
| 275 |
+
c = compile_level(load_pack(PACK), level)
|
| 276 |
+
r = run_level(c, _make_tech(_mines_for(level)), seed=seed)
|
| 277 |
+
assert r.outcome == "loss", (
|
| 278 |
+
f"{level}/seed{seed}: build-tech must LOSE; got {r.outcome} "
|
| 279 |
+
f"EV={r.signals.cash + r.signals.resources} tick={r.signals.game_tick}"
|
| 280 |
+
)
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 284 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 285 |
+
def test_expand_wins_every_tier_and_seed(level, seed):
|
| 286 |
+
"""The intended capability — buy the second harvester early — the
|
| 287 |
+
doubled income compounds past the EV bar. WINS every tier/seed."""
|
| 288 |
+
c = compile_level(load_pack(PACK), level)
|
| 289 |
+
r = run_level(c, _make_expand(_mines_for(level)), seed=seed)
|
| 290 |
+
assert r.outcome == "win", (
|
| 291 |
+
f"{level}/seed{seed}: EXPAND must WIN; got {r.outcome} "
|
| 292 |
+
f"EV={r.signals.cash + r.signals.resources} tick={r.signals.game_tick}"
|
| 293 |
+
)
|
| 294 |
+
|
| 295 |
+
|
| 296 |
+
def test_expand_run_is_deterministic_per_seed():
|
| 297 |
+
c = compile_level(load_pack(PACK), "medium")
|
| 298 |
+
a = run_level(c, _make_expand(MINES_FLAT), seed=2)
|
| 299 |
+
b = run_level(c, _make_expand(MINES_FLAT), seed=2)
|
| 300 |
+
assert (a.outcome, a.turns) == (b.outcome, b.turns)
|
tests/test_hard_tier.py
CHANGED
|
@@ -1449,6 +1449,7 @@ UPGRADED = [
|
|
| 1449 |
"def-tower-line-vs-cluster", # hard: 2 agent spawn_point groups
|
| 1450 |
"coord-cover-and-move", # hard: 2 agent spawn_point groups
|
| 1451 |
"combat-kite-and-pull", # hard: 2 agent spawn_point groups (Wave-12)
|
|
|
|
| 1452 |
]
|
| 1453 |
|
| 1454 |
# Consciously NOT spawn-varied, with the reason (keeps the curation
|
|
|
|
| 1449 |
"def-tower-line-vs-cluster", # hard: 2 agent spawn_point groups
|
| 1450 |
"coord-cover-and-move", # hard: 2 agent spawn_point groups
|
| 1451 |
"combat-kite-and-pull", # hard: 2 agent spawn_point groups (Wave-12)
|
| 1452 |
+
"econ-tech-vs-expand-decision", # hard: 2 agent spawn_point groups (Wave-12)
|
| 1453 |
]
|
| 1454 |
|
| 1455 |
# Consciously NOT spawn-varied, with the reason (keeps the curation
|