Spaces:
Running
Running
feat(scenario): build-defensive-tower-line — pbox-line choke defense (ERQA / MicroRTS anchor)
Browse files
openra_bench/scenarios/packs/build-defensive-tower-line.yaml
ADDED
|
@@ -0,0 +1,258 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
meta:
|
| 2 |
+
id: build-defensive-tower-line
|
| 3 |
+
title: 'Build a Defensive Tower LINE Across the Choke (Not a Cluster, Not a Scatter)'
|
| 4 |
+
capability: reasoning
|
| 5 |
+
real_world_meaning: >
|
| 6 |
+
Where do you commit your defensive structures when the threat is
|
| 7 |
+
funnelled through a known corridor whose width matters? Military
|
| 8 |
+
perimeter doctrine and firewall rule design both say: spread the
|
| 9 |
+
coverage across the full width of the corridor so no enemy unit
|
| 10 |
+
can slip past on an unguarded row. A single dense cluster wastes
|
| 11 |
+
overlapping fire on one cell while the corridor edges stay open;
|
| 12 |
+
a scatter across the map leaves the corridor itself uncovered.
|
| 13 |
+
The win predicate makes the LINE topology load-bearing — total
|
| 14 |
+
pillbox count alone is not enough; ≥1 pillbox must sit on EACH of
|
| 15 |
+
the corridor's vertical rungs (the four narrow sub-regions that
|
| 16 |
+
span y=18..22 at the choke column).
|
| 17 |
+
robotics_analogue: >
|
| 18 |
+
Network firewall / Web Application Firewall rule placement: when
|
| 19 |
+
every connection MUST traverse a known ingress (the public WAN
|
| 20 |
+
edge / the only API gateway), the right architecture is one rule
|
| 21 |
+
per protocol/port across the FULL inspection surface, not three
|
| 22 |
+
duplicated rules on one port while the rest stay open. Likewise a
|
| 23 |
+
physical perimeter patrol covers the WHOLE corridor width — a
|
| 24 |
+
cluster at one waypoint or a scatter across unrelated nodes both
|
| 25 |
+
leave the actual approach lane traversable.
|
| 26 |
+
benchmark_anchor:
|
| 27 |
+
- "ERQA"
|
| 28 |
+
- "MicroRTS defense"
|
| 29 |
+
- "military perimeter"
|
| 30 |
+
author: openra-bench
|
| 31 |
+
|
| 32 |
+
# rush-hour-arena (128×40). The map has a narrow lane around y≈18..22
|
| 33 |
+
# at mid-map (x≈60); the `rusher` scripted bot charges the agent's
|
| 34 |
+
# centroid (agent fact on the west), so its path is forced THROUGH
|
| 35 |
+
# that lane on every seed. Agent base pre-placed at the west (fact +
|
| 36 |
+
# tent + powr) so the pbox queue is ready turn 1.
|
| 37 |
+
# Pack-level cash is overridden per-level (4/4/4 pbox at 600cr each;
|
| 38 |
+
# medium/hard get tighter constraints, not more pbox — see below).
|
| 39 |
+
# An unarmed high-HP enemy `fact` far east keeps the engine alive
|
| 40 |
+
# past rusher annihilation so the win/fail check actually runs.
|
| 41 |
+
# SISTER PACK: def-tower-line-vs-cluster inverts the topology bar to
|
| 42 |
+
# enforce CLUSTER (graph min-cut doctrine); this pack enforces LINE
|
| 43 |
+
# (corridor-width perimeter doctrine). The two together discriminate
|
| 44 |
+
# whether the model understands the FORCING GEOMETRY: a chokepoint
|
| 45 |
+
# (single cell on a wide approach → cluster) vs a corridor (full
|
| 46 |
+
# vertical width that any one row can leak through → line).
|
| 47 |
+
base_map: rush-hour-arena
|
| 48 |
+
starting_cash: 2400
|
| 49 |
+
|
| 50 |
+
base:
|
| 51 |
+
agent:
|
| 52 |
+
faction: allies
|
| 53 |
+
enemy:
|
| 54 |
+
faction: soviet
|
| 55 |
+
bot_type: rusher
|
| 56 |
+
tools:
|
| 57 |
+
- observe
|
| 58 |
+
- build
|
| 59 |
+
- place_building
|
| 60 |
+
- move_units
|
| 61 |
+
- attack_unit
|
| 62 |
+
- attack_move
|
| 63 |
+
- stop
|
| 64 |
+
planning: true
|
| 65 |
+
# No interrupts — perimeter design is a STATIC up-front decision
|
| 66 |
+
# (the corridor is known a priori, the rush composition is fixed).
|
| 67 |
+
# Dropping interrupts also makes the tick budget deterministic
|
| 68 |
+
# (each step is exactly 90 ticks ⇒ max_turns is a hard tick
|
| 69 |
+
# budget that the `after_ticks` fail clause reliably bites in).
|
| 70 |
+
termination:
|
| 71 |
+
max_ticks: 12000
|
| 72 |
+
actors:
|
| 73 |
+
# Pre-placed agent base on the WEST so rusher path is forced
|
| 74 |
+
# through the mid-map corridor on the way to the fact centroid.
|
| 75 |
+
- {type: fact, owner: agent, position: [10, 20]}
|
| 76 |
+
- {type: tent, owner: agent, position: [14, 18]}
|
| 77 |
+
- {type: powr, owner: agent, position: [14, 22]}
|
| 78 |
+
|
| 79 |
+
levels:
|
| 80 |
+
# ── EASY ── bare LINE skill. Budget covers exactly 4 pbox (2400cr).
|
| 81 |
+
# Win requires ≥1 pbox in EACH of the 4 corridor rungs (y=18,19,21,22
|
| 82 |
+
# at x=60, radius 0.5 — only the exact rung cell counts, so a
|
| 83 |
+
# and a cluster on y=20 misses ALL FOUR). The intended LINE places
|
| 84 |
+
# one pbox per rung (4 rungs × 1 pbox = budget exhausted). A cluster
|
| 85 |
+
# at (60,20) satisfies the count clause but FAILS each rung; a
|
| 86 |
+
# random scatter (e.g. 4 pbox near the base) misses every rung; a
|
| 87 |
+
# stall loses on the count clause AND the fact razed by the rush.
|
| 88 |
+
# max_turns 60 ⇒ reachable tick 93+90·59 = 5403; deadline 5400.
|
| 89 |
+
easy:
|
| 90 |
+
description: >
|
| 91 |
+
A rusher band must traverse the narrow corridor at x=60,
|
| 92 |
+
y=18..22 to reach your base on the west. Build 4 pillboxes
|
| 93 |
+
(pbox — 600cr each, budget exactly 2400) AND place ONE on each
|
| 94 |
+
of the four corridor rungs (at (60,18), (60,19),
|
| 95 |
+
(60,21), (60,22)) so the rusher cannot slip past on any row.
|
| 96 |
+
A cluster on the middle of the corridor satisfies the count
|
| 97 |
+
but FAILS every rung; a random scatter near the base fails
|
| 98 |
+
every rung; a pure-army layout (no pbox) fails the count and
|
| 99 |
+
lets the rush raze the fact. Your fact must survive.
|
| 100 |
+
starting_cash: 2400
|
| 101 |
+
overrides:
|
| 102 |
+
actors:
|
| 103 |
+
- {type: fact, owner: agent, position: [10, 20]}
|
| 104 |
+
- {type: tent, owner: agent, position: [14, 18]}
|
| 105 |
+
- {type: powr, owner: agent, position: [14, 22]}
|
| 106 |
+
# 4 mobile defenders pre-placed (light wave).
|
| 107 |
+
- {type: e1, owner: agent, position: [12, 19], stance: 2}
|
| 108 |
+
- {type: e1, owner: agent, position: [12, 21], stance: 2}
|
| 109 |
+
- {type: e1, owner: agent, position: [16, 19], stance: 2}
|
| 110 |
+
- {type: e1, owner: agent, position: [16, 21], stance: 2}
|
| 111 |
+
# Light rusher band at far east — forced through the corridor
|
| 112 |
+
# on the way to agent fact centroid (10, 20).
|
| 113 |
+
- {type: e1, owner: enemy, position: [100, 20], stance: 3, count: 4}
|
| 114 |
+
- {type: e3, owner: enemy, position: [102, 20], stance: 3, count: 1}
|
| 115 |
+
# Unarmed high-HP marker (anti-DRAW): keeps the episode alive
|
| 116 |
+
# past full rusher elimination so the win/fail check fires.
|
| 117 |
+
- {type: fact, owner: enemy, position: [120, 20]}
|
| 118 |
+
win_condition:
|
| 119 |
+
all_of:
|
| 120 |
+
- building_count_gte: {type: pbox, n: 4}
|
| 121 |
+
- building_in_region: {type: pbox, x: 60, y: 18, radius: 0.5, count: 1}
|
| 122 |
+
- building_in_region: {type: pbox, x: 60, y: 19, radius: 0.5, count: 1}
|
| 123 |
+
- building_in_region: {type: pbox, x: 60, y: 21, radius: 0.5, count: 1}
|
| 124 |
+
- building_in_region: {type: pbox, x: 60, y: 22, radius: 0.5, count: 1}
|
| 125 |
+
- building_count_gte: {type: fact, n: 1}
|
| 126 |
+
- within_ticks: 5400
|
| 127 |
+
fail_condition:
|
| 128 |
+
any_of:
|
| 129 |
+
- after_ticks: 5401
|
| 130 |
+
- not: {building_count_gte: {type: fact, n: 1}}
|
| 131 |
+
max_turns: 60
|
| 132 |
+
|
| 133 |
+
# ── MEDIUM ── +1 axis: HEAVY wave (rusher band ~doubled) with
|
| 134 |
+
# the SAME tight 4-pbox budget. The cluster, scatter, and pure-army
|
| 135 |
+
# layouts still all lose the same way as easy; the difference is
|
| 136 |
+
# that the LINE has to assemble FASTER before the heavier rush
|
| 137 |
+
# breaks through the pre-placed defenders. max_turns 60 ⇒
|
| 138 |
+
# reachable tick 5403; deadline 5400.
|
| 139 |
+
medium:
|
| 140 |
+
description: >
|
| 141 |
+
Same forced rusher corridor at x=60, y=18..22. Build 4 pillboxes
|
| 142 |
+
(budget 2400cr = exactly 4 pbox at 600 each) AND place ONE on
|
| 143 |
+
each of the four corridor rungs (at (60,18),
|
| 144 |
+
(60,19), (60,21), (60,22)). The rush wave is heavier than easy
|
| 145 |
+
— the LINE must assemble fast. A cluster, a scatter, and a
|
| 146 |
+
pure-army layout all lose; the fact must survive.
|
| 147 |
+
starting_cash: 2400
|
| 148 |
+
overrides:
|
| 149 |
+
actors:
|
| 150 |
+
- {type: fact, owner: agent, position: [10, 20]}
|
| 151 |
+
- {type: tent, owner: agent, position: [14, 18]}
|
| 152 |
+
- {type: powr, owner: agent, position: [14, 22]}
|
| 153 |
+
# 5 mobile base defenders + a 4-rifle corridor screen at the
|
| 154 |
+
# choke entrance (x=58) so the LINE has time to assemble
|
| 155 |
+
# behind the screen against the heavier wave. The screen
|
| 156 |
+
# WILL be ground down by the rush; only the LINE topology
|
| 157 |
+
# finishes the survivors before the fact dies.
|
| 158 |
+
- {type: e1, owner: agent, position: [12, 19], stance: 2}
|
| 159 |
+
- {type: e1, owner: agent, position: [12, 21], stance: 2}
|
| 160 |
+
- {type: e1, owner: agent, position: [16, 19], stance: 2}
|
| 161 |
+
- {type: e1, owner: agent, position: [16, 21], stance: 2}
|
| 162 |
+
- {type: e1, owner: agent, position: [14, 20], stance: 2}
|
| 163 |
+
- {type: e1, owner: agent, position: [58, 18], stance: 2}
|
| 164 |
+
- {type: e1, owner: agent, position: [58, 19], stance: 2}
|
| 165 |
+
- {type: e1, owner: agent, position: [58, 21], stance: 2}
|
| 166 |
+
- {type: e1, owner: agent, position: [58, 22], stance: 2}
|
| 167 |
+
# Heavier rusher band (one more e1 than easy, same e3
|
| 168 |
+
# grenadier). The tighter timing pressure — not more cash
|
| 169 |
+
# for more pbox — is what makes medium harder than easy.
|
| 170 |
+
# Band staged at x=115 (vs easy x=100) so the LINE has time
|
| 171 |
+
# to fully assemble behind the corridor screen before the
|
| 172 |
+
# heavier band engages it.
|
| 173 |
+
- {type: e1, owner: enemy, position: [115, 20], stance: 3, count: 5}
|
| 174 |
+
- {type: e3, owner: enemy, position: [117, 20], stance: 3, count: 1}
|
| 175 |
+
# Anti-DRAW marker.
|
| 176 |
+
- {type: fact, owner: enemy, position: [120, 20]}
|
| 177 |
+
win_condition:
|
| 178 |
+
all_of:
|
| 179 |
+
- building_count_gte: {type: pbox, n: 4}
|
| 180 |
+
- building_in_region: {type: pbox, x: 60, y: 18, radius: 0.5, count: 1}
|
| 181 |
+
- building_in_region: {type: pbox, x: 60, y: 19, radius: 0.5, count: 1}
|
| 182 |
+
- building_in_region: {type: pbox, x: 60, y: 21, radius: 0.5, count: 1}
|
| 183 |
+
- building_in_region: {type: pbox, x: 60, y: 22, radius: 0.5, count: 1}
|
| 184 |
+
- building_count_gte: {type: fact, n: 1}
|
| 185 |
+
- within_ticks: 5400
|
| 186 |
+
fail_condition:
|
| 187 |
+
any_of:
|
| 188 |
+
- after_ticks: 5401
|
| 189 |
+
- not: {building_count_gte: {type: fact, n: 1}}
|
| 190 |
+
max_turns: 60
|
| 191 |
+
|
| 192 |
+
# ── HARD ── +1 axis: TWO spawn_point groups so the agent base
|
| 193 |
+
# latitude flips by seed (NORTH (10,12) vs SOUTH (10,28)). The
|
| 194 |
+
# rusher band is symmetric across y=20 and ALWAYS places (enemy
|
| 195 |
+
# actors don't honour spawn_point — CLAUDE.md), so the corridor
|
| 196 |
+
# column at x=60 remains the choke for both seeds, but the rush
|
| 197 |
+
# geometry approaches each base from a different bearing. The
|
| 198 |
+
# corridor is still y=18..22 at x=60 (it's a fixed map feature) so
|
| 199 |
+
# the LINE topology is identical across seeds — what flips is the
|
| 200 |
+
# agent's interpretation of "which corridor" to defend (the NORTH
|
| 201 |
+
# spawn could be tempted to cover y=14..18, the SOUTH spawn to
|
| 202 |
+
# cover y=22..26; both are WRONG — the corridor itself is at
|
| 203 |
+
# y=18..22 regardless of base latitude). max_turns 70 ⇒ reachable
|
| 204 |
+
# tick 93+90·69 = 6303; deadline 6300.
|
| 205 |
+
hard:
|
| 206 |
+
description: >
|
| 207 |
+
Agent base latitude flips between NORTH (y=12) and SOUTH (y=28)
|
| 208 |
+
by seed. Build 4 pillboxes (budget 2400cr = exactly 4 pbox at
|
| 209 |
+
600 each) AND place ONE on each of the four corridor rungs
|
| 210 |
+
(at (60,18), (60,19), (60,21), (60,22)). The
|
| 211 |
+
corridor at x=60 y=18..22 is a fixed map feature — covering
|
| 212 |
+
the rows next to your base instead (y=14..18 for NORTH, y=22..26
|
| 213 |
+
for SOUTH) FAILS the rung clauses. The fact must survive.
|
| 214 |
+
starting_cash: 2400
|
| 215 |
+
overrides:
|
| 216 |
+
actors:
|
| 217 |
+
# spawn_point 0 — NORTH base at y=12. Fact at (10, 12);
|
| 218 |
+
# tent/powr offset west so they aren't directly on the
|
| 219 |
+
# rusher path.
|
| 220 |
+
- {type: fact, owner: agent, position: [10, 12], spawn_point: 0}
|
| 221 |
+
- {type: tent, owner: agent, position: [6, 12], spawn_point: 0}
|
| 222 |
+
- {type: powr, owner: agent, position: [6, 14], spawn_point: 0}
|
| 223 |
+
- {type: e1, owner: agent, position: [12, 12], stance: 2, spawn_point: 0}
|
| 224 |
+
- {type: e1, owner: agent, position: [12, 13], stance: 2, spawn_point: 0}
|
| 225 |
+
- {type: e1, owner: agent, position: [12, 11], stance: 2, spawn_point: 0}
|
| 226 |
+
- {type: e1, owner: agent, position: [14, 12], stance: 2, spawn_point: 0}
|
| 227 |
+
- {type: e1, owner: agent, position: [8, 12], stance: 2, spawn_point: 0}
|
| 228 |
+
# spawn_point 1 — SOUTH base at y=28 (mirror across y=20).
|
| 229 |
+
- {type: fact, owner: agent, position: [10, 28], spawn_point: 1}
|
| 230 |
+
- {type: tent, owner: agent, position: [6, 28], spawn_point: 1}
|
| 231 |
+
- {type: powr, owner: agent, position: [6, 26], spawn_point: 1}
|
| 232 |
+
- {type: e1, owner: agent, position: [12, 28], stance: 2, spawn_point: 1}
|
| 233 |
+
- {type: e1, owner: agent, position: [12, 27], stance: 2, spawn_point: 1}
|
| 234 |
+
- {type: e1, owner: agent, position: [12, 29], stance: 2, spawn_point: 1}
|
| 235 |
+
- {type: e1, owner: agent, position: [14, 28], stance: 2, spawn_point: 1}
|
| 236 |
+
- {type: e1, owner: agent, position: [8, 28], stance: 2, spawn_point: 1}
|
| 237 |
+
# Enemies don't honour spawn_point (engine: oramap.rs).
|
| 238 |
+
# Symmetric rusher band at y=20 (mid-latitude); charges agent
|
| 239 |
+
# centroid so its path crosses the x=60 corridor on every
|
| 240 |
+
# seed regardless of which base latitude was picked.
|
| 241 |
+
- {type: e1, owner: enemy, position: [100, 20], stance: 3, count: 6}
|
| 242 |
+
- {type: e3, owner: enemy, position: [102, 20], stance: 3, count: 2}
|
| 243 |
+
# Anti-DRAW marker.
|
| 244 |
+
- {type: fact, owner: enemy, position: [120, 20]}
|
| 245 |
+
win_condition:
|
| 246 |
+
all_of:
|
| 247 |
+
- building_count_gte: {type: pbox, n: 4}
|
| 248 |
+
- building_in_region: {type: pbox, x: 60, y: 18, radius: 0.5, count: 1}
|
| 249 |
+
- building_in_region: {type: pbox, x: 60, y: 19, radius: 0.5, count: 1}
|
| 250 |
+
- building_in_region: {type: pbox, x: 60, y: 21, radius: 0.5, count: 1}
|
| 251 |
+
- building_in_region: {type: pbox, x: 60, y: 22, radius: 0.5, count: 1}
|
| 252 |
+
- building_count_gte: {type: fact, n: 1}
|
| 253 |
+
- within_ticks: 6300
|
| 254 |
+
fail_condition:
|
| 255 |
+
any_of:
|
| 256 |
+
- after_ticks: 6301
|
| 257 |
+
- not: {building_count_gte: {type: fact, n: 1}}
|
| 258 |
+
max_turns: 70
|
openra_bench/scenarios/packs/econ-harvester-pathing-optimization.yaml
ADDED
|
@@ -0,0 +1,279 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
meta:
|
| 2 |
+
id: econ-harvester-pathing-optimization
|
| 3 |
+
title: Economy — Harvester Pathing Optimization (Split-Routing M/M/c)
|
| 4 |
+
capability: reasoning
|
| 5 |
+
real_world_meaning: >
|
| 6 |
+
Split-routing under heterogeneous round-trip cost. The agent owns
|
| 7 |
+
TWO harvesters and faces TWO ore patches: A — a NEAR small patch
|
| 8 |
+
(rich per-cell but quickly exhausted by two collectors stacking on
|
| 9 |
+
it) — and B — a FAR larger patch (slow per-cycle but doesn't
|
| 10 |
+
saturate). The optimal policy sends ONE harvester to A and ONE to
|
| 11 |
+
B (the "split route"), so the proc receives a steady alternating
|
| 12 |
+
delivery stream from two short and one long round-trip cycle. The
|
| 13 |
+
LAZY/BRUTE alternatives both fail: piling BOTH harvs on A pumps
|
| 14 |
+
early cash but never enters B's region (the routing constraint in
|
| 15 |
+
the win predicate goes unmet); piling BOTH on B earns barely any
|
| 16 |
+
income within the deadline (long round-trip dominates). Stalling
|
| 17 |
+
earns zero. The capability under test is the OR vehicle-routing /
|
| 18 |
+
SC2 worker-distribution insight: distribute collectors across
|
| 19 |
+
heterogeneous sources to balance round-trip throughput, not to
|
| 20 |
+
monocrop the best-looking node.
|
| 21 |
+
robotics_analogue: >
|
| 22 |
+
Fleet vehicle-routing across a multi-server queueing system
|
| 23 |
+
(M/M/c): two pickup stations at different distances from a depot,
|
| 24 |
+
each modelled as a server with its own service rate; the optimal
|
| 25 |
+
fleet allocation puts exactly one vehicle per server so the depot
|
| 26 |
+
queue alternates arrivals from heterogeneous cycle times rather
|
| 27 |
+
than stacking all vehicles on the lowest-latency server (which
|
| 28 |
+
saturates the server and starves throughput) or the
|
| 29 |
+
highest-throughput server (which dominates the cycle and starves
|
| 30 |
+
the depot in early windows).
|
| 31 |
+
author: wave7-fleet
|
| 32 |
+
benchmark_anchor:
|
| 33 |
+
- "SC2 worker distribution"
|
| 34 |
+
- "OR vehicle routing"
|
| 35 |
+
- "M/M/c queueing"
|
| 36 |
+
|
| 37 |
+
# ENGINE NOTE (verified 2026-05-20 against installed openra_train wheel,
|
| 38 |
+
# post-S0/S1 harvest income — Task #14, scripted run_level seeds 1-4):
|
| 39 |
+
#
|
| 40 |
+
# 1. Per-harvester yields over 4500 ticks (50 turns) on rush-hour-arena,
|
| 41 |
+
# proc at (12,18), 2 harvs at (14,18)/(14,20):
|
| 42 |
+
# A=(16,18) NEAR — ~9000 cr/harv/4500t (1 harv: 8000; 2 harvs: 18000)
|
| 43 |
+
# B=(60,18) MID — ~1500 cr/harv/4500t (1 harv: 1000-2000)
|
| 44 |
+
# B=(80,18) FAR — ~1000 cr/harv/4500t (1 harv: 1000)
|
| 45 |
+
# The 2-harv yield on A is HIGHER than 1A+1B in raw credits, BUT
|
| 46 |
+
# the win predicate REQUIRES a harv inside BOTH region A AND region
|
| 47 |
+
# B (`units_of_type_in_region_gte` clauses). The "stack on A"
|
| 48 |
+
# policy never enters B's region (the harvest order persists, the
|
| 49 |
+
# harv shuttles between A and proc), so the routing clause fails
|
| 50 |
+
# even at 18000 cr. This is the discrimination teeth: the win
|
| 51 |
+
# predicate enforces SPLIT routing as a hard constraint, not as a
|
| 52 |
+
# cash-output preference.
|
| 53 |
+
#
|
| 54 |
+
# 2. Empirical (seed 1, scripted policies):
|
| 55 |
+
# stall 0 cr — LOSS every tier (no cash, no region cover)
|
| 56 |
+
# 2-on-A 18000 cr but 0 harvs in B — LOSS every tier
|
| 57 |
+
# 2-on-B 2000 cr (FAR) — LOSS every tier (cash < bar)
|
| 58 |
+
# 1A+1B (split) 4500-6000 cr + both region clauses fire — WIN
|
| 59 |
+
# every tier; WIN fires at ~turn 25 (easy, B nearer)
|
| 60 |
+
# or turn 35 (medium/hard, B farther) when the
|
| 61 |
+
# loaded harv from B passes through B's radius
|
| 62 |
+
# while the A harv is at A.
|
| 63 |
+
#
|
| 64 |
+
# 3. The `harvest` order with an explicit target cell PERSISTS — the
|
| 65 |
+
# harv shuttles patch ↔ proc on that route. To pivot to a different
|
| 66 |
+
# patch the agent must re-issue `harvest` with new coords. This is
|
| 67 |
+
# the routing knob the model uses.
|
| 68 |
+
#
|
| 69 |
+
# 4. Tick budget: engine advances ~90 ticks per decision turn.
|
| 70 |
+
# Easy/medium use max_turns=50 → ceiling 4503 → within_ticks =
|
| 71 |
+
# after_ticks = 4500 (non-finisher LOSES, not draws).
|
| 72 |
+
# Hard uses max_turns=50 → ceiling 4503 → within_ticks =
|
| 73 |
+
# after_ticks = 4500 (same alignment).
|
| 74 |
+
#
|
| 75 |
+
# 5. The pre-placed proc + fact + harvs trigger ConquestVictoryConditions
|
| 76 |
+
# — without a persistent enemy actor the engine auto-`done`s on
|
| 77 |
+
# "all enemies dead". An unarmed e1 at (120,36) stance:0 sits well
|
| 78 |
+
# outside the harvest envelope so it can't be killed; the win/fail
|
| 79 |
+
# predicate evaluates cleanly to the bar.
|
| 80 |
+
#
|
| 81 |
+
# 6. Hard tier spawn round-robin (verified seeds 1-4): seeds 1,3
|
| 82 |
+
# pick spawn_point 1 (SOUTH base, proc at y=28); seeds 2,4 pick
|
| 83 |
+
# spawn_point 0 (NORTH base, proc at y=14). The neutral mines
|
| 84 |
+
# place at the SAME four cells across both spawns (CLAUDE.md:
|
| 85 |
+
# "spawn_point filter applies ONLY to AGENT actors"), but the
|
| 86 |
+
# SPAWN-MATCHED A/B pair flips per seed: NORTH → A=(16,14)
|
| 87 |
+
# B=(80,14); SOUTH → A=(16,28) B=(80,28). A memorised "always
|
| 88 |
+
# split (16,14)+(80,14)" policy LOSES on the SOUTH seeds (1,3)
|
| 89 |
+
# because the harvs head north and never enter the SOUTH-matched
|
| 90 |
+
# regions of the active win-predicate disjunct. The win predicate
|
| 91 |
+
# is `any_of` over the two spawn-matched (A,B) pairs.
|
| 92 |
+
|
| 93 |
+
base_map: rush-hour-arena
|
| 94 |
+
starting_cash: 0
|
| 95 |
+
|
| 96 |
+
base:
|
| 97 |
+
agent:
|
| 98 |
+
faction: allies
|
| 99 |
+
enemy:
|
| 100 |
+
faction: soviet
|
| 101 |
+
tools:
|
| 102 |
+
- observe
|
| 103 |
+
- harvest
|
| 104 |
+
- move_units
|
| 105 |
+
- stop
|
| 106 |
+
planning: true
|
| 107 |
+
termination:
|
| 108 |
+
max_ticks: 40000
|
| 109 |
+
actors:
|
| 110 |
+
# Pre-placed agent base + 2 harvs centred on row y=18..20.
|
| 111 |
+
- {type: fact, owner: agent, position: [10, 22]}
|
| 112 |
+
- {type: proc, owner: agent, position: [12, 18]}
|
| 113 |
+
- {type: harv, owner: agent, position: [14, 18]}
|
| 114 |
+
- {type: harv, owner: agent, position: [14, 20]}
|
| 115 |
+
# Default patches (overridden per level for tier-specific geometry).
|
| 116 |
+
- {type: mine, owner: neutral, position: [16, 18]}
|
| 117 |
+
- {type: mine, owner: neutral, position: [80, 18]}
|
| 118 |
+
# Inert enemy marker far from the harvest envelope keeps the
|
| 119 |
+
# episode from auto-terminating on "all enemies dead" before the
|
| 120 |
+
# win/fail predicate is evaluated.
|
| 121 |
+
- {type: e1, owner: enemy, position: [120, 36], stance: 0}
|
| 122 |
+
|
| 123 |
+
levels:
|
| 124 |
+
easy:
|
| 125 |
+
description: >
|
| 126 |
+
Two patches and two harvesters. A — NEAR patch at (16,18) — is
|
| 127 |
+
rich per-cell; B — MID patch at (60,18) — is slower per round-
|
| 128 |
+
trip. The win predicate requires economy_value ≥ 4000 AND at
|
| 129 |
+
least one harvester in EACH patch's region (radius 6). Stacking
|
| 130 |
+
both harvs on A pumps 18000 cr but never enters B's region —
|
| 131 |
+
LOSS. Stacking both on B earns only ~2000 cr — LOSS. The split
|
| 132 |
+
(1 harv to A, 1 to B) clears both routing clauses AND the modest
|
| 133 |
+
cash bar — WIN. The capability asked is "use BOTH patches with
|
| 134 |
+
one harv each, don't monocrop the best-looking one".
|
| 135 |
+
starting_cash: 0
|
| 136 |
+
overrides:
|
| 137 |
+
actors:
|
| 138 |
+
- {type: fact, owner: agent, position: [10, 22]}
|
| 139 |
+
- {type: proc, owner: agent, position: [12, 18]}
|
| 140 |
+
- {type: harv, owner: agent, position: [14, 18]}
|
| 141 |
+
- {type: harv, owner: agent, position: [14, 20]}
|
| 142 |
+
- {type: mine, owner: neutral, position: [16, 18]}
|
| 143 |
+
- {type: mine, owner: neutral, position: [60, 18]}
|
| 144 |
+
- {type: e1, owner: enemy, position: [120, 36], stance: 0}
|
| 145 |
+
win_condition:
|
| 146 |
+
all_of:
|
| 147 |
+
- economy_value_gte: 4000
|
| 148 |
+
- has_building: fact
|
| 149 |
+
- units_of_type_in_region_gte:
|
| 150 |
+
type: harv
|
| 151 |
+
x: 16
|
| 152 |
+
y: 18
|
| 153 |
+
radius: 6
|
| 154 |
+
n: 1
|
| 155 |
+
- units_of_type_in_region_gte:
|
| 156 |
+
type: harv
|
| 157 |
+
x: 60
|
| 158 |
+
y: 18
|
| 159 |
+
radius: 6
|
| 160 |
+
n: 1
|
| 161 |
+
- within_ticks: 4500
|
| 162 |
+
# ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES.
|
| 163 |
+
fail_condition:
|
| 164 |
+
any_of:
|
| 165 |
+
- after_ticks: 4500
|
| 166 |
+
- not: {has_building: fact}
|
| 167 |
+
- not: {has_building: proc}
|
| 168 |
+
max_turns: 50
|
| 169 |
+
medium:
|
| 170 |
+
description: >
|
| 171 |
+
Same split-routing capability as easy, but B is pushed out to
|
| 172 |
+
(80,18) (FAR) so 2-on-B yields only ~2000 cr/4500t and the bar
|
| 173 |
+
is tightened to 5000 cr. The capability asked is identical —
|
| 174 |
+
one harv per patch — but the deadline now bites harder on a
|
| 175 |
+
monocrop policy: 2-on-A still earns the most credits (18000)
|
| 176 |
+
but the B-region clause is unsatisfied so the win never fires;
|
| 177 |
+
2-on-B clears the routing clause but only earns ~2000 cr; the
|
| 178 |
+
split (1 each) clears both at ~6000 cr around turn 35.
|
| 179 |
+
starting_cash: 0
|
| 180 |
+
win_condition:
|
| 181 |
+
all_of:
|
| 182 |
+
- economy_value_gte: 5000
|
| 183 |
+
- has_building: fact
|
| 184 |
+
- units_of_type_in_region_gte:
|
| 185 |
+
type: harv
|
| 186 |
+
x: 16
|
| 187 |
+
y: 18
|
| 188 |
+
radius: 6
|
| 189 |
+
n: 1
|
| 190 |
+
- units_of_type_in_region_gte:
|
| 191 |
+
type: harv
|
| 192 |
+
x: 80
|
| 193 |
+
y: 18
|
| 194 |
+
radius: 6
|
| 195 |
+
n: 1
|
| 196 |
+
- within_ticks: 4500
|
| 197 |
+
# ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES.
|
| 198 |
+
fail_condition:
|
| 199 |
+
any_of:
|
| 200 |
+
- after_ticks: 4500
|
| 201 |
+
- not: {has_building: fact}
|
| 202 |
+
- not: {has_building: proc}
|
| 203 |
+
max_turns: 50
|
| 204 |
+
hard:
|
| 205 |
+
description: >
|
| 206 |
+
Two spawn groups round-robined per seed: NORTH base (y=14) and
|
| 207 |
+
SOUTH base (y=28). Four neutral mines stay fixed at
|
| 208 |
+
(16,14)/(16,28)/(80,14)/(80,28); the SPAWN-MATCHED (A,B) pair
|
| 209 |
+
flips per seed — NORTH → A=(16,14), B=(80,14); SOUTH →
|
| 210 |
+
A=(16,28), B=(80,28). The win predicate is `any_of` over the
|
| 211 |
+
two spawn-matched split clauses, so a memorised "always send to
|
| 212 |
+
(16,14)+(80,14)" policy LOSES on the SOUTH seeds because the
|
| 213 |
+
harvs never enter the SOUTH-matched regions of the satisfied
|
| 214 |
+
disjunct. The capability asked is "identify your own spawn,
|
| 215 |
+
then split-route to the MATCHED A and B regions" — a route-
|
| 216 |
+
planning task that cannot be memorised from a single seed.
|
| 217 |
+
starting_cash: 0
|
| 218 |
+
overrides:
|
| 219 |
+
actors:
|
| 220 |
+
# spawn_point 0 — base NORTH (proc + harvs around y=14).
|
| 221 |
+
- {type: fact, owner: agent, position: [10, 14], spawn_point: 0}
|
| 222 |
+
- {type: proc, owner: agent, position: [12, 14], spawn_point: 0}
|
| 223 |
+
- {type: harv, owner: agent, position: [14, 14], spawn_point: 0}
|
| 224 |
+
- {type: harv, owner: agent, position: [14, 15], spawn_point: 0}
|
| 225 |
+
# spawn_point 1 — base SOUTH (proc + harvs around y=28).
|
| 226 |
+
- {type: fact, owner: agent, position: [10, 28], spawn_point: 1}
|
| 227 |
+
- {type: proc, owner: agent, position: [12, 28], spawn_point: 1}
|
| 228 |
+
- {type: harv, owner: agent, position: [14, 28], spawn_point: 1}
|
| 229 |
+
- {type: harv, owner: agent, position: [14, 29], spawn_point: 1}
|
| 230 |
+
# Four shared patches — identical to both spawns; the MATCHED
|
| 231 |
+
# (A,B) pair flips per seed (NORTH → (16,14)+(80,14); SOUTH →
|
| 232 |
+
# (16,28)+(80,28)).
|
| 233 |
+
- {type: mine, owner: neutral, position: [16, 14]}
|
| 234 |
+
- {type: mine, owner: neutral, position: [80, 14]}
|
| 235 |
+
- {type: mine, owner: neutral, position: [16, 28]}
|
| 236 |
+
- {type: mine, owner: neutral, position: [80, 28]}
|
| 237 |
+
- {type: e1, owner: enemy, position: [120, 36], stance: 0}
|
| 238 |
+
win_condition:
|
| 239 |
+
all_of:
|
| 240 |
+
- has_building: fact
|
| 241 |
+
- within_ticks: 4500
|
| 242 |
+
- any_of:
|
| 243 |
+
# NORTH-matched split.
|
| 244 |
+
- all_of:
|
| 245 |
+
- economy_value_gte: 5000
|
| 246 |
+
- units_of_type_in_region_gte:
|
| 247 |
+
type: harv
|
| 248 |
+
x: 16
|
| 249 |
+
y: 14
|
| 250 |
+
radius: 6
|
| 251 |
+
n: 1
|
| 252 |
+
- units_of_type_in_region_gte:
|
| 253 |
+
type: harv
|
| 254 |
+
x: 80
|
| 255 |
+
y: 14
|
| 256 |
+
radius: 6
|
| 257 |
+
n: 1
|
| 258 |
+
# SOUTH-matched split.
|
| 259 |
+
- all_of:
|
| 260 |
+
- economy_value_gte: 5000
|
| 261 |
+
- units_of_type_in_region_gte:
|
| 262 |
+
type: harv
|
| 263 |
+
x: 16
|
| 264 |
+
y: 28
|
| 265 |
+
radius: 6
|
| 266 |
+
n: 1
|
| 267 |
+
- units_of_type_in_region_gte:
|
| 268 |
+
type: harv
|
| 269 |
+
x: 80
|
| 270 |
+
y: 28
|
| 271 |
+
radius: 6
|
| 272 |
+
n: 1
|
| 273 |
+
# ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES.
|
| 274 |
+
fail_condition:
|
| 275 |
+
any_of:
|
| 276 |
+
- after_ticks: 4500
|
| 277 |
+
- not: {has_building: fact}
|
| 278 |
+
- not: {has_building: proc}
|
| 279 |
+
max_turns: 50
|
tests/test_build_defensive_tower_line.py
ADDED
|
@@ -0,0 +1,297 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""build-defensive-tower-line scenario family, full loop on Rust.
|
| 2 |
+
|
| 3 |
+
The pack tests DEFENSIVE PERIMETER TOPOLOGY: when the threat is funnelled
|
| 4 |
+
through a known corridor whose WIDTH matters (y=18..22 at x=60), the right
|
| 5 |
+
architecture is one pbox per row across the FULL corridor width (a LINE),
|
| 6 |
+
NOT a cluster on the centre row and NOT a scatter near the base. This is
|
| 7 |
+
the sibling/inverse of `def-tower-line-vs-cluster` (which forces a
|
| 8 |
+
CLUSTER at a single bottleneck cell); together the two packs discriminate
|
| 9 |
+
whether the model understands the FORCING GEOMETRY (single-cell chokepoint
|
| 10 |
+
vs corridor-width approach).
|
| 11 |
+
|
| 12 |
+
Anchors: ERQA spatial commit / MicroRTS defense placement / military
|
| 13 |
+
perimeter (firewall rule placement).
|
| 14 |
+
|
| 15 |
+
The win predicate makes the LINE topology load-bearing — total pbox
|
| 16 |
+
count alone is not enough:
|
| 17 |
+
|
| 18 |
+
* `building_count_gte:{pbox, n:4}` ⇒ the agent built the full budget;
|
| 19 |
+
* `building_in_region:{pbox, x:60, y:Y, radius:0.5, count:1}` for each
|
| 20 |
+
of the four corridor rungs Y ∈ {18,19,21,22} ⇒ exactly one pbox per
|
| 21 |
+
row across the corridor (a tiny radius 0.5 means only the exact cell
|
| 22 |
+
counts, so a cluster on (60,20) misses ALL FOUR rungs and a scatter
|
| 23 |
+
near the base misses all four);
|
| 24 |
+
* `building_count_gte:{fact,n:1}` (present-tense — `has_building` is
|
| 25 |
+
the one-shot "ever-seen" set, see CLAUDE.md footgun);
|
| 26 |
+
* `within_ticks` paired with `after_ticks` in the fail clause ⇒ a
|
| 27 |
+
non-finisher is a real reachable timeout LOSS (no interrupts on this
|
| 28 |
+
pack ⇒ each step is exactly 90 ticks, so max_turns is a hard tick
|
| 29 |
+
budget that the `after_ticks` deadline reliably bites in).
|
| 30 |
+
|
| 31 |
+
The scripted-policy validations prove deterministically that:
|
| 32 |
+
|
| 33 |
+
* the intended LINE policy (one pbox at each of the four corridor rung
|
| 34 |
+
cells) WINS every level + every hard seed (1..4);
|
| 35 |
+
* stall / random-4-pbox (4 pboxes placed near the base, away from the
|
| 36 |
+
corridor) both LOSE every level + every hard seed — a real LOSS,
|
| 37 |
+
not a draw;
|
| 38 |
+
* the hard tier defines ≥2 spawn_point groups (NORTH base y=12 / SOUTH
|
| 39 |
+
base y=28) so a memorised base-relative placement cannot generalise.
|
| 40 |
+
"""
|
| 41 |
+
|
| 42 |
+
from __future__ import annotations
|
| 43 |
+
|
| 44 |
+
import pytest
|
| 45 |
+
|
| 46 |
+
pytest.importorskip("openra_train", reason="Rust env wheel not installed")
|
| 47 |
+
pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
|
| 48 |
+
|
| 49 |
+
from openra_bench.eval_core import run_level
|
| 50 |
+
from openra_bench.scenarios import load_pack
|
| 51 |
+
from openra_bench.scenarios.loader import PACKS_DIR, compile_level
|
| 52 |
+
|
| 53 |
+
PACK = PACKS_DIR / "build-defensive-tower-line.yaml"
|
| 54 |
+
LEVELS = ("easy", "medium", "hard")
|
| 55 |
+
SEEDS = (1, 2, 3, 4)
|
| 56 |
+
|
| 57 |
+
# Required corridor rung cells (the four "rungs" spanning y=18..22 at
|
| 58 |
+
# the choke column x=60; y=20 is the middle that the LINE topology
|
| 59 |
+
# leaves open by construction — placing on y=20 won't satisfy any rung).
|
| 60 |
+
RUNGS = [(60, 18), (60, 19), (60, 21), (60, 22)]
|
| 61 |
+
|
| 62 |
+
# Cells used by the "random-4-pbox" wrong-topology policy: 4 pboxes
|
| 63 |
+
# clustered near the base rather than along the corridor. None of these
|
| 64 |
+
# lie inside ANY rung region (radius 0.5 around the rung cells), so the
|
| 65 |
+
# region clauses are all unsatisfied.
|
| 66 |
+
RANDOM_CELLS_NEAR_BASE = [(20, 18), (22, 20), (24, 22), (26, 19)]
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
# ── scripted policies ────────────────────────────────────────────────
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def stall(rs, C):
|
| 73 |
+
"""Observe-only — the agent never spends. Fact gets razed by the
|
| 74 |
+
rush AND the count/region clauses are never satisfied."""
|
| 75 |
+
return [C.observe()]
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
def make_line():
|
| 79 |
+
"""Intended LINE topology: one pbox at EACH of the four corridor
|
| 80 |
+
rung cells (60,18) (60,19) (60,21) (60,22)."""
|
| 81 |
+
|
| 82 |
+
def policy(rs, C):
|
| 83 |
+
own_b = rs.get("own_buildings") or []
|
| 84 |
+
n = sum(1 for b in own_b if b.get("type") == "pbox")
|
| 85 |
+
prod = rs.get("production") or []
|
| 86 |
+
prod_items = [p.get("item") for p in prod if isinstance(p, dict)]
|
| 87 |
+
# Once 4 pboxes are up, idle (the win clause re-evaluates each turn).
|
| 88 |
+
if n >= len(RUNGS):
|
| 89 |
+
return [C.observe()]
|
| 90 |
+
cmds = []
|
| 91 |
+
if "pbox" not in prod_items:
|
| 92 |
+
cmds.append(C.build("pbox"))
|
| 93 |
+
cmds.append(C.place_building("pbox", RUNGS[n][0], RUNGS[n][1]))
|
| 94 |
+
return cmds
|
| 95 |
+
|
| 96 |
+
return policy
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def make_random_4_pbox():
|
| 100 |
+
"""WRONG TOPOLOGY: 4 pboxes placed near the base (not at the
|
| 101 |
+
corridor rungs). Satisfies `building_count_gte:{pbox,n:4}` but
|
| 102 |
+
FAILS every rung region (none of the cells lie in any rung's
|
| 103 |
+
radius-0.5 disk), so the win predicate cannot fire."""
|
| 104 |
+
|
| 105 |
+
def policy(rs, C):
|
| 106 |
+
own_b = rs.get("own_buildings") or []
|
| 107 |
+
n = sum(1 for b in own_b if b.get("type") == "pbox")
|
| 108 |
+
prod = rs.get("production") or []
|
| 109 |
+
prod_items = [p.get("item") for p in prod if isinstance(p, dict)]
|
| 110 |
+
if n >= len(RANDOM_CELLS_NEAR_BASE):
|
| 111 |
+
return [C.observe()]
|
| 112 |
+
cmds = []
|
| 113 |
+
if "pbox" not in prod_items:
|
| 114 |
+
cmds.append(C.build("pbox"))
|
| 115 |
+
cmds.append(
|
| 116 |
+
C.place_building(
|
| 117 |
+
"pbox",
|
| 118 |
+
RANDOM_CELLS_NEAR_BASE[n][0],
|
| 119 |
+
RANDOM_CELLS_NEAR_BASE[n][1],
|
| 120 |
+
)
|
| 121 |
+
)
|
| 122 |
+
return cmds
|
| 123 |
+
|
| 124 |
+
return policy
|
| 125 |
+
|
| 126 |
+
|
| 127 |
+
# ── scenario-shape invariants ────────────────────────────────────────
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
def test_pack_compiles_with_three_levels_and_rusher_bot():
|
| 131 |
+
pack = load_pack(PACK)
|
| 132 |
+
assert pack.meta.id == "build-defensive-tower-line"
|
| 133 |
+
assert pack.meta.capability == "reasoning"
|
| 134 |
+
assert set(pack.levels) == {"easy", "medium", "hard"}
|
| 135 |
+
# Required-by-spec benchmark anchors.
|
| 136 |
+
anchors = pack.meta.benchmark_anchor
|
| 137 |
+
assert "ERQA" in anchors, anchors
|
| 138 |
+
assert "MicroRTS defense" in anchors, anchors
|
| 139 |
+
assert "military perimeter" in anchors, anchors
|
| 140 |
+
# Rusher bot wired through (charges agent centroid → forces the
|
| 141 |
+
# rush path through the corridor on every seed).
|
| 142 |
+
for lvl in LEVELS:
|
| 143 |
+
c = compile_level(pack, lvl)
|
| 144 |
+
assert c.map_supported
|
| 145 |
+
bot = getattr(c.scenario.enemy, "bot_type", None) or getattr(
|
| 146 |
+
c.scenario.enemy, "bot", None
|
| 147 |
+
)
|
| 148 |
+
assert str(bot).lower() == "rusher", (lvl, bot)
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def test_starting_cash_is_exact_pbox_budget():
|
| 152 |
+
"""The cash is intentionally tight (4 pbox at 600 each = 2400 on
|
| 153 |
+
every level, zero slack). A model that spends on units OR extra
|
| 154 |
+
power runs out before the count clause is satisfied."""
|
| 155 |
+
pack = load_pack(PACK)
|
| 156 |
+
for lvl in LEVELS:
|
| 157 |
+
c = compile_level(pack, lvl)
|
| 158 |
+
assert c.starting_cash == 2400, (lvl, c.starting_cash)
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 162 |
+
def test_every_level_has_a_reachable_timeout_fail(level):
|
| 163 |
+
"""Non-win must be a real LOSS: the `after_ticks` fail clause must
|
| 164 |
+
be strictly below the tick reachable at max_turns. No interrupts on
|
| 165 |
+
this pack ⇒ each step is exactly 90 ticks (max tick = 93+90·(N-1))."""
|
| 166 |
+
c = compile_level(load_pack(PACK), level)
|
| 167 |
+
assert c.fail_condition is not None
|
| 168 |
+
fc = c.fail_condition.model_dump(exclude_none=True)
|
| 169 |
+
deadline = None
|
| 170 |
+
for clause in fc.get("any_of", []) or []:
|
| 171 |
+
if "after_ticks" in clause:
|
| 172 |
+
deadline = int(clause["after_ticks"])
|
| 173 |
+
assert deadline is not None, f"{level}: no after_ticks fail clause"
|
| 174 |
+
reachable = 93 + 90 * (c.max_turns - 1)
|
| 175 |
+
assert deadline < reachable, (
|
| 176 |
+
f"{level}: deadline {deadline} unreachable within "
|
| 177 |
+
f"{c.max_turns} turns (max tick {reachable}) → draw degeneracy"
|
| 178 |
+
)
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
def test_fact_alive_clause_uses_present_tense_predicate():
|
| 182 |
+
"""The fact-survival clause must use the PRESENT-TENSE predicate
|
| 183 |
+
(`building_count_gte:{type:fact,n:1}`) rather than `has_building`,
|
| 184 |
+
which is a one-shot "ever seen" set that stays true after the fact
|
| 185 |
+
is destroyed (a documented CLAUDE.md footgun). Otherwise the rush
|
| 186 |
+
razing the fact would not trigger a LOSS."""
|
| 187 |
+
for lvl in LEVELS:
|
| 188 |
+
c = compile_level(load_pack(PACK), lvl)
|
| 189 |
+
fc = c.fail_condition.model_dump(exclude_none=True)
|
| 190 |
+
fact_clauses = [
|
| 191 |
+
clause for clause in fc.get("any_of", []) or []
|
| 192 |
+
if isinstance(clause, dict)
|
| 193 |
+
and isinstance(clause.get("not"), dict)
|
| 194 |
+
and "building_count_gte" in (clause["not"] or {})
|
| 195 |
+
and (clause["not"]["building_count_gte"] or {}).get("type") == "fact"
|
| 196 |
+
]
|
| 197 |
+
assert fact_clauses, f"{lvl}: missing present-tense fact-alive fail clause"
|
| 198 |
+
|
| 199 |
+
|
| 200 |
+
def test_win_requires_one_pbox_per_corridor_rung():
|
| 201 |
+
"""The LINE-enforcement contract: every level's win clause requires
|
| 202 |
+
exactly one pbox in EACH of the four corridor rungs at x=60
|
| 203 |
+
y∈{18,19,21,22}. A cluster on the centre row (y=20) misses all four
|
| 204 |
+
rungs because each rung region has radius 0.5 (cell-exact)."""
|
| 205 |
+
for lvl in LEVELS:
|
| 206 |
+
c = compile_level(load_pack(PACK), lvl)
|
| 207 |
+
wc = c.win_condition.model_dump(exclude_none=True)
|
| 208 |
+
rungs_seen = set()
|
| 209 |
+
for clause in wc.get("all_of", []) or []:
|
| 210 |
+
br = clause.get("building_in_region")
|
| 211 |
+
if (
|
| 212 |
+
isinstance(br, dict)
|
| 213 |
+
and br.get("type") == "pbox"
|
| 214 |
+
and int(br.get("x", -1)) == 60
|
| 215 |
+
and int(br.get("count", 0)) == 1
|
| 216 |
+
and float(br.get("radius", 0)) <= 1.0
|
| 217 |
+
):
|
| 218 |
+
rungs_seen.add(int(br["y"]))
|
| 219 |
+
assert rungs_seen == {18, 19, 21, 22}, (
|
| 220 |
+
f"{lvl}: corridor rungs y∈{{18,19,21,22}} required, got {sorted(rungs_seen)}"
|
| 221 |
+
)
|
| 222 |
+
|
| 223 |
+
|
| 224 |
+
def test_hard_has_two_spawn_point_groups():
|
| 225 |
+
"""Hard-tier contract: ≥2 distinct agent spawn_point groups so a
|
| 226 |
+
memorised relative-to-base placement that lands in the same world
|
| 227 |
+
cell on every seed cannot generalise."""
|
| 228 |
+
c = compile_level(load_pack(PACK), "hard")
|
| 229 |
+
groups = {
|
| 230 |
+
a.spawn_point for a in c.scenario.actors
|
| 231 |
+
if a.owner == "agent" and a.spawn_point is not None
|
| 232 |
+
}
|
| 233 |
+
assert groups == {0, 1}, groups
|
| 234 |
+
# In-bounds check (rush-hour-arena playable y ≈ 2..38, x ≈ 2..126):
|
| 235 |
+
for a in c.scenario.actors:
|
| 236 |
+
x, y = a.position
|
| 237 |
+
assert 2 <= x <= 126 and 2 <= y <= 38, (a.type, a.position)
|
| 238 |
+
|
| 239 |
+
|
| 240 |
+
# ── solvency: intended LINE wins every level + every hard seed ───────
|
| 241 |
+
|
| 242 |
+
|
| 243 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 244 |
+
def test_intended_line_wins_every_level_and_seed(level):
|
| 245 |
+
c = compile_level(load_pack(PACK), level)
|
| 246 |
+
for seed in SEEDS:
|
| 247 |
+
r = run_level(c, make_line(), seed=seed)
|
| 248 |
+
assert r.outcome == "win", (
|
| 249 |
+
f"{level} seed{seed}: intended LINE topology must WIN; "
|
| 250 |
+
f"got {r.outcome} (tick={r.signals.game_tick}, "
|
| 251 |
+
f"kills={r.signals.units_killed}, "
|
| 252 |
+
f"lost={r.signals.units_lost}, "
|
| 253 |
+
f"buildings={r.signals.own_buildings})"
|
| 254 |
+
)
|
| 255 |
+
|
| 256 |
+
|
| 257 |
+
# ── no-cheat: every lazy / wrong-topology policy LOSES (not draws) ───
|
| 258 |
+
|
| 259 |
+
|
| 260 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 261 |
+
@pytest.mark.parametrize(
|
| 262 |
+
"policy_name,policy_factory",
|
| 263 |
+
[
|
| 264 |
+
("stall", lambda: stall),
|
| 265 |
+
("random_4_pbox", lambda: make_random_4_pbox()),
|
| 266 |
+
],
|
| 267 |
+
)
|
| 268 |
+
def test_lazy_and_wrong_topology_policies_lose_every_level_and_seed(
|
| 269 |
+
level, policy_name, policy_factory
|
| 270 |
+
):
|
| 271 |
+
"""Stall (rush razes fact AND clock runs out with no pbox) and
|
| 272 |
+
random-4-pbox (count satisfied but every rung region unsatisfied,
|
| 273 |
+
so the win never fires and the clock runs out) must ALL LOSE on
|
| 274 |
+
every level + every seed — no draw."""
|
| 275 |
+
c = compile_level(load_pack(PACK), level)
|
| 276 |
+
fn = policy_factory()
|
| 277 |
+
for seed in SEEDS:
|
| 278 |
+
r = run_level(c, fn, seed=seed)
|
| 279 |
+
assert r.outcome == "loss", (
|
| 280 |
+
f"{level} seed{seed} {policy_name}: must LOSE (real fail, "
|
| 281 |
+
f"not a draw); got {r.outcome} (tick={r.signals.game_tick}, "
|
| 282 |
+
f"buildings={r.signals.own_buildings})"
|
| 283 |
+
)
|
| 284 |
+
|
| 285 |
+
|
| 286 |
+
# ── determinism ──────────────────────────────────────────────────────
|
| 287 |
+
|
| 288 |
+
|
| 289 |
+
def test_intended_run_is_deterministic_on_easy():
|
| 290 |
+
c = compile_level(load_pack(PACK), "easy")
|
| 291 |
+
a = run_level(c, make_line(), seed=3)
|
| 292 |
+
b = run_level(c, make_line(), seed=3)
|
| 293 |
+
assert (a.outcome, a.turns, a.signals.units_killed) == (
|
| 294 |
+
b.outcome,
|
| 295 |
+
b.turns,
|
| 296 |
+
b.signals.units_killed,
|
| 297 |
+
), "same seed must be deterministic"
|
tests/test_econ_harvester_pathing_optimization.py
ADDED
|
@@ -0,0 +1,358 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Full contributor-loop validation for econ-harvester-pathing-optimization.
|
| 2 |
+
|
| 3 |
+
The pack tests harvester SPLIT-routing under heterogeneous round-trip
|
| 4 |
+
cost (OR vehicle-routing / SC2 worker-distribution / M/M/c queueing).
|
| 5 |
+
TWO harvesters, TWO patches — A near and B far. The optimal policy
|
| 6 |
+
sends ONE harv to A and ONE to B; the win predicate encodes the
|
| 7 |
+
routing requirement as `units_of_type_in_region_gte` clauses on BOTH
|
| 8 |
+
patch regions plus a modest cash bar.
|
| 9 |
+
|
| 10 |
+
Bar (per CLAUDE.md "no defect, no cheat"):
|
| 11 |
+
- stall LOSES every tier (no income, no region cover).
|
| 12 |
+
- 2-on-A LOSES every tier — pumps more credits than the bar but never
|
| 13 |
+
enters B's region, so the routing clause fails.
|
| 14 |
+
- 2-on-B LOSES every tier — clears B's region but the FAR round-trip
|
| 15 |
+
earns ~2000 cr, well below the cash bar.
|
| 16 |
+
- 1A+1B (intended SPLIT) WINS every tier and every hard seed.
|
| 17 |
+
- Hard: a memorised single-pair "always send to (16,14)+(80,14)"
|
| 18 |
+
policy LOSES on the spawn-mismatched seeds (the `any_of` disjunct
|
| 19 |
+
for that spawn pair never matches).
|
| 20 |
+
- The capability-policy that IDENTIFIES the matched (A,B) pair from
|
| 21 |
+
the harvs' Y row and splits accordingly WINS every hard seed.
|
| 22 |
+
"""
|
| 23 |
+
|
| 24 |
+
from __future__ import annotations
|
| 25 |
+
|
| 26 |
+
import pytest
|
| 27 |
+
|
| 28 |
+
pytest.importorskip("openra_train", reason="Rust env wheel not installed")
|
| 29 |
+
|
| 30 |
+
pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
|
| 31 |
+
from openra_bench.eval_core import run_level
|
| 32 |
+
from openra_bench.scenarios import load_pack
|
| 33 |
+
from openra_bench.scenarios.loader import PACKS_DIR, compile_level
|
| 34 |
+
|
| 35 |
+
PACK = PACKS_DIR / "econ-harvester-pathing-optimization.yaml"
|
| 36 |
+
|
| 37 |
+
# Easy / medium geometry.
|
| 38 |
+
EASY_A = (16, 18)
|
| 39 |
+
EASY_B = (60, 18)
|
| 40 |
+
MED_A = (16, 18)
|
| 41 |
+
MED_B = (80, 18)
|
| 42 |
+
|
| 43 |
+
# Hard tier: four patches, NORTH and SOUTH spawn-matched pairs.
|
| 44 |
+
HARD_NA = (16, 14)
|
| 45 |
+
HARD_NB = (80, 14)
|
| 46 |
+
HARD_SA = (16, 28)
|
| 47 |
+
HARD_SB = (80, 28)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
# ---------------------------------------------------------------- policies
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def stall_policy(rs, Command):
|
| 54 |
+
return [Command.observe()]
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def _make_alloc(targets):
|
| 58 |
+
"""Send harv[i] (in id order) to targets[i] every turn. The
|
| 59 |
+
`harvest` order persists so re-issuing is idempotent; passing
|
| 60 |
+
`None` for a slot leaves that harv idle (used by the stall-ish
|
| 61 |
+
one-harv probes)."""
|
| 62 |
+
def f(rs, Command):
|
| 63 |
+
harvs = sorted(
|
| 64 |
+
(u for u in rs.get("units_summary", []) if u.get("type") == "harv"),
|
| 65 |
+
key=lambda u: u["id"],
|
| 66 |
+
)
|
| 67 |
+
cmds = []
|
| 68 |
+
for h, t in zip(harvs, targets):
|
| 69 |
+
if t is not None:
|
| 70 |
+
cmds.append(Command.harvest([str(h["id"])], *t))
|
| 71 |
+
return cmds or [Command.observe()]
|
| 72 |
+
return f
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
def _make_smart_hard():
|
| 76 |
+
"""Hard-tier intended policy: identify the matched (A,B) pair from
|
| 77 |
+
the harvs' Y row (NORTH base → harvs at y=14..15 → split to
|
| 78 |
+
(16,14)+(80,14); SOUTH base → y=28..29 → split to (16,28)+(80,28))."""
|
| 79 |
+
def f(rs, Command):
|
| 80 |
+
harvs = sorted(
|
| 81 |
+
(u for u in rs.get("units_summary", []) if u.get("type") == "harv"),
|
| 82 |
+
key=lambda u: u["id"],
|
| 83 |
+
)
|
| 84 |
+
if not harvs:
|
| 85 |
+
return [Command.observe()]
|
| 86 |
+
y = harvs[0]["cell_y"]
|
| 87 |
+
if y < 21:
|
| 88 |
+
targets = [HARD_NA, HARD_NB]
|
| 89 |
+
else:
|
| 90 |
+
targets = [HARD_SA, HARD_SB]
|
| 91 |
+
return [
|
| 92 |
+
Command.harvest([str(h["id"])], *t)
|
| 93 |
+
for h, t in zip(harvs, targets)
|
| 94 |
+
]
|
| 95 |
+
return f
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
# ---------------------------------------------------------------- helpers
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def _run(level, policy_factory, seed=1):
|
| 102 |
+
c = compile_level(load_pack(PACK), level)
|
| 103 |
+
assert c.map_supported, "rush-hour-arena terrain must be present"
|
| 104 |
+
policy = policy_factory() if callable(policy_factory) else policy_factory
|
| 105 |
+
return c, run_level(c, policy, seed=seed)
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
def _ev(res):
|
| 109 |
+
return res.signals.cash + res.signals.resources
|
| 110 |
+
|
| 111 |
+
|
| 112 |
+
# ---------------------------------------------------------------- structural
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
def test_pack_loads_and_meta_active():
|
| 116 |
+
pack = load_pack(PACK)
|
| 117 |
+
assert pack.meta.status == "active"
|
| 118 |
+
assert pack.meta.id == "econ-harvester-pathing-optimization"
|
| 119 |
+
assert pack.meta.capability == "reasoning"
|
| 120 |
+
anchors = pack.meta.benchmark_anchor
|
| 121 |
+
# The task's three named real-world / benchmark anchors.
|
| 122 |
+
assert any("SC2" in a and "worker" in a for a in anchors), anchors
|
| 123 |
+
assert any("OR" in a and "routing" in a for a in anchors), anchors
|
| 124 |
+
assert any("M/M/c" in a for a in anchors), anchors
|
| 125 |
+
|
| 126 |
+
|
| 127 |
+
def test_all_tiers_have_reachable_deadlines():
|
| 128 |
+
"""tick-alignment idiom: within_ticks ≤ ceiling AND
|
| 129 |
+
after_ticks ≤ ceiling AND within_ticks == after_ticks (so a
|
| 130 |
+
non-finisher LOSES, not draws)."""
|
| 131 |
+
pack = load_pack(PACK)
|
| 132 |
+
|
| 133 |
+
def _find_within(node):
|
| 134 |
+
"""Recurse through nested all_of/any_of to find the
|
| 135 |
+
within_ticks leaf in the win condition."""
|
| 136 |
+
if isinstance(node, dict):
|
| 137 |
+
if "within_ticks" in node:
|
| 138 |
+
return int(node["within_ticks"])
|
| 139 |
+
for k in ("all_of", "any_of"):
|
| 140 |
+
if k in node:
|
| 141 |
+
for c in node[k]:
|
| 142 |
+
v = _find_within(c)
|
| 143 |
+
if v is not None:
|
| 144 |
+
return v
|
| 145 |
+
return None
|
| 146 |
+
|
| 147 |
+
for lvl in ("easy", "medium", "hard"):
|
| 148 |
+
L = pack.levels[lvl]
|
| 149 |
+
ceiling = 93 + 90 * (L.max_turns - 1)
|
| 150 |
+
wt = _find_within(L.win_condition.model_dump())
|
| 151 |
+
ft = next(
|
| 152 |
+
int(c["after_ticks"])
|
| 153 |
+
for c in L.fail_condition.model_dump()["any_of"]
|
| 154 |
+
if "after_ticks" in c
|
| 155 |
+
)
|
| 156 |
+
assert wt is not None, f"{lvl}: within_ticks missing from win"
|
| 157 |
+
assert wt <= ceiling, f"{lvl}: within_ticks {wt} > ceiling {ceiling}"
|
| 158 |
+
assert ft <= ceiling, f"{lvl}: after_ticks {ft} > ceiling {ceiling}"
|
| 159 |
+
assert wt == ft, (
|
| 160 |
+
f"{lvl}: within_ticks {wt} != after_ticks {ft} "
|
| 161 |
+
"(non-finisher must LOSE, not draw)"
|
| 162 |
+
)
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
def test_hard_has_two_seed_driven_spawn_groups():
|
| 166 |
+
"""Hard tier must define ≥2 agent spawn_point groups (the
|
| 167 |
+
UPGRADED contract — a single memorised opening can't generalise)."""
|
| 168 |
+
c = compile_level(load_pack(PACK), "hard")
|
| 169 |
+
sp = {
|
| 170 |
+
(a.spawn_point if a.spawn_point is not None else 0)
|
| 171 |
+
for a in c.scenario.actors
|
| 172 |
+
if a.owner == "agent"
|
| 173 |
+
}
|
| 174 |
+
assert len(sp) >= 2, (
|
| 175 |
+
f"hard must define ≥2 agent spawn_point groups; got {sorted(sp)}"
|
| 176 |
+
)
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
# ---------------------------------------------------------------- EASY
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
def test_easy_stall_loses():
|
| 183 |
+
_, res = _run("easy", lambda: stall_policy)
|
| 184 |
+
assert res.outcome == "loss", (
|
| 185 |
+
f"stall must LOSE easy; got {res.outcome} ev={_ev(res)}"
|
| 186 |
+
)
|
| 187 |
+
|
| 188 |
+
|
| 189 |
+
def test_easy_both_to_a_loses_despite_high_cash():
|
| 190 |
+
"""The crucial discrimination: 2-on-A earns ~18000 cr (well over
|
| 191 |
+
the 4000 bar) but never enters B's region — the routing clause
|
| 192 |
+
fails so the win predicate as a whole fails. LOSS, not WIN."""
|
| 193 |
+
_, res = _run("easy", lambda: _make_alloc([EASY_A, EASY_A]))
|
| 194 |
+
assert res.outcome == "loss", (
|
| 195 |
+
f"2-on-A must LOSE easy (no harv in B region); "
|
| 196 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 197 |
+
)
|
| 198 |
+
assert _ev(res) >= 8000, (
|
| 199 |
+
f"2-on-A should still HAVE earned a lot of cash "
|
| 200 |
+
f"(routing clause is the teeth, not cash); ev={_ev(res)}"
|
| 201 |
+
)
|
| 202 |
+
|
| 203 |
+
|
| 204 |
+
def test_easy_both_to_b_loses_on_cash():
|
| 205 |
+
"""2-on-B clears B's region but the FAR round-trip earns ~2000 cr
|
| 206 |
+
over 4500 ticks — well below the 4000 bar. LOSS."""
|
| 207 |
+
_, res = _run("easy", lambda: _make_alloc([EASY_B, EASY_B]))
|
| 208 |
+
assert res.outcome == "loss", (
|
| 209 |
+
f"2-on-B must LOSE easy (~2000 ev < 4000 bar); "
|
| 210 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 211 |
+
)
|
| 212 |
+
|
| 213 |
+
|
| 214 |
+
def test_easy_split_wins():
|
| 215 |
+
"""The intended split (1 harv to A, 1 harv to B) clears both
|
| 216 |
+
routing clauses AND the modest cash bar — WIN."""
|
| 217 |
+
_, res = _run("easy", lambda: _make_alloc([EASY_A, EASY_B]))
|
| 218 |
+
assert res.outcome == "win", (
|
| 219 |
+
f"1A+1B split must WIN easy; got {res.outcome} ev={_ev(res)}"
|
| 220 |
+
)
|
| 221 |
+
|
| 222 |
+
|
| 223 |
+
def test_easy_split_wins_either_assignment():
|
| 224 |
+
"""The assignment of which harv-id goes where doesn't matter —
|
| 225 |
+
routing is symmetric. Sanity check the reversed assignment also
|
| 226 |
+
wins (catches a hidden id-ordering dependency)."""
|
| 227 |
+
_, res = _run("easy", lambda: _make_alloc([EASY_B, EASY_A]))
|
| 228 |
+
assert res.outcome == "win", (
|
| 229 |
+
f"1B+1A split must WIN easy; got {res.outcome} ev={_ev(res)}"
|
| 230 |
+
)
|
| 231 |
+
|
| 232 |
+
|
| 233 |
+
# ---------------------------------------------------------------- MEDIUM
|
| 234 |
+
|
| 235 |
+
|
| 236 |
+
def test_medium_stall_loses():
|
| 237 |
+
_, res = _run("medium", lambda: stall_policy)
|
| 238 |
+
assert res.outcome == "loss", (
|
| 239 |
+
f"stall must LOSE medium; got {res.outcome} ev={_ev(res)}"
|
| 240 |
+
)
|
| 241 |
+
|
| 242 |
+
|
| 243 |
+
def test_medium_both_to_a_loses_despite_high_cash():
|
| 244 |
+
"""2-on-A still earns the most credits (~18000) but the B-region
|
| 245 |
+
clause is unsatisfied. LOSS."""
|
| 246 |
+
_, res = _run("medium", lambda: _make_alloc([MED_A, MED_A]))
|
| 247 |
+
assert res.outcome == "loss", (
|
| 248 |
+
f"2-on-A must LOSE medium (no harv in B region); "
|
| 249 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 250 |
+
)
|
| 251 |
+
|
| 252 |
+
|
| 253 |
+
def test_medium_both_to_b_loses_on_cash():
|
| 254 |
+
"""B at (80,18) is FAR — 2 harvs there earn only ~2000 cr over
|
| 255 |
+
4500 ticks, well below the 5000 bar."""
|
| 256 |
+
_, res = _run("medium", lambda: _make_alloc([MED_B, MED_B]))
|
| 257 |
+
assert res.outcome == "loss", (
|
| 258 |
+
f"2-on-B must LOSE medium (~2000 ev < 5000 bar); "
|
| 259 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 260 |
+
)
|
| 261 |
+
|
| 262 |
+
|
| 263 |
+
def test_medium_split_wins():
|
| 264 |
+
_, res = _run("medium", lambda: _make_alloc([MED_A, MED_B]))
|
| 265 |
+
assert res.outcome == "win", (
|
| 266 |
+
f"1A+1B split must WIN medium; got {res.outcome} ev={_ev(res)}"
|
| 267 |
+
)
|
| 268 |
+
|
| 269 |
+
|
| 270 |
+
def test_medium_split_wins_either_assignment():
|
| 271 |
+
_, res = _run("medium", lambda: _make_alloc([MED_B, MED_A]))
|
| 272 |
+
assert res.outcome == "win", (
|
| 273 |
+
f"1B+1A split must WIN medium; got {res.outcome} ev={_ev(res)}"
|
| 274 |
+
)
|
| 275 |
+
|
| 276 |
+
|
| 277 |
+
# ---------------------------------------------------------------- HARD
|
| 278 |
+
|
| 279 |
+
|
| 280 |
+
@pytest.mark.parametrize("seed", [1, 2, 3, 4])
|
| 281 |
+
def test_hard_stall_loses_every_seed(seed):
|
| 282 |
+
_, res = _run("hard", lambda: stall_policy, seed=seed)
|
| 283 |
+
assert res.outcome == "loss", (
|
| 284 |
+
f"stall must LOSE hard/seed{seed}; got {res.outcome} ev={_ev(res)}"
|
| 285 |
+
)
|
| 286 |
+
|
| 287 |
+
|
| 288 |
+
@pytest.mark.parametrize("seed", [1, 2, 3, 4])
|
| 289 |
+
def test_hard_both_to_matched_a_loses_every_seed(seed):
|
| 290 |
+
"""Stacking both harvs on the NORTH-near patch (16,14) is a "use
|
| 291 |
+
only A" policy. On a NORTH spawn it clears the A-region clause
|
| 292 |
+
but the NORTH B-region is empty. On a SOUTH spawn it's outright
|
| 293 |
+
far from both matched regions. LOSS every seed."""
|
| 294 |
+
_, res = _run("hard", lambda: _make_alloc([HARD_NA, HARD_NA]), seed=seed)
|
| 295 |
+
assert res.outcome == "loss", (
|
| 296 |
+
f"2-on-A(north) must LOSE hard/seed{seed}; "
|
| 297 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 298 |
+
)
|
| 299 |
+
|
| 300 |
+
|
| 301 |
+
@pytest.mark.parametrize("seed", [1, 2, 3, 4])
|
| 302 |
+
def test_hard_both_to_matched_b_loses_every_seed(seed):
|
| 303 |
+
"""Mirror: 2-on-B(north) — far patch only, never enters A's
|
| 304 |
+
region (on either spawn) and earns ~1000 cr."""
|
| 305 |
+
_, res = _run("hard", lambda: _make_alloc([HARD_NB, HARD_NB]), seed=seed)
|
| 306 |
+
assert res.outcome == "loss", (
|
| 307 |
+
f"2-on-B(north) must LOSE hard/seed{seed}; "
|
| 308 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 309 |
+
)
|
| 310 |
+
|
| 311 |
+
|
| 312 |
+
def test_hard_memorised_north_split_loses_on_south_spawn_seeds():
|
| 313 |
+
"""A model that memorises "always split (16,14)+(80,14)" loses on
|
| 314 |
+
the SOUTH-spawn seeds (1 and 3 per round-robin) because the harvs
|
| 315 |
+
head to NORTH-matched cells and never enter the SOUTH-matched
|
| 316 |
+
regions — the SOUTH disjunct fails (no harvs in its regions) and
|
| 317 |
+
the NORTH disjunct fails on cash (the harvs are too far from
|
| 318 |
+
their own proc to round-trip efficiently)."""
|
| 319 |
+
for seed in (1, 3):
|
| 320 |
+
_, res = _run("hard", lambda: _make_alloc([HARD_NA, HARD_NB]), seed=seed)
|
| 321 |
+
assert res.outcome == "loss", (
|
| 322 |
+
f"memorised-NORTH-split must LOSE hard/seed{seed} (SOUTH spawn); "
|
| 323 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 324 |
+
)
|
| 325 |
+
|
| 326 |
+
|
| 327 |
+
def test_hard_memorised_south_split_loses_on_north_spawn_seeds():
|
| 328 |
+
"""Symmetric: memorising "always split (16,28)+(80,28)" loses on
|
| 329 |
+
NORTH-spawn seeds 2 and 4."""
|
| 330 |
+
for seed in (2, 4):
|
| 331 |
+
_, res = _run("hard", lambda: _make_alloc([HARD_SA, HARD_SB]), seed=seed)
|
| 332 |
+
assert res.outcome == "loss", (
|
| 333 |
+
f"memorised-SOUTH-split must LOSE hard/seed{seed} (NORTH spawn); "
|
| 334 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 335 |
+
)
|
| 336 |
+
|
| 337 |
+
|
| 338 |
+
@pytest.mark.parametrize("seed", [1, 2, 3, 4])
|
| 339 |
+
def test_hard_smart_spawn_matched_split_wins_every_seed(seed):
|
| 340 |
+
"""The intended capability — identify the spawn-matched (A,B)
|
| 341 |
+
pair from the agent's harv Y position, then split-route to BOTH
|
| 342 |
+
matched patches — WINS every seed cleanly."""
|
| 343 |
+
_, res = _run("hard", _make_smart_hard, seed=seed)
|
| 344 |
+
assert res.outcome == "win", (
|
| 345 |
+
f"SMART spawn-matched split must WIN hard/seed{seed}; "
|
| 346 |
+
f"got {res.outcome} ev={_ev(res)}"
|
| 347 |
+
)
|
| 348 |
+
|
| 349 |
+
|
| 350 |
+
# ---------------------------------------------------------------- determinism
|
| 351 |
+
|
| 352 |
+
|
| 353 |
+
def test_outcomes_are_deterministic_per_seed():
|
| 354 |
+
"""Same seed, same policy → identical outcome and ev."""
|
| 355 |
+
c = compile_level(load_pack(PACK), "medium")
|
| 356 |
+
a = run_level(c, _make_alloc([MED_A, MED_B]), seed=2)
|
| 357 |
+
b = run_level(c, _make_alloc([MED_A, MED_B]), seed=2)
|
| 358 |
+
assert (a.outcome, a.turns, _ev(a)) == (b.outcome, b.turns, _ev(b))
|
tests/test_hard_tier.py
CHANGED
|
@@ -117,6 +117,14 @@ UPGRADED = [
|
|
| 117 |
# flips per seed ((16,14) for NORTH, (16,28) for SOUTH), so a
|
| 118 |
# memorised "always send to (16,14)" cannot generalise.
|
| 119 |
"econ-multi-patch-allocation",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
# Group F opening greenfield seed (Wave-4): cold-start from a
|
| 121 |
# single MCV with no buildings / no harvester / no income. Hard
|
| 122 |
# tier defines 2 agent spawn_point groups (NORTH (20,14) / SOUTH
|
|
|
|
| 117 |
# flips per seed ((16,14) for NORTH, (16,28) for SOUTH), so a
|
| 118 |
# memorised "always send to (16,14)" cannot generalise.
|
| 119 |
"econ-multi-patch-allocation",
|
| 120 |
+
# Wave-7 econ reasoning: split-routing under heterogeneous
|
| 121 |
+
# round-trip cost (OR vehicle-routing / SC2 worker-distribution /
|
| 122 |
+
# M/M/c). Hard defines two agent spawn_point groups (NORTH base
|
| 123 |
+
# y=14 / SOUTH base y=28) round-robined by seed; four neutral
|
| 124 |
+
# mines stay fixed but the SPAWN-MATCHED (A,B) pair flips per
|
| 125 |
+
# seed (NORTH → (16,14)+(80,14); SOUTH → (16,28)+(80,28)), so a
|
| 126 |
+
# memorised single-pair split cannot generalise.
|
| 127 |
+
"econ-harvester-pathing-optimization",
|
| 128 |
# Group F opening greenfield seed (Wave-4): cold-start from a
|
| 129 |
# single MCV with no buildings / no harvester / no income. Hard
|
| 130 |
# tier defines 2 agent spawn_point groups (NORTH (20,14) / SOUTH
|