yxc20098 commited on
Commit
6defda6
·
1 Parent(s): 5f5dde4

feat(scenario): tech-turtle-defensive-tech — fortify-then-tech (SC2 turtle macro / military fortify-then-research anchor)

Browse files

Wave-4 TURTLE node of the tech triple (alongside tech-aggro-all-in
AGGRO and tech-balanced-econ-then-tech BALANCED). Allied agent must
fortify the lane FIRST (3 pbox + 1 gun) and THEN tech up (weap +
dome), in that order, under real attack pressure from a scripted
hunt-bot enemy band of e1+e3 closing on the base.

Idiom: Wave-2 then:[A,B,C,D] happened-before composite enforces the
defence-then-tech sequencing:
1. building_count_gte: {type: pbox, n: 3|8}
2. building_count_gte: {type: gun, n: 1}
3. has_building: weap
4. has_building: dome

Engine-prereq nuance: gun requires weap, so the natural build order
becomes pbox×3, weap (clause 3 latent), gun (clause 2 latches, clause
3 greedily latches), dome — the chain still encodes 'defence FIRST
then tech' because pbox×3 must be observed before clause 2 can
advance, and the hunt-bot pressure is the discriminator against
pure-tech-skip-defence (no defences ⇒ chain stuck and/or fact razed).

Real-world anchors:
- SC2 turtle macro / 'turtle to victory' defensive-tech timing
- Military fortify-then-research doctrine
- Cold War nuclear-shield-then-research analogue
- Risk-averse capital allocation: hedge first, THEN bet

Discriminations verified by scripted policies (tests/test_tech_turtle
_defensive_tech.py × seeds 1..4 × 3 levels):
- stall (observe only): LOSS (clock + fail)
- pure-aggro-skip-defence (race weap+dome, no pbox/gun): LOSS
(chain never advances past phase 1; or fact razed)
- only-defence-no-tech (spam pbox forever): LOSS (chain stuck
at phase 1; no weap/dome ever)
- intended fortify-then-tech (pbox×3, weap, gun, dome): WIN

Difficulty axis:
- easy: 4× e1 hunt at x=95, generous clock 7200
- medium: 4× e1 + 1× e3 hunt at x=105, units_lost_lte: 6
- hard: 2 hunt bands (north+south) at x=100, pbox×8 bar,
≥2 spawn_point groups (NORTH/SOUTH base) round-robined by seed

Pre-placed agent base: fact + tent + powr + 2nd powr (for pbox×3
power drain) + proc + harv + mine (so income flows from turn 1
and the chain can be funded inside the 3000 starting_cash + ~95
cr/turn harv income window).

Model smoke (Together Qwen3.6-Plus, medium s=1): loss
composite=0.3675 (perception=0.0342 reasoning=0.6548 action=1.0
objective=0.9999) — clean run, no crash.

Hard tier registered in tests/test_hard_tier.py::UPGRADED (the
spawn-variation contract: ≥2 agent spawn_point groups round-
robined by seed).

openra_bench/scenarios/packs/tech-turtle-defensive-tech.yaml ADDED
@@ -0,0 +1,342 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # tech-turtle-defensive-tech.yaml
2
+ #
3
+ # REASONING capability — Wave-4 TURTLE variant of the tech triple.
4
+ # Siblings: `tech-aggro-all-in` (AGGRO — race tech, skip everything
5
+ # else) and `tech-balanced-econ-then-tech` (BALANCED — econ floor
6
+ # first, then tech). This pack is the TURTLE cell: build defences
7
+ # FIRST, then tech up safely behind them. A scripted `hunt` enemy
8
+ # band of e1+e3 closes on the agent base by ~tick 1500 and will
9
+ # raze ANY tech building (or the construction yard itself) unless
10
+ # the lane is hardened first.
11
+ #
12
+ # The Wave-2 `then:[A,B,C,D]` happened-before composite enforces the
13
+ # defence-then-tech sequencing:
14
+ # 1. building_count_gte: {type: pbox, n: 3} # phase 1 — anti-inf wall
15
+ # 2. building_count_gte: {type: gun, n: 1} # phase 2 — anti-armour cap
16
+ # 3. has_building: weap # phase 3 — tech step 1
17
+ # 4. has_building: dome # phase 4 — tech step 2
18
+ #
19
+ # IMPORTANT engine-prereq nuance: `gun` (allied gun turret) requires
20
+ # `weap` (engine prereq). So the natural BUILD order is:
21
+ # pbox×3 (clause 1 latches — defences up)
22
+ # weap (clause 3 satisfied but NOT latched — chain blocked at 2)
23
+ # gun×1 (clause 2 latches; clause 3 greedily latches in same eval)
24
+ # dome (clause 4 latches — chain complete)
25
+ # The `then` chain still encodes "defence FIRST then tech": pbox×3
26
+ # must be observed BEFORE the chain can advance past phase 1, so any
27
+ # pure-tech skip cannot satisfy phase 1 → chain stuck. The hunt-bot
28
+ # pressure is the discriminator against "skip pbox and race tech":
29
+ # without 3 pbox up before the hunt band arrives (~tick 1500), the
30
+ # tech buildings (or the fact itself) get razed and the
31
+ # `not has_building: fact` fail clause fires.
32
+ #
33
+ # Real-world anchors:
34
+ # - SC2 turtle macro / "turtle to victory" defensive-tech timing
35
+ # - Military fortify-then-research doctrine (Cold War nuclear-shield
36
+ # -then-research analogue): build the shield BEFORE the new
37
+ # capability, so the new capability isn't lost before it ships.
38
+ # - Risk-averse capital allocation: hedge first (downside cap), THEN
39
+ # bet (upside).
40
+ #
41
+ # DISCRIMINATIONS (bar from CLAUDE.md — no defect, no cheat):
42
+ # - stall (observe only): LOSS — clock + hunt razes fact.
43
+ # - pure-aggro-skip-defence (race weap+dome, no pbox/gun): LOSS —
44
+ # hunt band reaches fact ~tick 1500 with no defences ⇒ fact razed
45
+ # ⇒ `not has_building: fact` fail fires.
46
+ # - build-only-defence-no-tech (only pbox spam, no weap/dome): LOSS
47
+ # — chain stuck at phase 1; clock expires.
48
+ # - intended defence-then-tech (pbox×3, weap, gun, dome): WIN.
49
+ #
50
+ # ENGINE FACTS used (CLAUDE.md / SCENARIO_QUALITY.md):
51
+ # - allied agent: `tent` (barracks) prereq for `pbox` (600cr)
52
+ # - `pbox` prereq `tent`; allied + 600cr each
53
+ # - `gun` prereq `weap` (engine-enforced); allied + 800cr
54
+ # - `weap` (~2000cr) prereq `proc`
55
+ # - `dome` (~1000cr) prereq `proc`
56
+ # - `proc` (1400cr) prereq `powr` — pre-placed harv harvests it
57
+ # - hunt bot: every unit attacks the nearest agent unit / building
58
+ # - inert enemy `fact` marker far east prevents auto-DRAW on
59
+ # enemy-elim (the hunt band would all die otherwise)
60
+ # - `then:` greedy advance — once a prior clause latches, any
61
+ # currently-true later clauses latch in the same evaluation
62
+ # (proven by tests/test_then_composite.py)
63
+ #
64
+ # Validate (no model / no network):
65
+ # python3 -m openra_bench.scenarios.validate \
66
+ # openra_bench/scenarios/packs/tech-turtle-defensive-tech.yaml
67
+
68
+ meta:
69
+ id: tech-turtle-defensive-tech
70
+ title: 'Tech Turtle — Fortify the Lane FIRST, THEN Tech Up Safely'
71
+ capability: reasoning
72
+ real_world_meaning: >
73
+ Defensive-tech sequencing under real attack pressure: build the
74
+ static defence line (pillboxes covering the lane mouth, then a
75
+ gun turret cap) BEFORE committing the budget to deeper tech
76
+ buildings (war factory, radar dome). A `hunt` enemy band is
77
+ closing on the base by ~tick 1500 and will raze any tech
78
+ building (or the construction yard) unless the lane is hardened
79
+ first. Pure aggro (race tech, skip defence) loses the base to
80
+ the hunt band; pure turtle (defences only, no tech) never
81
+ satisfies the tech clauses; only the fortify-then-research
82
+ sequence wins inside the budget.
83
+ robotics_analogue: >
84
+ Capability development under live adversarial threat: ship the
85
+ operational hardening (perimeter, monitoring, redundancy) BEFORE
86
+ rolling out the new capability that the adversary is incentivised
87
+ to disrupt. A roadmap that ships the capability without first
88
+ standing up the perimeter loses the capability the moment the
89
+ adversary arrives; a roadmap that only ever hardens never ships
90
+ the capability the mission requires.
91
+ benchmark_anchor:
92
+ - "SC2 turtle macro / defensive-tech timing"
93
+ - "military strategic-defense doctrine (fortify before research)"
94
+ - "Cold War nuclear-shield-then-research analogue"
95
+ - "risk-averse capital allocation: hedge first then bet"
96
+ author: openra-bench
97
+
98
+ base_map: rush-hour-arena
99
+
100
+ base:
101
+ agent:
102
+ faction: allies
103
+ enemy:
104
+ faction: soviet
105
+ bot_type: hunt
106
+ tools:
107
+ - observe
108
+ - build
109
+ - place_building
110
+ - harvest
111
+ - move_units
112
+ - attack_unit
113
+ - attack_move
114
+ - stop
115
+ planning: true
116
+ termination:
117
+ max_ticks: 40000
118
+
119
+ starting_cash: 3000
120
+
121
+ levels:
122
+ # ── EASY ─────────────────────────────────────────────────────────
123
+ # Bare turtle-then-tech rehearsal: a smaller hunt band (4× e1)
124
+ # closing slowly from the mid-map, generous clock. The chain still
125
+ # bites (pbox×3 must be observed-before gun-1 / weap / dome) but
126
+ # the income window is forgiving. Stall / pure-aggro / pure-turtle
127
+ # all LOSE; intended fortify-then-tech WINS.
128
+ # max_turns 90 → reachable tick 93 + 90·89 = 8103 ≥ 7200.
129
+ easy:
130
+ description: >
131
+ A small hunt-band of 4 enemy rifle infantry is closing on your
132
+ base from the mid-map. Build the defence line FIRST, then tech
133
+ up — in this order: (1) build at least 3 pillboxes (pbox), then
134
+ (2) build at least 1 gun turret (gun — needs a war factory),
135
+ (3) own a war factory (weap), (4) own a radar dome (dome). The
136
+ pre-placed refinery and harvester give you income. If you skip
137
+ defences the hunt band razes your base; if you only build
138
+ defences and never tech, the chain stalls; stalling loses on
139
+ the clock. Win = the defence-then-tech chain completed before
140
+ tick 7200, with the construction yard still standing and ≥1
141
+ own unit.
142
+ starting_cash: 3000
143
+ overrides:
144
+ actors:
145
+ # Agent base — fact + tent + powr per spec, PLUS proc + harv +
146
+ # mine so income flows from turn 1 (the tech chain needs
147
+ # ~5600 cash on top of the ~1800 for the 3 pbox; without harv
148
+ # income the bare starting_cash cannot fund the chain). Layout
149
+ # puts vulnerable buildings (fact, proc) WEST and the screen
150
+ # / pbox lane mouth to the EAST so the hunt band hits the
151
+ # defences BEFORE the income building.
152
+ - {type: fact, owner: agent, position: [10, 20]}
153
+ - {type: tent, owner: agent, position: [8, 18]}
154
+ - {type: powr, owner: agent, position: [6, 20]}
155
+ # Second powr — three pbox each drain -20 power and the
156
+ # tech buildings (weap −30, dome −40) push the surplus
157
+ # negative without it; underpowered queues construct very
158
+ # slowly, so weap never completes inside the clock.
159
+ - {type: powr, owner: agent, position: [6, 22]}
160
+ - {type: proc, owner: agent, position: [6, 16]}
161
+ - {type: harv, owner: agent, position: [8, 22]}
162
+ # Mine to the WEST of base so the harv has somewhere to go
163
+ # without crossing the lane mouth (the hunt band's lane).
164
+ - {type: mine, owner: neutral, position: [2, 20]}
165
+ # Three starting riflemen guarding the EAST lane mouth — the
166
+ # first line that engages the hunt band and gives pbox×3 time
167
+ # to come online.
168
+ - {type: e1, owner: agent, position: [16, 20], stance: 3, count: 3}
169
+ # Hunt band — 4× e1 starting east at x=95. At ~3–4 cells/turn
170
+ # the band needs ~20+ turns (~tick 2200) to reach the lane
171
+ # mouth at x≈16 (where the screen sits) — enough budget for
172
+ # pbox×3 (build/place ≈ 15 turns) to stand up first. The
173
+ # starting riflemen blunt the leading edge for a few more
174
+ # turns.
175
+ - {type: e1, owner: enemy, position: [95, 20], stance: 3, count: 4}
176
+ # Unarmed high-HP enemy marker far east — prevents engine
177
+ # auto-DRAW on enemy-elimination once the hunt band dies
178
+ # (CLAUDE.md footgun).
179
+ - {type: fact, owner: enemy, position: [120, 20]}
180
+ win_condition:
181
+ all_of:
182
+ - then:
183
+ id: defense-then-tech-easy
184
+ clauses:
185
+ - {building_count_gte: {type: pbox, n: 3}}
186
+ - {building_count_gte: {type: gun, n: 1}}
187
+ - {has_building: weap}
188
+ - {has_building: dome}
189
+ - {within_ticks: 7200}
190
+ - {own_units_gte: 1}
191
+ fail_condition:
192
+ any_of:
193
+ - {after_ticks: 7201}
194
+ - {not: {has_building: fact}}
195
+ max_turns: 90
196
+
197
+ # ── MEDIUM ───────────────────────────────────────────────────────
198
+ # +1 controlled variable: HEAVIER hunt band (6× e1 + e3 mix) closing
199
+ # FROM CLOSER, plus an attrition cap (units_lost_lte: 4). The hunt
200
+ # band reaches the lane mouth faster, so the pbox-first phase must
201
+ # be executed promptly — over-investing in tech before the wall
202
+ # bleeds the small starting unit pool past the cap.
203
+ # max_turns 80 → reachable tick 93 + 90·79 = 7203 ≥ 7200.
204
+ medium:
205
+ description: >
206
+ A heavier hunt-band — 4 rifle infantry plus 1 rocket soldier
207
+ (e3) — is closing on your base from the far east. Build the
208
+ defence line FIRST, then tech up — in this order: (1) ≥3
209
+ pillboxes (pbox), (2) ≥1 gun turret (gun — needs war factory),
210
+ (3) war factory (weap), (4) radar dome (dome). Same fail
211
+ conditions as easy plus an attrition cap: you may lose at most
212
+ 6 of your own units. Skipping defences loses the base to the
213
+ hunt band; building only defences never satisfies the tech
214
+ chain; stalling loses on the clock; sloppy turtling that
215
+ bleeds the starting infantry past 6 lost also loses.
216
+ starting_cash: 3000
217
+ overrides:
218
+ actors:
219
+ - {type: fact, owner: agent, position: [10, 20]}
220
+ - {type: tent, owner: agent, position: [8, 18]}
221
+ - {type: powr, owner: agent, position: [6, 20]}
222
+ - {type: powr, owner: agent, position: [6, 22]}
223
+ - {type: proc, owner: agent, position: [6, 16]}
224
+ - {type: harv, owner: agent, position: [8, 22]}
225
+ - {type: mine, owner: neutral, position: [2, 20]}
226
+ # Starting defenders: 6 e1 + 2 e3 at the EAST lane mouth to
227
+ # blunt the hunt band's leading edge while the pbox wall
228
+ # stands up. The attrition cap (units_lost_lte: 6) is
229
+ # measurable from this real baseline and adds the "don't
230
+ # over-bleed the screen" axis.
231
+ - {type: e1, owner: agent, position: [16, 20], stance: 3, count: 6}
232
+ - {type: e3, owner: agent, position: [16, 22], stance: 3, count: 2}
233
+ # Hunt band — heavier (4 e1 + 2 e3) starting far east at
234
+ # x=105 so the band needs ~25 turns (~tick 2800) to reach
235
+ # the lane mouth at x≈18. The pbox×3 wall (built ≈ tick
236
+ # 1500) and starting screen blunt it before it reaches the
237
+ # fact / proc. Hunt is the discriminator against pure-aggro
238
+ # (no defences ⇒ fact razed ⇒ fail).
239
+ - {type: e1, owner: enemy, position: [105, 20], stance: 3, count: 4}
240
+ - {type: e3, owner: enemy, position: [107, 20], stance: 3, count: 1}
241
+ # Inert enemy `fact` marker (anti-DRAW).
242
+ - {type: fact, owner: enemy, position: [120, 20]}
243
+ win_condition:
244
+ all_of:
245
+ - then:
246
+ id: defense-then-tech-medium
247
+ clauses:
248
+ - {building_count_gte: {type: pbox, n: 3}}
249
+ - {building_count_gte: {type: gun, n: 1}}
250
+ - {has_building: weap}
251
+ - {has_building: dome}
252
+ - {within_ticks: 7200}
253
+ - {own_units_gte: 1}
254
+ - {units_lost_lte: 6}
255
+ fail_condition:
256
+ any_of:
257
+ - {after_ticks: 7201}
258
+ - {not: {has_building: fact}}
259
+ - {not: {units_lost_lte: 6}}
260
+ max_turns: 80
261
+
262
+ # ── HARD ─────────────────────────────────────────────────────────
263
+ # +1 controlled variable: TWO hunt bands (8× total) hitting from
264
+ # TWO latitudes (north y=10 and south y=30), AND ≥2 spawn_point
265
+ # groups so the agent base latitude varies by seed. The pbox bar
266
+ # is raised to 8 to cover both lanes (4 per lane). Same clock; same
267
+ # attrition cap.
268
+ # max_turns 80 → reachable tick 93 + 90·79 = 7203 ≥ 7200.
269
+ hard:
270
+ description: >
271
+ Two hunt-bands hit your base from two different latitudes
272
+ (north and south). Your base stages from a seed-chosen
273
+ latitude — a single memorised pillbox cell cannot generalise.
274
+ Build the defence line FIRST, then tech up — in this order:
275
+ (1) ≥8 pillboxes (pbox — enough to cover both lanes),
276
+ (2) ≥1 gun turret (gun — needs war factory), (3) war factory
277
+ (weap), (4) radar dome (dome). Same attrition cap (≤6 own
278
+ lost) and clock (tick 7200) as medium. Skip defences, only-
279
+ defences, or over-attrition all lose; only the full fortify-
280
+ then-tech chain wins.
281
+ starting_cash: 3000
282
+ overrides:
283
+ actors:
284
+ # NORTH base spawn (spawn_point 0). Layout: vulnerable
285
+ # buildings (fact, proc) to the WEST; screen + future pbox
286
+ # lane mouth to the EAST so the hunt band hits defences
287
+ # BEFORE the income building.
288
+ - {type: fact, owner: agent, position: [10, 14], spawn_point: 0}
289
+ - {type: tent, owner: agent, position: [8, 12], spawn_point: 0}
290
+ - {type: powr, owner: agent, position: [6, 14], spawn_point: 0}
291
+ # Extra powr per spawn — 8 pbox is -160 power; needed.
292
+ - {type: powr, owner: agent, position: [6, 16], spawn_point: 0}
293
+ - {type: powr, owner: agent, position: [8, 10], spawn_point: 0}
294
+ - {type: proc, owner: agent, position: [6, 10], spawn_point: 0}
295
+ - {type: harv, owner: agent, position: [8, 16], spawn_point: 0}
296
+ - {type: e1, owner: agent, position: [16, 14], stance: 3, count: 6, spawn_point: 0}
297
+ - {type: e3, owner: agent, position: [16, 16], stance: 3, count: 2, spawn_point: 0}
298
+ # SOUTH base spawn (spawn_point 1) — symmetric.
299
+ - {type: fact, owner: agent, position: [10, 26], spawn_point: 1}
300
+ - {type: tent, owner: agent, position: [8, 28], spawn_point: 1}
301
+ - {type: powr, owner: agent, position: [6, 26], spawn_point: 1}
302
+ - {type: powr, owner: agent, position: [6, 24], spawn_point: 1}
303
+ - {type: powr, owner: agent, position: [8, 30], spawn_point: 1}
304
+ - {type: proc, owner: agent, position: [6, 30], spawn_point: 1}
305
+ - {type: harv, owner: agent, position: [8, 24], spawn_point: 1}
306
+ - {type: e1, owner: agent, position: [16, 26], stance: 3, count: 6, spawn_point: 1}
307
+ - {type: e3, owner: agent, position: [16, 24], stance: 3, count: 2, spawn_point: 1}
308
+ # Two ore patches, one per spawn cluster (engine doesn't
309
+ # filter neutral actors by spawn_point — both placed). Mines
310
+ # west of base so the harv loop doesn't cross the lane.
311
+ - {type: mine, owner: neutral, position: [2, 14]}
312
+ - {type: mine, owner: neutral, position: [2, 26]}
313
+ # Two hunt bands at distinct latitudes (both ALWAYS placed
314
+ # — enemy actors with no spawn_point are not filtered). The
315
+ # hunt bot drives each unit at the nearest agent target,
316
+ # so whichever lane the agent's base occupies will be hit
317
+ # by one band; the OTHER band will also turn in (the seed-
318
+ # varied spawn means the agent can't pre-commit to one lane).
319
+ - {type: e1, owner: enemy, position: [100, 10], stance: 3, count: 3}
320
+ - {type: e3, owner: enemy, position: [102, 10], stance: 3, count: 1}
321
+ - {type: e1, owner: enemy, position: [100, 30], stance: 3, count: 3}
322
+ - {type: e3, owner: enemy, position: [102, 30], stance: 3, count: 1}
323
+ # Inert enemy `fact` marker (anti-DRAW).
324
+ - {type: fact, owner: enemy, position: [120, 20]}
325
+ win_condition:
326
+ all_of:
327
+ - then:
328
+ id: defense-then-tech-hard
329
+ clauses:
330
+ - {building_count_gte: {type: pbox, n: 8}}
331
+ - {building_count_gte: {type: gun, n: 1}}
332
+ - {has_building: weap}
333
+ - {has_building: dome}
334
+ - {within_ticks: 7200}
335
+ - {own_units_gte: 1}
336
+ - {units_lost_lte: 6}
337
+ fail_condition:
338
+ any_of:
339
+ - {after_ticks: 7201}
340
+ - {not: {has_building: fact}}
341
+ - {not: {units_lost_lte: 6}}
342
+ max_turns: 80
tests/test_hard_tier.py CHANGED
@@ -170,6 +170,13 @@ UPGRADED = [
170
  # flips per seed (an off-axis diagonal busts the tick budget
171
  # and brushes the wrong-corner patrol).
172
  "mfb-base-1-defend-base-2-build",
 
 
 
 
 
 
 
173
  ]
174
 
175
  # Consciously NOT spawn-varied, with the reason (keeps the curation
 
170
  # flips per seed (an off-axis diagonal busts the tick budget
171
  # and brushes the wrong-corner patrol).
172
  "mfb-base-1-defend-base-2-build",
173
+ # Wave-4 TURTLE node of the tech triple (SC2 turtle macro /
174
+ # military fortify-before-research doctrine anchor). Hard defines
175
+ # two agent spawn_point groups (NORTH base / SOUTH base) so the
176
+ # base latitude varies by seed; the two hunt bands (north + south)
177
+ # always place and hit whichever latitude the agent occupies, so
178
+ # a memorised pbox-lane placement cannot generalise across seeds.
179
+ "tech-turtle-defensive-tech",
180
  ]
181
 
182
  # Consciously NOT spawn-varied, with the reason (keeps the curation
tests/test_tech_turtle_defensive_tech.py ADDED
@@ -0,0 +1,375 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """tech-turtle-defensive-tech pack — full no-cheat validation on Rust.
2
+
3
+ Wave-4 TURTLE tech-triple pack. Uses the Wave-2 `then:[A,B,C,D]`
4
+ happened-before composite to enforce the defence-then-tech order:
5
+ 1. building_count_gte: {type: pbox, n: 3|8} # phase 1 — anti-inf wall
6
+ 2. building_count_gte: {type: gun, n: 1} # phase 2 — anti-armour cap
7
+ 3. has_building: weap # phase 3 — tech step 1
8
+ 4. has_building: dome # phase 4 — tech step 2
9
+
10
+ Bar (per CLAUDE.md): the intended fortify-then-tech policy must WIN
11
+ on every (level, seed); stall / pure-aggro-skip-defence / build-only-
12
+ defence-no-tech must LOSE on every (level, seed). No draws.
13
+
14
+ Scenario shape:
15
+ - rush-hour-arena, allied agent, soviet hunt-bot enemy band.
16
+ - Pre-placed agent base: fact + tent + powr + proc + harv + mine
17
+ + starting riflemen (so income flows from turn 1 and `own_units_
18
+ gte:1` is satisfied immediately).
19
+ - Inert enemy `fact` marker far east prevents auto-DRAW on
20
+ enemy-elim.
21
+ - Hunt band closes on agent base by ~tick 1500 and would raze the
22
+ fact unless pbox×3 (medium: pbox×3, hard: pbox×8) is up first.
23
+ - hard: ≥2 spawn_point groups (north / south) per the hard-tier
24
+ contract.
25
+ """
26
+
27
+ from __future__ import annotations
28
+
29
+ import pytest
30
+
31
+ pytest.importorskip("openra_train", reason="Rust env wheel not installed")
32
+
33
+ from openra_bench.eval_core import run_level
34
+ from openra_bench.scenarios import load_pack
35
+ from openra_bench.scenarios.loader import PACKS_DIR, compile_level
36
+
37
+ PACK = PACKS_DIR / "tech-turtle-defensive-tech.yaml"
38
+ LEVELS = ("easy", "medium", "hard")
39
+ SEEDS = (1, 2, 3, 4)
40
+
41
+
42
+ # ── Policies ──────────────────────────────────────────────────────
43
+
44
+
45
+ def _stall_policy():
46
+ """Idles every turn — must LOSE (clock + hunt razes fact)."""
47
+ def pol(obs, Cmd):
48
+ return [Cmd.observe()]
49
+ return pol
50
+
51
+
52
+ def _pure_aggro_skip_defence_policy():
53
+ """Race weap+dome, never build a pbox/gun. The hunt band reaches
54
+ the base by ~tick 1500 with no defences ⇒ fact razed ⇒
55
+ `not has_building: fact` fail fires ⇒ LOSS."""
56
+ def pol(obs, Cmd):
57
+ own_b = {b["type"] for b in (obs.get("own_buildings", []) or [])}
58
+ prod = obs.get("production", []) or []
59
+ base = next((b for b in (obs.get("own_buildings", []) or [])
60
+ if b["type"] == "fact"), None)
61
+ cmds = []
62
+ if "weap" not in own_b and "weap" not in prod:
63
+ cmds.append(Cmd.build("weap"))
64
+ if base and "weap" not in own_b:
65
+ cmds.append(Cmd.place_building(
66
+ "weap", base["cell_x"] + 6, base["cell_y"]
67
+ ))
68
+ if "dome" not in own_b and "dome" not in prod:
69
+ cmds.append(Cmd.build("dome"))
70
+ if base and "dome" not in own_b:
71
+ cmds.append(Cmd.place_building(
72
+ "dome", base["cell_x"] + 8, base["cell_y"] + 2
73
+ ))
74
+ if not cmds:
75
+ cmds.append(Cmd.observe())
76
+ return cmds
77
+ return pol
78
+
79
+
80
+ def _only_defence_no_tech_policy():
81
+ """Spam pbox forever — never build weap/dome. The then-chain
82
+ cannot advance past phase 2 (gun needs weap, which is never
83
+ built); clock expires ⇒ LOSS."""
84
+ state = {"placed": 0}
85
+
86
+ def pol(obs, Cmd):
87
+ own_b_list = obs.get("own_buildings", []) or []
88
+ prod = obs.get("production", []) or []
89
+ base = next((b for b in own_b_list if b["type"] == "fact"), None)
90
+ cmds = []
91
+ if "pbox" not in prod:
92
+ cmds.append(Cmd.build("pbox"))
93
+ if base:
94
+ # Spread pbox placements so they don't collide.
95
+ dx = 3 + (state["placed"] % 4)
96
+ dy = -2 + (state["placed"] // 4)
97
+ cmds.append(Cmd.place_building(
98
+ "pbox", base["cell_x"] + dx, base["cell_y"] + dy
99
+ ))
100
+ state["placed"] += 1
101
+ if not cmds:
102
+ cmds.append(Cmd.observe())
103
+ return cmds
104
+ return pol
105
+
106
+
107
+ def _intended_defence_then_tech_policy(pbox_target: int):
108
+ """The intended TURTLE capability play:
109
+ turn 1: kick the pre-placed harvester onto the mine
110
+ phase 1: queue pbox one at a time until ≥pbox_target standing
111
+ phase 2: once phase 1 done, build weap (gun's engine prereq)
112
+ phase 3: once weap up, build gun (this latches the then-chain
113
+ clause 2; weap latches greedily as clause 3)
114
+ phase 4: build dome (clause 4 latches → chain complete)
115
+ The fortify-first ordering survives the hunt band's arrival
116
+ at ~tick 1500.
117
+ """
118
+ state = {
119
+ "placed_pbox": 0,
120
+ "weap_attempts": 0,
121
+ "gun_attempts": 0,
122
+ "dome_attempts": 0,
123
+ "harv_kicked": False,
124
+ }
125
+
126
+ def pol(obs, Cmd):
127
+ ob = obs.get("own_buildings", []) or []
128
+ own_b = {b["type"] for b in ob}
129
+ pbox_have = sum(1 for b in ob if b["type"] == "pbox")
130
+ prod = obs.get("production", []) or []
131
+ cash = int(obs.get("cash", 0) or 0)
132
+ units = obs.get("units_summary", []) or []
133
+ base = next((b for b in ob if b["type"] == "fact"), None)
134
+ bx = base["cell_x"] if base else 10
135
+ by = base["cell_y"] if base else 20
136
+ cmds = []
137
+
138
+ # Kick the pre-placed harvester at the nearest mine. Mine is
139
+ # WEST of base (at bx-8) in every scenario (easy/medium:
140
+ # (2,20); hard: (2,14)/(2,26)) — placed west so the harv loop
141
+ # doesn't cross the lane the hunt band is closing down.
142
+ if not state["harv_kicked"]:
143
+ harv = next((u for u in units if u.get("type") == "harv"), None)
144
+ if harv:
145
+ cmds.append(Cmd.harvest([harv["id"]], bx - 8, by))
146
+ state["harv_kicked"] = True
147
+
148
+ # Phase 1: pbox first. Queue ONE at a time (wait for it to
149
+ # complete before queueing the next) so we don't over-build
150
+ # before income arrives. Place each pbox on a distinct cell.
151
+ in_q_pbox = "pbox" in prod
152
+ if pbox_have < pbox_target and not in_q_pbox and cash >= 600:
153
+ cmds.append(Cmd.build("pbox"))
154
+ # Place whenever a pbox is ready (we only have at most one
155
+ # queued at a time so a single place per turn is enough).
156
+ if pbox_have < pbox_target:
157
+ i = state["placed_pbox"]
158
+ row = -3 + 2 * (i % 4) # y offsets: -3,-1,+1,+3
159
+ col = 3 + (i // 4) # x offsets: 3,4,5,…
160
+ cmds.append(Cmd.place_building(
161
+ "pbox", bx + col, by + row
162
+ ))
163
+ state["placed_pbox"] += 1
164
+
165
+ # Phase 2/3: once pbox wall is up, build weap (gun's prereq).
166
+ # weap needs proc (pre-placed) so the queue accepts it.
167
+ # Retry placement on a sliding offset until the building
168
+ # actually completes (the engine silently blocks placements
169
+ # that collide with existing footprints / units).
170
+ if pbox_have >= pbox_target and "weap" not in own_b \
171
+ and "weap" not in prod and cash >= 2000:
172
+ cmds.append(Cmd.build("weap"))
173
+ if pbox_have >= pbox_target and "weap" not in own_b:
174
+ i = state["weap_attempts"]
175
+ cmds.append(Cmd.place_building(
176
+ "weap", bx + 8 + (i % 6), by - 4 + (i // 6)
177
+ ))
178
+ state["weap_attempts"] += 1
179
+
180
+ # gun needs weap — queue it once weap is up.
181
+ gun_have = sum(1 for b in ob if b["type"] == "gun")
182
+ if "weap" in own_b and gun_have < 1 and "gun" not in prod \
183
+ and cash >= 800:
184
+ cmds.append(Cmd.build("gun"))
185
+ if "weap" in own_b and gun_have < 1:
186
+ i = state["gun_attempts"]
187
+ cmds.append(Cmd.place_building(
188
+ "gun", bx + 8 + (i % 6), by + 4 + (i // 6)
189
+ ))
190
+ state["gun_attempts"] += 1
191
+
192
+ # dome needs proc (pre-placed); build in parallel once weap
193
+ # is up — but keep the spec's gun-before-dome ordering so the
194
+ # then-chain advances cleanly.
195
+ if "weap" in own_b and gun_have >= 1 and "dome" not in own_b \
196
+ and "dome" not in prod and cash >= 1000:
197
+ cmds.append(Cmd.build("dome"))
198
+ if "weap" in own_b and gun_have >= 1 and "dome" not in own_b:
199
+ i = state["dome_attempts"]
200
+ cmds.append(Cmd.place_building(
201
+ "dome", bx + 12 + (i % 4), by + (i // 4)
202
+ ))
203
+ state["dome_attempts"] += 1
204
+
205
+ if not cmds:
206
+ cmds.append(Cmd.observe())
207
+ return cmds
208
+ return pol
209
+
210
+
211
+ # ── Pack-shape tests (cheap; do not run the engine) ──────────────
212
+
213
+
214
+ def test_pack_compiles_with_three_levels():
215
+ pack = load_pack(PACK)
216
+ assert pack.meta.id == "tech-turtle-defensive-tech"
217
+ assert pack.meta.capability == "reasoning"
218
+ assert set(pack.levels) == {"easy", "medium", "hard"}
219
+
220
+
221
+ def test_meta_benchmark_anchor_set():
222
+ pack = load_pack(PACK)
223
+ anchors = pack.meta.benchmark_anchor or []
224
+ assert any("turtle" in a for a in anchors), anchors
225
+ assert any("fortify" in a or "shield" in a for a in anchors), anchors
226
+ assert any("hedge" in a or "risk" in a for a in anchors), anchors
227
+
228
+
229
+ def test_hard_tier_has_seed_driven_spawn_groups():
230
+ c = compile_level(load_pack(PACK), "hard")
231
+ sp = {a.spawn_point for a in c.scenario.actors if a.owner == "agent"}
232
+ assert len(sp) >= 2, f"hard needs ≥2 spawn groups, got {sp}"
233
+
234
+
235
+ def test_every_level_has_fail_condition():
236
+ pack = load_pack(PACK)
237
+ for lvl in LEVELS:
238
+ c = compile_level(pack, lvl)
239
+ assert c.fail_condition is not None, f"{lvl} missing fail_condition"
240
+
241
+
242
+ def test_then_composite_used_in_win():
243
+ """Confirms the defence-then-tech chain wired through to the
244
+ compiled win condition (the whole point of the TURTLE pack)."""
245
+ for lvl in LEVELS:
246
+ c = compile_level(load_pack(PACK), lvl)
247
+ win = c.win_condition.model_dump(exclude_none=True)
248
+ ao = win.get("all_of") or []
249
+ assert any("then" in clause for clause in ao), (
250
+ f"{lvl} win has no then-composite: {win}"
251
+ )
252
+ then_clause = next(cl["then"] for cl in ao if "then" in cl)
253
+ clauses = then_clause["clauses"]
254
+ # Must be exactly: pbox → gun → weap → dome.
255
+ assert "building_count_gte" in clauses[0]
256
+ assert clauses[0]["building_count_gte"]["type"] == "pbox"
257
+ assert "building_count_gte" in clauses[1]
258
+ assert clauses[1]["building_count_gte"]["type"] == "gun"
259
+ assert "has_building" in clauses[2] and clauses[2]["has_building"] == "weap"
260
+ assert "has_building" in clauses[3] and clauses[3]["has_building"] == "dome"
261
+
262
+
263
+ def test_tick_budget_aligned_with_max_turns():
264
+ """within_ticks must be reachable inside max_turns."""
265
+ pack = load_pack(PACK)
266
+ for lvl in LEVELS:
267
+ level_def = pack.levels[lvl]
268
+ max_turns = level_def.max_turns
269
+ reachable = 93 + 90 * (max_turns - 1)
270
+ win = compile_level(pack, lvl).win_condition.model_dump(exclude_none=True)
271
+
272
+ def _collect(node, key, out):
273
+ if isinstance(node, dict):
274
+ if key in node:
275
+ out.append(node[key])
276
+ for v in node.values():
277
+ _collect(v, key, out)
278
+ elif isinstance(node, list):
279
+ for v in node:
280
+ _collect(v, key, out)
281
+ wts = []
282
+ _collect(win, "within_ticks", wts)
283
+ assert wts, f"{lvl} has no within_ticks leaf (no clock teeth)"
284
+ for wt in wts:
285
+ assert wt <= reachable, (
286
+ f"{lvl} within_ticks={wt} > reachable={reachable} "
287
+ f"(max_turns={max_turns}) — deadline never bites ⇒ draw"
288
+ )
289
+
290
+
291
+ # ── Engine-bound tests (parameterised over seeds 1..4) ────────────
292
+
293
+
294
+ def _pbox_target(level: str) -> int:
295
+ return 8 if level == "hard" else 3
296
+
297
+
298
+ @pytest.mark.parametrize("seed", SEEDS)
299
+ @pytest.mark.parametrize("level", LEVELS)
300
+ def test_intended_defence_then_tech_wins(level, seed):
301
+ """The intended TURTLE policy (pbox first, then weap+gun+dome)
302
+ must WIN on every (level, seed). Load-bearing solvency test."""
303
+ c = compile_level(load_pack(PACK), level)
304
+ res = run_level(
305
+ c, _intended_defence_then_tech_policy(_pbox_target(level)), seed=seed
306
+ )
307
+ own_b = res.signals.own_building_types
308
+ assert res.outcome == "win", (
309
+ f"intended defence-then-tech must WIN on {level} s={seed}; "
310
+ f"got {res.outcome} turns={res.turns} "
311
+ f"own_buildings={own_b} cash={res.signals.cash} "
312
+ f"units_lost={res.signals.units_lost} "
313
+ f"then_progress={getattr(res.signals, 'then_progress', {})}"
314
+ )
315
+
316
+
317
+ @pytest.mark.parametrize("seed", SEEDS)
318
+ @pytest.mark.parametrize("level", LEVELS)
319
+ def test_stall_loses(level, seed):
320
+ """Do-nothing must LOSE on every (level, seed)."""
321
+ c = compile_level(load_pack(PACK), level)
322
+ res = run_level(c, _stall_policy(), seed=seed)
323
+ assert res.outcome == "loss", (
324
+ f"stall must LOSE on {level} s={seed}; got {res.outcome}"
325
+ )
326
+
327
+
328
+ @pytest.mark.parametrize("seed", SEEDS)
329
+ @pytest.mark.parametrize("level", LEVELS)
330
+ def test_pure_aggro_skip_defence_loses(level, seed):
331
+ """Pure aggro (race weap+dome without any defence) must LOSE:
332
+ hunt band reaches the base ~tick 1500, no pbox/gun to blunt it
333
+ ⇒ fact razed ⇒ `not has_building: fact` fail fires (or the
334
+ then-chain never advances past phase 1 ⇒ clock expires)."""
335
+ c = compile_level(load_pack(PACK), level)
336
+ res = run_level(c, _pure_aggro_skip_defence_policy(), seed=seed)
337
+ own_b = res.signals.own_building_types
338
+ assert res.outcome == "loss", (
339
+ f"pure-aggro-skip-defence must LOSE on {level} s={seed}; got "
340
+ f"{res.outcome} own_buildings={own_b} "
341
+ f"then_progress={getattr(res.signals, 'then_progress', {})}"
342
+ )
343
+
344
+
345
+ @pytest.mark.parametrize("seed", SEEDS)
346
+ @pytest.mark.parametrize("level", LEVELS)
347
+ def test_only_defence_no_tech_loses(level, seed):
348
+ """Build only pbox forever — never weap/dome. Phase 2 (gun)
349
+ requires weap (engine prereq), which is never built; the
350
+ then-chain stalls at phase 1; clock expires ⇒ LOSS."""
351
+ c = compile_level(load_pack(PACK), level)
352
+ res = run_level(c, _only_defence_no_tech_policy(), seed=seed)
353
+ own_b = res.signals.own_building_types
354
+ assert res.outcome == "loss", (
355
+ f"only-defence-no-tech must LOSE on {level} s={seed}; got "
356
+ f"{res.outcome} own_buildings={own_b} "
357
+ f"then_progress={getattr(res.signals, 'then_progress', {})}"
358
+ )
359
+ # Sanity: tech buildings never built.
360
+ assert "weap" not in own_b, (
361
+ f"only-defence-no-tech built weap on {level} s={seed}"
362
+ )
363
+ assert "dome" not in own_b, (
364
+ f"only-defence-no-tech built dome on {level} s={seed}"
365
+ )
366
+
367
+
368
+ @pytest.mark.parametrize("seed", SEEDS)
369
+ def test_hard_seeds_produce_distinct_starts(seed):
370
+ """Hard's two spawn_point groups must round-robin per seed."""
371
+ c = compile_level(load_pack(PACK), "hard")
372
+ res = run_level(c, _stall_policy(), seed=seed)
373
+ assert res.outcome == "loss" # stall must lose
374
+ sp = {a.spawn_point for a in c.scenario.actors if a.owner == "agent"}
375
+ assert len(sp) >= 2