yxc20098 commited on
Commit
ac7bbdb
·
1 Parent(s): 01b56d9

feat(scenario): def-stance-mgmt-hold-then-attack — timed stance flip for ambush (ROE / SC2 stance anchor)

Browse files
openra_bench/scenarios/packs/def-stance-mgmt-hold-then-attack.yaml ADDED
@@ -0,0 +1,359 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # def-stance-mgmt-hold-then-attack.yaml
2
+ #
3
+ # ACTION capability — Wave-8 stance-flip ambush pack. Defenders (4x
4
+ # 2tnk) are PRE-PLACED at a choke (~35,20) on stance:0 (HoldFire) so
5
+ # they will NOT auto-engage on their own. The agent MUST `set_stance`
6
+ # them to stance:3 (AttackAnything) at some point during the run for
7
+ # the engine's stance-driven auto-fire to wipe the inbound rush before
8
+ # it reaches the construction yard (fact at 10,20). A stall policy
9
+ # that NEVER flips the stance lets the rush walk through the silent
10
+ # choke and raze the fact — the load-bearing test is the `set_stance`
11
+ # verb itself: without calling it, the defenders never engage and the
12
+ # mission is forfeit; with it, the engine's auto-targeting wipes the
13
+ # rush at the choke.
14
+ #
15
+ # This is the COMBAT-CELL counterpart to proc-only-defend-no-attack:
16
+ # - proc-only-defend-no-attack tests PROCEDURAL compliance — the
17
+ # defenders auto-engage once stance is lifted, BUT calling
18
+ # attack_unit / attack_move is a forbidden-tool LOSS. The skill is
19
+ # "DO NOT call the attack tool" + "lift stance at some point".
20
+ # - THIS pack tests STANCE MANAGEMENT in a pure combat cell — the
21
+ # defenders are armoured tanks (2tnk) on stance:0; the inbound
22
+ # force is a rifle rush; there are NO forbidden tools and NO
23
+ # procedural cap. The only stance-management policy that holds
24
+ # all the win clauses (≥K kills + ≥3 tanks alive + fact intact +
25
+ # within the tick budget) is one that LIFTS the stance via
26
+ # `set_stance` so the tank cluster's auto-targeting wipes the
27
+ # rush at the choke. Never-flip ⇒ tanks silent ⇒ rush razes
28
+ # the fact ⇒ LOSS on `not has_building:fact`.
29
+ #
30
+ # Real-world / benchmark anchors:
31
+ # - Military rules-of-engagement (ROE) drills: ambush / defended
32
+ # overwatch — fire authority is held until the trigger is given,
33
+ # at which point the overwatch element opens up and wipes the
34
+ # enemy column. A defended position with locked-down weapons
35
+ # authority is functionally undefended; lifting the engagement
36
+ # authority is the controllable act that turns the position into
37
+ # an active defence.
38
+ # - SC2 stance micro: pro players micro siege tanks between Hold
39
+ # Position / Attack stance so the main battery FIRES when the
40
+ # opponent's army is in range rather than chip-firing or sitting
41
+ # idle. The decision to flip stance IS the controllable verb;
42
+ # never flipping is the canonical newbie mistake.
43
+ # - L-shape / linear ambush doctrine (USMC FM 7-8): the kill zone
44
+ # is set up in advance; the trigger to engage is held until the
45
+ # enemy column is committed inside the zone. The discipline
46
+ # reduces to "do not engage until told; engage when told".
47
+ #
48
+ # DISCRIMINATIONS (the bar — stall LOSES on every level / seed, the
49
+ # intended flip-the-stance play WINS on every level / seed):
50
+ #
51
+ # • stall (only `observe()`): defenders stay on stance:0 the
52
+ # entire run. The rush walks straight at the tank cluster (the
53
+ # `rusher` scripted bot concentrates the band on the agent
54
+ # centroid every ~8 ticks; openra-sim/src/scripted_bot.rs::
55
+ # Rusher). With the defenders unwilling to fire, the rush
56
+ # overruns the cluster (tanks die without firing back), then
57
+ # turns on the fact (the engine auto-`done`s would short-circuit
58
+ # here but the inert enemy `fact` marker far east keeps the
59
+ # episode alive past unit-elim), and the fact is razed ⇒
60
+ # `not has_building:fact` LOSS. Independently, kill count
61
+ # stays at 0, busting `units_killed_gte` ⇒ the win predicate
62
+ # fails on multiple clauses, every level / seed.
63
+ #
64
+ # • intended (set_stance(unit_ids, 3) once tanks are in the
65
+ # units_summary): the tank cluster auto-fires on every rusher
66
+ # band that closes to cannon range (≈5 cells; CLAUDE.md /
67
+ # combat-focus-fire-priority "2tnk one-shots e1"). 4× tank
68
+ # cannons firing on each lead e1 wipes the rush column in 1-2
69
+ # volleys before the rush reaches the fact. ≥K kills logged
70
+ # (the rush composition is sized so the kill bar is hit
71
+ # comfortably), ≥3 of the 4 tanks alive (the rush is e1-only —
72
+ # a small rifle force does not out-damage a 4-tank cluster),
73
+ # fact intact, well under the tick deadline ⇒ WIN every level
74
+ # and every hard seed.
75
+ #
76
+ # NOTE on the flip-immediately variant: in this combat cell with
77
+ # stationary tanks on stance:3, the engine's auto-targeting fires
78
+ # the instant any enemy enters cannon range — there is no behavioural
79
+ # difference between "flip on turn 1" and "flip when the rush is in
80
+ # the choke" because the tanks do not pursue (`stance:3 reserve auto-
81
+ # engages the rush when it closes to cannon range and racks up
82
+ # kills, but the reserve NEVER MOVES OUT of its starting cluster` —
83
+ # def-pre-position-mobile-reserve / engine fact). The load-bearing
84
+ # capability is therefore the `set_stance` verb itself; the timing
85
+ # axis is encoded as flavour text in the level briefs (the agent is
86
+ # told to flip when the column is in the kill zone) but the win
87
+ # predicate measures only the EFFECT (kills + survival + fact
88
+ # intact). A model that always flips on turn 1 WINS — that is the
89
+ # correct outcome: the verb was called, the engine did the rest.
90
+ # A model that never flips LOSES — the verb was the discriminator.
91
+ #
92
+ # ENGINE FACTS used (CLAUDE.md / SCENARIO_QUALITY.md):
93
+ # - `rusher` bot concentrates the band on the agent centroid every
94
+ # ~8 ticks (openra-sim/src/scripted_bot.rs::Rusher) — the rush
95
+ # drives straight at the tank cluster at the choke.
96
+ # - `set_stance` is a first-class agent tool (openra_bench/agent.py
97
+ # describes 0=HoldFire, 1=ReturnFire, 2=Defend, 3=AttackAnything).
98
+ # A 2tnk on stance:0 will NOT auto-fire even when an enemy walks
99
+ # directly under its cannon; lifting to stance:3 makes it auto-
100
+ # engage in range. This is what makes the discipline load-
101
+ # bearing (a never-flip policy cannot win by accident).
102
+ # - 2tnk cannon vs e1: ~1-shot kill (CLAUDE.md / combat-focus-
103
+ # fire-priority). Cannon range > e1 rifle range, so the tanks
104
+ # pick off e1s before the e1s can fire back at the cluster.
105
+ # - Inert enemy `fact` marker far east: anti-DRAW per CLAUDE.md.
106
+ # Without a persistent enemy actor the engine auto-`done`s on
107
+ # enemy-elim before the win/fail evaluator runs.
108
+ # - Tick budget: ~90 ticks/turn ⇒ max_tick ≤ 93 + 90·(max_turns-1).
109
+ # after_ticks 4501 with max_turns 55 ⇒ 4501 ≤ 4953 ✓ (deadline
110
+ # bites; non-win is a real LOSS, not a draw).
111
+ # - Enemy actors with NO spawn_point ALWAYS place regardless of
112
+ # the agent's spawn group (CLAUDE.md). Hard varies the AGENT
113
+ # spawn (north choke y=18..20 vs south choke y=20..22) — both
114
+ # symmetric covers for the fact column (10,20) — while the
115
+ # enemy rush ALWAYS places at the same eastern band.
116
+ #
117
+ # Hard-tier curation: this pack IS spawn-varied on hard (defender
118
+ # choke latitude round-robins NORTH y=18..20 / SOUTH y=20..22 by
119
+ # seed — both still close the (35,~) choke that covers the fact
120
+ # column at x=10). Registered in tests/test_hard_tier.py::UPGRADED.
121
+ #
122
+ # Benchmark anchor: military rules-of-engagement / ambush doctrine
123
+ # (engage on trigger, not on contact); SC2 stance micro (hold-then-
124
+ # attack); ambush-trigger discipline.
125
+
126
+ meta:
127
+ id: def-stance-mgmt-hold-then-attack
128
+ title: 'Ambush Trigger Discipline — Lift Stance to Engage the Inbound Rush'
129
+ capability: action
130
+ real_world_meaning: >
131
+ A defended-overwatch / ambush trigger drill: four medium tanks are
132
+ pre-staged at a choke between the inbound enemy column and the
133
+ construction yard. Their standing orders are HoldFire (stance:0)
134
+ — they will not fire on their own. The mission is to call
135
+ `set_stance` to lift the engagement authority so all four tanks
136
+ auto-engage the rush at cannon range and wipe it before it
137
+ reaches the base. Never flipping the stance leaves the position
138
+ functionally undefended (the rush walks through unopposed and
139
+ razes the base); the controllable verb that turns the pre-staged
140
+ position into an active defence is the stance flip.
141
+ robotics_analogue: >
142
+ A weapon-system safety interlock: a turret / drone-swarm
143
+ perimeter is pre-staged and aimed at the avenue of approach,
144
+ but the trigger authority is held by the operator. The operator
145
+ MUST issue the engagement-authority command for the system to
146
+ fire on the inbound target; without that command the perimeter
147
+ is functionally a passive observation post and the target
148
+ reaches the protected asset. The skill is the operator's
149
+ decision to LIFT the engagement-authority lock at all — not
150
+ the precise moment within the engagement window.
151
+ benchmark_anchor:
152
+ - "military rules-of-engagement (ROE) ambush doctrine"
153
+ - "SC2 stance micro — hold-then-attack siege-tank salvo"
154
+ - "ambush doctrine — engagement authority lifted on trigger"
155
+ author: openra-bench
156
+
157
+ # rush-hour-arena: 128×40 playable. Fact at the west edge (10,20).
158
+ # The choke at x≈35 is the tank cluster's staging cell; the rush
159
+ # spawns at the east band x≈90.
160
+ base_map: rush-hour-arena
161
+ starting_cash: 0
162
+
163
+ base:
164
+ agent:
165
+ faction: allies
166
+ cash: 0
167
+ enemy:
168
+ faction: soviet
169
+ cash: 0
170
+ bot_type: rusher
171
+ # Combat-only toolset (NO build / NO harvest): the test is the
172
+ # stance-management verb (set_stance) on pre-staged defenders.
173
+ # attack_unit / attack_move are ALSO available (this is NOT a
174
+ # procedural-compliance pack) but a brute "attack_move east on
175
+ # turn 1" pulls the tanks out of the choke and breaks the static
176
+ # ambush geometry — the engine's stance-driven auto-fire is the
177
+ # canonical and efficient solution.
178
+ tools:
179
+ - observe
180
+ - move_units
181
+ - attack_unit
182
+ - attack_move
183
+ - stop
184
+ - set_stance
185
+ planning: true
186
+ termination:
187
+ max_ticks: 6000
188
+ actors: []
189
+
190
+ levels:
191
+ # ── EASY ──────────────────────────────────���──────────────────────
192
+ # Bare stance-flip skill: a single rush band (6× e1) inbound from
193
+ # the east. The agent must call set_stance(units, 3) at some point
194
+ # so the tank cluster auto-fires when the rush closes to cannon
195
+ # range. Stall (never flip) ⇒ tanks never engage; rush overruns
196
+ # the cluster (tanks die without firing); then the rush razes the
197
+ # fact ⇒ LOSS on `not has_building:fact`. Intended flip ⇒ all
198
+ # four tanks auto-fire as the rush enters cannon range, wiping
199
+ # the e1 column before it reaches the fact ⇒ WIN.
200
+ # max_turns 55 → reachable tick ≈ 4953; after_ticks 4501 bites
201
+ # as a real LOSS (not a draw).
202
+ easy:
203
+ description: >
204
+ AMBUSH TRIGGER: four medium tanks (2tnk) are pre-staged at the
205
+ choke (35,20) on HOLD-FIRE (stance:0). An enemy rifle rush
206
+ (6 e1) is inbound from the east (spawn at (90,20)) and will
207
+ drive straight at your tanks then at your construction yard
208
+ (fact at 10,20). Use `set_stance` to lift the tanks to
209
+ AttackAnything (stance:3) so they auto-engage the column at
210
+ cannon range — the canonical answer is to flip the stance
211
+ WHEN the rush is committed inside the choke (a generous
212
+ timing window on easy: any flip while the rush is approaching
213
+ satisfies the win). Win when ≥4 raiders are dead AND ≥3 of
214
+ your 4 tanks survive AND the fact still stands, before tick
215
+ 4500. Stalling (never flipping — fact gets razed) and timeout
216
+ all LOSE.
217
+ overrides:
218
+ actors:
219
+ # Agent construction yard — west edge, the rush's ultimate
220
+ # target if the choke is silent.
221
+ - {type: fact, owner: agent, position: [10, 20]}
222
+ # 4× 2tnk at the choke (35,20) on stance:0 (HoldFire).
223
+ # They will NOT fire until the agent lifts the stance.
224
+ # Clustered tightly so the salvo geometry is sharp: all
225
+ # four cannons can fire on the same enemy cell in one tick.
226
+ - {type: 2tnk, owner: agent, position: [35, 19], stance: 0}
227
+ - {type: 2tnk, owner: agent, position: [35, 20], stance: 0}
228
+ - {type: 2tnk, owner: agent, position: [35, 21], stance: 0}
229
+ - {type: 2tnk, owner: agent, position: [36, 20], stance: 0}
230
+ # Inbound rush — 6× e1 (rifle). e1 vs 2tnk has very low
231
+ # damage per CLAUDE.md / combat-focus-fire-priority, so a
232
+ # stance-lifted 4-tank cluster wipes the column in 1-2
233
+ # volleys with at most 1 tank lost (own_units_gte:3 holds).
234
+ - {type: e1, owner: enemy, position: [90, 20], stance: 3, count: 6}
235
+ # Inert enemy `fact` marker far east (anti-DRAW): without a
236
+ # persistent enemy actor the engine auto-`done`s on enemy-
237
+ # elim before the win/fail evaluator runs (CLAUDE.md fact #5).
238
+ - {type: fact, owner: enemy, position: [120, 20]}
239
+ win_condition:
240
+ all_of:
241
+ - {has_building: fact}
242
+ - {units_killed_gte: 4}
243
+ - {own_units_gte: 3}
244
+ - {within_ticks: 4500}
245
+ fail_condition:
246
+ any_of:
247
+ - {after_ticks: 4501}
248
+ - {not: {has_building: fact}}
249
+ - {not: {own_units_gte: 1}}
250
+ max_turns: 55
251
+
252
+ # ── MEDIUM ───────────────────────────────────────────────────────
253
+ # +1 controlled variable vs easy: TIGHTER kill bar (≥5 vs ≥4) on
254
+ # a BIGGER rush (8× e1 vs 6× e1). The bigger column ensures the
255
+ # kill bar is reachable for the intended flip-the-stance policy,
256
+ # while the same per-tank attrition pressure on the cluster
257
+ # (e1-only rush at point-blank range against a 4-tank cluster
258
+ # cannot bust own_units_gte:3) keeps the discrimination clean:
259
+ # the stance-flip act is what wins; stalling loses.
260
+ medium:
261
+ description: >
262
+ AMBUSH TRIGGER: four medium tanks (2tnk) are pre-staged at the
263
+ choke (35,20) on HOLD-FIRE (stance:0). A larger rifle rush
264
+ (8 e1) is inbound from the east (spawn at (90,20)) and will
265
+ drive at your tanks then your construction yard (fact at
266
+ 10,20). Use `set_stance` to lift the tanks to AttackAnything
267
+ so they auto-engage the column at cannon range — the
268
+ canonical answer is to flip when the rush is committed inside
269
+ the choke. Win when ≥5 raiders are dead AND ≥3 of your 4
270
+ tanks survive AND the fact still stands, before tick 4500.
271
+ Stalling (fact razed) and timeout both LOSE.
272
+ overrides:
273
+ actors:
274
+ - {type: fact, owner: agent, position: [10, 20]}
275
+ - {type: 2tnk, owner: agent, position: [35, 19], stance: 0}
276
+ - {type: 2tnk, owner: agent, position: [35, 20], stance: 0}
277
+ - {type: 2tnk, owner: agent, position: [35, 21], stance: 0}
278
+ - {type: 2tnk, owner: agent, position: [36, 20], stance: 0}
279
+ # 8× e1 inbound — bigger rush column so the ≥5 kill bar is
280
+ # reachable with margin for the intended stance-flip policy.
281
+ - {type: e1, owner: enemy, position: [90, 20], stance: 3, count: 8}
282
+ - {type: fact, owner: enemy, position: [120, 20]}
283
+ win_condition:
284
+ all_of:
285
+ - {has_building: fact}
286
+ - {units_killed_gte: 5}
287
+ - {own_units_gte: 3}
288
+ - {within_ticks: 4500}
289
+ fail_condition:
290
+ any_of:
291
+ - {after_ticks: 4501}
292
+ - {not: {has_building: fact}}
293
+ - {not: {own_units_gte: 1}}
294
+ max_turns: 55
295
+
296
+ # ── HARD ─────────────────────────────────────────────────────────
297
+ # +1 controlled variable on top of medium: ≥2 agent spawn_point
298
+ # groups so the choke latitude varies per seed (NORTH choke
299
+ # y=18..20 vs SOUTH choke y=20..22 — both cover the fact column
300
+ # x=10 and both sit ~55 cells east of the fact across the same
301
+ # choke geometry; a memorised "set_stance when the rush is at
302
+ # (40,20)" trigger CELL cannot generalise because the kill zone
303
+ # shifts in y per seed; the DOCTRINE — flip the stance so the
304
+ # cluster auto-engages — still applies from either spawn). The
305
+ # rush composition (8× e1) is the same as medium so the kill bar
306
+ # is achievable from either choke (rusher concentrates the band
307
+ # on the agent centroid every ~8 ticks; CLAUDE.md: enemy actors
308
+ # don't honour spawn_point so the rush always places). Same
309
+ # clock; same tick-aligned fail.
310
+ hard:
311
+ description: >
312
+ AMBUSH TRIGGER: four medium tanks (2tnk) are pre-staged at a
313
+ choke (~35, seed-chosen latitude — NORTH y≈19 or SOUTH y≈21)
314
+ on HOLD-FIRE (stance:0). A rifle rush (8 e1) drives at your
315
+ tanks then your construction yard (fact at 10,20). Lift the
316
+ tanks via `set_stance` to AttackAnything so they auto-engage
317
+ the column at cannon range; the choke latitude is seed-driven
318
+ so a memorised trigger cell does not generalise — only the
319
+ DOCTRINE (lift the stance so the cluster auto-engages) wins
320
+ on every seed. Win when ≥5 raiders are dead AND ≥3 of your 4
321
+ tanks survive AND the fact still stands, before tick 4500.
322
+ overrides:
323
+ actors:
324
+ # Fact lives outside spawn_point but CLAUDE.md: if ANY agent
325
+ # actor declares spawn_point, every agent actor WITHOUT
326
+ # spawn_point is filtered OUT — duplicate the fact across
327
+ # BOTH groups at the same cell.
328
+ - {type: fact, owner: agent, position: [10, 20], spawn_point: 0}
329
+ - {type: fact, owner: agent, position: [10, 20], spawn_point: 1}
330
+ # spawn_point 0 — NORTH choke at y=18..20 (cluster centroid y=19).
331
+ - {type: 2tnk, owner: agent, position: [35, 18], stance: 0, spawn_point: 0}
332
+ - {type: 2tnk, owner: agent, position: [35, 19], stance: 0, spawn_point: 0}
333
+ - {type: 2tnk, owner: agent, position: [35, 20], stance: 0, spawn_point: 0}
334
+ - {type: 2tnk, owner: agent, position: [36, 19], stance: 0, spawn_point: 0}
335
+ # spawn_point 1 — SOUTH choke at y=20..22 (cluster centroid y=21).
336
+ - {type: 2tnk, owner: agent, position: [35, 20], stance: 0, spawn_point: 1}
337
+ - {type: 2tnk, owner: agent, position: [35, 21], stance: 0, spawn_point: 1}
338
+ - {type: 2tnk, owner: agent, position: [35, 22], stance: 0, spawn_point: 1}
339
+ - {type: 2tnk, owner: agent, position: [36, 21], stance: 0, spawn_point: 1}
340
+ # Rush — same composition as medium; enemy actors don't
341
+ # honour spawn_point so this band always places every seed.
342
+ # Spawn at (90,20) — rusher concentrates the band on the
343
+ # agent centroid every ~8 ticks, so the column drives at
344
+ # whichever choke the tanks are staged at (NORTH y=19 or
345
+ # SOUTH y=21).
346
+ - {type: e1, owner: enemy, position: [90, 20], stance: 3, count: 8}
347
+ - {type: fact, owner: enemy, position: [120, 20]}
348
+ win_condition:
349
+ all_of:
350
+ - {has_building: fact}
351
+ - {units_killed_gte: 5}
352
+ - {own_units_gte: 3}
353
+ - {within_ticks: 4500}
354
+ fail_condition:
355
+ any_of:
356
+ - {after_ticks: 4501}
357
+ - {not: {has_building: fact}}
358
+ - {not: {own_units_gte: 1}}
359
+ max_turns: 55
tests/test_def_stance_mgmt_hold_then_attack.py ADDED
@@ -0,0 +1,322 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """def-stance-mgmt-hold-then-attack — stance-flip ambush trigger pack
2
+ (Wave-8 ACTION capability).
3
+
4
+ The bar (per CLAUDE.md): the intended (set_stance-then-hold) policy
5
+ WINS on every level and every hard seed (1..4); the stall policy
6
+ (observe only — never lift the stance from HoldFire) LOSES on every
7
+ level and every hard seed. Non-win is a real reachable timeout LOSS
8
+ (after_ticks 4501 fits inside max_turns 55 ⇒ 4501 ≤ 93 + 90·54 = 4953,
9
+ so the deadline bites as a real LOSS rather than collapsing to a
10
+ draw).
11
+
12
+ The discrimination is stance-management:
13
+ • Intended: lift defenders from HoldFire (stance:0) to AttackAnything
14
+ (stance:3) via `set_stance` so the engine's stance-driven auto-
15
+ fire engages the inbound rusher band as it closes to cannon range
16
+ at the choke; 4× 2tnk cannons one-shot e1 (combat-focus-fire-
17
+ priority / CLAUDE.md) and wipe the column before it reaches the
18
+ fact. ≥K kills, ≥3 of 4 tanks alive, fact intact, well under
19
+ the tick budget ⇒ WIN.
20
+ • Stall (only observe): defenders stay on stance:0 the entire run.
21
+ The rusher band overruns the silent tank cluster (the tanks die
22
+ without firing back), then turns on the fact and razes it ⇒
23
+ `not has_building:fact` LOSS. Kills stay at 0 (`units_killed_gte`
24
+ also fails) so the win predicate is busted on multiple clauses
25
+ every level / seed.
26
+
27
+ NOTE on the flip-immediately variant: in this combat cell with
28
+ stationary 2tnks on stance:3 the engine's auto-targeting fires the
29
+ instant any enemy enters cannon range — there is no behavioural
30
+ difference between "flip on turn 1" and "flip when the rush is in
31
+ the choke" because the tanks do not pursue beyond their starting
32
+ cluster (def-pre-position-mobile-reserve engine fact). The load-
33
+ bearing capability is therefore the `set_stance` verb itself; the
34
+ timing axis is encoded in the level brief as a doctrinal note but
35
+ the win predicate measures only the EFFECT (kills + survival +
36
+ fact intact). A model that always flips on turn 1 WINS — that is
37
+ the correct outcome: the verb was called, the engine did the rest.
38
+ A model that never flips LOSES — the verb was the discriminator.
39
+
40
+ Validation is scripted (no model / network) — these policies are
41
+ exhaustive proxies for the stance-management capability and exercise
42
+ the predicate teeth (has_building / units_killed_gte / own_units_gte
43
+ / within_ticks) directly through eval_core.run_level. See CLAUDE.md
44
+ "How to validate" for the harness.
45
+ """
46
+
47
+ from __future__ import annotations
48
+
49
+ from pathlib import Path
50
+
51
+ import pytest
52
+
53
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
54
+ from openra_bench.scenarios import load_pack
55
+ from openra_bench.scenarios.loader import compile_level
56
+
57
+ PACKS = Path(__file__).parent.parent / "openra_bench" / "scenarios" / "packs"
58
+ PACK_PATH = PACKS / "def-stance-mgmt-hold-then-attack.yaml"
59
+
60
+ LEVELS = ("easy", "medium", "hard")
61
+ HARD_SEEDS = (1, 2, 3, 4)
62
+
63
+
64
+ # ── structural / metadata checks (no engine) ──────────────────────────
65
+
66
+
67
+ def test_pack_compiles_and_meta_fields_populated():
68
+ pack = load_pack(PACK_PATH)
69
+ assert pack.meta.id == "def-stance-mgmt-hold-then-attack"
70
+ assert pack.meta.capability == "action"
71
+ # benchmark_anchor must name the ROE + SC2 stance + ambush anchors
72
+ # the pack is built against (suite-enforced by
73
+ # test_benchmark_anchor_required, but spot-checked here too).
74
+ anchors = pack.meta.benchmark_anchor
75
+ assert isinstance(anchors, list) and anchors
76
+ joined = " ".join(anchors).lower()
77
+ assert "roe" in joined or "rules-of-engagement" in joined or "rules of engagement" in joined
78
+ assert "sc2" in joined or "stance" in joined
79
+ assert "ambush" in joined
80
+ # meta required-prose fields populated and on-theme.
81
+ rwm = pack.meta.real_world_meaning.lower()
82
+ assert "stance" in rwm or "hold" in rwm or "set_stance" in rwm
83
+ assert "ambush" in rwm or "trigger" in rwm or "overwatch" in rwm
84
+ for lvl in LEVELS:
85
+ c = compile_level(pack, lvl)
86
+ assert c.map_supported
87
+ assert c.win_condition is not None and c.fail_condition is not None
88
+
89
+
90
+ def test_allowlist_includes_set_stance():
91
+ """The base `tools:` allowlist must include `set_stance` — it is
92
+ the verb under test (without it the agent cannot lift defenders
93
+ from HoldFire and the scenario is unsolvable)."""
94
+ pack = load_pack(PACK_PATH)
95
+ base = pack.base if isinstance(pack.base, dict) else pack.base.dict()
96
+ tools = set(base.get("tools") or [])
97
+ assert "set_stance" in tools, (
98
+ "set_stance MUST be on the allowlist — it is the agent's only "
99
+ "way to lift the pre-staged defenders from HoldFire and the "
100
+ "load-bearing verb the pack tests"
101
+ )
102
+ assert "observe" in tools
103
+
104
+
105
+ def test_defenders_start_on_holdfire_at_the_choke():
106
+ """The defenders (2tnk) on every level must be pre-placed at the
107
+ choke on stance:0 (HoldFire). If they start on any other stance
108
+ the scenario degenerates (a stall WINS because auto-engage fires
109
+ without the agent doing anything — the verb is no longer load-
110
+ bearing)."""
111
+ pack = load_pack(PACK_PATH)
112
+ for lvl in LEVELS:
113
+ c = compile_level(pack, lvl)
114
+ tanks = [
115
+ a for a in c.scenario.actors
116
+ if a.owner == "agent" and a.type == "2tnk"
117
+ ]
118
+ assert tanks, f"{lvl}: no 2tnk defenders found"
119
+ for t in tanks:
120
+ stance = getattr(t, "stance", None)
121
+ assert stance == 0, (
122
+ f"{lvl}: defender 2tnk at {t.position} starts on stance "
123
+ f"{stance}, expected 0 (HoldFire) — otherwise the "
124
+ f"set_stance verb is not load-bearing (a stall policy "
125
+ f"would win by auto-engagement)"
126
+ )
127
+
128
+
129
+ def test_each_level_has_a_reachable_timeout_fail():
130
+ """Non-win must be a real LOSS, not a draw. The `after_ticks` fail
131
+ must be strictly below the tick reachable at max_turns (≤90 ticks/
132
+ turn ⇒ tick ≤ 93 + 90·(max_turns-1))."""
133
+ pack = load_pack(PACK_PATH)
134
+ for lvl in LEVELS:
135
+ c = compile_level(pack, lvl)
136
+ fc = c.fail_condition.model_dump(exclude_none=True)
137
+ deadline = None
138
+ for clause in fc.get("any_of", []) or []:
139
+ if "after_ticks" in clause:
140
+ deadline = int(clause["after_ticks"])
141
+ assert deadline is not None, f"{lvl}: no after_ticks fail clause"
142
+ reachable = 93 + 90 * (c.max_turns - 1)
143
+ assert deadline < reachable, (
144
+ f"{lvl}: deadline {deadline} unreachable within "
145
+ f"{c.max_turns} turns (max tick {reachable}) → draw degeneracy"
146
+ )
147
+ # within_ticks in the win clause must ALSO be reachable (else
148
+ # a passing intended policy times out as a draw too).
149
+ wc = c.win_condition.model_dump(exclude_none=True)
150
+ deadline_win = None
151
+ for clause in wc.get("all_of", []) or []:
152
+ if "within_ticks" in clause:
153
+ deadline_win = int(clause["within_ticks"])
154
+ assert deadline_win is not None, f"{lvl}: no within_ticks win clause"
155
+ assert deadline_win < reachable, (
156
+ f"{lvl}: win deadline {deadline_win} unreachable within "
157
+ f"{c.max_turns} turns"
158
+ )
159
+
160
+
161
+ def test_win_predicate_carries_the_task_idiom():
162
+ """The win clause MUST be of the form
163
+ `all_of[ has_building:fact, units_killed_gte:K, own_units_gte:3,
164
+ within_ticks:T ]` — matches the Wave-8 scenario brief and ensures
165
+ the four load-bearing axes (asset survival + engagement + force
166
+ preservation + clock) are all measured."""
167
+ pack = load_pack(PACK_PATH)
168
+ for lvl in LEVELS:
169
+ c = compile_level(pack, lvl)
170
+ wc = c.win_condition.model_dump(exclude_none=True)
171
+ flat = str(wc)
172
+ assert "has_building" in flat and "fact" in flat, (lvl, wc)
173
+ assert "units_killed_gte" in flat, (lvl, wc)
174
+ assert "own_units_gte" in flat, (lvl, wc)
175
+ assert "within_ticks" in flat, (lvl, wc)
176
+
177
+
178
+ def test_fail_predicate_carries_the_task_idiom():
179
+ """The fail clause MUST be `any_of[ after_ticks:T+1,
180
+ not has_building:fact, not own_units_gte:1 ]` — every non-win
181
+ pathway (timeout, base lost, force wiped) is a real LOSS."""
182
+ pack = load_pack(PACK_PATH)
183
+ for lvl in LEVELS:
184
+ c = compile_level(pack, lvl)
185
+ fc = c.fail_condition.model_dump(exclude_none=True)
186
+ flat = str(fc)
187
+ assert "after_ticks" in flat, (lvl, fc)
188
+ assert "has_building" in flat and "fact" in flat, (lvl, fc)
189
+ assert "own_units_gte" in flat, (lvl, fc)
190
+
191
+
192
+ def test_hard_defines_two_agent_spawn_point_groups():
193
+ """Hard-tier contract (UPGRADED): ≥2 distinct seed-driven agent
194
+ spawn_point groups so the choke latitude rotates by seed (anti-
195
+ memorisation of an absolute trigger cell — the doctrine must
196
+ generalise across either spawn)."""
197
+ pack = load_pack(PACK_PATH)
198
+ c = compile_level(pack, "hard")
199
+ groups = {
200
+ (a.spawn_point if a.spawn_point is not None else 0)
201
+ for a in c.scenario.actors
202
+ if a.owner == "agent"
203
+ }
204
+ assert groups == {0, 1}, (
205
+ f"hard must define ≥2 agent spawn_point groups for seed-driven "
206
+ f"start variation; got {sorted(groups)}"
207
+ )
208
+ # In-bounds check (rush-hour-arena playable y ≈ 2..38).
209
+ for a in c.scenario.actors:
210
+ x, y = a.position
211
+ assert 2 <= x <= 126 and 2 <= y <= 38, (a.type, a.position)
212
+
213
+
214
+ def test_pack_is_in_upgraded_not_in_not_applicable():
215
+ """Round-trip check: this pack IS spawn-varied on hard, so it must
216
+ appear in UPGRADED and NOT in NOT_APPLICABLE."""
217
+ from tests.test_hard_tier import NOT_APPLICABLE, UPGRADED
218
+
219
+ assert "def-stance-mgmt-hold-then-attack" in UPGRADED, (
220
+ "pack defines ≥2 spawn groups on hard; must appear in UPGRADED"
221
+ )
222
+ assert "def-stance-mgmt-hold-then-attack" not in NOT_APPLICABLE, (
223
+ "pack must not be in NOT_APPLICABLE — it IS spawn-varied"
224
+ )
225
+
226
+
227
+ # ── engine-driven scripted policies (intended WINS, stall LOSES) ────
228
+
229
+
230
+ def _intended_policy(rs, Command):
231
+ """Lift defenders from HoldFire (stance:0) to AttackAnything
232
+ (stance:3) every turn. The tank cluster's auto-targeting then
233
+ engages the rusher band the instant it closes to cannon range
234
+ (the tanks don't pursue beyond the choke — def-pre-position-
235
+ mobile-reserve engine fact — so the salvo happens at the choke
236
+ regardless of which turn the flip is issued). The agent never
237
+ issues attack_unit / attack_move; the kills are the engine's
238
+ stance-driven auto-fire."""
239
+ units = [
240
+ u for u in (rs.get("units_summary", []) or [])
241
+ if u.get("type") == "2tnk"
242
+ ]
243
+ if not units:
244
+ return [Command.observe()]
245
+ ids = [str(u["id"]) for u in units]
246
+ return [Command.set_stance(ids, 3), Command.observe()]
247
+
248
+
249
+ def _stall_policy(rs, Command):
250
+ """Issue only observe(); never lift the stance, never move. The
251
+ defenders stay on stance:0 (HoldFire) — they never engage; the
252
+ rusher band overruns the silent cluster (tanks die without
253
+ firing), then turns on the fact and razes it ⇒ `not has_building:
254
+ fact` LOSS. Kills stay at 0 (independent fail on units_killed_gte)
255
+ so the win predicate is busted on multiple clauses."""
256
+ return [Command.observe()]
257
+
258
+
259
+ @pytest.mark.parametrize("level", LEVELS)
260
+ def test_intended_policy_wins_every_level_and_seed(level):
261
+ """Intended set_stance(3) wins on every level; on hard, across all
262
+ 4 seeds (UPGRADED contract)."""
263
+ pytest.importorskip("openra_train")
264
+ from openra_bench.eval_core import run_level
265
+
266
+ c = compile_level(load_pack(PACK_PATH), level)
267
+ seeds = HARD_SEEDS if level == "hard" else (1,)
268
+ for s in seeds:
269
+ res = run_level(c, _intended_policy, seed=s)
270
+ assert res.outcome == "win", (
271
+ f"{level} seed={s}: intended set_stance(3) policy must WIN, "
272
+ f"got {res.outcome} after {res.turns} turns "
273
+ f"(killed={res.signals.units_killed}, "
274
+ f"lost={res.signals.units_lost}, "
275
+ f"tick={res.signals.game_tick}, "
276
+ f"buildings={res.signals.own_buildings})"
277
+ )
278
+
279
+
280
+ @pytest.mark.parametrize("level", LEVELS)
281
+ def test_stall_policy_loses_every_level_and_seed(level):
282
+ """Pure-observe stall: defenders stay on HoldFire, rusher overruns
283
+ the cluster, fact razed ⇒ LOSS on every level / seed. Non-win
284
+ must be a real LOSS, not a draw — `after_ticks 4501` reaches
285
+ within `max_turns 55` (≤93 + 90·54 = 4953)."""
286
+ pytest.importorskip("openra_train")
287
+ from openra_bench.eval_core import run_level
288
+
289
+ c = compile_level(load_pack(PACK_PATH), level)
290
+ seeds = HARD_SEEDS if level == "hard" else (1,)
291
+ for s in seeds:
292
+ res = run_level(c, _stall_policy, seed=s)
293
+ assert res.outcome == "loss", (
294
+ f"{level} seed={s}: stall must LOSE (fact razed or force "
295
+ f"wiped), got {res.outcome} "
296
+ f"(killed={res.signals.units_killed}, "
297
+ f"lost={res.signals.units_lost}, "
298
+ f"tick={res.signals.game_tick})"
299
+ )
300
+ # Stall must rack up exactly 0 kills (defenders never fire on
301
+ # HoldFire) — this is the proof that `set_stance` is the load-
302
+ # bearing verb (a "kill the rush via auto-engage" play is
303
+ # impossible without lifting the stance).
304
+ assert res.signals.units_killed == 0, (
305
+ f"{level} seed={s}: stall (no set_stance call) killed "
306
+ f"{res.signals.units_killed} units — defenders should be "
307
+ f"silent on HoldFire (verb is not load-bearing if kills > 0)"
308
+ )
309
+
310
+
311
+ def test_intended_run_is_deterministic_on_easy():
312
+ """Same seed must reproduce the same outcome + turn count — the
313
+ bench's seed-based reproducibility contract."""
314
+ pytest.importorskip("openra_train")
315
+ from openra_bench.eval_core import run_level
316
+
317
+ c = compile_level(load_pack(PACK_PATH), "easy")
318
+ a = run_level(c, _intended_policy, seed=2)
319
+ b = run_level(c, _intended_policy, seed=2)
320
+ assert (a.outcome, a.turns) == (b.outcome, b.turns), (
321
+ "same seed must be deterministic"
322
+ )