yxc20098 commited on
Commit
9b275f8
·
1 Parent(s): b1c6156

feat(scenario): proc-conditional-branch-action — scout then branch IF/ELSE (τ²-bench / IFBench conditional anchor)

Browse files
openra_bench/scenarios/packs/proc-conditional-branch-action.yaml ADDED
@@ -0,0 +1,441 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # proc-conditional-branch-action — Wave-11
2
+ #
3
+ # REASONING capability — CONDITIONAL PROCEDURE EXECUTION.
4
+ # The brief is an explicit IF/ELSE runbook:
5
+ #
6
+ # "Scout the map. IF the enemy outpost is on the NORTH flank,
7
+ # send your strike force NORTH and destroy it. ELSE (the enemy
8
+ # outpost is on the SOUTH flank) send your strike force SOUTH
9
+ # and destroy it."
10
+ #
11
+ # The agent does not know a priori which branch is live — it must
12
+ # (1) SCOUT to observe the condition, (2) BRANCH on what it sees,
13
+ # (3) EXECUTE the matching action. A fixed unconditional policy
14
+ # ("always attack NORTH") is correct on the seeds where the
15
+ # condition picks NORTH and wrong on the seeds where it picks
16
+ # SOUTH — there it commits the army to an empty flank, never
17
+ # reaches the live outpost, and times out. Only the
18
+ # scout-then-branch discipline generalises across every seed.
19
+ #
20
+ # This is the conditional-tool-use / runbook-branching primitive:
21
+ # read the runtime condition, dispatch to the matching procedure.
22
+ # The win is encoded as an explicit two-branch `any_of` IF/ELSE —
23
+ # exactly one branch is satisfiable per seed, selected by what the
24
+ # scout reveals.
25
+ #
26
+ # ── Seed axis (the Wave-9 enemy-side `spawn_point` per-owner
27
+ # activation, CLAUDE.md): the ENEMY outpost + its guard cluster
28
+ # rotate FLANK per seed across TWO spawn_point groups —
29
+ # spawn_point=0 → enemy outpost NORTH → branch A (attack N)
30
+ # spawn_point=1 → enemy outpost SOUTH → branch B (attack S)
31
+ # The agent base is FIXED and identical every seed (no agent
32
+ # actor declares spawn_point → the per-owner agent filter is
33
+ # inactive → every agent actor passes through unconditionally).
34
+ # `distinct_enemy_spawn_points` exposes the two groups and the
35
+ # env round-robins the seed across them (seeds 1..4 → spawn
36
+ # groups 1,0,1,0 — seeds 1/3 SOUTH, seeds 2/4 NORTH).
37
+ #
38
+ # ── Win idiom — explicit IF/ELSE branch:
39
+ # any_of:
40
+ # # BRANCH A — outpost NORTH: scout, then raze the N outpost.
41
+ # - all_of:
42
+ # - then: [ buildings_discovered_gte:1, # scout
43
+ # enemy_key_buildings_destroyed_in_region{ # raze
44
+ # tent, NORTH region } ]
45
+ # - building_count_gte:{fact,1}
46
+ # - within_ticks:T
47
+ # # BRANCH B — outpost SOUTH: scout, then raze the S outpost.
48
+ # - all_of:
49
+ # - then: [ buildings_discovered_gte:1, # scout
50
+ # enemy_key_buildings_destroyed_in_region{ # raze
51
+ # tent, SOUTH region } ]
52
+ # - building_count_gte:{fact,1}
53
+ # - within_ticks:T
54
+ # Fail: after_ticks:T+1 OR not building_count_gte:{fact,1}.
55
+ #
56
+ # `enemy_key_buildings_destroyed_in_region` is region-scoped: it
57
+ # latches only when an enemy `tent` is destroyed WITHIN radius of
58
+ # the named region. The enemy outpost `tent` exists on ONLY ONE
59
+ # flank per seed (the seed-varying spawn group), so:
60
+ # - On a NORTH seed: only branch A is satisfiable — there is no
61
+ # SOUTH tent for branch B's clause 2 to ever latch.
62
+ # - On a SOUTH seed: only branch B is satisfiable — mirror.
63
+ # A fixed "always attack NORTH" policy on a SOUTH seed sends the
64
+ # strike force to the empty NORTH flank: it discovers nothing
65
+ # there (branch A clause 1 never latches — `buildings_discovered`
66
+ # only counts a building the agent has actually seen), it razes
67
+ # nothing (branch A clause 2 impossible — no NORTH tent), branch
68
+ # B clause 2 needs a SOUTH tent razed but the army is at the
69
+ # wrong end of the map → the deadline passes → `after_ticks`
70
+ # fires → real timeout LOSS (the enemy is `guard`, it holds post
71
+ # and never force-wipes the agent, so non-completion is a clean
72
+ # clock LOSS, never a draw).
73
+ #
74
+ # ── Four-policy bar (every seed 1..4):
75
+ # • stall (observe only) → LOSS. No tent razed; the
76
+ # `after_ticks` deadline fires.
77
+ # • always-branch-A (attack N) → LOSS on the SOUTH seeds
78
+ # (1, 3): the army crosses to an empty NORTH flank, no NORTH
79
+ # tent to raze, the SOUTH tent stays up → deadline LOSS. WINS
80
+ # the NORTH seeds (2, 4) — but the hard tier requires winning
81
+ # EVERY seed → a 2-of-4 play is a structural LOSS.
82
+ # • always-branch-B (attack S) → mirror: LOSS on NORTH seeds.
83
+ # • intended scout-then-branch → WIN every seed. Drive a
84
+ # jeep to the live flank, read which side the enemy outpost is
85
+ # on, send the strike force to THAT flank, raze the outpost.
86
+ #
87
+ # ── Engine guardrails (CLAUDE.md / SCENARIO_QUALITY.md):
88
+ # - Enemy-side `spawn_point:` activates the per-owner filter
89
+ # (Wave-9, openra-data/src/oramap.rs). The persistent far-east
90
+ # enemy `fact` marker (anti-DRAW: prevents engine auto-`done`
91
+ # on enemy-elim before the win/fail evaluator runs, since
92
+ # razing the outpost `tent` would otherwise auto-`done` the
93
+ # episode) is duplicated across BOTH enemy spawn groups at
94
+ # identical coords so it lands every seed.
95
+ # - The pack-wide enemy `bot_type: guard` makes the enemy hold
96
+ # post: the outpost cluster does not roam, so a wrong-flank or
97
+ # stall play is a clean clock LOSS (deadline) rather than a
98
+ # force-wipe — and the agent base is never threatened, so
99
+ # `building_count_gte:fact` is a stable (always-true unless the
100
+ # agent itself loses the fact) clause that simply guards the
101
+ # anti-DRAW invariant.
102
+ # - `building_count_gte:{fact,1}` (NOT `has_building:fact`) for
103
+ # the "still standing" clause: `has_building` reads
104
+ # `own_building_types` which ACCUMULATES across the episode and
105
+ # stays true after the fact is razed; `building_count_gte`
106
+ # reads the live `own_buildings` list and bites correctly.
107
+ # - `after_ticks` lives only in the fail clause (it is
108
+ # structurally incompatible with a WIN clause under
109
+ # ConquestVictoryConditions — CLAUDE.md). Timed semantics are
110
+ # `within_ticks` in the win + `after_ticks` in the fail.
111
+ # - `enemy_key_buildings_destroyed_in_region` requires the agent
112
+ # to have SEEN the tent and then have a unit within ~6 cells
113
+ # when it falls — i.e. the strike force must physically reach
114
+ # and raze the outpost (`attack_move` reliably executes this;
115
+ # pinned by the smoke runs).
116
+ # - Tick budget: max_turns 66 → reachable tick 93 + 90·65 = 5943
117
+ # ≥ 5401 (deadline 5400 +1), so a non-finisher is a real
118
+ # reachable timeout LOSS, never a draw. The intended one-flank
119
+ # commit razes the outpost by tick ~2000 (smoke-verified),
120
+ # well inside 5400.
121
+ # - `enemy_building_spotted` / `enemy_unit_spotted` interrupts:
122
+ # re-decide the instant the scout reveals the live flank.
123
+ # - Starter combat units (`2tnk` strike force + `jeep` scouts)
124
+ # satisfy `own_units_gte:1` from t=0 so the unit-less mis-fire
125
+ # footgun does not trip.
126
+ # - `2tnk` / `jeep` scenario actors place from t=0 — this is a
127
+ # branch-SELECTION puzzle, not a build-order puzzle, so the
128
+ # whole strike force is pre-placed.
129
+ # - Map: rush-hour-arena (128×40, playable x∈[2..126], y∈[2..38];
130
+ # Rust-loadable, no map-gen needed).
131
+ #
132
+ # Real-world / benchmark anchors:
133
+ # - τ²-bench conditional tool-use: the correct tool call is
134
+ # gated on an observed runtime condition; a policy that fixes
135
+ # the call without reading the condition fails the branch.
136
+ # - IFBench conditional instruction: "IF <condition> THEN do X
137
+ # ELSE do Y" — compliance requires evaluating the antecedent.
138
+ # - Operational runbook branching: an incident runbook with an
139
+ # IF/ELSE fork (symptom A → procedure A, symptom B → procedure
140
+ # B); executing the wrong procedure is a misfire.
141
+ #
142
+ # Validate (no model / no network):
143
+ # python3 -m openra_bench.scenarios.validate \
144
+ # openra_bench/scenarios/packs/proc-conditional-branch-action.yaml
145
+
146
+ meta:
147
+ id: proc-conditional-branch-action
148
+ title: 'Scout, Then Branch — Conditional IF/ELSE Procedure Execution'
149
+ capability: reasoning
150
+ real_world_meaning: >
151
+ Conditional procedure execution: the operator is handed an
152
+ explicit IF/ELSE runbook ("IF the enemy outpost is NORTH,
153
+ attack NORTH; ELSE attack SOUTH") but does not know which
154
+ antecedent is live. The capability under test is the
155
+ read-then-dispatch loop — scout to OBSERVE the condition,
156
+ BRANCH on what is seen, then EXECUTE the matching procedure. A
157
+ fixed unconditional policy executes one branch on faith and is
158
+ wrong on every seed where the other branch is correct.
159
+ robotics_analogue: >
160
+ Conditional tool-use / runbook branching under uncertainty: an
161
+ incident-response agent follows a decision tree whose fork is
162
+ gated on a sensor reading — symptom A dispatches procedure A,
163
+ symptom B dispatches procedure B. Skipping the diagnostic step
164
+ and committing to one procedure is correct half the time and a
165
+ misfire the other half. The robust doctrine is observe-the-
166
+ condition-then-branch, never branch-on-faith.
167
+ benchmark_anchor:
168
+ - 'τ²-bench conditional tool-use'
169
+ - 'IFBench conditional instruction'
170
+ - 'runbook branching'
171
+ author: openra-bench
172
+
173
+ base_map: rush-hour-arena
174
+
175
+ starting_cash: 1500
176
+
177
+ base:
178
+ agent:
179
+ faction: allies
180
+ # `guard` — the enemy outpost cluster holds post; it does not
181
+ # roam, so a wrong-flank or stall play is a clean clock LOSS and
182
+ # the agent base is never threatened.
183
+ enemy:
184
+ faction: soviet
185
+ bot_type: guard
186
+ tools:
187
+ - observe
188
+ - move_units
189
+ - attack_unit
190
+ - attack_move
191
+ - stop
192
+ planning: true
193
+ # Re-decide the instant the scout reveals the live flank (enemy
194
+ # outpost or guard unit spotted).
195
+ interrupts:
196
+ enemy_building_spotted: true
197
+ enemy_unit_spotted: true
198
+ termination:
199
+ max_ticks: 12000
200
+ actors:
201
+ # Base-pack default (anti-empty-actors sanity for the loader);
202
+ # each level overrides with the level-specific layout.
203
+ - {type: fact, owner: agent, position: [20, 20]}
204
+
205
+ levels:
206
+ # ── EASY ─────────────────────────────────────────────────────────
207
+ # The bare IF/ELSE skill, condition NORTH, condition pre-revealed.
208
+ # The two jeeps are pre-positioned FORWARD-NORTH (y=8) so a single
209
+ # passive observe on turn 1 already sees the enemy NORTH outpost.
210
+ # The model still has to BRANCH: read "outpost is NORTH", send the
211
+ # strike force NORTH, raze the outpost. The SOUTH branch is
212
+ # structurally dead (no SOUTH outpost) so an "always SOUTH" policy
213
+ # commits the army to an empty flank and times out even on easy.
214
+ # max_turns 66 → reachable tick 5943 ≥ 5401.
215
+ easy:
216
+ description: >
217
+ Runbook: scout the map, then BRANCH. IF the enemy outpost is
218
+ on the NORTH flank, send your strike force NORTH and destroy
219
+ it; ELSE (outpost on the SOUTH flank) send it SOUTH and
220
+ destroy it. On this level the enemy is on the NORTH flank and
221
+ your two scout jeeps are already forward-NORTH in sight of
222
+ the enemy outpost. Send your four medium tanks NORTH and
223
+ destroy the enemy outpost (a barracks at the north edge).
224
+ Keep your construction yard standing. Win before tick 5400.
225
+ Committing the tanks to the wrong (SOUTH) flank leaves them
226
+ stranded on an empty lane and you time out; stalling loses on
227
+ the clock.
228
+ overrides:
229
+ actors:
230
+ # Agent base — construction yard (the anti-DRAW / "still
231
+ # standing" anchor; no production needed, the strike force
232
+ # is pre-placed).
233
+ - {type: fact, owner: agent, position: [20, 20]}
234
+ # Strike force — four medium tanks at the base centroid.
235
+ - {type: 2tnk, owner: agent, position: [18, 20], stance: 1}
236
+ - {type: 2tnk, owner: agent, position: [22, 20], stance: 1}
237
+ - {type: 2tnk, owner: agent, position: [18, 22], stance: 1}
238
+ - {type: 2tnk, owner: agent, position: [22, 22], stance: 1}
239
+ # Two scout jeeps pre-positioned FORWARD-NORTH at y=8 —
240
+ # within vision (~7) of the enemy outpost. A turn-1 observe
241
+ # latches `buildings_discovered_gte:1`; easy teaches the
242
+ # BRANCH without requiring scout movement.
243
+ - {type: jeep, owner: agent, position: [20, 8], stance: 0}
244
+ - {type: jeep, owner: agent, position: [22, 8], stance: 0}
245
+ # Enemy NORTH outpost — a `tent` (barracks) at the playable
246
+ # edge (y=4) with a small guard cluster. `guard` bot holds
247
+ # post; the cluster does not roam.
248
+ - {type: tent, owner: enemy, position: [20, 4]}
249
+ - {type: e1, owner: enemy, position: [20, 6], stance: 2, count: 3}
250
+ # Inert enemy `fact` marker far east — anti-DRAW per
251
+ # CLAUDE.md (razing the outpost tent would otherwise
252
+ # auto-`done` the episode before the win/fail evaluator
253
+ # runs). No spawn_point on easy: single composition.
254
+ - {type: fact, owner: enemy, position: [120, 20]}
255
+ win_condition:
256
+ any_of:
257
+ # BRANCH A — outpost NORTH: scout, then raze the N outpost.
258
+ - all_of:
259
+ - then:
260
+ id: branch-north-easy
261
+ clauses:
262
+ - {buildings_discovered_gte: 1}
263
+ - {enemy_key_buildings_destroyed_in_region: {types: [tent], x: 20, y: 4, radius: 12}}
264
+ - {building_count_gte: {type: fact, n: 1}}
265
+ - {within_ticks: 5400}
266
+ # BRANCH B — outpost SOUTH: scout, then raze the S outpost.
267
+ # Present so the win schema is the same explicit IF/ELSE
268
+ # every tier. On easy the enemy is NORTH, so there is no
269
+ # SOUTH tent — clause 2 of this branch can never latch, and
270
+ # an "always SOUTH" play strands its army and times out.
271
+ - all_of:
272
+ - then:
273
+ id: branch-south-easy
274
+ clauses:
275
+ - {buildings_discovered_gte: 1}
276
+ - {enemy_key_buildings_destroyed_in_region: {types: [tent], x: 20, y: 36, radius: 12}}
277
+ - {building_count_gte: {type: fact, n: 1}}
278
+ - {within_ticks: 5400}
279
+ fail_condition:
280
+ any_of:
281
+ - {after_ticks: 5401}
282
+ - {not: {building_count_gte: {type: fact, n: 1}}}
283
+ max_turns: 66
284
+
285
+ # ── MEDIUM ───────────────────────────────────────────────────────
286
+ # +1 controlled variable: the condition is FOGGED. The two jeeps
287
+ # now start AT the base (y=20), NOT forward — the NORTH outpost at
288
+ # (20, 4) is 16 cells away, well outside passive vision (~7). A
289
+ # no-scout play cannot latch `buildings_discovered_gte:1`. The
290
+ # intended play is `move_units([jeep1, jeep2], 20, 8)` to enter
291
+ # outpost sight, read "outpost is NORTH", THEN commit the strike
292
+ # force NORTH. Condition is still NORTH (medium adds only the
293
+ # SCOUT requirement; hard adds the per-seed flip).
294
+ # max_turns 66 → reachable tick 5943 ≥ 5401.
295
+ medium:
296
+ description: >
297
+ Runbook: scout, then BRANCH. IF the enemy outpost is NORTH,
298
+ send your strike force NORTH and destroy it; ELSE send it
299
+ SOUTH. The enemy flank is NOT given — you must SCOUT to find
300
+ it. You start with a construction yard, four medium tanks,
301
+ and TWO scout jeeps at the base. Drive a jeep to discover the
302
+ enemy outpost, read which flank it is on, then send the four
303
+ tanks to THAT flank and destroy the outpost (a barracks at
304
+ the map edge). Keep your construction yard standing. Win
305
+ before tick 5400. Skipping the scout leaves the branch chain
306
+ stuck at clause 1; committing the tanks to the wrong flank
307
+ strands them on an empty lane and you time out.
308
+ overrides:
309
+ actors:
310
+ - {type: fact, owner: agent, position: [20, 20]}
311
+ - {type: 2tnk, owner: agent, position: [18, 20], stance: 1}
312
+ - {type: 2tnk, owner: agent, position: [22, 20], stance: 1}
313
+ - {type: 2tnk, owner: agent, position: [18, 22], stance: 1}
314
+ - {type: 2tnk, owner: agent, position: [22, 22], stance: 1}
315
+ # Jeeps AT the base — must drive to scout. Distance from
316
+ # (24, 20) to the NORTH outpost (20, 4) is √(16+256)≈16,
317
+ # far out of jeep vision (~7).
318
+ - {type: jeep, owner: agent, position: [24, 20], stance: 0}
319
+ - {type: jeep, owner: agent, position: [26, 20], stance: 0}
320
+ # Enemy NORTH outpost — outside passive vision; the scout
321
+ # MUST drive a jeep north to see it.
322
+ - {type: tent, owner: enemy, position: [20, 4]}
323
+ - {type: e1, owner: enemy, position: [20, 6], stance: 2, count: 3}
324
+ - {type: fact, owner: enemy, position: [120, 20]}
325
+ win_condition:
326
+ any_of:
327
+ - all_of:
328
+ - then:
329
+ id: branch-north-medium
330
+ clauses:
331
+ - {buildings_discovered_gte: 1}
332
+ - {enemy_key_buildings_destroyed_in_region: {types: [tent], x: 20, y: 4, radius: 12}}
333
+ - {building_count_gte: {type: fact, n: 1}}
334
+ - {within_ticks: 5400}
335
+ - all_of:
336
+ - then:
337
+ id: branch-south-medium
338
+ clauses:
339
+ - {buildings_discovered_gte: 1}
340
+ - {enemy_key_buildings_destroyed_in_region: {types: [tent], x: 20, y: 36, radius: 12}}
341
+ - {building_count_gte: {type: fact, n: 1}}
342
+ - {within_ticks: 5400}
343
+ fail_condition:
344
+ any_of:
345
+ - {after_ticks: 5401}
346
+ - {not: {building_count_gte: {type: fact, n: 1}}}
347
+ max_turns: 66
348
+
349
+ # ── HARD ─────────────────────────────────────────────────────────
350
+ # +1 controlled variable (the Wave-9 enemy-side `spawn_point`
351
+ # axis): the enemy outpost ROTATES FLANK per seed across TWO
352
+ # enemy spawn_point groups —
353
+ # spawn_point=0 → outpost NORTH (tent (20, 4)) → branch A live.
354
+ # spawn_point=1 → outpost SOUTH (tent (20, 36)) → branch B live.
355
+ # The agent base is FIXED at (20, 20) every seed (no agent
356
+ # spawn_point). `distinct_enemy_spawn_points` exposes the two
357
+ # groups; the env round-robins the seed (seeds 1..4 → groups
358
+ # 1,0,1,0 — seeds 1/3 SOUTH, seeds 2/4 NORTH).
359
+ #
360
+ # A memorised "always attack NORTH" opener WINS the NORTH seeds
361
+ # (2, 4) and LOSES the SOUTH seeds (1, 3): the strike force
362
+ # crosses to an empty NORTH flank, branch A clause 1 never
363
+ # latches (nothing to discover NORTH), branch A clause 2 is
364
+ # impossible (no NORTH tent), branch B clause 2 needs the SOUTH
365
+ # tent razed but the army is at the wrong end of the map → the
366
+ # deadline passes → real timeout LOSS. Mirror for "always SOUTH".
367
+ # Only the scout-then-branch policy wins every seed.
368
+ #
369
+ # The persistent far-east enemy `fact` marker is duplicated
370
+ # across BOTH enemy spawn groups (CLAUDE.md per-owner filter: an
371
+ # enemy actor without spawn_point would be filtered out once any
372
+ # enemy declares one).
373
+ # max_turns 66 → reachable tick 5943 ≥ 5401.
374
+ hard:
375
+ description: >
376
+ Runbook: scout, then BRANCH. The enemy outpost could be on
377
+ the NORTH flank OR the SOUTH flank — chosen by seed, and NOT
378
+ given to you. Drive a scout jeep to discover which flank the
379
+ enemy outpost is on. IF it is NORTH, send your four medium
380
+ tanks NORTH and destroy it; ELSE (outpost SOUTH) send them
381
+ SOUTH and destroy it (the outpost is a barracks at the map
382
+ edge). Keep your construction yard standing. Win before tick
383
+ 5400. A memorised "always attack NORTH" (or "always SOUTH")
384
+ opener loses on the seeds where the outpost is on the other
385
+ flank — the strike force is stranded on an empty lane and the
386
+ live outpost is never razed.
387
+ overrides:
388
+ actors:
389
+ # ── AGENT BASE — fixed every seed (no spawn_point).
390
+ - {type: fact, owner: agent, position: [20, 20]}
391
+ - {type: 2tnk, owner: agent, position: [18, 18], stance: 1}
392
+ - {type: 2tnk, owner: agent, position: [22, 18], stance: 1}
393
+ - {type: 2tnk, owner: agent, position: [18, 22], stance: 1}
394
+ - {type: 2tnk, owner: agent, position: [22, 22], stance: 1}
395
+ # Two scout jeeps AT the base — must drive to whichever
396
+ # flank to read the condition.
397
+ - {type: jeep, owner: agent, position: [24, 20], stance: 0}
398
+ - {type: jeep, owner: agent, position: [26, 20], stance: 0}
399
+
400
+ # ── ENEMY spawn_point=0 — outpost massing NORTH ──
401
+ # Branch A is the live branch.
402
+ - {type: tent, owner: enemy, position: [20, 4], spawn_point: 0}
403
+ - {type: e1, owner: enemy, position: [20, 6], stance: 2, count: 3, spawn_point: 0}
404
+
405
+ # ── ENEMY spawn_point=1 — outpost massing SOUTH ──
406
+ # Branch B is the live branch.
407
+ - {type: tent, owner: enemy, position: [20, 36], spawn_point: 1}
408
+ - {type: e1, owner: enemy, position: [20, 34], stance: 2, count: 3, spawn_point: 1}
409
+
410
+ # ── Persistent far-east enemy `fact` marker (anti-DRAW).
411
+ # Duplicated across BOTH enemy spawn groups at identical
412
+ # coords — once any enemy declares spawn_point the per-owner
413
+ # filter activates and an unmarked enemy actor would be
414
+ # dropped (CLAUDE.md).
415
+ - {type: fact, owner: enemy, position: [120, 20], spawn_point: 0}
416
+ - {type: fact, owner: enemy, position: [120, 20], spawn_point: 1}
417
+ win_condition:
418
+ any_of:
419
+ # BRANCH A — outpost NORTH: scout, then raze the N outpost.
420
+ - all_of:
421
+ - then:
422
+ id: branch-north-hard
423
+ clauses:
424
+ - {buildings_discovered_gte: 1}
425
+ - {enemy_key_buildings_destroyed_in_region: {types: [tent], x: 20, y: 4, radius: 12}}
426
+ - {building_count_gte: {type: fact, n: 1}}
427
+ - {within_ticks: 5400}
428
+ # BRANCH B — outpost SOUTH: scout, then raze the S outpost.
429
+ - all_of:
430
+ - then:
431
+ id: branch-south-hard
432
+ clauses:
433
+ - {buildings_discovered_gte: 1}
434
+ - {enemy_key_buildings_destroyed_in_region: {types: [tent], x: 20, y: 36, radius: 12}}
435
+ - {building_count_gte: {type: fact, n: 1}}
436
+ - {within_ticks: 5400}
437
+ fail_condition:
438
+ any_of:
439
+ - {after_ticks: 5401}
440
+ - {not: {building_count_gte: {type: fact, n: 1}}}
441
+ max_turns: 66
tests/test_proc_conditional_branch_action.py ADDED
@@ -0,0 +1,188 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """proc-conditional-branch-action pack — full no-cheat validation.
2
+
3
+ Wave-11 REASONING pack: conditional IF/ELSE procedure execution.
4
+ The brief is an explicit runbook fork — "scout the map; IF the
5
+ enemy outpost is on the NORTH flank, attack NORTH; ELSE attack
6
+ SOUTH". The agent must (1) SCOUT to observe which flank the
7
+ outpost is on, (2) BRANCH on that observation, (3) EXECUTE the
8
+ matching action (drive the strike force to the live flank and
9
+ raze the outpost).
10
+
11
+ The win is an explicit two-branch `any_of` IF/ELSE: each branch
12
+ is a `then`-chain `[buildings_discovered_gte:1,
13
+ enemy_key_buildings_destroyed_in_region{tent, flank}]` gated on a
14
+ region-scoped raze. The enemy outpost `tent` exists on only ONE
15
+ flank per seed (the Wave-9 enemy-side `spawn_point` axis), so
16
+ exactly one branch is satisfiable per seed.
17
+
18
+ Bar (per CLAUDE.md):
19
+ • stall → LOSS every level/seed (after_ticks).
20
+ • always-attack-NORTH → LOSS on the seeds where the outpost is
21
+ SOUTH (hard seeds 1, 3); WINS only the NORTH seeds.
22
+ • always-attack-SOUTH → mirror: LOSS on the NORTH seeds.
23
+ • scout-then-branch → WIN every level + seed.
24
+ A fixed unconditional opener wins exactly one flank; only the
25
+ scout-then-branch policy generalises across the per-seed flip.
26
+ """
27
+
28
+ from __future__ import annotations
29
+
30
+ import pytest
31
+
32
+ pytest.importorskip("openra_train", reason="Rust env wheel not installed")
33
+ pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
34
+
35
+ from openra_bench.eval_core import run_level
36
+ from openra_bench.scenarios import load_pack
37
+ from openra_bench.scenarios.loader import PACKS_DIR, compile_level
38
+
39
+ PACK = PACKS_DIR / "proc-conditional-branch-action.yaml"
40
+ LEVELS = ("easy", "medium", "hard")
41
+ SEEDS = (1, 2, 3, 4)
42
+ # On hard the env round-robins the enemy spawn_point per seed:
43
+ # seeds 1/3 → spawn_point 1 (outpost SOUTH), seeds 2/4 → spawn_point
44
+ # 0 (outpost NORTH). easy/medium have a single fixed NORTH outpost.
45
+ HARD_NORTH_SEEDS = (2, 4)
46
+ HARD_SOUTH_SEEDS = (1, 3)
47
+
48
+
49
+ # ── Policies ──────────────────────────────────────────────────────
50
+
51
+
52
+ def _stall_policy():
53
+ """Do nothing — must LOSE on after_ticks every level/seed."""
54
+ def pol(obs, Cmd):
55
+ return [Cmd.observe()]
56
+ return pol
57
+
58
+
59
+ def _fixed_flank_policy(flank: str):
60
+ """Always send the whole force to `flank` without scouting — a
61
+ fixed unconditional branch. Correct on the seeds where the
62
+ outpost is on `flank`, a stranded-army timeout LOSS otherwise."""
63
+ outpost_y = 4 if flank == "north" else 36
64
+
65
+ def pol(obs, Cmd):
66
+ units = obs.get("units_summary", []) or []
67
+ cmds = [
68
+ Cmd.attack_move([str(u["id"])], 20, outpost_y) for u in units
69
+ ]
70
+ return cmds or [Cmd.observe()]
71
+
72
+ return pol
73
+
74
+
75
+ def _scout_then_branch_policy():
76
+ """The intended capability: scout both flanks, branch on the
77
+ observed enemy, then commit the strike force to the live flank."""
78
+ state = {"flank": None}
79
+
80
+ def _observed_flank(obs):
81
+ ys = [
82
+ e["cell_y"]
83
+ for e in (obs.get("enemy_summary") or [])
84
+ if e.get("cell_y") is not None
85
+ ]
86
+ ys += [
87
+ b["cell_y"]
88
+ for b in (obs.get("enemy_buildings_summary") or [])
89
+ if b.get("cell_y") is not None
90
+ ]
91
+ if not ys:
92
+ return None
93
+ return "north" if sum(ys) / len(ys) < 20 else "south"
94
+
95
+ def pol(obs, Cmd):
96
+ units = obs.get("units_summary", []) or []
97
+ jeeps = [u for u in units if u.get("type") == "jeep"]
98
+ tanks = [u for u in units if u.get("type") == "2tnk"]
99
+ flank = _observed_flank(obs)
100
+ if flank and state["flank"] is None:
101
+ state["flank"] = flank
102
+ # Phase 1 — SCOUT: split the jeeps to probe both flanks.
103
+ if state["flank"] is None:
104
+ cmds = [
105
+ Cmd.move_units([str(j["id"])], 20, 6 if i == 0 else 34)
106
+ for i, j in enumerate(jeeps)
107
+ ]
108
+ return cmds or [Cmd.observe()]
109
+ # Phase 2/3 — BRANCH + EXECUTE: drive the force to the live
110
+ # flank and raze the outpost.
111
+ outpost_y = 4 if state["flank"] == "north" else 36
112
+ cmds = [
113
+ Cmd.attack_move([str(u["id"])], 20, outpost_y)
114
+ for u in tanks + jeeps
115
+ ]
116
+ return cmds or [Cmd.observe()]
117
+
118
+ return pol
119
+
120
+
121
+ # ── Tests ─────────────────────────────────────────────────────────
122
+
123
+
124
+ @pytest.mark.parametrize("level", LEVELS)
125
+ @pytest.mark.parametrize("seed", SEEDS)
126
+ def test_stall_loses(level, seed):
127
+ """A do-nothing policy must lose on the deadline — no draw."""
128
+ c = compile_level(load_pack(PACK), level)
129
+ res = run_level(c, _stall_policy(), seed=seed)
130
+ assert res.outcome == "loss", (
131
+ f"{level}/seed{seed}: stall must LOSE, got {res.outcome}"
132
+ )
133
+
134
+
135
+ @pytest.mark.parametrize("level", LEVELS)
136
+ @pytest.mark.parametrize("seed", SEEDS)
137
+ def test_scout_then_branch_wins(level, seed):
138
+ """The intended scout-then-branch policy must win every seed."""
139
+ c = compile_level(load_pack(PACK), level)
140
+ res = run_level(c, _scout_then_branch_policy(), seed=seed)
141
+ assert res.outcome == "win", (
142
+ f"{level}/seed{seed}: scout-then-branch must WIN, got {res.outcome}"
143
+ )
144
+
145
+
146
+ @pytest.mark.parametrize("seed", SEEDS)
147
+ def test_fixed_flank_loses_on_opposite_seed_hard(seed):
148
+ """On hard the outpost flank rotates per seed. A fixed
149
+ unconditional opener must LOSE on the seeds where the outpost
150
+ is on the other flank (and may win the matching seeds) — a
151
+ 2-of-4 play is a structural LOSS under the hard-tier contract."""
152
+ c = compile_level(load_pack(PACK), "hard")
153
+ north = run_level(c, _fixed_flank_policy("north"), seed=seed)
154
+ south = run_level(c, _fixed_flank_policy("south"), seed=seed)
155
+ if seed in HARD_NORTH_SEEDS:
156
+ assert north.outcome == "win", (
157
+ f"hard/seed{seed}: outpost is NORTH — always-NORTH should win"
158
+ )
159
+ assert south.outcome == "loss", (
160
+ f"hard/seed{seed}: outpost is NORTH — always-SOUTH must LOSE"
161
+ )
162
+ else: # HARD_SOUTH_SEEDS
163
+ assert south.outcome == "win", (
164
+ f"hard/seed{seed}: outpost is SOUTH — always-SOUTH should win"
165
+ )
166
+ assert north.outcome == "loss", (
167
+ f"hard/seed{seed}: outpost is SOUTH — always-NORTH must LOSE"
168
+ )
169
+
170
+
171
+ def test_hard_enemy_spawn_axis_has_two_groups():
172
+ """The hard tier must define ≥2 enemy-side spawn_point groups
173
+ (the Wave-9 per-owner seed axis) — the agent base stays fixed."""
174
+ c = compile_level(load_pack(PACK), "hard")
175
+ enemy_sps = {
176
+ a.spawn_point
177
+ for a in c.scenario.actors
178
+ if a.owner == "enemy" and a.spawn_point is not None
179
+ }
180
+ assert len(enemy_sps) >= 2, (
181
+ f"hard needs ≥2 enemy spawn_point groups, got {sorted(enemy_sps)}"
182
+ )
183
+ agent_sps = {
184
+ a.spawn_point
185
+ for a in c.scenario.actors
186
+ if a.owner == "agent" and a.spawn_point is not None
187
+ }
188
+ assert not agent_sps, "agent base must be fixed (no agent spawn_point)"