Spaces:
Running
Running
feat(scenario): scout-and-report — recon and extract (military intelligence / SC2 / drone-surveillance anchor)
Browse files
openra_bench/scenarios/packs/scout-and-report.yaml
ADDED
|
@@ -0,0 +1,254 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# scout-and-report.yaml
|
| 2 |
+
#
|
| 3 |
+
# PERCEPTION capability — reconnaissance-and-EXTRACT cell. The agent
|
| 4 |
+
# must (a) reach a specific FAR enemy building to surface it AND
|
| 5 |
+
# (b) bring the WHOLE scout team back to the base region. Distinct
|
| 6 |
+
# from the sibling perception packs:
|
| 7 |
+
#
|
| 8 |
+
# * scout-and-survive — 1 jeep, ANY return; the SCOUT must
|
| 9 |
+
# survive the discovery round-trip.
|
| 10 |
+
# * scout-detect-enemy-tech — observe N distinct enemy buildings;
|
| 11 |
+
# NO return-to-base clause.
|
| 12 |
+
# * perception-frontier-reading — discover N fog pockets, no return.
|
| 13 |
+
# * perception-target-vs-fog — infer WHICH pocket holds the target.
|
| 14 |
+
#
|
| 15 |
+
# This pack is the only perception cell that requires BOTH:
|
| 16 |
+
# (i) a discovered enemy building (reveal a SPECIFIC far target), AND
|
| 17 |
+
# (ii) the full team (n:2) returns to the base region INTACT (no
|
| 18 |
+
# losses — `units_lost_lte:0`).
|
| 19 |
+
#
|
| 20 |
+
# Real-world anchor: military reconnaissance with mandatory recovery
|
| 21 |
+
# AND full-team intact extraction (intel-and-extract ops); SC2 recon-
|
| 22 |
+
# and-report where losing any unit ruins the report channel; drone
|
| 23 |
+
# surveillance with return-to-base + zero-attrition acceptance.
|
| 24 |
+
#
|
| 25 |
+
# Win idiom: `buildings_discovered_gte:1 AND units_in_region_gte
|
| 26 |
+
# {start, n:2} AND within_ticks:K AND units_lost_lte:0`. The discovery
|
| 27 |
+
# clause forces the round-trip out to the far fact; the n:2 return
|
| 28 |
+
# clause forces the WHOLE team back (one jeep stranded east = LOSS);
|
| 29 |
+
# units_lost_lte:0 forces a NO-LOSS extraction (any jeep dies = LOSS).
|
| 30 |
+
# Brute-attacking the en-route defenders kills at least one jeep
|
| 31 |
+
# (jeeps vs an e1 stack — low DPS, fragile chassis) → bust the
|
| 32 |
+
# units_lost_lte:0 tooth. Stalling busts the clock. One-way commit
|
| 33 |
+
# busts the n:2 return clause.
|
| 34 |
+
#
|
| 35 |
+
# Validate:
|
| 36 |
+
# cd /Users/berta/Projects/OpenRA-Bench && \
|
| 37 |
+
# python3 -m openra_bench.scenarios.validate \
|
| 38 |
+
# openra_bench/scenarios/packs/scout-and-report.yaml
|
| 39 |
+
|
| 40 |
+
meta:
|
| 41 |
+
id: scout-and-report
|
| 42 |
+
title: 'Scout and Report — Reveal the Far Target, Then Extract the Team'
|
| 43 |
+
capability: perception
|
| 44 |
+
real_world_meaning: >
|
| 45 |
+
Military intelligence reconnaissance with mandatory full-team
|
| 46 |
+
recovery. The scout team must surface a specific FAR enemy
|
| 47 |
+
installation AND bring the entire team back INTACT for the
|
| 48 |
+
intelligence to be delivered. Real intel-and-extract doctrine
|
| 49 |
+
treats neither bare discovery NOR a partial-team return as a
|
| 50 |
+
success — the channel is the report (requires return) AND the
|
| 51 |
+
asset is the team (requires zero losses). The pack tests
|
| 52 |
+
whether the agent treats "saw the target" or "one jeep made it
|
| 53 |
+
home" as success rather than the binding "discovered AND full
|
| 54 |
+
team extracted" criterion.
|
| 55 |
+
robotics_analogue: >
|
| 56 |
+
Multi-drone reconnaissance overflight with return-to-base + zero-
|
| 57 |
+
attrition acceptance: the mission counts as a success only if the
|
| 58 |
+
platforms reveal the named target AND ALL platforms recover. A
|
| 59 |
+
one-way commit that surfaces the target but loses any drone is a
|
| 60 |
+
partial failure, not a win — the report channel requires both the
|
| 61 |
+
data AND the platforms back at base for the next mission.
|
| 62 |
+
benchmark_anchor:
|
| 63 |
+
- "military intelligence reconnaissance (recon-and-extract)"
|
| 64 |
+
- "SC2 scout-and-return for info"
|
| 65 |
+
- "drone surveillance with return-to-base"
|
| 66 |
+
- "intel ops: gather then withdraw"
|
| 67 |
+
author: "openra-bench"
|
| 68 |
+
|
| 69 |
+
base_map: rush-hour-arena
|
| 70 |
+
|
| 71 |
+
base:
|
| 72 |
+
# The advertised tool surface includes attack_unit (the model CAN
|
| 73 |
+
# try to attack), but using it on the defenders is the failure
|
| 74 |
+
# mode the pack is designed to punish — any combat engagement
|
| 75 |
+
# with the e1 stacks bleeds at least one jeep and busts
|
| 76 |
+
# units_lost_lte:0.
|
| 77 |
+
agent: {faction: allies}
|
| 78 |
+
enemy: {faction: soviet}
|
| 79 |
+
tools: [move_units, attack_unit, stop]
|
| 80 |
+
planning: true
|
| 81 |
+
termination: {max_ticks: 10000}
|
| 82 |
+
actors:
|
| 83 |
+
# 2× jeep scout team at the west spawn — the start region the
|
| 84 |
+
# team must return to (centre of the units_in_region clause on
|
| 85 |
+
# easy/medium; spawn-matched corner regions on hard).
|
| 86 |
+
- {type: jeep, owner: agent, position: [10, 20], stance: 1, count: 2}
|
| 87 |
+
|
| 88 |
+
levels:
|
| 89 |
+
# ── EASY ────────────────────────────────────────────────────────
|
| 90 |
+
# 2 stationary infantry pickets on the direct east lane (y=20).
|
| 91 |
+
# The fact (discovery target) sits at the far east at (110, 20).
|
| 92 |
+
# Pickets are easy to detour around (one row north or south
|
| 93 |
+
# clears them). Generous clock — even a slow detour-and-return
|
| 94 |
+
# by both jeeps wins. Bruting bleeds a jeep; stalling busts the
|
| 95 |
+
# clock; reaching the fact but stopping east loses on n:2 return.
|
| 96 |
+
easy:
|
| 97 |
+
description: >
|
| 98 |
+
Two jeep scouts at the west (10, 20). The discovery target —
|
| 99 |
+
a specific far enemy fact — sits at (110, 20). Two enemy
|
| 100 |
+
rifle infantry block the direct east lane (around x=60, y=20).
|
| 101 |
+
Drive AROUND the pickets, reveal the fact, and bring BOTH
|
| 102 |
+
jeeps BACK to the start region (radius 5 around (10, 20))
|
| 103 |
+
before the clock expires. Any jeep loss FAILS the mission
|
| 104 |
+
(zero attrition). Bruting the pickets bleeds a jeep (low
|
| 105 |
+
jeep DPS vs infantry); stopping after the discovery in
|
| 106 |
+
enemy territory loses on the n:2 return clause; stalling
|
| 107 |
+
loses on the clock.
|
| 108 |
+
overrides:
|
| 109 |
+
actors:
|
| 110 |
+
- {type: jeep, owner: agent, position: [10, 20], stance: 1, count: 2}
|
| 111 |
+
# Two stationary pickets directly on the y=20 lane — react
|
| 112 |
+
# if approached but don't push (stance:1).
|
| 113 |
+
- {type: e1, owner: enemy, position: [60, 20], stance: 1}
|
| 114 |
+
- {type: e1, owner: enemy, position: [62, 21], stance: 1}
|
| 115 |
+
# Discovery target — the FAR fact (unarmed, persistent so
|
| 116 |
+
# the episode does not auto-end on accidental nearby kills).
|
| 117 |
+
- {type: fact, owner: enemy, position: [110, 20]}
|
| 118 |
+
# A second persistent enemy building so destroying the fact
|
| 119 |
+
# alone can't auto-`done` the run before the return + no-
|
| 120 |
+
# loss clauses are evaluated.
|
| 121 |
+
- {type: barr, owner: enemy, position: [113, 22]}
|
| 122 |
+
win_condition:
|
| 123 |
+
all_of:
|
| 124 |
+
- {buildings_discovered_gte: 1}
|
| 125 |
+
- {units_in_region_gte: {x: 10, y: 20, radius: 5, n: 2}}
|
| 126 |
+
- {units_lost_lte: 0}
|
| 127 |
+
- {within_ticks: 4500}
|
| 128 |
+
fail_condition:
|
| 129 |
+
any_of:
|
| 130 |
+
- {after_ticks: 4501}
|
| 131 |
+
- {not: {units_lost_lte: 0}}
|
| 132 |
+
- {not: {own_units_gte: 1}}
|
| 133 |
+
max_turns: 52
|
| 134 |
+
|
| 135 |
+
# ── MEDIUM ──────────────────────────────────────────────────────
|
| 136 |
+
# +1 controlled variable: a DEFENDED BOTTLENECK. Four stationary
|
| 137 |
+
# infantry pickets span y=18..22 around x=60 — the central lane
|
| 138 |
+
# is closed, the scout pair MUST detour through y<14 or y>26 to
|
| 139 |
+
# make the round trip. The clock still admits the long detour by
|
| 140 |
+
# both jeeps, but not a stall and not a brute attempt (4× e1
|
| 141 |
+
# kills a jeep easily).
|
| 142 |
+
medium:
|
| 143 |
+
description: >
|
| 144 |
+
Two jeep scouts at the west (10, 20). The discovery target —
|
| 145 |
+
a specific far enemy fact — sits at (110, 20). FOUR enemy
|
| 146 |
+
rifle infantry form a defended bottleneck across the central
|
| 147 |
+
lane (around x=58..62, y=16..22), closing the direct east
|
| 148 |
+
route. Detour AROUND the bottleneck (y<14 or y>26), reveal
|
| 149 |
+
the fact, and bring BOTH jeeps BACK to the start region
|
| 150 |
+
(radius 5 around (10, 20)) before the clock. Any jeep loss
|
| 151 |
+
FAILS the mission. Bruting four e1 with jeeps loses; the
|
| 152 |
+
central lane is closed. Stalling loses on the clock; one-way
|
| 153 |
+
commits that strand a jeep east lose on the n:2 return.
|
| 154 |
+
overrides:
|
| 155 |
+
actors:
|
| 156 |
+
- {type: jeep, owner: agent, position: [10, 20], stance: 1, count: 2}
|
| 157 |
+
# Four-infantry bottleneck spanning the central lane.
|
| 158 |
+
# stance:1 — react/defend in place; they don't pursue out
|
| 159 |
+
# of range, but a jeep that walks into the cluster eats
|
| 160 |
+
# four-on-one and dies.
|
| 161 |
+
- {type: e1, owner: enemy, position: [58, 18], stance: 1}
|
| 162 |
+
- {type: e1, owner: enemy, position: [60, 20], stance: 1}
|
| 163 |
+
- {type: e1, owner: enemy, position: [62, 22], stance: 1}
|
| 164 |
+
- {type: e1, owner: enemy, position: [60, 16], stance: 1}
|
| 165 |
+
- {type: fact, owner: enemy, position: [110, 20]}
|
| 166 |
+
- {type: barr, owner: enemy, position: [113, 22]}
|
| 167 |
+
win_condition:
|
| 168 |
+
all_of:
|
| 169 |
+
- {buildings_discovered_gte: 1}
|
| 170 |
+
- {units_in_region_gte: {x: 10, y: 20, radius: 5, n: 2}}
|
| 171 |
+
- {units_lost_lte: 0}
|
| 172 |
+
- {within_ticks: 4500}
|
| 173 |
+
fail_condition:
|
| 174 |
+
any_of:
|
| 175 |
+
- {after_ticks: 4501}
|
| 176 |
+
- {not: {units_lost_lte: 0}}
|
| 177 |
+
- {not: {own_units_gte: 1}}
|
| 178 |
+
max_turns: 52
|
| 179 |
+
|
| 180 |
+
# ── HARD ────────────────────────────────────────────────────────
|
| 181 |
+
# +1 more controlled variable: TWO SEED-VARIED SPAWN GROUPS
|
| 182 |
+
# (N start vs S start) AND a SECOND SCATTERED DEFENDER GROUP so
|
| 183 |
+
# the in-route geometry differs per seed. The bottleneck is also
|
| 184 |
+
# THICKENED with two flank pickets so the inner detour bands
|
| 185 |
+
# (y≈12 / y≈28) are no longer safe — only the FAR edge of the
|
| 186 |
+
# map matching the spawn corner (y≈4 for N start, y≈35 for S
|
| 187 |
+
# start) clears the bottleneck. The two return regions are
|
| 188 |
+
# disjoint (N is (10, 8), S is (10, 32)); a wrong-corner return
|
| 189 |
+
# doesn't satisfy the clause. The discovery target plus a second
|
| 190 |
+
# persistent enemy building remain at the eastern edge so the
|
| 191 |
+
# engine doesn't auto-`done` on a chance kill.
|
| 192 |
+
hard:
|
| 193 |
+
description: >
|
| 194 |
+
Two jeep scouts at the west, started at the NORTH (10, 8) or
|
| 195 |
+
SOUTH (10, 32) corner by seed. The discovery target — a
|
| 196 |
+
specific far enemy fact — sits at (110, 20). SIX stationary
|
| 197 |
+
infantry thicken the central bottleneck (spanning y=14..24
|
| 198 |
+
around x=58..62) and TWO additional flank pickets (one
|
| 199 |
+
north, one south) sit forward at x≈50 so the inner detour
|
| 200 |
+
bands are also contested. The scout team must pick the FAR
|
| 201 |
+
edge of the map matching its spawn corner (y≈4 lane for a
|
| 202 |
+
NORTH start, y≈35 lane for a SOUTH start), weave east to
|
| 203 |
+
surface the fact, climb back to that edge, and RETURN BOTH
|
| 204 |
+
jeeps to the matching corner region (radius 5 around (10, 8)
|
| 205 |
+
NORTH or (10, 32) SOUTH). Any jeep loss FAILS the mission.
|
| 206 |
+
Bruting the bottleneck loses; the wrong-corner edge attrites
|
| 207 |
+
the jeeps before they cross; stalling loses on the clock.
|
| 208 |
+
overrides:
|
| 209 |
+
actors:
|
| 210 |
+
# Two seed-varied agent spawn corners (N vs S). count:2 at
|
| 211 |
+
# each so each spawn group gets a 2-jeep team.
|
| 212 |
+
- {type: jeep, owner: agent, position: [10, 8], stance: 1, count: 2, spawn_point: 0}
|
| 213 |
+
- {type: jeep, owner: agent, position: [10, 32], stance: 1, count: 2, spawn_point: 1}
|
| 214 |
+
# Thickened central bottleneck: four central pickets (as in
|
| 215 |
+
# medium) plus two flank pickets at y=14 / y=24 so the
|
| 216 |
+
# inner detour bands (y≈12 / y≈28) cannot weave through in
|
| 217 |
+
# e1's react range (~5 cells). The far-edge bands at
|
| 218 |
+
# y≈4..5 (north) and y≈34..35 (south) remain safely clear.
|
| 219 |
+
- {type: e1, owner: enemy, position: [58, 18], stance: 1}
|
| 220 |
+
- {type: e1, owner: enemy, position: [60, 20], stance: 1}
|
| 221 |
+
- {type: e1, owner: enemy, position: [62, 22], stance: 1}
|
| 222 |
+
- {type: e1, owner: enemy, position: [60, 16], stance: 1}
|
| 223 |
+
- {type: e1, owner: enemy, position: [60, 14], stance: 1}
|
| 224 |
+
- {type: e1, owner: enemy, position: [60, 24], stance: 1}
|
| 225 |
+
# Second scattered defender group — one forward picket per
|
| 226 |
+
# band at x≈50, placed in the INNER detour band (y=12 /
|
| 227 |
+
# y=28) so they punish a near-edge weave (y≈10 / y≈30) but
|
| 228 |
+
# stay safely OUT of react range from the FAR edge lanes
|
| 229 |
+
# the intended policy uses: y=12 → y=4 is 8 cells (clear),
|
| 230 |
+
# y=28 → y=35 is 7 cells (clear), both well past the
|
| 231 |
+
# ~5-cell e1 react radius. A scout that tries to cut a
|
| 232 |
+
# shorter inner detour walks right into them.
|
| 233 |
+
- {type: e1, owner: enemy, position: [50, 12], stance: 1}
|
| 234 |
+
- {type: e1, owner: enemy, position: [50, 28], stance: 1}
|
| 235 |
+
- {type: fact, owner: enemy, position: [110, 20]}
|
| 236 |
+
- {type: barr, owner: enemy, position: [113, 22]}
|
| 237 |
+
# Hard win: discover + return BOTH jeeps to the spawn-matching
|
| 238 |
+
# corner. `any_of` over the two corners — the team returns to
|
| 239 |
+
# its OWN corner. The full-team-intact bar (units_lost_lte:0)
|
| 240 |
+
# still applies.
|
| 241 |
+
win_condition:
|
| 242 |
+
all_of:
|
| 243 |
+
- {buildings_discovered_gte: 1}
|
| 244 |
+
- any_of:
|
| 245 |
+
- {units_in_region_gte: {x: 10, y: 8, radius: 5, n: 2}}
|
| 246 |
+
- {units_in_region_gte: {x: 10, y: 32, radius: 5, n: 2}}
|
| 247 |
+
- {units_lost_lte: 0}
|
| 248 |
+
- {within_ticks: 4500}
|
| 249 |
+
fail_condition:
|
| 250 |
+
any_of:
|
| 251 |
+
- {after_ticks: 4501}
|
| 252 |
+
- {not: {units_lost_lte: 0}}
|
| 253 |
+
- {not: {own_units_gte: 1}}
|
| 254 |
+
max_turns: 52
|
tests/test_hard_tier.py
CHANGED
|
@@ -387,6 +387,19 @@ UPGRADED = [
|
|
| 387 |
# and a memorised "always defend at y=N" opening cannot
|
| 388 |
# generalise.
|
| 389 |
"econ-contention-with-enemy",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 390 |
]
|
| 391 |
|
| 392 |
# Consciously NOT spawn-varied, with the reason (keeps the curation
|
|
|
|
| 387 |
# and a memorised "always defend at y=N" opening cannot
|
| 388 |
# generalise.
|
| 389 |
"econ-contention-with-enemy",
|
| 390 |
+
# Wave-6 perception pack — military intelligence reconnaissance
|
| 391 |
+
# (recon-and-extract) / SC2 scout-and-return for info / drone
|
| 392 |
+
# surveillance with return-to-base / intel ops: gather then withdraw.
|
| 393 |
+
# Sibling of scout-and-survive (which permits ANY return of a single
|
| 394 |
+
# jeep); this pack requires the WHOLE 2-jeep team to extract INTACT
|
| 395 |
+
# (`units_lost_lte:0` + `units_in_region_gte n:2`) AFTER revealing
|
| 396 |
+
# the specific far enemy fact. Hard defines two agent spawn_point
|
| 397 |
+
# groups (NORTH (10,8) / SOUTH (10,32)) round-robined by seed; the
|
| 398 |
+
# win clause is `any_of` over the two matching corner-return
|
| 399 |
+
# regions, and a second scattered defender group at the inner
|
| 400 |
+
# bands (y=12 / y=28) punishes the near-edge shortcut so a
|
| 401 |
+
# memorised single-detour-band opening cannot generalise.
|
| 402 |
+
"scout-and-report",
|
| 403 |
]
|
| 404 |
|
| 405 |
# Consciously NOT spawn-varied, with the reason (keeps the curation
|
tests/test_scout_and_report.py
ADDED
|
@@ -0,0 +1,371 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""scout-and-report — perception pack with REVEAL + FULL-TEAM EXTRACT.
|
| 2 |
+
|
| 3 |
+
Sibling to scout-and-survive (which requires only ANY return of a
|
| 4 |
+
single jeep). This pack adds the BOTH-jeep extraction bar and the
|
| 5 |
+
specific-far-target reveal bar:
|
| 6 |
+
|
| 7 |
+
* 2× jeep scout team (one jeep = the asset, two = the report channel)
|
| 8 |
+
* far enemy fact at (110, 20) (the specific target)
|
| 9 |
+
* win = buildings_discovered_gte:1 AND n:2 units in start region
|
| 10 |
+
AND units_lost_lte:0 AND within_ticks:4500
|
| 11 |
+
|
| 12 |
+
Scripted policies cover the four bar-defining outcomes per CLAUDE.md
|
| 13 |
+
"no defect, no cheat":
|
| 14 |
+
|
| 15 |
+
* stall → LOSS (clock; nothing discovered)
|
| 16 |
+
* one-way scout (no return) → LOSS (n:2 return clause unmet)
|
| 17 |
+
* brute attack the defenders → LOSS (units_lost_lte:0 — jeep
|
| 18 |
+
dies to picket return-fire)
|
| 19 |
+
* intended discover-and-return → WIN (every level, seeds 1..4)
|
| 20 |
+
|
| 21 |
+
Hard tier additionally rotates the agent spawn corner between NORTH
|
| 22 |
+
(10, 8) and SOUTH (10, 32) by seed; the intended policy routes per-
|
| 23 |
+
corner and returns BOTH jeeps to the SAME corner. The pack's hard
|
| 24 |
+
win clause is `any_of` over the two corner regions so a correct
|
| 25 |
+
round trip to either corner satisfies it.
|
| 26 |
+
"""
|
| 27 |
+
|
| 28 |
+
from __future__ import annotations
|
| 29 |
+
|
| 30 |
+
from pathlib import Path
|
| 31 |
+
|
| 32 |
+
import pytest
|
| 33 |
+
|
| 34 |
+
pytest.importorskip("openra_train", reason="Rust env wheel not installed")
|
| 35 |
+
|
| 36 |
+
pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
|
| 37 |
+
from openra_bench.eval_core import run_level
|
| 38 |
+
from openra_bench.scenarios import load_pack
|
| 39 |
+
from openra_bench.scenarios.loader import compile_level
|
| 40 |
+
|
| 41 |
+
PACK_PATH = (
|
| 42 |
+
Path(__file__).parent.parent
|
| 43 |
+
/ "openra_bench"
|
| 44 |
+
/ "scenarios"
|
| 45 |
+
/ "packs"
|
| 46 |
+
/ "scout-and-report.yaml"
|
| 47 |
+
)
|
| 48 |
+
|
| 49 |
+
LEVELS = ("easy", "medium", "hard")
|
| 50 |
+
SEEDS = (1, 2, 3, 4)
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
# ── Pack-shape tests (cheap; no engine) ───────────────────────────
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def test_pack_compiles_with_three_levels():
|
| 57 |
+
pack = load_pack(PACK_PATH)
|
| 58 |
+
assert pack.meta.id == "scout-and-report"
|
| 59 |
+
assert pack.meta.capability == "perception"
|
| 60 |
+
assert set(pack.levels) == {"easy", "medium", "hard"}
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def test_meta_benchmark_anchor_set():
|
| 64 |
+
"""Seed-taxonomy contract: anchors must call out the military-
|
| 65 |
+
intel / SC2 recon-and-report / drone-surveillance framing."""
|
| 66 |
+
pack = load_pack(PACK_PATH)
|
| 67 |
+
anchors = pack.meta.benchmark_anchor or []
|
| 68 |
+
assert any("military intelligence" in a for a in anchors), anchors
|
| 69 |
+
assert any("SC2 scout" in a for a in anchors), anchors
|
| 70 |
+
assert any("drone surveillance" in a for a in anchors), anchors
|
| 71 |
+
assert any("intel ops" in a for a in anchors), anchors
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def test_every_level_has_fail_condition():
|
| 75 |
+
"""No silent draws — every level emits a real LOSS on timeout
|
| 76 |
+
or scout-death."""
|
| 77 |
+
pack = load_pack(PACK_PATH)
|
| 78 |
+
for lvl in LEVELS:
|
| 79 |
+
c = compile_level(pack, lvl)
|
| 80 |
+
assert c.fail_condition is not None, f"{lvl} missing fail_condition"
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
def test_hard_tier_has_seed_driven_spawn_groups():
|
| 84 |
+
"""Hard must define ≥2 agent spawn_point groups (the binding
|
| 85 |
+
contract from tests/test_hard_tier.py::UPGRADED)."""
|
| 86 |
+
c = compile_level(load_pack(PACK_PATH), "hard")
|
| 87 |
+
sp = {a.spawn_point for a in c.scenario.actors if a.owner == "agent"}
|
| 88 |
+
assert len(sp) >= 2, f"hard needs ≥2 spawn groups, got {sp}"
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def test_tick_budget_aligned_with_max_turns():
|
| 92 |
+
"""within_ticks/after_ticks must be reachable inside max_turns.
|
| 93 |
+
Engine advances ~90 ticks/turn → reachable max = 93 + 90·(N-1)."""
|
| 94 |
+
pack = load_pack(PACK_PATH)
|
| 95 |
+
for lvl in LEVELS:
|
| 96 |
+
level_def = pack.levels[lvl]
|
| 97 |
+
max_turns = level_def.max_turns
|
| 98 |
+
reachable = 93 + 90 * (max_turns - 1)
|
| 99 |
+
win = compile_level(pack, lvl).win_condition.model_dump(exclude_none=True)
|
| 100 |
+
|
| 101 |
+
def _collect(node, key, out):
|
| 102 |
+
if isinstance(node, dict):
|
| 103 |
+
if key in node:
|
| 104 |
+
out.append(node[key])
|
| 105 |
+
for v in node.values():
|
| 106 |
+
_collect(v, key, out)
|
| 107 |
+
elif isinstance(node, list):
|
| 108 |
+
for v in node:
|
| 109 |
+
_collect(v, key, out)
|
| 110 |
+
|
| 111 |
+
wts: list[int] = []
|
| 112 |
+
_collect(win, "within_ticks", wts)
|
| 113 |
+
assert wts, f"{lvl} has no within_ticks leaf"
|
| 114 |
+
for wt in wts:
|
| 115 |
+
assert wt <= reachable, (
|
| 116 |
+
f"{lvl} within_ticks={wt} > reachable={reachable} "
|
| 117 |
+
f"(max_turns={max_turns}) — deadline never bites ⇒ draw"
|
| 118 |
+
)
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def test_return_clause_n2_present_in_win():
|
| 122 |
+
"""Defining feature: the win must enforce n:2 (full-team) return
|
| 123 |
+
at the start region, not just ANY jeep returning."""
|
| 124 |
+
pack = load_pack(PACK_PATH)
|
| 125 |
+
for lvl in LEVELS:
|
| 126 |
+
win = compile_level(pack, lvl).win_condition.model_dump(exclude_none=True)
|
| 127 |
+
found: list = []
|
| 128 |
+
|
| 129 |
+
def _walk(node):
|
| 130 |
+
if isinstance(node, dict):
|
| 131 |
+
if "units_in_region_gte" in node:
|
| 132 |
+
found.append(node["units_in_region_gte"])
|
| 133 |
+
for v in node.values():
|
| 134 |
+
_walk(v)
|
| 135 |
+
elif isinstance(node, list):
|
| 136 |
+
for v in node:
|
| 137 |
+
_walk(v)
|
| 138 |
+
|
| 139 |
+
_walk(win)
|
| 140 |
+
assert found, f"{lvl}: missing units_in_region_gte return clause"
|
| 141 |
+
for spec in found:
|
| 142 |
+
assert int(spec.get("n", 1)) >= 2, (
|
| 143 |
+
f"{lvl}: units_in_region_gte must require n>=2 (full team), "
|
| 144 |
+
f"got {spec}"
|
| 145 |
+
)
|
| 146 |
+
|
| 147 |
+
|
| 148 |
+
def test_units_lost_lte_zero_in_win():
|
| 149 |
+
"""The full-team-intact bar: units_lost_lte:0 must be a win clause
|
| 150 |
+
(the no-attrition extraction teeth)."""
|
| 151 |
+
pack = load_pack(PACK_PATH)
|
| 152 |
+
for lvl in LEVELS:
|
| 153 |
+
win = compile_level(pack, lvl).win_condition.model_dump(exclude_none=True)
|
| 154 |
+
found: list = []
|
| 155 |
+
|
| 156 |
+
def _walk(node):
|
| 157 |
+
if isinstance(node, dict):
|
| 158 |
+
if "units_lost_lte" in node:
|
| 159 |
+
found.append(node["units_lost_lte"])
|
| 160 |
+
for v in node.values():
|
| 161 |
+
_walk(v)
|
| 162 |
+
elif isinstance(node, list):
|
| 163 |
+
for v in node:
|
| 164 |
+
_walk(v)
|
| 165 |
+
|
| 166 |
+
_walk(win)
|
| 167 |
+
assert found, f"{lvl}: missing units_lost_lte clause (no attrition teeth)"
|
| 168 |
+
assert 0 in [int(x) for x in found], (
|
| 169 |
+
f"{lvl}: units_lost_lte must include 0 (zero-attrition extract); "
|
| 170 |
+
f"got {found}"
|
| 171 |
+
)
|
| 172 |
+
|
| 173 |
+
|
| 174 |
+
# ── Scripted policies ─────────────────────────────────────────────
|
| 175 |
+
|
| 176 |
+
|
| 177 |
+
def _stall(_rs, Command):
|
| 178 |
+
return [Command.observe()]
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
def _one_way_scout(rs, Command):
|
| 182 |
+
"""Push BOTH jeeps east to near the fact and STOP there. Discovers
|
| 183 |
+
the fact but never returns ⇒ units_in_region_gte clause unmet ⇒
|
| 184 |
+
LOSS. Models the "discovery is the goal" misread that the pack
|
| 185 |
+
exists to catch."""
|
| 186 |
+
units = rs.get("units_summary", []) or []
|
| 187 |
+
if not units:
|
| 188 |
+
return [Command.observe()]
|
| 189 |
+
cmds = []
|
| 190 |
+
for u in units:
|
| 191 |
+
uid = str(u["id"])
|
| 192 |
+
ux, _uy = u["cell_x"], u["cell_y"]
|
| 193 |
+
if ux < 50:
|
| 194 |
+
# Climb to the y=4 edge lane to get past the central
|
| 195 |
+
# bottleneck (works on every level), then run east.
|
| 196 |
+
cmds.append(Command.move_units([uid], target_x=50, target_y=4))
|
| 197 |
+
elif ux < 105:
|
| 198 |
+
cmds.append(Command.move_units([uid], target_x=108, target_y=4))
|
| 199 |
+
elif ux < 108:
|
| 200 |
+
# Dip down to the fact's row to reveal it.
|
| 201 |
+
cmds.append(Command.move_units([uid], target_x=110, target_y=20))
|
| 202 |
+
else:
|
| 203 |
+
# ARRIVED — stop in enemy territory, never return.
|
| 204 |
+
cmds.append(Command.stop([uid]))
|
| 205 |
+
return cmds
|
| 206 |
+
|
| 207 |
+
|
| 208 |
+
def _brute_attack(rs, Command):
|
| 209 |
+
"""Drive both jeeps straight east through the picket stack at
|
| 210 |
+
x=58..62, y=18..22. The jeeps auto-fire opportunistically in
|
| 211 |
+
range and draw return fire from the picket stack. Jeep DPS vs an
|
| 212 |
+
infantry stack is poor; at least one jeep dies before crossing,
|
| 213 |
+
so units_lost_lte:0 fires the LOSS clause."""
|
| 214 |
+
units = rs.get("units_summary", []) or []
|
| 215 |
+
if not units:
|
| 216 |
+
return [Command.observe()]
|
| 217 |
+
cmds = []
|
| 218 |
+
for u in units:
|
| 219 |
+
uid = str(u["id"])
|
| 220 |
+
cmds.append(Command.move_units([uid], target_x=110, target_y=20))
|
| 221 |
+
return cmds
|
| 222 |
+
|
| 223 |
+
|
| 224 |
+
def _intended_discover_and_return(rs, Command):
|
| 225 |
+
"""Both jeeps: climb to the spawn-matching edge lane, run east,
|
| 226 |
+
dip down to (110, 20) to surface the fact, climb back to the
|
| 227 |
+
edge lane, traverse west, drop to the spawn cell. The team
|
| 228 |
+
returns INTACT to the SAME spawn corner.
|
| 229 |
+
|
| 230 |
+
Phases:
|
| 231 |
+
1. Outbound — climb to edge lane (y≈4 from NORTH, y≈35 from
|
| 232 |
+
SOUTH), run east, dip to reveal the fact at (110, 20).
|
| 233 |
+
2. Return — climb back to the edge lane (so the west traverse
|
| 234 |
+
doesn't pass through the central bottleneck at y=14..24
|
| 235 |
+
on hard), traverse west on the edge, drop to the start
|
| 236 |
+
cell.
|
| 237 |
+
"""
|
| 238 |
+
state = _intended_discover_and_return._state # type: ignore[attr-defined]
|
| 239 |
+
units = rs.get("units_summary", []) or []
|
| 240 |
+
if not units:
|
| 241 |
+
return [Command.observe()]
|
| 242 |
+
|
| 243 |
+
# Latch each jeep's start cell once (its actual spawn) so the
|
| 244 |
+
# return trip targets its OWN spawn corner.
|
| 245 |
+
if not state["starts"]:
|
| 246 |
+
# Use ALL units' spawn cells (the agent's persistent ID set
|
| 247 |
+
# is stable across turns).
|
| 248 |
+
for u in units:
|
| 249 |
+
state["starts"][str(u["id"])] = (u["cell_x"], u["cell_y"])
|
| 250 |
+
|
| 251 |
+
# Detect discovery (fact in enemy_buildings_summary).
|
| 252 |
+
eb = rs.get("enemy_buildings_summary", []) or []
|
| 253 |
+
if any(b.get("type") == "fact" for b in eb):
|
| 254 |
+
state["discovered"] = True
|
| 255 |
+
|
| 256 |
+
cmds = []
|
| 257 |
+
for u in units:
|
| 258 |
+
uid = str(u["id"])
|
| 259 |
+
ux, uy = u["cell_x"], u["cell_y"]
|
| 260 |
+
sx, sy = state["starts"].get(uid, (ux, uy))
|
| 261 |
+
# Spawn-matching edge lane.
|
| 262 |
+
edge_y = 35 if sy >= 26 else 4
|
| 263 |
+
|
| 264 |
+
if not state["discovered"]:
|
| 265 |
+
if ux < 25:
|
| 266 |
+
cmds.append(Command.move_units([uid], target_x=25, target_y=edge_y))
|
| 267 |
+
elif ux < 105:
|
| 268 |
+
cmds.append(Command.move_units([uid], target_x=108, target_y=edge_y))
|
| 269 |
+
else:
|
| 270 |
+
# Final dip down to the fact's row to reveal it.
|
| 271 |
+
cmds.append(Command.move_units([uid], target_x=110, target_y=20))
|
| 272 |
+
else:
|
| 273 |
+
# Return: climb back to edge lane first, then traverse
|
| 274 |
+
# west, then drop to the start cell.
|
| 275 |
+
if abs(uy - edge_y) > 4 and ux > 100:
|
| 276 |
+
cmds.append(Command.move_units([uid], target_x=ux, target_y=edge_y))
|
| 277 |
+
elif ux > 25:
|
| 278 |
+
cmds.append(Command.move_units([uid], target_x=20, target_y=edge_y))
|
| 279 |
+
else:
|
| 280 |
+
cmds.append(Command.move_units([uid], target_x=sx, target_y=sy))
|
| 281 |
+
return cmds
|
| 282 |
+
|
| 283 |
+
|
| 284 |
+
_intended_discover_and_return._state = { # type: ignore[attr-defined]
|
| 285 |
+
"starts": {},
|
| 286 |
+
"discovered": False,
|
| 287 |
+
}
|
| 288 |
+
|
| 289 |
+
|
| 290 |
+
def _reset_intended_state():
|
| 291 |
+
_intended_discover_and_return._state["starts"] = {} # type: ignore[attr-defined]
|
| 292 |
+
_intended_discover_and_return._state["discovered"] = False # type: ignore[attr-defined]
|
| 293 |
+
|
| 294 |
+
|
| 295 |
+
# ── Solvency: intended WINS every (level, seed) ───────────────────
|
| 296 |
+
|
| 297 |
+
|
| 298 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 299 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 300 |
+
def test_intended_discover_and_return_wins(level, seed):
|
| 301 |
+
_reset_intended_state()
|
| 302 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 303 |
+
res = run_level(c, _intended_discover_and_return, seed=seed)
|
| 304 |
+
assert res.outcome == "win", (
|
| 305 |
+
f"intended discover+return must WIN on {level} s={seed}; "
|
| 306 |
+
f"got {res.outcome} tick={res.signals.game_tick} "
|
| 307 |
+
f"bds={len(res.signals.enemy_buildings_seen_ids)} "
|
| 308 |
+
f"lost={res.signals.units_lost}"
|
| 309 |
+
)
|
| 310 |
+
|
| 311 |
+
|
| 312 |
+
# ── Stability: every non-win pattern is a real reachable LOSS ─────
|
| 313 |
+
|
| 314 |
+
|
| 315 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 316 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 317 |
+
def test_stall_loses(level, seed):
|
| 318 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 319 |
+
res = run_level(c, _stall, seed=seed)
|
| 320 |
+
assert res.outcome == "loss", (
|
| 321 |
+
f"stall must LOSE on {level} s={seed}; got {res.outcome}"
|
| 322 |
+
)
|
| 323 |
+
|
| 324 |
+
|
| 325 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 326 |
+
@pytest.mark.parametrize("level", LEVELS)
|
| 327 |
+
def test_one_way_scout_loses(level, seed):
|
| 328 |
+
"""One-way commit: discovers the fact but stops in enemy
|
| 329 |
+
territory. The n:2 return clause is never satisfied ⇒ LOSS on
|
| 330 |
+
the clock, never a draw."""
|
| 331 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 332 |
+
res = run_level(c, _one_way_scout, seed=seed)
|
| 333 |
+
assert res.outcome == "loss", (
|
| 334 |
+
f"one-way scout must LOSE on {level} s={seed}; got {res.outcome} "
|
| 335 |
+
f"bds={len(res.signals.enemy_buildings_seen_ids)} "
|
| 336 |
+
f"lost={res.signals.units_lost}"
|
| 337 |
+
)
|
| 338 |
+
|
| 339 |
+
|
| 340 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 341 |
+
@pytest.mark.parametrize("level", ["medium", "hard"])
|
| 342 |
+
def test_brute_attack_loses_medium_and_hard(level, seed):
|
| 343 |
+
"""Brute through the picket stack: jeep DPS is insufficient vs
|
| 344 |
+
the 4+× e1 bottleneck (medium/hard); at least one jeep dies to
|
| 345 |
+
return-fire ⇒ units_lost_lte:0 fires ⇒ LOSS.
|
| 346 |
+
|
| 347 |
+
Easy is intentionally permissive (only 2 pickets) — a brute
|
| 348 |
+
play CAN survive on easy. The strict bar is medium/hard."""
|
| 349 |
+
c = compile_level(load_pack(PACK_PATH), level)
|
| 350 |
+
res = run_level(c, _brute_attack, seed=seed)
|
| 351 |
+
assert res.outcome == "loss", (
|
| 352 |
+
f"brute attack must LOSE on {level} s={seed}; got {res.outcome} "
|
| 353 |
+
f"lost={res.signals.units_lost} "
|
| 354 |
+
f"bds={len(res.signals.enemy_buildings_seen_ids)}"
|
| 355 |
+
)
|
| 356 |
+
|
| 357 |
+
|
| 358 |
+
# ── Spawn variation contract (hard) ───────────────────────────────
|
| 359 |
+
|
| 360 |
+
|
| 361 |
+
@pytest.mark.parametrize("seed", SEEDS)
|
| 362 |
+
def test_hard_seeds_produce_distinct_starts(seed):
|
| 363 |
+
"""Hard's two spawn_point groups must actually round-robin —
|
| 364 |
+
seeds 1..4 collectively must produce ≥2 distinct starts. The
|
| 365 |
+
spawn-variation contract is enforced cross-seed by
|
| 366 |
+
tests/test_hard_tier.py::test_curated_hard_still_compiles_and_runs."""
|
| 367 |
+
c = compile_level(load_pack(PACK_PATH), "hard")
|
| 368 |
+
res = run_level(c, _stall, seed=seed)
|
| 369 |
+
assert res.outcome == "loss"
|
| 370 |
+
sp = {a.spawn_point for a in c.scenario.actors if a.owner == "agent"}
|
| 371 |
+
assert len(sp) >= 2
|